Author Topic: Serial Sensor Project  (Read 569 times)

0 Members and 1 Guest are viewing this topic.

Offline TheBushPilot

  • Senior Member
  • **
  • Posts: 60
  • Storm chaser, struggling Edlog user
Serial Sensor Project
« on: August 14, 2022, 01:07:13 AM »
For a "fun" "little" project, I tasked myself with getting a data logger and serial sensor to talk to each other. This'll only take a week or two right? :? Not even close! \:D/ Over the span of a few months I managed to learn the basics of serial communications and old Edlog programming language Campbell Scientific data loggers used to use. The struggle was real trying to get this stuff to work up to this point. I want to kick my younger self for not doing the obvious and learn the fundamentals (circa 2019). Crazy what a little background knowledge will do. Sometimes it's as simple as doing a google search of the basics before you get into something.

Any who, here we are, trudging out of the valley of despair.

There is not a lot of documentation on the serial control port I/O function aside from what's found in manuals. Even so, it's uh, very limited and niche toward use case. Of course granted what's more niche than configuring a CR23X and 61302L to communicate asynchronously via RS-232 ASCII through Edlog? I suppose you can't really blame the internet for not having resources on such an obscure thing lol. Despite this, it was done, somehow.

3:  Port Serial I/O (P15)
 1: 1        Reps
 2: 00       I/O Configuration Code (index (--) to disable RX buffering)
 3: 0000     TX after CTS or Delay (0.01 sec units) before TX
 4: 00       Control Ports Configuration Code
 5: 0000     Start Loc for TX [ ______________ ]
 6: 0000     Number of Locs to TX
 7: 0000     Termination Character for RX
 8: 0000     RX Buffer Size or Max Chars to RX if Par 2 indexed (--)
 9: 0000     Time Out for CTS (TX) and/or RX (0.01 sec units)
10: 0000     Start Loc for RX [ ______________ ]
11: 1.0      Multiplier for RX
12: 0.0      Offset for RX

That right there has been the headache and frustration of my life for the last 7 1/2 weeks. Sure once you know what you're doing it's a total cakewalk, but man, that learning curve is steeper than the Gateway Arch. Same for Edlog, great program but really only once you know what you're doing.

The configuration:

As already mentioned, R. M. Young 61302L barometric pressure sensor and Campbell Scientific CR23X Micrologger. These are communicating via asynchronous ASCII RS-232. Logging at 1 second intervals, and using the panel temp to correct for mean sea-level pressure as I don't have an external temperature sensor. The sensor itself is kinda off..? I suppose. After manually reverting all values except for calibration points, it still reports being slightly off from neighboring stations so I said screw it and made an additive offset after running it through the hypsometric equation. And sure, my elevation is different from neighboring stations, but I had tested it before when I used to live near an airport and was still off.

[ You are not allowed to view attachments ]

11:  Z=F x 10^n (P30)
 1: 1.443    F
 2: 00       n, Exponent of 10
 3: 8        Z Loc [ A              ]

12:  Z=F x 10^n (P30)
 1: 274.593  F
 2: 00       n, Exponent of 10
 3: 9        Z Loc [ C              ]

13:  Z=F x 10^n (P30)
 1: -5.257   F
 2: 00       n, Exponent of 10
 3: 12       Z Loc [ EXP            ]

14:  Z=F x 10^n (P30)
 1: 1.0      F
 2: 00       n, Exponent of 10
 3: 13       Z Loc [ ONE            ]

15:  Z=F x 10^n (P30)
 1: 5.0      F
 2: 00       n, Exponent of 10
 3: 17       Z Loc [ FIVE           ]

16:  Z=X+Y (P33)
 1: 4        X Loc [ panel_T_c      ]
 2: 9        Y Loc [ C              ]
 3: 10       Z Loc [ B              ]

17:  Z=X/Y (P38)
 1: 8        X Loc [ A              ]
 2: 10       Y Loc [ B              ]
 3: 11       Z Loc [ D              ]

18:  Z=X-Y (P35)
 1: 13       X Loc [ ONE            ]
 2: 11       Y Loc [ D              ]
 3: 14       Z Loc [ E              ]

19:  Z=X^Y (P47)
 1: 14       X Loc [ E              ]
 2: 12       Y Loc [ EXP            ]
 3: 15       Z Loc [ F              ]

20:  Z=X*Y (P36)
 1: 3        X Loc [ bp_raw         ]
 2: 15       Y Loc [ F              ]
 3: 16       Z Loc [ bp_msl_raw     ]

21:  Z=X-Y (P35)
 1: 16       X Loc [ bp_msl_raw     ]
 2: 17       Y Loc [ FIVE           ]
 3: 5        Z Loc [ MSLP           ]

22:  Z=X (P31)
 1: 3        X Loc [ bp_raw         ]
 2: 2        Z Loc [ StPr           ]

Now that, that right there was very fun figuring out. Since Edlog is a pretty basic program, it doesn't do certain operations right out of the gate. This requires the user to do it the long way. Very, very tedious long way. But hey, it works. :lol: (I will probably inevitably make a separate thread on the breakdown of these calculations.)

So now it reports correct pressure. Allegedly...

Great, the logger and sensor are communicating without issue. Let's go home. Haha, not so fast. Oooone small problem: packet loss. Sure the thrill of seeing the little numbers change on the screen was super because HEY! it's working and ya know, it's never worked before until now. But alas, a different problem still loomed above with seemingly no resolve.

 [ You are not allowed to view attachments ]

Exhibit A: What I just about went insane over. Those streaks you see there on that graph represent when the logger had a table overrun. Where essentially the program execution took too long to complete so it cut it off early, leaving part or no value in the table location. Sure, you're logging at hourly or even minute intervals this is not a problem as both devices have time to communicate. It gets interesting when you bring that interval time down to seconds. Or in my case, second intervals. How do we remedy this?

Sure a baud rate of 4800 is more than enough for an input string of like 11 characters (61302L outputs "1000.00CRLF"). Which is right, to a point. While around 97% of the time it logs correctly, that other 3% is undoubtedly problematic (and annoying). Both in post processing and graphically, I'd rather not have to deal with noise. Plus if you can do it right the first time why not? Right? Hence why through dumb luck managed to create a filter within the logger program itself to parse out the erroneous data values. It separates these values but also I guess replaces said missing value with the next data point to maintain continuity..? I'm not entirely sure how it works but it does and am not changing it. :roll:

4:  If (X<=>F) (P89)
 1: 1        X Loc [ raw            ]
 2: 4        <
 3: 500.00   F
 4: 30       Then Do

     5:  Data Table (P84)^13164
      1: 0.0      Seconds into Interval
      2: 0        Each Time This Instruction Executes (with Time Stamp)
      3: 0.0      (0 = auto allocate, -x = redirect to inloc x)
      4: PACKET_LOSS               Table Name

     6:  Sample (P70)^24602
      1: 1        Reps
      2: 1        Loc [ raw            ]

7:  Else (P94)

8:  Data Table (P84)^13964
 1: 0.0      Seconds into Interval
 2: -32      Each Time This Instruction Executes (without Time Stamp)
 3: -3       (0 = auto allocate, -x = redirect to inloc x)
 4:                           Table Name

9:  Sample (P70)^30901
 1: 1        Reps
 2: 1        Loc [ raw            ]

10:  End (P95)

Where the IF statement checks if the value is below that of the minimum the pressure sensor is able to report. Then, that bad value is saved to a separate data table where one can view at a later time. The ELSE denotes that if the value checks as good, it will be reassigned to the raw string input and sent down the line to be processed through MSL correction. In doing this, the actual value saved to final storage is filtered to only report valid data. Why it up for discussion, but does and will be left as is. :grin: So far it (tested duration of like 8 hours thus far) has proven to be very reliable.

 [ You are not allowed to view attachments ]

And after all is set and done, the 2 boxes magically output a barometric pressure value every second of both station and mean sea-level. The realized end goal of this project was to understand serial sensors and data logger communications. I look to apply these concepts to some mobile mesonets that utilize serial sensors and eventually some "glorified home ASOS station". Since we don't have the luxury of government issued grants for the latest and greatest data loggers, being able to use older but still completely usable systems will be monumental in getting us started with a project my friends and I have going.

Supplemental information on how I got these two devices to work together.

Edit: I realize now this is more of a software discussion than hardware lol.
« Last Edit: August 14, 2022, 10:37:32 AM by TheBushPilot »
"There is always more than one thing influencing anything we are trying to measure." ~ Sherman Fredrickson
"Do it right or don't do it at all."

Met Instruments Project