Topic: Hardware/Software Tinkerer, buy vs build

[impala454]

Hi all, long time lurker on these forums and finally registered.  I'm a software developer by day and electronics/software tinkerer in my spare time.  I'm also a weather nerd and live in the Houston, Texas area (where we tend to have rather interesting weather!).  I have looked into building my own piece-meal kind of weather station as well as looked at some of the top off the shelf stations that come recommended on this board.  All that, combined with the layout of my house/backyard, derives into a set of requirements and questions I'm asking for help with from you fine folks.


Requirements

• Rain measurement - Able to measure Hurricane Harvey levels rain like we got in 2017 (e.g. 6" in a single hour).
• Temperature - Able to accurately measure from the top of a pole in full sun (assuming some sort of fan on it?).
• Wired - Both power and data need to be wired.
• Cost - I don't mind spending $500-600 on something that fulfills all my needs, but I do not want to spend money on inside consoles or other bits that do data parsing/display.  That's the part I can/want to do myself.

Desirements (yes it's a legit engineering term  )

• Sturdy - I'd like the station to be able to handle hurricanes, that's the most interesting data to get!  The last thing I want to do is have to go pull down my weather station because a storm is coming.
• Upkeep - I understand these are precision devices and need maintenance, but would like to keep it to a minimum (what's reasonable?).
• Software - If I buy an off the shelf station, I would prefer it be one someone has already sniffed/decoded, preferably via open source software.  I don't mind writing said software but it's always easier when someone else has already done it.  If it's ease of software vs accuracy/sturdiness though, I'd choose the latter and write the software myself.

Questions

• Mounting - Much of my yard is shaded by trees, leaving the "best as far as I know" location re-using an old satellite dish mount, which I would then mount a pole to (with extra support if needed).  It would always receive full sun (except at night time har har ).  Assuming this is okay, how high would I need to mount the station from the roof?  Does it need to go over the highest roof point?  Is there a worry of temperatures being wrong due to the heat absorption of the roof?  Or wind readings being inaccurate due to the roof itself?  I could potentially mount it about 10' away from the roof but it would still be pretty close.
• Components - Fellow electronics nerds, do you know what components the off the shelf stations are made of?  E.g. I saw the SHT31 is used in some of them for temperature and is available from mouser for $8 as of this writing.  If I were to piece-meal a station together which raw components would be best?  If all I bought was an electronic rain gauge which stand-alone one is best (keeping in mind the requirements above)?
• Consoles/Displays - I notice many of the off the shelf stations come bundled with interior displays or devices which decomm the data and upload to various services for you.  I have no desire to spend money on these if I can avoid it.  Are there vendors I can go to and only purchase the instrument itself without those "extras"?
• Existing Projects/Documentation - I would plan to fully document a build, but would love to see others' builds if they are widely available.  I saw the Other Weather Station Hardware section but no concise lists or anything.
• Critters - Do folks have problems with critters defiling their weather station?  I have IP cameras at my house and occasionally have to knock down wasp nests or dirt daubers, is that a big problem for anyone?  Do birds regularly try to land on your station?

I think that's all for this huge post for now.  Any feedback/help is very much appreciated.

[Devin_Noel]

I have a stack of Raspberry Pi in my office & actually had a pile of sensors breadboarded & working on one for an automate & monitor the wife's chicken coop project. I ended up just shelving it & going with an off the shelf solution (SmartThings based). Im an IT security professional & the prospect of Internet connected code I'm capable of cranking out was just to ugly. I'm sure some of the vendors are worse security wise, but there is a better chance they will maintain firmware than i will.

I'm here looking at replacemement plans for my Accurite 5 in 1 station that died a few months ago. I got it, or put it on my Christmas list over building my own for the same reasons I shelved my home rolled coop automation project. My only big quandary at the moment is Accurite abandoning the 5 in 1 system & it's old Internet connectivity bridge for the new Atlas system. Not happy about that or not being able to reuse working parts with Atlas. But still looking at a new Atlas.

Sent from my SM-N950U using Tapatalk

[pfletch101]

I like my Davis VP2, which is hooked up (via its console and a logger) to a Raspberry running weewx, but I don't have a wide enough experience of other hardware to know whether there is something better for you. No automatic rain recording device will retain high accuracy in hurricane-level rain, and it is even arguable whether there is a 'gold standard' under these extreme conditions. Otherwise, some comments on some of your specific questions:

Questions

Mounting - Much of my yard is shaded by trees, leaving the "best as far as I know" location re-using an old satellite dish mount, which I would then mount a pole to (with extra support if needed).  It would always receive full sun (except at night time har har ).  Assuming this is okay, how high would I need to mount the station from the roof?  Does it need to go over the highest roof point?  Is there a worry of temperatures being wrong due to the heat absorption of the roof?  Or wind readings being inaccurate due to the roof itself?  I could potentially mount it about 10' away from the roof but it would still be pretty close.

If you want reasonably accurate component readings, you need to have your anemometer relatively high up in clear air (e.g. 5' above a roof peak), and the rest of your instruments about 5' above the ground, ideally over turf and a reasonable distance away from buildings and trees. The realities of your property may prevent ideal placement, but measuring temperature and humidity near or above a building is usually a bad idea.

Components - Fellow electronics nerds, do you know what components the off the shelf stations are made of?  E.g. I saw the SHT31 is used in some of them for temperature and is available from mouser for $8 as of this writing.  If I were to piece-meal a station together which raw components would be best?  If all I bought was an electronic rain gauge which stand-alone one is best (keeping in mind the requirements above)?

I am also something of an electronics nerd, but I question whether 'rolling your own' weather station is worth the time and effort - not so much from the electronic viewpoint, but mainly from the 'packaging' viewpoint. Getting valid readings depends as much on a housing design that exposes sensors as closely as possible to ambient conditions while protecting them from the potential bad consequences of this exposure. Weather station manufacturers spend a good deal of time and effort getting this balance right. You are very unlikely to do it better.

Consoles/Displays - I notice many of the off the shelf stations come bundled with interior displays or devices which decomm the data and upload to various services for you.  I have no desire to spend money on these if I can avoid it.  Are there vendors I can go to and only purchase the instrument itself without those "extras"?

Yes, but how much is the time you will then spend writing and/or debugging the software that is decoding the data worth? I view my display as a place to get a quick overview of the weather and a housing for the logger that feeds my record-keeping and -displaying software.

Critters - Do folks have problems with critters defiling their weather station?  I have IP cameras at my house and occasionally have to knock down wasp nests or dirt daubers, is that a big problem for anyone?  Do birds regularly try to land on your station?

Yes! (at least to part of the question). I have lost two anemometer cables to attacks by squirrels (I think - I didn't actually see the varmints chewing on the cables). Since I don't climb on roofs (so fixing that sort of problem means hiring a roofer) and don't 'publish' my observations, I now have my anemometer in a much less than ideal location where there is a short cable run and I can get at it, but in my next house I will be running the anemometer wiring through armored cable. I have not had a lot of problems with birds, but another reason for having your rain measuring device where you can get at it is that you will need to keep it clean and clear - whether from bird debris, leaves, or insects.

[impala454]

Thanks pfletch, that's some great info!  A few comments/questions:

No automatic rain recording device will retain high accuracy in hurricane-level rain, and it is even arguable whether there is a 'gold standard' under these extreme conditions.

Interesting.  Does this include the ridiculous expensive ones too?  E.g. I go to the  Harris County Flood website to see all their rain gauges on occasion, what types of gauges do you suppose they use?  Or is it just generally accepted that they'll be less accurate during a hurricane?  All that said, I was eyeballing the aluminum Rainwise RGA (pdf datasheet) as a possibility.  Are there any other go-tos?  How would that compare to the Davis VP2?

The realities of your property may prevent ideal placement, but measuring temperature and humidity near or above a building is usually a bad idea.

The realities of my property (plus the WAF) mean likely one of those will have to occur.  So given the choices of up high on a pole above the roof, or 5' above the ground yet very close to the house, which would you choose (for the non-anemometer sensors)?

Getting valid readings depends as much on a housing design that exposes sensors as closely as possible to ambient conditions while protecting them from the potential bad consequences of this exposure.

Copy for sure on that one, I suppose this mainly only affects temp/humidity/pressure sensors right?  I have been looking at the general idea of the Stevenson screen and saw that you can purchase pre-built ones or might build my own.  All that said I appreciate the amount of extra work that would be and will take that into account in the choice between buy or build.

Yes, but how much is the time you will then spend writing and/or debugging the software that is decoding the data worth? I view my display as a place to get a quick overview of the weather and a housing for the logger that feeds my record-keeping and -displaying software.

I actually really enjoy the software/debugging part so it's worth it for me.  I already have a setup which includes a server running a database and Grafana instance for various other sensors around the house.  Does the weewx software allow you to grab all the raw data and ship it elsewhere?  And does it require having the console or can you read directly from the sensor package?

Other notes taken- Love the idea of armored cable or at least PVC to protect it.  Also good to know on the rain gauge cleaning and such.  I had it in my mind that that could be up on the pole above the roof but I'd be much less likely to maintain it if I had to get it down too often.


I'm still not sold either way, buy vs build, but this is some great information to get going.  I can sympathize with what @Devin_Noel said.  I too have a box or three of sensors, micros, breadboards, etc, though I do complete a project every now and then.  I suppose it'll also come down to cost and if I think I can build something better (I saw there are a lot of newer components available).  That said here's a few components I've taken note of so far if anyone cares to chime in:

Components

• Sensiron SHT85 (temp + humidity) - datasheet (pdf)
• TE MS5611 (pressure) - datasheet (pdf) - if this one looks to be too much of a pain I'd go with the next option,
• Bosch BMP280 - datasheet (pdf)
• Rainwise RGA - datasheet (pdf) - I don't see pricing info but have emailed them.  If the Aluminum one doesn't make sense I'd go with the plastic 111.  I'd definitely like to hear about any other options too.

Off The Shelf
So far I've looked at these as off the shelf options (just based off searches here/googling).  Any others I should think about, give the requirements?

• AcuRite Atlas
• Davis Vantage Pro 2
• Ambient Weather WS-2902 Osprey

[johnd]

One option I'd be tempted to consider would be getting a cabled Davis VP2 ISS (the complete sensor assembly, but excluding the console) which delivers its data on an RS422 connection. Although Davis are good about publicly documenting the API from their loggers (fitted to a console), AFAIK they've never disclosed the internal API for ISS to console communications. However, several users have sniffed out the message format from the wireless ISS units and I believe that the cabled units use a similar if not identical API.

NB I'm suggesting you consider a cabled ISS rather than wireless because it sounds like you're happy with the principle of using cable from outside to in, and it also avoids the extra complication of setting up a compatible wireless receiver (although a Smartbedded Meteostick is a neat option for a preconfigured wireless receiver for a wireless VP2). You'll see that there are several options for using commercially available kit to generate the raw data while leaving you plenty of scope to further process the raw data as you see fit.

[pfletch101]

Thanks pfletch, that's some great info!  A few comments/questions:

No automatic rain recording device will retain high accuracy in hurricane-level rain, and it is even arguable whether there is a 'gold standard' under these extreme conditions.

Interesting.  Does this include the ridiculous expensive ones too?  E.g. I go to the  Harris County Flood website to see all their rain gauges on occasion, what types of gauges do you suppose they use?  Or is it just generally accepted that they'll be less accurate during a hurricane?  All that said, I was eyeballing the aluminum Rainwise RGA (pdf datasheet) as a possibility.  Are there any other go-tos?  How would that compare to the Davis VP2?

All tipping bucket devices have accuracy problems at extreme rain rates because of the difficulty of accounting properly for rain flow while the bucket is in the process of tipping. Obviously, the heavier the rain, the greater proportion of the total measuring time is accounted for by the transitions. The other problem with hurricanes is wind. All rainfall measuring systems work best in reasonably calm conditions. The rain amounts collected in very high wind conditions may depend more on details of airflow patterns over and around the bucket than on how much water is in a given volume of that air.

The realities of your property may prevent ideal placement, but measuring temperature and humidity near or above a building is usually a bad idea.

The realities of my property (plus the WAF) mean likely one of those will have to occur.  So given the choices of up high on a pole above the roof, or 5' above the ground yet very close to the house, which would you choose (for the non-anemometer sensors)?

It is really a judgement call, based on what parameters are most important to you. T/H will be most distorted by a rooftop location - rainfall probably somewhat less so. Rainfall measurements are likely to be seriously distorted by a location too close to a building - T/H measurements less so, particularly if you are not next to a south wall. If you want to post a Google view of your site, others more expert than I may be able to suggest possible locations.

Yes, but how much is the time you will then spend writing and/or debugging the software that is decoding the data worth? I view my display as a place to get a quick overview of the weather and a housing for the logger that feeds my record-keeping and -displaying software.

I actually really enjoy the software/debugging part so it's worth it for me.  I already have a setup which includes a server running a database and Grafana instance for various other sensors around the house.  Does the weewx software allow you to grab all the raw data and ship it elsewhere?  And does it require having the console or can you read directly from the sensor package?

weewx saves all the data in a standard database - SQLite on the Pi by default, but you can configure it to use MySQL/MariaDB. You do normally need to have a console with a logger or another intermediate device as an intermediary between the station and your software, but see JohnD's post.

[Storm017]

Have my Wireless Davis VP2 ISS (Wireless) interfacing a RPI3 B+, via a NESDR Smart SDR. Loaded on the RPI3 B+ is the latest copy of Weewx (3.9.2) using weewx-rtldavis as the driver. Have several additional components interfacing the RPI3 B+ via I2C. They included a DS3231 Real Time Clock; a BME280; and a AS3935 Module. All data is being saved locally using the default DB (SQLite3) and being displayed locally via Apache2 web server. The Weewx skin installed is weewx-Belchertown with MQTT enabled.

Using the same concept to interface my Acurite 5 N 1 sensor array and various 433MHz sensors to an additional RPI3 B+.

[impala454]

Storm017: That's pretty awesome.  I'm already using a SDR to read my utility meters (with the rtlamr library).  That lightning module looks pretty nice too.  I'll add that to my possibility list.  Did you have to build a fancy antenna for it?  I always figured I'd join the Blitzortung group at some point but not there yet.

johnd: The cabled one sounds good to me, when you say just getting the Davis VP2 ISS, do you mean this one?  It pictures some kind of console with it.  Is that the thing I would talk RS422 to, or you mean the ISS itself?  I didn't see any choices for the ISS without that console.

pfletch101: Thanks again for all your help!  Yeah the solution may point more towards the DIY project since in that case I could split up the pieces and put each one in its optimal place.  That said, here's a google maps photo, my house in the middle (north is up).  On the roof would be ideal, but there is an open void that might work, assuming it would pass WAF (which is generally a bad assumption )

[mwall]

Does the weewx software allow you to grab all the raw data and ship it elsewhere?  And does it require having the console or can you read directly from the sensor package?

you can use the weewx-influx extension to save data directly into an influxdb timeseries database, and from the plot directly in grafana dashboards. 

https://github.com/matthewwall/weewx-influx

as of 22sep2019 there are 24 uploaders listed on the weewx wiki.

https://github.com/weewx/weewx/wiki

you can read data directly from the sensors using the weewx-sdr driver if the sensors emit data using an encoding understood by rtl_433.  otherwise, if the console has a communication interface, there is probably a weewx driver that will read data from the console

https://github.com/matthewwall/weewx-sdr

there are also weewx drivers that collect data from various solar controllers (sunny web box, enphase, outback mate, tracer, victron), power monitors such as the per-circuit brultech GEM, ultrasonic sensors (for tank depth, snow depth, etc), particulate sensors (sds011).  you can integrate the data using weewx itself, or by feeding into an MQTT or influxdb or mysql system.  (we had postgres working at one point, but it has not been maintained/updated)

m

[Storm017]

Storm017: Did you have to build a fancy antenna for it?  I always figured I'd join the Blitzortung group at some point but not there yet.

Did not build an external antenna for the module.  Overall the AS3935 module works fine, just need to have it in an area that is not to noisy. I can detect lightning over 20 miles away. I do get 1 or 2 false positives every couple of days, but I think that is way the board configured for the environment it is located in.

For Davis Products check out https://www.scaledinstruments.com/product-category/davis-instruments/

[johnd]

johnd: The cabled one sounds good to me, when you say just getting the Davis VP2 ISS, do you mean this one?  It pictures some kind of console with it.  Is that the thing I would talk RS422 to, or you mean the ISS itself?  I didn't see any choices for the ISS without that console.

I mean just the ISS itself, eg 6322C - see eg:

https://www.scaledinstruments.com/shop/davis-instruments/sensor-suites/vantage-pro2-series-sensor-suites/cabled-vantage-pro2-series-sensor-suites/davis-6322c-cabled-vantage-pro2-iss/

[impala454]

mwall: Thanks!  Looks like a lot of good open source support.

Storm017: Awesome, I'll definitely check out the module.  The building a crazy antenna is what's kept me from joining the Blitzortung crowd.

johnd: Good deal, thanks, that's a much nicer price.  What exactly does the +$200 calibration get you?  If you don't choose that option do I need to calibrate it somehow when I get it?

[johnd]

johnd: Good deal, thanks, that's a much nicer price.  What exactly does the +$200 calibration get you?  If you don't choose that option do I need to calibrate it somehow when I get it?

Don't worry about it - it's simply a calibration check. It's there mostly because some professional buyers want documented confirmation that an individual station example meets its published specification. But I really wouldn't bother.

[impala454]

Awesome.  I work at NASA so trust me I understand paying for useless documentation 

[johnd]

And I should have added/underlined: If you do decide to go down the 6322C route I've suggested or something similar, then you may need to do some detective work on the RS422 packet formats. There's no polling of the sensors required, the ISS simply emits various data packets at timed intervals. There's quite a bit of info online, especially about the wireless format and it shouldn't be difficult to do anyway, but identifying and parsing the various binary packets will be stage 1 of any development project.

[pfletch101]

And I should have added/underlined: If you do decide to go down the 6322C route I've suggested or something similar, then you may need to do some detective work on the RS422 packet formats. There's no polling of the sensors required, the ISS simply emits various data packets at timed intervals. There's quite a bit of info online, especially about the wireless format and it shouldn't be difficult to do anyway, but identifying and parsing the various binary packets will be stage 1 of any development project.

I'm not sure that I would minimize the difficulty of dealing directly with the Vantage Pro ISS's wireless transmissions. The frequency jumping protocol makes things more interesting  and I haven't seen any 'off the shelf' driver software for it (though it is a long time since I looked at the question).

[impala454]

And I should have added/underlined: If you do decide to go down the 6322C route I've suggested or something similar, then you may need to do some detective work on the RS422 packet formats. There's no polling of the sensors required, the ISS simply emits various data packets at timed intervals. There's quite a bit of info online, especially about the wireless format and it shouldn't be difficult to do anyway, but identifying and parsing the various binary packets will be stage 1 of any development project.

I'm not sure that I would minimize the difficulty of dealing directly with the Vantage Pro ISS's wireless transmissions. The frequency jumping protocol makes things more interesting  and I haven't seen any 'off the shelf' driver software for it (though it is a long time since I looked at the question).

I believe johnd has been suggesting this whole time (inline with my requests) to use the cabled ISS.  Though it's still no small task if someone else hasn't already done it.

Side question then:  If I choose not to buy the console, but then decide later I need it, can I just buy that part?

[CW2274]

And I should have added/underlined: If you do decide to go down the 6322C route I've suggested or something similar, then you may need to do some detective work on the RS422 packet formats. There's no polling of the sensors required, the ISS simply emits various data packets at timed intervals. There's quite a bit of info online, especially about the wireless format and it shouldn't be difficult to do anyway, but identifying and parsing the various binary packets will be stage 1 of any development project.

I'm not sure that I would minimize the difficulty of dealing directly with the Vantage Pro ISS's wireless transmissions. The frequency jumping protocol makes things more interesting  and I haven't seen any 'off the shelf' driver software for it (though it is a long time since I looked at the question).

Side question then:  If I choose not to buy the console, but then decide later I need it, can I just buy that part?

Yes. Most all of us shop Davis stuff here:

https://www.scaledinstruments.com/product-category/davis-instruments/parts/

[Storm017]

I'm not sure that I would minimize the difficulty of dealing directly with the Vantage Pro ISS's wireless transmissions. The frequency jumping protocol makes things more interesting  and I haven't seen any 'off the shelf' driver software for it (though it is a long time since I looked at the question).

@pfletch101 Have no issues with capturing Vantage Pro ISS's wireless transmissions. I'm using the weewx-rtldavis driver.  Thanks goes out to Lucas for working on the driver. https://www.wxforum.net/index.php?topic=36267.0
https://github.com/lheijst/weewx-rtldavis

[impala454]

Just FYI in case anyone is following, the folks at Rainwise got back to me and said the Rainview RGA (pdf link) is $445.00 vs the Rainview 111 is $69.95 (without the counter).  The main difference being the RGA is aluminum and "The assemblies inside are similar but there are better bearings with the RGA."

Do folks here have any experience with comparing the Rainview 111 vs the Davis version or others?

[pfletch101]

I'm not sure that I would minimize the difficulty of dealing directly with the Vantage Pro ISS's wireless transmissions. The frequency jumping protocol makes things more interesting  and I haven't seen any 'off the shelf' driver software for it (though it is a long time since I looked at the question).

@pfletch101 Have no issues with capturing Vantage Pro ISS's wireless transmissions. I'm using the weewx-rtldavis driver.  Thanks goes out to Lucas for working on the driver. https://www.wxforum.net/index.php?topic=36267.0
https://github.com/lheijst/weewx-rtldavis

I have been trying to follow that thread, but it has not been clear to me whether US hardware/frequencies is/are now fully supported.

[CW2274]

Do folks here have any experience with comparing the Rainview 111 vs the Davis version or others?

Several of us. The RW is unquestionably the superior tipper. It has a larger catch, 8" vs 6.5", twice as accurate on paper (2% vs 4%), and is more consistent. Not only, the RW is easily hardwired into the VP2 ISS, works exactly the same as the Davis tipper.

[Storm017]

I have been trying to follow that thread, but it has not been clear to me whether US hardware/frequencies is/are now fully supported.

@pfletch101.  I don't have every sensor available for the Vantage Pro2 , but so far for what I have the driver is tracking against what is being displayed on the console.

[johnd]

I believe johnd has been suggesting this whole time (inline with my requests) to use the cabled ISS.  Though it's still no small task if someone else hasn't already done it.

Just to be clear: Decoding the wireless packets has two parts:

1. Configuring an SDR to receive the wireless transmissions accurately, with good S/N and no/minimal packet loss;

2. Decoding the individual sensor messages to extract the raw data;

People have done all of this with varying degrees of success (MB Pro and Meteostick seem very successful) so all of it is demonstrably possible and documented in various places. My suggestion of using 6332C was to eliminate step [1] altogether (other than setting up a working RS422 interface), which just leaves step [2]. I've seen comments that the sensor message format for the cabled interface is identical to that for the wireless interface, but I've yet to come across anything online that provides proof or documented detail. However, various users and indeed businesses seem to have achieved it successfully (ie with cabled ISS units) so I can't believe that it's a major obstacle - indeed I'd suggest that it's likely to be pretty straightforward.

Edit: I suspect that pp6-7 of https://www.smartbedded.com/files/manual-en.pdf might be a good guide, though units are likely different.

Side question then:  If I choose not to buy the console, but then decide later I need it, can I just buy that part?

Yes. Virtually all Davis components and spares are readily available online, eg from Scaled Instruments.

[impala454]

johnd: Yes I definitely understood (but thanks for the clarification).  I have a fair amount of experience configuring SDRs so that's not a huge deal, my desire for a wired setup is more just from a personal preference.  If I'm installing a device of any kind around my house I wire it if at all humanly possible.  Regarding what's out there, I found this write up for directly reading a Davis anemometer via arduino, so I have some confidence it can be done without too much trouble.