Author Topic: Upload Minimum Interval Limitations  (Read 1064 times)

0 Members and 1 Guest are viewing this topic.

Offline havtrail

  • Senior Contributor
  • ****
  • Posts: 206
    • Haverford Weather Station
Upload Minimum Interval Limitations
« on: August 28, 2018, 11:12:22 AM »
I am putting this in this topic area because it generally pertains to a comparison of limits on how often one can upload data from a PWS (mainly cellular or cloud-server based) to a commercial hosting site.

As an Onset HOBO RX3003-based weather station, I am limited to a minimum interval for connecting and uploading of every 10 minutes, and that the station's logging interval must be half of that (5 minutes) or less. At these most-frequently allowed settings, We are only using about 20% of the monthly cellular data allotment under the HOBOlink plan, which would indicate I could upload much more frequently without exceeding that limit. So I called Onset and asked if our minimum connection interval could be halved to 5 minutes. I was told no. In fact, they mentioned that I could pop my own SIM card into the unit, but on second thought, they said, even having done that, the logger unit would still only connect every 10 minutes minimum.

After hanging up, I got the usual follow-up customer survey email, so I wrote the following in the comments section. I have waited to post this here to make sure any anger of the moment has gone away, even though what I wrote was, I believe, entirely based on fact.

Response to Onset follow-up survey Aug. 8, 2018:

"Onset HOBOlink seems to limit cellular data upload connections to a minimum interval of every 10 minutes for no apparent technical reason. I am told I cannot do a more-frequent upload to HOBOlink and then Weather Underground, yet I am only using 20% of the monthly permitted data volume. This is contributing to our Weather Underground-based kiosk and on-line displays of "present" conditions to be regularly up to an hour or more behind. This is unacceptable for public display. All others in my immediate area who use WU, with less expensive monitoring stations and software / hosting, are updating between every 10 seconds and 1 minute. Once every 10 minutes does not suffice for public information, environmental awareness and STEM encouragement purposes. I must use WU for the display because the HOBOlink display is not "kiosk capable" (auto-updating, attention-getting, flexible in format, and controllable in auto-display). I am dissatisfied with Onset's HOBOlink service limitations and will be highlighting these factual limitations publicly now, so that other potential weather monitoring system purchasers can be aware of them as part of their decision-making. This does not even include the multi-day HOBOlink server and AT&T outages that I have experienced over the last few months since launching our unit in mid-April.

"That said, the RX3003 device itself has been working well at gathering data, and Tech Support rep [name was here] has been very professional in his manner. But they are both working within a system the limitations of which are readily apparent to someone trying to make good use of it for public display and awareness."

In looking about, I noticed that Davis' weatherlink.com server/host package, for example, offers a 5 minute connection interval, for less than the $300 annual fee of Onset's best package.

Rich K.
Onset HOBO RX2102 Cellular
https://www.havtrail.com/weather/
NEWA https://newa.cornell.edu Haverford, PA

Offline galfert

  • Global Moderator
  • Forecaster
  • *****
  • Posts: 6822
Re: Upload Minimum Interval Limitations
« Reply #1 on: August 28, 2018, 03:03:44 PM »
I think this is a good sub to have posted this on. Thanks for sharing. This will get the word out best as anything I can think of.

That said, you may only be using 20% of their plan, but there are probably reasons why they won't let you up the interval. See they are paying AT&T in bulk. So if you up your transmission frequency and others do too then they have to renegotiate and pay more to AT&T. Still no excuse for limiting sending every 10 minutes.

But to play devil's advocate, there are two ways of using a weather station. Live view and for historical data analysis. It seems that the hardware you have with its limited data plan is more geared to just historical data analysis rather than for live viewing. I upload to Weather Underground every 15 seconds but I also save to my SQL database only every 10 minutes. For me to produce custom graphs all I need is a data point every 10 minutes. I have thought about making it every 5 minutes but I've yet to decide to make that change. I upload to CWOP every 5 minutes, AWEKAS every 5 minutes, WeatherForYou every 10 minutes (graphs look terrible if less, too cluttered).

Weather Underground even though I may upload every 15 seconds they throw it all out and only keep data every 5 minutes. So the 15 seconds is only good for live viewing. Which makes sense and that is a good thing.

In conclusion I think 5 minutes is the reasonable upload frequency for data archiving and analysis purposes, and probably good enough for some sort of mildly delayed almost live semblance of what just occurred.

By only uploading every 10 minutes you might be missing some wind gusts. So I'd pressure them to reconsider.


« Last Edit: August 28, 2018, 03:11:41 PM by galfert »
Ecowitt GW1000 | Meteobridge on Raspberry Pi
WU: KFLWINTE111  |  PWSweather: KFLWINTE111
CWOP: FW3708  |  AWEKAS: 14814
Windy: pws-f075acbe
Weather Underground Issue Tracking
Tele-Pole

Offline havtrail

  • Senior Contributor
  • ****
  • Posts: 206
    • Haverford Weather Station
Re: Upload Minimum Interval Limitations
« Reply #2 on: August 28, 2018, 03:20:31 PM »
Thanks for the insights. Is there a standard or a recommended logging interval for CWOPS or AWEKAS participants?

Rich K.
Onset HOBO RX2102 Cellular
https://www.havtrail.com/weather/
NEWA https://newa.cornell.edu Haverford, PA

Offline galfert

  • Global Moderator
  • Forecaster
  • *****
  • Posts: 6822
Re: Upload Minimum Interval Limitations
« Reply #3 on: August 28, 2018, 06:31:16 PM »
Both CWOP and AWEKAS the fastest upload frequency is 5 minutes.

CWOP used to recommend every 10 minutes. Then it became every 9 or 11 minutes (because of server problems with everyone trying to upload at the full exact 05, 10 minute marks of the hour). But newest documentation I found said 5 minutes is okay.
I haven't really read the documentation on AWEKAS and what they recommend. I just noticed that 5 minutes was the fastest allowed so I stuck with that.

My Meteobridge that I use to upload for most of these doesn't upload at the exact time interval configured. Instead it is always a few seconds off delayed so after an hour or so the actual minute mark changes between one hour and the next. What I'm saying is that it doesn't try and keep uploading at the same exact minute marks hour to hour...it drifts by a few seconds and then eventually you end up on a different minute mark for that hour. Except for mySQL database which I set for the full 10 minute marks...I could change that though with a quick setting.

WeatherForYou (PWS) actually allows every single minute uploads. But their graphs get completely unreadable at that frequency. I played with it and I liked 10 minute results the best because even with every 5 minutes or less it was all scrunched up.

« Last Edit: August 28, 2018, 08:50:46 PM by galfert »
Ecowitt GW1000 | Meteobridge on Raspberry Pi
WU: KFLWINTE111  |  PWSweather: KFLWINTE111
CWOP: FW3708  |  AWEKAS: 14814
Windy: pws-f075acbe
Weather Underground Issue Tracking
Tele-Pole