Author Topic: Wunderground data volumes  (Read 1214 times)

0 Members and 1 Guest are viewing this topic.

Offline johnmn

  • Member
  • *
  • Posts: 29
Wunderground data volumes
« on: July 07, 2017, 08:26:00 PM »
Hi. I'm trying to find out how I can minimise the data traffic between my Meteobridge and Wunderground. I have my updates happening every 15 minutes, so that means nearly 3000 GET posts to wunderground a month.

This (plus reponses and other overheads, whatever they might be) seems to be using up to 130MB of data a month, which works out to over 33KB per update!

This seems like a lot, and I'd like to cut it down if I could. Can anyone offer suggestions or observations? Wunderground do not seem to be very helpful in this area.

Thanks.

Offline docbee

  • Moderator
  • Forecaster
  • *****
  • Posts: 853
    • smartbedded
Re: Wunderground data volumes
« Reply #1 on: July 14, 2017, 05:46:25 PM »
WU uploads packets are much smaller, about 200 bytes I would say.
founder of smartbedded.com - home of meteohub, meteoplug, meteobridge, meteostick

Offline johnmn

  • Member
  • *
  • Posts: 29
Re: Wunderground data volumes
« Reply #2 on: July 14, 2017, 11:11:29 PM »
That's what I have always thought, based on my understanding of the PWS upload protocol. Lately I have been wondering what the returned response might be.

[A test from my PC suggests that all that is returned is "success<CR/LF>"]

Otherwise I can't explain what all the data use is down to!

Thanks for your response. This is a frustrating puzzle.

Offline eee3

  • Member
  • *
  • Posts: 3
Re: Wunderground data volumes
« Reply #3 on: August 01, 2017, 12:36:39 PM »
johnmn, something else is eating up your data.  A typical upload via the WU PWS upload protocol is under 2KB (including the protocol overhead).  If you’re sending over TLS it’s a little larger but still nowhere near the numbers you’ve calculated.  You should be able to do 5 minute intervals and stay well under your limit.

Offline johnmn

  • Member
  • *
  • Posts: 29
Re: Wunderground data volumes
« Reply #4 on: August 01, 2017, 11:53:55 PM »
Thanks eee3.

As a result of a thread in the Wunderground message area, (http://www.wxforum.net/index.php?topic=32557.0) I used Wireshark to monitor the packet traffic, and found a (frankly) ridiculous amount of protocol stuff, including what seemed to be messages associated with redirecting to Amazon Web Services, is inflating the traffic by as much as fifteen times!

It seems at this moment sending ~300 bytes up and 10 back is not possible.  :sad:

Wunderground have (just today) verified that nothing can be done to reduce the data volume (although I don't really know what they looked at).

I may end up configuring the Meteobridge to send data to our own website and log and display the data there. I can exert more control there.

Thanks for your response