Author Topic: Grabbing my data from Weatherlink.com  (Read 7407 times)

0 Members and 1 Guest are viewing this topic.

Offline andywin

  • Member
  • *
  • Posts: 6
Grabbing my data from Weatherlink.com
« on: August 31, 2013, 05:27:05 PM »
I’m running a Davis VP2 using Weatherlink IP. I’m looking to download my raw station data from weatherlink.com without having to use the Weatherlink software (I want to import data into a database and don't want to rely on Weatherlink). I know that I can get the recent conditions using an XML request: http://weatherlink.com/xml.php?user=yourusername&pass=yourpassword. Is there a way to download the archived data from weatherlink.com (without having to use weatherlink)?
Alternatively, I’m also testing out Meteobridge loaded on a TP-Link TL-MR3020 on a separate VP2 console. Again, I can set up pushes of current and recent data, but am trying to figure out a way to export or access several days worth of data in case internet goes down.
Thanks

Offline Weather Display

  • Forecaster
  • *****
  • Posts: 2611
    • West Coast Road Weather Data
Re: Grabbing my data from Weatherlink.com
« Reply #1 on: August 31, 2013, 05:33:15 PM »
I think Davis would frown on this myself
Brian
info@weather-display.com
http://www.weather-display.com

Offline johnd

  • Forecaster
  • *****
  • Posts: 4849
    • www.weatherstations.co.uk
Re: Grabbing my data from Weatherlink.com
« Reply #2 on: September 01, 2013, 04:33:07 AM »
Is there a way to download the archived data from weatherlink.com (without having to use weatherlink)?

Yes: Consult the Davis SerialTechRef document.

@Brian: I'm not sure that Davis do/would object provided the service isn't abused. (If they do consider you to be abusing it then they can disable your access pretty smartly.) The weatherlink.com server - presumably some sort of small cluster or private cloud - is scaled to be handling uploads from (currently) 13000 plus users every minute and due to grow further now that Connect has been launched. And there must be the expectation of a corresponding level of data downloads (typically as web pages of course, and what's the difference between a web page file and a simple text file, except that the latter will be smaller).

You can't request data in this way without providing your password and Davis might have a reasonable expectation that you would keep this private. So provided you (as the primary user) were the only person requesting the archive download and if you were to ask for an archive data update no more than once an hour (not much point in doing it more often anyway, the archive uploads are only hourly!), but perhaps daily, then the server load will be quite small. If you do want to disseminate the data more widely and automatically then you would need to use your own PC/server as the source of any further copies. (Edit: Actually I don't mean copies as such, more that if you do wish to make the data available more widely it's likely to be in the form of e.g. further processed web pages, which would automatically put the load on your server, not the Davis server.)
« Last Edit: September 01, 2013, 05:30:09 AM by johnd »
Prodata Weather Systems
Prodata's FAQ/support site for Davis stations
Includes many details on 6313 Weatherlink console.
UK Davis Premier Dealer - All Davis stations, accessories and spares
Cambridge UK

Sorry, but I don't usually have time to help with individual issues by email unless you are a Prodata customer. Please post your issue in the relevant forum section here & I will comment there if I have anything useful to add.

Offline andywin

  • Member
  • *
  • Posts: 6
Re: Grabbing my data from Weatherlink.com
« Reply #3 on: September 02, 2013, 10:50:18 PM »
Thanks Johnd. I was searching through an earlier SerialTechRef version, which was why I couldn't find anything about it.
For the benefit of anyone else interested, the web download protocol can be found at http://www.davisnet.com/support/weather/download/VantageSerialProtocolDocs_v261.pdf - pages 36-37. Next challenge will be to figure out how to parse out a binary encoded octet stream of the raw 52 byte archive records.
It was actually Davis that suggested I download my data directly from the WL.com site. For current conditions, they asked that I not send the query faster than every 10 minutes. For the archived data, I'd probably only do it once or twice per day.

Offline d_l

  • Forecaster
  • *****
  • Posts: 1700
  • Slide Mtn - Mt Rose
    • Thomas Creek Estates neighborhood weather
Re: Grabbing my data from Weatherlink.com
« Reply #4 on: September 26, 2013, 04:24:25 PM »
I don't understand.  Can the Meteobridge not query the WLIP directly and also allow the WLIP to upload to weatherlink.com if you wish?  The Meteohub can handle this easily.
--Dave--

Wireless VP2 w/ solar, 24hr FARS, Heater, (Envoy-WLIP)*3-Meteohub, plus custom VP2 @ 26', WL 6.0.4, WU & W4U=KNVRENO37 NetcamXL

People always talk about the weather, but they never do anything about it.  Not me.  I'm gonna measure it.  https://www.tceweather.com

Offline jruys

  • Yo
  • Senior Contributor
  • ****
  • Posts: 221
    • Highlands Ranch Weather
Re: Grabbing my data from Weatherlink.com
« Reply #5 on: November 13, 2013, 10:39:55 PM »
If you need help parsing the 52 byte record I can probably help you if you are using .net.  I have code to parse the WLK file structures which are pretty close to the wetherlink.com record structure.

Offline inconvergent

  • Member
  • *
  • Posts: 1
    • inconvergent
Re: Grabbing my data from Weatherlink.com
« Reply #6 on: January 06, 2014, 01:43:44 PM »
For the benefit of anyone else interested, the web download protocol can be found at http://www.davisnet.com/support/weather/download/VantageSerialProtocolDocs_v261.pdf - pages 36-37.

Thanks for that. I've been looking for this for some time. Now to parse it in Python ...
weather site: http://nordaasen.no