WXforum.net

Weather Related Organizations => WeatherUnderground => Topic started by: frankpc on July 22, 2019, 11:52:53 PM

Title: Data obtained with API not current
Post by: frankpc on July 22, 2019, 11:52:53 PM
The data contained within the results of the API query I'm using is not current.  I am using the following format as an example:

https://api.weather.com/v2/pws/history/all?stationId=KKSSHAWN68&format=xml&units=m&date=20190722&apiKey=123456789

I just ran it at 2200 local time.  And the most recent time in the xml file is 0500.

If I run: https://api.weather.com/v2/pws/history/all?stationId=KKSSHAWN68&format=xml&units=m&date=20190721&apiKey=123456789 , (yesterday) all 24 hours are returned.

Other PWS' data obtained from the API may be 10 hours behind.  the delay time varies a lot.

Is there an API query available to yield all data in XML format beginning at midnight local time? 

I need to obtain all 24 hours of PWS data from the previous day at 0500 in the morning.  And it seems the data for the previous day isn't available at 5 AM in the morning.

Help with this would be appreciated. Thanks!
Title: Re: Data obtained with API not current
Post by: rlee171 on July 25, 2019, 11:15:48 AM
Frankpc didn't get any responses to his question. I am curious as well what the proper format is for looking at our data using the API...  My Dashboard is showing good data when at times my wxwuhistory shows empty or partial data. Would like to compare the two. Just for the fun of it I sent an email to WU and Victoria replied and asked how she could help. Sure would like to pose the question of the mis-match in data but need to confirm it first.
Title: Re: Data obtained with API not current
Post by: pjschaffer on July 25, 2019, 12:05:47 PM
Frank,

I just gave it a try with the following:

https://api.weather.com/v2/pws/history/all?stationId=KCASANTA813&format=xml&units=e&date=20190725&apiKey=*********************

The return was everything today up until the time I ran it.  I haven't seen the issue you're describing.  I typically use json as a data format, but that really shouldn't matter.

Just for the heck of it, I tried your query as well:

https://api.weather.com/v2/pws/history/all?stationId=KKSSHAWN68&format=xml&units=m&date=20190725&apiKey=***********

This query also returned all data from midnight to current.

These queries were run at around 9am PDT.  I'm not sure if the results would be different before 5am.
Title: Re: Data obtained with API not current
Post by: wvdkuil on July 25, 2019, 05:17:16 PM
Frank,

I just gave it a try with the following:

https://api.weather.com/v2/pws/history/all?stationId=KCASANTA813&format=xml&units=e&date=20190725&apiKey=*********************

The return was everything today up until the time I ran it.  I haven't seen the issue you're describing.  I typically use json as a data format, but that really shouldn't matter.

Just for the heck of it, I tried your query as well:

https://api.weather.com/v2/pws/history/all?stationId=KKSSHAWN68&format=xml&units=m&date=20190725&apiKey=***********

This query also returned all data from midnight to current.

These queries were run at around 9am PDT.  I'm not sure if the results would be different before 5am.
when you run the query multiple times, say every hour, the data is exactly the same every time.
So, only the first time you run today’s query, the data is current, all other times it is old.
When running the same query on another computer that result is again current, but all query’s after the first again the same data.
Both computers return their own not current data.

I am using CURL and request .json

Wim
Title: Re: Data obtained with API not current
Post by: saratogaWX on July 25, 2019, 05:31:15 PM
It also seems to be a cache issue with the requests.  They're returning a header for
https://api.weather.com/v2/pws/history/all?stationId=KCASARAT1&format=json&units=e&date=20190725&apiKey=*apikey* of

Quote
HTTP/2.0 200 OK
content-type: application/json; charset=UTF-8
access-control-allow-origin: *
vary: Accept-Encoding
x-trace-token: 1058076e-622f-4ad0-9807-6e622f8ad081
x-region: us-east-1
surrogate-control: ESI/1.0
content-encoding: gzip
cache-control: max-age=74058
date: Thu, 25 Jul 2019 21:17:37 GMT
content-length: 4671
X-Firefox-Spdy: h2
even though the request had
Quote
Pragma: no-cache
Cache-Control: no-cache
in it.
This seems to happen with browsers and with curl (with the added headers for no cache).  GRRR..
Right now at 14:30, my latest data seems to be from 10:30 for today.
Title: Re: Data obtained with API not current
Post by: pjschaffer on July 25, 2019, 06:16:57 PM
It also seems to be a cache issue with the requests.  They're returning a header for


I can confirm that I'm seeing the same thing, although not consistently.  Very frustrating
Title: Re: Data obtained with API not current
Post by: saratogaWX on July 26, 2019, 02:10:41 PM
I've sent Victoria a note about the missing API data in July, 2019.  Oddly enough, the old WXDailyHistory.asp page does return the full month's data, so there's a disconnect somewhere inside the api.weather.com servers (or unrefreshed Akamai cache servers associated with api.weather.com).