Author Topic: Data obtained with API not current  (Read 375 times)

0 Members and 1 Guest are viewing this topic.

Offline frankpc

  • Lakepoint Weather
  • Member
  • *
  • Posts: 35
  • Build it and they will come.
Data obtained with API not current
« on: July 22, 2019, 11:52:53 PM »
The data contained within the results of the API query I'm using is not current.  I am using the following format as an example:

https://api.weather.com/v2/pws/history/all?stationId=KKSSHAWN68&format=xml&units=m&date=20190722&apiKey=123456789

I just ran it at 2200 local time.  And the most recent time in the xml file is 0500.

If I run: https://api.weather.com/v2/pws/history/all?stationId=KKSSHAWN68&format=xml&units=m&date=20190721&apiKey=123456789 , (yesterday) all 24 hours are returned.

Other PWS' data obtained from the API may be 10 hours behind.  the delay time varies a lot.

Is there an API query available to yield all data in XML format beginning at midnight local time? 

I need to obtain all 24 hours of PWS data from the previous day at 0500 in the morning.  And it seems the data for the previous day isn't available at 5 AM in the morning.

Help with this would be appreciated. Thanks!
WU: Lakepointe KKSSHAWN122
.-- -... -.... --.- .-- .---

Offline rlee171

  • Senior Member
  • **
  • Posts: 76
    • Peaceful Hills Weather
Re: Data obtained with API not current
« Reply #1 on: July 25, 2019, 11:15:48 AM »
Frankpc didn't get any responses to his question. I am curious as well what the proper format is for looking at our data using the API...  My Dashboard is showing good data when at times my wxwuhistory shows empty or partial data. Would like to compare the two. Just for the fun of it I sent an email to WU and Victoria replied and asked how she could help. Sure would like to pose the question of the mis-match in data but need to confirm it first.
Davis Vantage Pro 2
Cumulus v.1.9.4
Hot Springs National Park, AR
hsnpar.com

Offline pjschaffer

  • Member
  • *
  • Posts: 5
    • PV System
Re: Data obtained with API not current
« Reply #2 on: July 25, 2019, 12:05:47 PM »
Frank,

I just gave it a try with the following:

https://api.weather.com/v2/pws/history/all?stationId=KCASANTA813&format=xml&units=e&date=20190725&apiKey=*********************

The return was everything today up until the time I ran it.  I haven't seen the issue you're describing.  I typically use json as a data format, but that really shouldn't matter.

Just for the heck of it, I tried your query as well:

https://api.weather.com/v2/pws/history/all?stationId=KKSSHAWN68&format=xml&units=m&date=20190725&apiKey=***********

This query also returned all data from midnight to current.

These queries were run at around 9am PDT.  I'm not sure if the results would be different before 5am.

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1445
    • My PWS at Leuven Belgium Europe
Re: Data obtained with API not current
« Reply #3 on: July 25, 2019, 05:17:16 PM »
Frank,

I just gave it a try with the following:

https://api.weather.com/v2/pws/history/all?stationId=KCASANTA813&format=xml&units=e&date=20190725&apiKey=*********************

The return was everything today up until the time I ran it.  I haven't seen the issue you're describing.  I typically use json as a data format, but that really shouldn't matter.

Just for the heck of it, I tried your query as well:

https://api.weather.com/v2/pws/history/all?stationId=KKSSHAWN68&format=xml&units=m&date=20190725&apiKey=***********

This query also returned all data from midnight to current.

These queries were run at around 9am PDT.  I'm not sure if the results would be different before 5am.
when you run the query multiple times, say every hour, the data is exactly the same every time.
So, only the first time you run today’s query, the data is current, all other times it is old.
When running the same query on another computer that result is again current, but all query’s after the first again the same data.
Both computers return their own not current data.

I am using CURL and request .json

Wim
Support for Leuven-Template & Scripts  at this forum
Support for the PWS_Dashboard (HWS-template) for WD a.o.  http://pwsdashboard.weather-template.com/
Script sponsors https://support.leuven-template.eu/userlistScripts.php
Template sponsors https://support.leuven-template.eu/userlist.php

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 7399
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Data obtained with API not current
« Reply #4 on: July 25, 2019, 05:31:15 PM »
It also seems to be a cache issue with the requests.  They're returning a header for
https://api.weather.com/v2/pws/history/all?stationId=KCASARAT1&format=json&units=e&date=20190725&apiKey=*apikey* of

Quote
HTTP/2.0 200 OK
content-type: application/json; charset=UTF-8
access-control-allow-origin: *
vary: Accept-Encoding
x-trace-token: 1058076e-622f-4ad0-9807-6e622f8ad081
x-region: us-east-1
surrogate-control: ESI/1.0
content-encoding: gzip
cache-control: max-age=74058
date: Thu, 25 Jul 2019 21:17:37 GMT
content-length: 4671
X-Firefox-Spdy: h2
even though the request had
Quote
Pragma: no-cache
Cache-Control: no-cache
in it.
This seems to happen with browsers and with curl (with the added headers for no cache).  GRRR..
Right now at 14:30, my latest data seems to be from 10:30 for today.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline pjschaffer

  • Member
  • *
  • Posts: 5
    • PV System
Re: Data obtained with API not current
« Reply #5 on: July 25, 2019, 06:16:57 PM »
It also seems to be a cache issue with the requests.  They're returning a header for


I can confirm that I'm seeing the same thing, although not consistently.  Very frustrating

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 7399
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Data obtained with API not current
« Reply #6 on: July 26, 2019, 02:10:41 PM »
I've sent Victoria a note about the missing API data in July, 2019.  Oddly enough, the old WXDailyHistory.asp page does return the full month's data, so there's a disconnect somewhere inside the api.weather.com servers (or unrefreshed Akamai cache servers associated with api.weather.com).
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP