Author Topic: WU-History now works with WU/TMC API... (Read 13450 times)

saratogaWX · « **on:** May 23, 2019, 04:52:54 PM »

Whew.. I've written a shim script to replace the now-missing WXDailyHistory.asp page on the WU website with a WXDailyHistory.php script on your website that returns the same formatted CSV file for use with Jim McMurry's (jcweather.us) script set. Only two files change if you have a working installation:

WU-History-inc.php - V3.4d - 23-May-2019 and
WXDailyHistory.php - V1.00 - 23-May-2019 (added file .. initial release).

The WXDailyHistory.php file uses the WU/TWC API to api.weather.com to retrieve (and cache monthly data), then format to the CSV file expected by WU-History-inc.php.

Released on GitHub at https://github.com/ktrue/WU-History distribution .. please read the README.md for more info.

Best regards,
Ken

txweather.org · « **Reply #1 on:** May 23, 2019, 04:56:44 PM »

Yeeeeeeeeeeeeehhhhhhhhaaaaaaaaaa!!!! Thanks Ken!!!!
You are da man!

tmabell · « **Reply #2 on:** May 24, 2019, 06:01:10 AM »

I must have fat-fingered something? This is all I get:

Quote

Sorry... the WU data for this date is not currently available. Please try again later.

The data seems to have been successfully cached:

(I've removed my API key for the moment).

txweather.org · « **Reply #3 on:** May 24, 2019, 10:08:14 AM »

Heads up for those who like to copy and paste...

The section in the readme of the github has a small typo:

Code: [Select]

If running in a Saratoga template, the following can be used in Settings.php instead of customizing WXDailyHistory.php

$SITE['WCAPIkey'] = 'specify-your-api-key-here';
$SITE['WCunits']  = 'e'; // ='e' for F,mph,inHg,in; ='m' for C,km/h,hPa,mm
$SITE['WUID']  = 'KCASARAT1' // your WU PWS ID
$SITE['tz']   = 'America/Los_Angeles'; // timezone
$SITE['cacheFileDir'] = './cache/';  // cache file directory

(the same WCAPIkey and WCunits used for the WC-forecast.php script are reused with this script)

You need a ";" at the end of $SITE['WUID'] = 'KCASARAT1'
Like this: $SITE['WUID'] = 'KCASARAT1';

Just a heads up

ed2kayak · « **Reply #4 on:** May 24, 2019, 10:21:51 AM »

Ken Thank you!!

saratogaWX · « **Reply #5 on:** May 24, 2019, 10:29:47 AM »

Quote from: txweather.org on May 24, 2019, 10:08:14 AM

Heads up for those who like to copy and paste...

The section in the readme of the github has a small typo:

Code: [Select]
If running in a Saratoga template, the following can be used in Settings.php instead of customizing WXDailyHistory.php $SITE['WCAPIkey'] = 'specify-your-api-key-here'; $SITE['WCunits'] = 'e'; // ='e' for F,mph,inHg,in; ='m' for C,km/h,hPa,mm $SITE['WUID'] = 'KCASARAT1' // your WU PWS ID $SITE['tz'] = 'America/Los_Angeles'; // timezone $SITE['cacheFileDir'] = './cache/'; // cache file directory (the same WCAPIkey and WCunits used for the WC-forecast.php script are reused with this script)
You need a ";" at the end of $SITE['WUID'] = 'KCASARAT1'
Like this: $SITE['WUID'] = 'KCASARAT1';

Just a heads up

Oops... thanks for finding the typo.. I've corrected the example on GitHub.

txweather.org · « **Reply #6 on:** May 24, 2019, 10:37:03 AM »

Thanks Ken!
Ken, Are the graphs coming and going normal?
My graphs display ok yesterday but today they dont.
https://www.txweather.org/wxwuhistory.php

saratogaWX · « **Reply #7 on:** May 24, 2019, 12:14:52 PM »

The graphs are likely to vanish permanently -- the WU script that creates them (cgi-bin/wxStationGraphAll ) is a CGI script (likely a Perl script) and is likely to be deprecated in the future.

saratogaWX · « **Reply #8 on:** May 24, 2019, 12:32:04 PM »

Quote from: tmabell on May 24, 2019, 06:01:10 AM

I must have fat-fingered something? This is all I get:

Quote
Sorry... the WU data for this date is not currently available. Please try again later.

https://mymishawakaweather.com/WU-History-Inc.php

The data seems to have been successfully cached:

https://mymishawakaweather.com/wxdailyhistory.php

(I've removed my API key for the moment).

Hi Tom,
Looking at view-source:https://mymishawakaweather.com/WU-History-Inc.php?debug=y shows:

Quote

' -->

While going to https://mymishawakaweather.com/WXDailyHistory.php?ID=KINMISHA2&month=05&day=24&year=2019&format=1&graphspan=day
does return data (as expected).

The curl timeout is troubling.. is your site running into a concurrent number of PHP processes allowed issue? Do your have a web application firewall that might be inhibiting the script from execution?

BTW.. I ran WXDailyHistory.php?graphspan=year and the cache files filled successfully in your ./cache/ for Jan-Apr, 2019 so the script works...

tmabell · « **Reply #9 on:** May 24, 2019, 12:54:16 PM »

Thanks for looking into this for me. Yes there is something going on that I have yet to identify but to answer your questions, I run my own server locally and have zero restrictions in place that would prevent the script form working. The firewall should not be an issue but I will look into it. All other scripts work fine. After seeing what you tried, I did the same thing from that machine and got the same result. However what I found interesting was that by using "localhost" as the source for the url as opposed to my web address, it did work. I have some investigative work to do on my end.

rlee171 · « **Reply #10 on:** May 25, 2019, 09:30:32 PM »

Ken, I have accomplished the updates... your documentation made it a cake walk for those of us still who have very minimal coding skills. I still have a problem that I was hoping it would be fixed with the new changes since it appears the reference to WUG-inc.day.php leads me to believe it may be WU related. I am using the HighCharts for my station graphs. I have attached a jpg of the error I am receiving. You have been plenty busy taking care of what has needed to be done, so I am not purposely asking you to solve this issue, it just seemed like an appropriate place to ask while lots of folks are focused on WU. Thank you sincerely for the additional time you have been spending especially taking the position of being our "spokesperson" and giving WU the luxury of not having to address us as an angry mob. If one of you guys (or gals) have a direction to point me on for this one, any advice would be greatly appreciated.

Ralph hsnpar.org

[ You are not allowed to view attachments ]

saratogaWX · « **Reply #11 on:** May 26, 2019, 01:15:43 AM »

You appear to be running on PHP Version: 5.4.19 which is quite old.. any chance of updating to 5.6 or 7.3?

The image shows you aren't trying to run my script (WXDailyHistory.php) on your site, but you're still configured to use the WU WXdailyHistory.asp page call, and that won't work anymore.

From the distribution, take a look at my WU-History-inc.php and make corresponding changes to your WUG-inc-day.php script.

rlee171 · « **Reply #12 on:** May 26, 2019, 06:16:16 PM »

I was able to get up to PHP Ver 7.2.7 - thank you for pointing that out. I am still working through the other changes you have addressed.
Thank you very much for your continued support.

Ralph

saratogaWX · « **Reply #13 on:** May 26, 2019, 06:24:59 PM »

I now see on the view-source of the page:

Quote

Sorry... the WU data for this date is not currently available. Please try again later.

It's odd that a 200-OK reply would result in no JSON content. Your https://www.wunderground.com/dashboard/pws/KARHOTSP16 PWS page shows data. Digging deeper, and running a view-source for WXDailyHistory.php with &debug=y shows

Quote

a 401 response. The API is returning content of

Quote

Access Denied
You don't have permission to access "http://api.weather.com/v2/pws/history/all?" on this server.

Reference #18.5cef6368.1558909171.1226b63

So I suspect your WU/TWC API Key is in error.. Please double check (by wunderground, Profile, API KEY, show and copy that key to use.

neavilag · « **Reply #14 on:** May 27, 2019, 01:08:26 AM »

Thanks for the script, I was hitting WU site to store my history data and present from my DB directly in my site. I did this via the CSV option, but I saw it stop working.

Seems based on what you have posted, the only option now is via the API, do you know the limits of this API if I want to fetch history every 10 minutes in order to have my latest values, will this hit the limits ?

thanks again.

txweather.org · « **Reply #15 on:** May 27, 2019, 01:28:56 AM »

Ken do we have to put this on a cron?

WXDailyHistory.php?&year=2019&graphspan=year&debug=y

I have pulled the data and I was able to see it but now is no longer displaying.

Sorry... the WU data for this date is not currently available. Please try again later.

rlee171 · « **Reply #16 on:** May 27, 2019, 04:59:32 PM »

Sorry... the WU data for this date is not currently available. Please try again later.[/quote]
It's odd that a 200-OK reply would result in no JSON content. Your https://www.wunderground.com/dashboard/pws/KARHOTSP16 PWS page shows data. Digging deeper, and running a view-source for WXDailyHistory.php with &debug=y shows

Quote

a 401 response. The API is returning content of

Quote

Access Denied
You don't have permission to access "http://api.weather.com/v2/pws/history/all?" on this server.

Reference #18.5cef6368.1558909171.1226b63

So I suspect your WU/TWC API Key is in error.. Please double check (by wunderground, Profile, API KEY, show and copy that key to use.
[/quote]

I double checked my key and it is correct, however there may be a problem with the way I placed it in the WXDailyHistory script.
It's there twice, I didn't know what to put in the " 2nd line - print "replacing. I only have the one key, nothing to replace.

if(preg_match('|specify|i',$WCAPIkey)) {
 print "Note: the WXDailyHistory script requires an API key from WeatherUnderground to operate. ";
 print "Visit <a href=\"https://www.wunderground.com/member/api-keys\">Weather Underground</a> to ";
 print "register for an API key. You must have a PWS submitting data to WU to acquire an API key.\n";
 if( isset($SITE['fcsturlWC']) ) {
 print "Insert in Settings.php an entry for: \n";
 print "\$SITE['WCAPIkey'] = '**key inserted here**'; \n";
 print "replacing **and also here** with your WU/TWC API key.\n";
 }
 return;

Thanks,
Ralph

BCJKiwi · « **Reply #17 on:** May 27, 2019, 07:59:14 PM »

Also only seeing:-
Sorry... the WU data for this date is not currently available. Please try again later.

All the historic data data has been downloaded into the cache including the current month in which the last day is 27/May 2019 (presumably the UTC date as it is the same date in the regularly downloaded wunderground forecast used in ajax dashboard - .cache/WC-forecast-json-0-m-en.txt ).

So, the api is correct (already in settings.php)

Any suggestions?

Thanks

Stegrie · « **Reply #18 on:** May 28, 2019, 12:29:40 PM »

The Script will not work yet

Yesterday got working, today came: Sorry... the WU data for this date is not currently available. Please try again later.
Can anybody look or help?
Cache work and has Data
WD send to WU correctly
I dont know what's wrong...

https://www.wetterstation-johanngeorgenstadt.de/wxwuhistory.php

Stefan

saratogaWX · « **Reply #19 on:** May 28, 2019, 04:10:10 PM »

I don’t see a $SITE[‘WCAPIkey’] entry in your Settings.php .. there’s a $SITE[‘WUAPIkey’] entry.. maybe that should be WCAPIkey ?

BCJKiwi · « **Reply #20 on:** May 28, 2019, 04:51:54 PM »

Ken,
Presume your reply is to Stegrie as I do have all the required WC and WU/TWC entries in settings.php

Thanks

saratogaWX · « **Reply #21 on:** May 28, 2019, 06:12:21 PM »

Quote from: BCJKiwi on May 28, 2019, 04:51:54 PM

Ken,
Presume your reply is to Stegrie as I do have all the required WC and WU/TWC entries in settings.php

Thanks

I think your issue is that WXDailyHistory.php is not in the same directory as wxwuhistory.php and WU-History-inc.php

I get file-not-found when trying to do https://silveracorn.nz/weather/WXDailyHistory.php?sce=view

baedkesj · « **Reply #22 on:** May 28, 2019, 07:31:17 PM »

Like a few others..I'm now having issues displaying data using this script. I also get "Sorry... the WU data for this date is not currently available. Please try again later." message. However I pretty sure I see files in the ./cache directory.

Link: https://www.hburg-wx.net/wxwuhistory.php

All help appreciated!
steve

zeppline · « **Reply #23 on:** May 28, 2019, 09:03:14 PM »

I have mine working after a couple of hours. I went back and read instructions. I was missing a file (WXDailyHistory.php). It work if you follow the instruction. Thanks Ken

txweather.org · « **Reply #24 on:** May 28, 2019, 09:09:45 PM »

Quote from: zeppline on May 28, 2019, 09:03:14 PM

I have mine working after a couple of hours. I went back and read instructions. I was missing a file (WXDailyHistory.php). It work if you follow the instruction. Thanks Ken

All of my files are in place. So this is not the case for all. What I have notice is that I have to run WXDailyHistory.php at least once a day.

News:

Author Topic: WU-History now works with WU/TMC API... (Read 13450 times)