WXforum.net

Web Weather => Weather Website PHP/AJAX scripting => Topic started by: daybreak on August 15, 2018, 11:04:32 AM

Title: advforecast2.php issue
Post by: daybreak on August 15, 2018, 11:04:32 AM
I noticed this on my page this morning.  http://www.daybreakimagery.com/weather/forecast.php (http://www.daybreakimagery.com/weather/forecast.php) Did something change or is it temporary?

Thanks,

Richard
Title: Re: advforecast2.php issue
Post by: saratogaWX on August 15, 2018, 11:36:07 AM
I noticed this on my page this morning.  http://www.daybreakimagery.com/weather/forecast.php (http://www.daybreakimagery.com/weather/forecast.php) Did something change or is it temporary?

Thanks,

Richard
That message
Quote
The detailed point forecast weather data is not currently available.
The zone forecast data for ILZ070 (Marion) will be displayed
until the point forecast data is again available.
If this persists, contact the NWS St. Louis/St. Charles, MO WFO
at 636-441-8467 or email at w-lsx.webmaster@noaa.gov
to have them update the point forecast for /gridpoints/LSX/139,79 on api.weather.gov
happens if there is a point forecast issue for your specific area.  Give it a few hours, and if it persists, call the WFO and alert them to the issue.
Title: Re: advforecast2.php issue
Post by: daybreak on August 15, 2018, 11:43:52 AM
Ok thanks Ken!

Richard
Title: Re: advforecast2.php issue
Post by: jgillett on August 15, 2018, 12:13:09 PM
Same issue here and in HI, CA, and NB. Haven't seen that message in a long time. Maybe not localized?
Title: Re: advforecast2.php issue
Post by: ctccbc on August 15, 2018, 12:20:00 PM
I think it is a national issue, I'm having it to.  Usually they clear up in a few hours, unless Ken has to make a change to the code.

Chris
Title: Re: advforecast2.php issue
Post by: Jasiu on August 15, 2018, 12:21:35 PM
It's only an issue with the JSON/API code (v5.09).  The old code is returning the point forecast.  So... they must be mucking with it.  (remember that the API is still in BETA).
Title: Re: advforecast2.php issue
Post by: saratogaWX on August 15, 2018, 12:24:47 PM
Looking at the view-source of the page, I see
Quote
<!-- point forecast more than 18hrs old (age h:m:s is 24:38:36) .. use Zone forecast instead -->
and that's why the Zone is shown instead.

They're a bit behind in the ingress of gridpoint data to the api.weather.gov service .. usually, that happens every 8 hours or so.
Title: Re: advforecast2.php issue
Post by: daybreak on August 15, 2018, 02:00:53 PM
My forecast is back to normal.

Richard
Title: Re: advforecast2.php issue
Post by: daybreak on August 22, 2018, 06:45:18 AM
My forecast page http://www.daybreakimagery.com/weather/forecast.php (http://www.daybreakimagery.com/weather/forecast.php) has been giving issues now for at least 18 hours again.  I don't know if they changed something?  When I look at my printable forecast on the NWS page it looks fine.  I've noticed others are having issues also.

Richard
Title: Re: advforecast2.php issue
Post by: UpstateWeather.com on August 22, 2018, 09:02:02 AM
My forecast page http://www.daybreakimagery.com/weather/forecast.php (http://www.daybreakimagery.com/weather/forecast.php) has been giving issues now for at least 18 hours again.  I don't know if they changed something?  When I look at my printable forecast on the NWS page it looks fine.  I've noticed others are having issues also.

Richard

I'm having the same problem. It was fine yesterday oddly enough, but today's same error.
Title: Re: advforecast2.php issue
Post by: Intheswamp on August 22, 2018, 09:08:11 AM
Yup.  Things are slightly askew for me this morning...

(https://farm2.staticflickr.com/1846/43290829225_f99918bb9a_z.jpg) (https://flic.kr/p/28XsPik)ForecastProblem_08222018 (https://flic.kr/p/28XsPik) by Intheswamp (https://www.flickr.com/photos/71532701@N07/), on Flickr
Title: Re: advforecast2.php issue
Post by: Otis on August 22, 2018, 10:03:33 AM
Yes mine as well this morning, the last forecast was from yesterday.
Title: Re: advforecast2.php issue
Post by: n7xrd on August 22, 2018, 10:59:02 AM
No forecast here since 8-20 @ 8:46PM
Title: Re: advforecast2.php issue
Post by: tshattuck on August 22, 2018, 11:13:01 AM
Mine is down also.

It would be nice if we could have a similar but abbreviated error message appear on our Index/Main page of our websites with the forecast icons at the bottom of the page.

My Point Forecast was working yesterday so I don't know why the zone forecast displayed is from August 15th. The Zone forecast on my Index page is not even close to being correct.

 [ You are not allowed to view attachments ]
Title: Re: advforecast2.php issue
Post by: dasman on August 22, 2018, 02:23:15 PM
I just tried ?force=1 and my forecast has finally updated.
Title: Re: advforecast2.php issue
Post by: Intheswamp on August 23, 2018, 12:20:52 AM
Just got home and checked BeeWeather and the forecasts are working again!!!  =D&gt;
Title: Re: advforecast2.php issue
Post by: tshattuck on August 23, 2018, 02:49:32 PM
Mine is also working again.
Title: Re: advforecast2.php issue
Post by: gwwilk on September 20, 2018, 04:45:20 PM
Something new here for my point forecast this afternoon:  it's unavailable via advforecast2.php (https://www.gwwilkins.org/temp-images/AdvForecast2-2018-09-20.png), as is a current zone forecast.

Yet this shows up (https://www.gwwilkins.org/temp-images/PointForecast-2018-09-20.png) when I go to OAX and click on my location (https://forecast.weather.gov/MapClick.php?x=181&y=196&site=oax&zmx=&zmy=&map_x=181&map_y=196).

Is there a new wrinkle here?

Edit 3:40 PM CDT 2018 Sept 20:  The advforecast2.php is now returning a more complete forecast, but yesterday's icons are still present.  Apparently the NWS is fiddling with these forecasts, and we have as yet to see the final results.  I think nothing is warranted at this point except sitting tight.
Title: Re: advforecast2.php issue
Post by: gwwilk on September 21, 2018, 09:39:32 AM
My advforecast2.php still gives me an old zone forecast, and the same is true for all of the OAX forecasts listed under my NWS Forecasts (https://www.gwwilkins.org/wxforecast.php).  Yet my point forecast displays on the NWS/OAX site (https://forecast.weather.gov/MapClick.php?lon=-96.63102149963379&lat=40.79054951505782).

I'm not sure what's going on here.  Any thoughts?
Title: Re: advforecast2.php issue
Post by: saratogaWX on September 21, 2018, 10:45:46 AM
There's a bit of apples:oranges here.  The website is using the V5 JSON forecast, and the point-forecast link you cited is for the 'old' (not V3) forecast, which seems to work.  Doing a view-source on your page with ?force=1 shows
Quote
<!-- advforecast2.php (JSON) - V5.09 - 26-May-2018 on PHP 5.4.19-->
<!-- RAW NWS URL: https://forecast.weather.gov/MapClick.php?CityName=Lincoln&state=NE&site=OAX&textField1=40.8164&textField2=-96.6882&e=0&TextType=2 -->
<!-- zone=NEZ066 -->
<!-- loaded URLcache from ./cache/forecast-URLcache.txt with 3 entries -->
<!-- corrected NWS API URL='https://api.weather.gov/points/40.8164,-96.6882/forecast' -->
<!-- replaced point URL with gridpoint URL from URLcache -->
<!-- curl fetching 'https://api.weather.gov/gridpoints/OAX/56,39/forecast' -->
<!-- HTTP stats:  RC=200 dest=23.45.184.117
      Times: dns=0.000 conn=0.010 pxfer=0.056 get=0.251 total=0.307 secs -->
<!-- loaded point-forecast https://api.weather.gov/points/40.8164,-96.6882/forecast - 10364 bytes -->
<!-- wrote cache file ./cache/forecast-NEZ066-0-json.txt -->
<!-- point forecast more than 18hrs old (age h:m:s is 63:37:11) .. use Zone forecast instead -->
<!-- curl fetching 'https://api.weather.gov/zones/JSON-LD/NEZ066/forecast' -->
<!-- HTTP stats:  RC=200 dest=23.45.184.117
      Times: dns=0.000 conn=0.010 pxfer=0.056 get=0.011 total=0.067 secs -->
<!-- loaded (Zone forecast) https://api.weather.gov/zones/JSON-LD/NEZ066/forecast - 4104 bytes -->
<!-- wrote cache file ./cache/forecast-NEZ066-0-json.txt -->
<!-- meta info re: point='40.8164,-96.6882' zone='NEZ066' metacache= './cache/forecast-NEZ066-0-json-meta.txt' -->
<!-- metaZoneURL='https://api.weather.gov/zones/forecast/NEZ066' -->
<!-- loaded meta info from ./cache/forecast-NEZ066-0-json-meta.txt -->
<!-- META
Array
(
    [city] => Lincoln
    [state] => NE
    [point] => 40.8164,-96.6882
    [forecastOfficeURL] => https://api.weather.gov/offices/OAX
    [forecastZoneURL] => https://api.weather.gov/zones/forecast/NEZ066
    [forecastZone] => NEZ066
    [forecastHourlyURL] => https://api.weather.gov/points/40.8164,-96.6882/forecast/hourly
    [forecastGridDataURL] => https://api.weather.gov/gridpoints/OAX/56,39
    [observationStationsURL] => https://api.weather.gov/points/40.8164,-96.6882/stations
    [countyZoneURL] => https://api.weather.gov/zones/county/NEC109
    [countyZone] => NEC109
    [fireWeatherZoneURL] => https://api.weather.gov/zones/fire/NEZ066
    [fireWeatherZone] => NEZ066
    [timeZone] => America/Chicago
    [radarStation] => KOAX
    [zoneName] => Lancaster
    [WFOname] => Omaha/Valley, NE
    [WFOphone] => 402-359-5166
    [WFOemail] => w-oax.webmaster@noaa.gov
)
 -->
<!-- ZONE forecast processing -->
which shows the issue is an old gridpoint forecast (more than 18 hrs) using the JSON API, so the script loads the Zone forecast instead.  For some reason, the NWS is not refreshing the gridpoint forecast with current data -- should be no more than 8 hours old.  It's an NWS issue with the api.weather.gov server.
Title: Re: advforecast2.php issue
Post by: gwwilk on September 21, 2018, 11:28:22 AM
Thanks for the educational and informative reply, as always, Ken!  I'll send an email to the zone forecast's nws address to see if something can be done about it.

I'll correct the 'Special Message' on my website.
Title: Re: advforecast2.php issue
Post by: saratogaWX on September 21, 2018, 07:03:23 PM
I'd suggest you revert to the advforecast2.php V4.05 (https://saratoga-weather.org/advforecast2-V4.05.php?sce=view) for the time being.  I've set up my testing site to use your Lincoln, NE URL so I can look too. 

I notice that the https://forecast-V3.weather.gov/ site (API driven) is going very slowly now, and sometimes failing (with 500-Server Error) on point forecast requests.  Glad that site is not production yet... :roll:
Title: Re: advforecast2.php issue
Post by: gwwilk on September 21, 2018, 07:44:59 PM
Thanks, Ken, for ferreting out the problem and suggesting the solution!  I've reverted to the smaller icons with v4.05 by modifying advforecast2.php (v4.05), DualImage.php, and Settings.php.

It works (somewhat in that I only have a single row of icons right now): https://www.gwwilkins.org/wxforecast.php

Title: Re: advforecast2.php issue
Post by: saratogaWX on September 22, 2018, 12:09:02 AM
The 4.x version of the script only does 1 row.. the 5.x version does two rows because more icons were available :)

Glad you've bypassed the issues with the NWS V3 forecast beta for now...
Title: Re: advforecast2.php issue
Post by: gwwilk on September 22, 2018, 09:36:11 AM
Aha!  I had forgotten about the single row in V4.xx.

Interestingly, 'check-fetch-times.php' now correctly prompts me to update my advforecast2.php to v5.09.  :-)  The link you provided, https://forecast-v3.weather.gov/, allows me to check on the current status of my OAX pre-production point forecasts.  They're still unavailable as of today. :-(
Title: Re: advforecast2.php issue
Post by: gateway2capecod on September 24, 2018, 06:18:50 PM
hello...


my friend jerry's forecast is also going haywire since last thurs/fri...although occasionally I see the correct data on his page, as of right now the older than 18 hours data is on his forecast page...



http://www.victoriatexasweather.net/wxforecast.php
Title: Re: advforecast2.php issue
Post by: saratogaWX on September 24, 2018, 06:51:27 PM
Quote
<!-- advforecast2.php (JSON) - V5.09 - 26-May-2018 on PHP 5.6.36-->
<!-- RAW NWS URL: http://forecast.weather.gov/MapClick.php?lat=28.82420&lon=-96.9829&unit=0&lg=english&FcstType=text&TextType=2 -->
<!-- zone=TXZ234 -->
<!-- loaded URLcache from ./cache/forecast-URLcache.txt with 1 entries -->
<!-- corrected NWS API URL='https://api.weather.gov/points/28.8242,-96.9829/forecast' -->
<!-- replaced point URL with gridpoint URL from URLcache -->
<!-- curl fetching 'https://api.weather.gov/gridpoints/CRP/128,77/forecast' -->
<!-- HTTP stats:  RC=200 dest=23.194.108.143 port=443 (from sce=10.24.3.8)
      Times: dns=0.000 conn=0.032 pxfer=0.107 get=0.985 total=1.091 secs -->
<!-- loaded point-forecast https://api.weather.gov/points/28.8242,-96.9829/forecast - 11105 bytes -->
<!-- wrote cache file ./cache/forecast-TXZ234-0-json.txt -->
<!-- point forecast more than 18hrs old (age h:m:s is 122:32:41) .. use Zone forecast instead -->
Even the Zone forecast on the new script is old.  I suggest he revert to the 4.05 version (https://saratoga-weather.org/advforecast2-V4.05.php?sce=view) for now.
Title: Re: advforecast2.php issue
Post by: miraculon on September 24, 2018, 08:03:12 PM
Same kind of problem here with old zone-only forecasts. Other local areas are pulling today's but not for my county (Presque Isle Co, MI).

See:

http://rogerscityweather.com/wxforecast.php?z=0 (http://rogerscityweather.com/wxforecast.php?z=0) Rogers City (3 days old)
http://rogerscityweather.com/wxforecast.php?z=1 (http://rogerscityweather.com/wxforecast.php?z=1) Onaway (seems to be only an hour old, but still "zone")
http://rogerscityweather.com/wxforecast.php?z=2 (http://rogerscityweather.com/wxforecast.php?z=2) Alpena (same story)
http://rogerscityweather.com/wxforecast.php?z=3 (http://rogerscityweather.com/wxforecast.php?z=3) Cheboygan (same here)

The zone forecast for my area is old, the others are not as bad. Still no point forecast though.

I tried reverting to V4.05, and got a blank-ish screen. Forcing the update didn't help.

I guess that I will just have to wait for NWS to fix it. I'll put a notice on my page about the issue.

Greg H.

Title: Re: advforecast2.php issue
Post by: Otis on September 24, 2018, 08:50:33 PM
Hey Greg, I was having same problem and did as Ken suggested - reverted to v4.05 and mine is working.
Cheboygan - https://lakehuronweather.org/wxforecast.php?z=3 (https://lakehuronweather.org/wxforecast.php?z=3)
Rogers City - https://lakehuronweather.org/wxforecast.php?z=18 (https://lakehuronweather.org/wxforecast.php?z=18)

So you did try clearing the cache right?
Title: Re: advforecast2.php issue
Post by: Jasiu on September 24, 2018, 08:51:47 PM

I tried reverting to V4.05, and got a blank-ish screen. Forcing the update didn't help.


The likely reason for that is the format of the forecast URL ($SITE['fcsturlNWS']) in your settings file. From a view-source on your site:

Code: [Select]
<!-- RAW NWS URL: https://forecast-v3.weather.gov/point/45.4071,-83.8129 -->

If you use a URL of the "MapClick" variety, it will work with the 4.x script and the 5.x script will convert it to the "point" format; i.e., it works with both.  Example:

Code: [Select]
<!-- RAW NWS URL: https://forecast.weather.gov/MapClick.php?CityName=Lexington&state=MA&site=MTR&textField1=42.4399&textField2=-71.2405&e=1&TextType=2 -->
Title: Re: advforecast2.php issue
Post by: saratogaWX on September 24, 2018, 08:53:58 PM
Correct.. the MapClick formatted point-printable forecast URLs are the only format that works with both old and new scripts.  The /point/ URLs only work with the V5.x script and the beta site.
Title: Re: advforecast2.php issue
Post by: gateway2capecod on September 24, 2018, 09:01:52 PM
Thanks for help....

got victoria texas forecast working for now with the older version script trick
Title: Re: advforecast2.php issue
Post by: miraculon on September 25, 2018, 08:52:07 AM
It looks like I needed both the V 4.05 and the Mapclick fix. It is working now.

Thanks for the suggestions.

Greg H.
Title: Re: advforecast2.php issue
Post by: gwwilk on September 25, 2018, 11:19:45 AM
Apparently they're still working on the Point Forecasts at https://forecast-v3.weather.gov/ because I received a valid and complete Point Forecast there earlier this morning.  I tried to implement it, and failed.  Then I rechecked the site and it didn't deliver once again.  I did receive a response yesterday from the W-Oax Webmaster - NOAA Service Account <w-oax.webmaster@noaa.gov> thanking me and stating that they would look into it.  I sent a service request over the weekend when all I got was an error message and service number to send along to them.
Title: Re: advforecast2.php issue
Post by: gwwilk on September 26, 2018, 10:37:14 AM
My Point Forecast is back up this morning (https://www.gwwilkins.org/wxforecast.php), but for how long remains to be seen.  Here's hoping...
Title: Re: advforecast2.php issue
Post by: Jasiu on September 28, 2018, 10:24:19 AM
I have been seeing a lot of fetch calls with the new API time out for weeks now. I've let the NWS folks know, so they are aware of it. I'll poke them again...
Title: Re: advforecast2.php issue
Post by: gwwilk on September 28, 2018, 10:47:38 AM
Poking the NWS can get results.  After my second email of the given service number to the address listed on https://forecast-v3.weather.gov/ following another failed attempt to retrieve a local Point Forecast there, a 'ticket' was created that was sent to 14 other noaa.gov individuals as well as myself.  Within 48 hours all of my Point Forecast problems were resolved!

Fingers crossed, it's still working as intended.
Title: Re: advforecast2.php issue
Post by: mikeym2m on September 28, 2018, 04:07:20 PM
Strange, I haven't had any problems with Ver 5.09 the last 10 - 14 days that everyone's having problems with.  :? Maybe a problem with some regions and not others?

Cheers

 :grin:

MikeyM
Title: Re: advforecast2.php issue
Post by: DaculaWeather on October 08, 2018, 06:12:50 AM
Mine has been working, now force 1 or 2 doesn't make any difference. Forecast stuck at Sept 26...
Title: Re: advforecast2.php issue
Post by: Otis on October 08, 2018, 07:06:05 AM
Using the latest script mine has been down for Days!  I have contacted the local NWS Office and they replied that they are working on it.  I have reverted to the 4.05 version of the script until it is fixed.
Title: Re: advforecast2.php issue
Post by: Intheswamp on October 08, 2018, 09:01:53 AM
My point forecast has been shifted to the zone forecast.  I can't remember if I checked the forecast yesterday or not, though, so I'm not sure when the point forecast stopped.  For me it looks like the Birmingham, Mobile/Pensacola, and Tallahassee NWS offices are down...Peachtree City (Atlanta) office appear to be up and running.  The Pensacola forecast appears to be completely down.  I'll wait and see what happens...seems to correct itself...eventually.  :???:
Title: Re: advforecast2.php issue
Post by: DaculaWeather on October 08, 2018, 09:07:02 AM
Atlanta is still not working for me, still stuck on Sept 26th. I'm not dropping back to an old script.
Title: Re: advforecast2.php issue
Post by: miraculon on October 08, 2018, 09:39:13 AM
My local zones in NE Lower Michigan are still OK using the v4.05 script.

Greg H.
Title: Re: advforecast2.php issue
Post by: Intheswamp on October 08, 2018, 12:38:36 PM
Atlanta is still not working for me, still stuck on Sept 26th. I'm not dropping back to an old script.
That's odd.  I just went to your website and I see what you're talking about.  But, if I pull the forecast up on my website it looks good.   http://www.beeweather.com/wxforecast.php?z=14   :?:
Title: Re: advforecast2.php issue
Post by: Intheswamp on October 08, 2018, 12:48:18 PM
Something else kind of screwy...  If I check the forecast for Andalusia, AL, which is covered by Mobile/Pensacola NWS and located about 40 miles SW of me, I get a good point forecast.  But if I check my local Rutledge forecast it gives me the zone forecast....and it Rutledge is covered by Mobile/Pensacola, too.  #-o

HOLD ON A MINUTE....!!!!!
Now my local forecast has dropped back to yesterday and Andalusia is showing Sept 4.....?????   #-o :shock:

Ok, I'm going to turn around and slowlyyyyyy walk away.......
Title: Re: advforecast2.php issue
Post by: DaculaWeather on October 08, 2018, 12:53:58 PM
LOL! If you don't, it will drive you crazy. :-)
Title: Re: advforecast2.php issue
Post by: Jasiu on October 08, 2018, 02:55:15 PM
Dacula,

Looks like you are running V5.04 of the script.  Latest/greatest is V5.09.

Yours:

Code: [Select]
<!-- advforecast2.php (JSON) - V5.04 - 11-Oct-2017 on PHP 7.1.22-->
Mine:

Code: [Select]
<!-- advforecast2.php (JSON) - V5.09 - 26-May-2018 on PHP 7.2.10-->
I strongly suggest upgrading.
Title: Re: advforecast2.php issue
Post by: saratogaWX on October 08, 2018, 03:24:44 PM
Correct.  The version history for the JSON script is:

//  Version 5.00 - 22-Apr-2017 - complete rewrite to use api.weather.gov JSON feeds for forecasts and alerts
//  Version 5.01 - 26-Apr-2017 - switch to use test NWS site for data waiting for June 19, 2017 prod cutover
//  Version 5.02 - 22-May-2017 - added temperature trend indicator
//  Version 5.03 - 16-Jul-2017 - if point forecast HTTP return code > 400, force zone forecast fetch
//  Version 5.04 - 11-Oct-2017 - fix for stale point-forecast via API to failover to Zone forecast
//  Version 5.05 - 27-Feb-2018 - more fixes for point-forecast/refetch fail->Zone failover
//  Version 5.06 - 12-Apr-2018 - fix PHP warning when no alerts available
//  Version 5.07 - 14-Apr-2018 - add caching for point->gridpoint forecast URLs + WFO info on Zone fcst
//  Version 5.08 - 25-May-2018 - fixes for point/zone JSON changes from api.weather.gov
//  Version 5.09 - 26-May-2018 - added new NWS API icons for tropical storm/hurricane

Based on the current api.weather.gov implementation, you need 5.09 to work correctly for their new website responses.
Title: Re: advforecast2.php issue
Post by: Intheswamp on October 08, 2018, 03:25:02 PM
LOL! If you don't, it will drive you crazy. :-)
Uh, I just checked BeeWeather and....it's back working.   :roll: =D&gt;
Title: Re: advforecast2.php issue
Post by: Otis on October 08, 2018, 03:35:21 PM
Same here, I just checked and the "new" version is working again so I again changed back to version 5.09.  Will just keep an eye on it.
Title: Re: advforecast2.php issue
Post by: Intheswamp on October 08, 2018, 03:54:27 PM
Uh, er, uh....well, the point forecast is gone again and the zone has reverted back to yesterday, 10-7-2018.  I'm outta here!!!!!  ](*,) ](*,)
Title: Re: advforecast2.php issue
Post by: wvdkuil on October 08, 2018, 04:15:08 PM
Same here, I just checked and the "new" version is working again so I again changed back to version 5.09.  Will just keep an eye on it.
I have my NWS scripts still in "test" with different lat-lon settings every few days. Currently for 3 Miles ENE of Chandler OK
In my day-time (your night-time) the v3-API is fully functional about  40-60% of the time.  In your day-time it functions for 75% or more of the time.  The most troublesome part is alerts-api https://api.weather.gov/alerts/active?point=35.7236,-96.8205
I included an automatic fall-back to v2-json  so that at least a normal forecast is shown when the new API is not available.
For this demo-site I use relative long timeouts (20 seconds) for the NWS scripts.  Also error-reporting is set to maximum to catch all errors.

Feel free to request a download  link (https://support.leuven-template.eu/scripts.php?lang=en) to test the scripts yourself.  All but NWS are full production versions.

Menu page for all forecasts:  https://www.weerstation-herent.be/wsfct4/

Demo links to my  "National Weather Service (v3) Weather forecast " :
Stand-alone version:  https://www.weerstation-herent.be/wsfct4/wsFctNoaaPPage4.php
In Saratoga-template:  https://www.weerstation-herent.be/saratoga/wx_lvn_noaaPP.php

ALL feedback and suggestions for improvements are welcome. I live on another continent and need local expertise to improve on these scripts. For me the metno,  darksky or WU forecasts are far easier to test compared to NWS.

Wim

Title: Re: advforecast2.php issue
Post by: Jasiu on October 08, 2018, 04:33:58 PM
I've been looking into the new API problems, and timeouts seem to be the biggest issue for me. Essentially, the https connection gets set up (or an existing connection is reused), the GET for the API endpoint is issued, the GET is ack-ed at the TCP level, and then.... crickets.

After some experimentation, I now have every fetch in a for loop with an increasing CURL timeout value for each retry, starting at one second and maxing out at nine.  It often takes multiple calls to get the data, but I have not yet seen a case where I don't eventually get the data.

For example, running the forecast script right now:

1) Point forecast fetch: Failed three times, worked on 4th. On the successful fetch, the total time spent was only 0.122 seconds.

2) Alert "point" fetch: Failed six times, worked on 7th.  0.124 seconds for the successful fetch.

EDIT: I was able to verify the actual traffic sent to the NWS server by running the script from my laptop. Nice that Mac OS has good-old Unix underneath.  :grin:
Title: Re: advforecast2.php issue
Post by: saratogaWX on October 08, 2018, 04:42:23 PM
I think it may be a caching problem on the NWS end.

The query for /point/<lat>,<long>/forecast is supposed to return a 301 to /gridpoint/<g1>,<g2>/forecast and a subsequent query to /gridpoint/<g1>,<g2>/forecast then either yields data (200 OK) or (404 Not Found) if the gridpoint data is not current/present.

In my 5.09 script, I cache the successful point->gridpoint URL and just use the gridpoint subsequently for the query.  Rebuilt if a force=1 (or equivalent) is done.

Given their issues with keeping the gridpoint forecasts loaded/current, and perhaps bugs with their caching service, it's not a big wonder to see why things aren't working reliably for api.weather.gov/forecast-v3.weather.gov right now.
Title: Re: advforecast2.php issue
Post by: DaculaWeather on October 08, 2018, 07:40:44 PM
SENIOR DUTY METEOROLOGIST NWS ADMINISTRATIVE MESSAGE
NWS NCEP CENTRAL OPERATIONS COLLEGE PARK MD
2323Z MON OCT 08 2018


NCO is aware of issues with products not updating on weather.gov,
specifically the WAWA map and WFO sites. NCO is fully engaged and
working to restore the products.
Title: Re: advforecast2.php issue
Post by: wvdkuil on October 09, 2018, 03:34:15 AM
This is an example of the 07:00 UTC  test of my  NWS scripts.
Extremely  long response times / time outs for the first URL of the 4 needed => results in no-data retrieved

https://api.weather.gov/
URL         time/seconds
1=points    5.1   20.0  4.7  4.3  20.0
2=offices   2.2   x     0.3  1.4  x
3=forecast  8.2   x    19.4  0.1  x
4=alerts   14.0   x     1.4  1.4  x


No idea why the access times vary so much, even for the same location in subsequent requests.
Wim

Detailed list of CURL access times:
Code: [Select]
At 09:00 CEST  /  07:00 UTC

======= test for 35.7236,-96.8205

[url] => https://api.weather.gov/points/35.7236,-96.8205 
    [total_time] => 5.163184
    [namelookup_time] => 0.008313
    [connect_time] => 0.010489
    [pretransfer_time] => 0.021661
    [starttransfer_time] => 5.163055

[url] => https://api.weather.gov/offices/OUN
    [total_time] => 2.203137
    [namelookup_time] => 2.0E-5
    [connect_time] => 0.001654
    [pretransfer_time] => 0.011606
    [starttransfer_time] => 2.203021

[url] => https://api.weather.gov/gridpoints/OUN/122,104/forecast 
    [total_time] => 8.208564
    [namelookup_time] => 2.2E-5
    [connect_time] => 0.001735
    [pretransfer_time] => 0.014604
    [starttransfer_time] => 8.208464

[url] => https://api.weather.gov/alerts/active?point=35.7236,-96.8205
    [total_time] => 14.004311
    [namelookup_time] => 2.0E-5
    [connect_time] => 0.002216
    [pretransfer_time] => 0.017703
    [starttransfer_time] => 14.004184

======= test for 41.1511,-74.3511

[url] => https://api.weather.gov/points/41.1511,-74.3511
    [total_time] => 20.011297
    [namelookup_time] => 0.023046
    [connect_time] => 0.043851
    [pretransfer_time] => 0.157907
Error  codes:Operation timed out after 20011 milliseconds with 0 bytes received

==> alternative v2 forecast trieved

[url] => https://forecast.weather.gov/MapClick.php?lat=41.1511&lon=-74.3511&FcstType=json
    [total_time] => 1.201338
    [namelookup_time] => 0.018547
    [connect_time] => 0.040375
    [pretransfer_time] => 0.229122
    [starttransfer_time] => 1.2013

=======  repeat same test for 41.1511,-74.3511

[url] => https://api.weather.gov/points/41.1511,-74.3511
    [total_time] => 4.740465
    [namelookup_time] => 0.024589
    [connect_time] => 0.045082
    [pretransfer_time] => 0.15167
    [starttransfer_time] => 4.7329

[url] => https://api.weather.gov/offices/OKX
    [total_time] => 0.361493
    [namelookup_time] => 2.0E-5
    [connect_time] => 0.021295
    [pretransfer_time] => 0.125193
    [starttransfer_time] => 0.361398

[url] => https://api.weather.gov/gridpoints/OKX/17,52/forecast
    [total_time] => 19.49374
    [namelookup_time] => 4.4E-5
    [connect_time] => 0.02147
    [pretransfer_time] => 0.127005
    [starttransfer_time] => 19.493604

[url] => https://api.weather.gov/alerts/active?point=41.1511,-74.3511
    [total_time] => 1.431452
    [namelookup_time] => 2.1E-5
    [connect_time] => 0.021789
    [pretransfer_time] => 0.129773
    [starttransfer_time] => 1.431374

=====  force RELOAD after 2 minutes after all data OK  for  41.1511,-74.3511

[url] => https://api.weather.gov/points/41.1511,-74.3511
    [total_time] => 4.351943
    [namelookup_time] => 0.008756
    [connect_time] => 0.028074
    [pretransfer_time] => 0.15786
    [starttransfer_time] => 4.351842

[url] => https://api.weather.gov/offices/OKX
    [total_time] => 1.431917
    [namelookup_time] => 3.3E-5
    [connect_time] => 0.023675
    [pretransfer_time] => 0.159705
    [starttransfer_time] => 1.43182

[url] => https://api.weather.gov/gridpoints/OKX/17,52/forecast
    [total_time] => 0.131973
    [namelookup_time] => 2.2E-5
    [connect_time] => 0.020575
    [pretransfer_time] => 0.109796
    [starttransfer_time] => 0.131874

[url] => https://api.weather.gov/alerts/active?point=41.1511,-74.3511
    [total_time] => 1.420042
    [namelookup_time] => 2.3E-5
    [connect_time] => 0.020782
    [pretransfer_time] => 0.099507
    [starttransfer_time] => 1.41994

=====  force RELOAD after 2 minutes after all data OK  for  41.1511,-74.3511

[url] => https://api.weather.gov/points/41.1511,-74.3511
    [total_time] => 20.026838
    [namelookup_time] => 0.021461
    [connect_time] => 0.042464
    [pretransfer_time] => 0.158886
Error  codes:Operation timed out after 20026 milliseconds
Title: Re: advforecast2.php issue
Post by: saratogaWX on October 09, 2018, 01:08:20 PM
I did a bit of 'dig' for api.weather.gov

Quote
10/09/18 09:58:46 dig api.weather.gov @ 208.67.222.222
Dig api.weather.gov@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for api.weather.gov type=255 class=1
  api.weather.gov CNAME (Canonical Name) sancert.weather.gov.edgekey.net


10/09/18 09:59:05 dig sancert.weather.gov.edgekey.net @ 208.67.222.222
Dig sancert.weather.gov.edgekey.net@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for sancert.weather.gov.edgekey.net type=255 class=1
  sancert.weather.gov.edgekey.net CNAME (Canonical Name) e278.dscg.akamaiedge.net


10/09/18 09:59:20 dig e278.dscg.akamaiedge.net @ 208.67.222.222
Dig e278.dscg.akamaiedge.net@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for e278.dscg.akamaiedge.net type=255 class=1
  e278.dscg.akamaiedge.net A (Address) 23.6.200.7
  e278.dscg.akamaiedge.net AAAA (IPv6 Address) 2001:559:19:2096:0:0:0:116
  e278.dscg.akamaiedge.net AAAA (IPv6 Address) 2001:559:19:208a:0:0:0:116

It looks like they're trying to use Akamai cloud for their load balancing/primary serving for requests and it's likely not set up quite correctly.  I got 26s for the same query.. all of it wait for response.  Not exactly a speedy response for a cloud caching service.
Title: Re: advforecast2.php issue
Post by: saratogaWX on October 10, 2018, 03:46:13 PM
Sigh... looks like BOTH api.weather.gov (the Beta/non-production service used by V5.09) AND forecast.weather.gov (used by V4.05) are using the same Akamai IP address, so ... no safe-harbor while they get the Akamai hosting issues fixed.  I'm seeing 25 second responses from forecast.weather.gov now too..\

Grumble.

Quote
10/10/18 12:38:32 dig api.weather.gov @ 208.67.222.222
Dig api.weather.gov@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for api.weather.gov type=255 class=1
  api.weather.gov CNAME (Canonical Name) sancert.weather.gov.edgekey.net

10/10/18 12:37:59 dig forecast.weather.gov @ 208.67.222.222
Dig forecast.weather.gov@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for forecast.weather.gov type=255 class=1
  forecast.weather.gov CNAME (Canonical Name) sancert.weather.gov.edgekey.net

10/10/18 12:38:42 dig sancert.weather.gov.edgekey.net @ 208.67.222.222
Dig sancert.weather.gov.edgekey.net@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for sancert.weather.gov.edgekey.net type=255 class=1
  sancert.weather.gov.edgekey.net CNAME (Canonical Name) e278.dscg.akamaiedge.net


10/10/18 12:38:51 dig e278.dscg.akamaiedge.net @ 208.67.222.222
Dig e278.dscg.akamaiedge.net@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for e278.dscg.akamaiedge.net type=255 class=1
  e278.dscg.akamaiedge.net AAAA (IPv6 Address) 2001:559:19:6081:0:0:0:116
  e278.dscg.akamaiedge.net AAAA (IPv6 Address) 2001:559:19:6083:0:0:0:116
  e278.dscg.akamaiedge.net A (Address) 23.197.96.136

Title: Re: advforecast2.php issue
Post by: Intheswamp on October 12, 2018, 08:19:52 AM
This morning...

Cities and towns reported by the Mobile/Pensacola office seems to be working well and is faster in loading than I've observed lately.

Birmingham/Calera office seems about the same, working well but might be a tad slower.

Tallahassee office....only zone forecast.  Understandable.

But, Mobile/Pens and B'ham/Calera seem to be doing good.
Title: Re: advforecast2.php issue
Post by: DaculaWeather on October 15, 2018, 02:17:56 PM
For me here in Atlanta, today is the first day that I've gotten the point forecast using this script, it has only displayed the zone forecast for months and months and months.