WXforum.net
Web Weather => Weather Website PHP/AJAX scripting => Topic started by: tshattuck on December 05, 2017, 05:48:29 PM
-
Hello:
I am utilizing the Saratoga-Weather.org php scripts and have an issue.
My default Point Forecast for Clay, NY has not been available many times over the past several weeks. I noticed several other locations (forecasts) which use the same Zone NYS018 however were available at the same time.
I went into the Settings.php file and moved location Syracuse, NY which also uses NYS018 to the top line and it stopped working the script states "The detailed point weather forecast is not currently available."
If I leave Syracuse on any line other than the first (default) it works - no error message. I should also note if I move Clay to another line in the script it works.
I also changed the zone while leaving Clay, NY in the default position and it worked.
I don't understand what is happening. Help!
Any assistance is appreciated.
-
It's likely an NWS issue with api.weather.gov and point-forecast data. When it happens again, check https://forecast-v3.weather.gov/point/43.1858,-76.1728 to see if the same thing is shown (that site uses the api.forecast.gov too).
-
Ken:
Thanks for the quick reply. I thought there was something wrong with the php script. I will keep watching for the error and check as you suggested.
-
Ken / Folks:
An update:
The same error has occurred two separate times today. The default forecast is currently not working and is displaying incorrect data. This is affecting my index page and the forecast page: http://yourlocalweather-clay-ny.com/wxforecast.php on my website. The problem appears to be with zone NYS018 only. I checked https://forecast-v3.weather.gov/point/43.1858,-76.1728 (https://forecast-v3.weather.gov/point/43.1858,-76.1728) and I do see a forecast.
This issue does not occur with any other forecasts on my forecast page. If I change the default to Syracuse, NY, which also uses NYS018 it also stops working. If I leave Syracuse down in my list of available forecast locations it works.
Is anyone else having this problem or is anyone aware of any maintenance the NWS is performing that is causing this issue?
Is there a work around for this problem? This has become very frustrating and it doesn't help my website. :!:
Regards,
-
The default forecast is now working. Go figure! I checked https://forecast-v3.weather.gov/point/43.1858,-76.1728 (https://forecast-v3.weather.gov/point/43.1858,-76.1728) and see that it now states "Current" were as before it stated "Not Current".
I hope the NWS gets this resolved soon this is driving me crazy. :roll:
I wish there was another way we could pull the data.
Thanks
-
Ummm... NYS018 is not a forecast Zone. NYZ018 is the zone forecast, but.. you have it correctly configured
Two places to look when it fails:
1) a view-source of the page will show what the script has accessed and processed.
2) look at the cache file itself ./cache/forecast-NYZ018-0-json.txt to see what it says.
A successful Point forecast load looks like <!-- advforecast2.php (JSON) - V5.04 - 11-Oct-2017 on PHP 7.0.25-->
<!-- RAW NWS URL: http://forecast.weather.gov/MapClick.php?CityName=Clay&state=NY&site=BGM&textField1=43.1858&textField2=-76.1728&e=1&TextType=2 -->
<!-- zone=NYZ018 -->
<!-- corrected NWS API URL='https://api.weather.gov/points/43.1858,-76.1728/forecast' -->
<!-- curl fetching 'https://api.weather.gov/points/43.1858,-76.1728/forecast' -->
<!-- HTTP stats: RC=200 dest=38.68.168.88 port=443 (from sce=50.62.177.13)
Times: dns=0.000 conn=0.011 pxfer=0.047 get=0.013 total=0.060 secs -->
<!-- loaded https://api.weather.gov/points/43.1858,-76.1728/forecast - 10406 bytes -->
<!-- wrote cache file ./cache/forecast-NYZ018-0-json.txt -->
<!-- meta info re: point='43.1858,-76.1728' zone='NYZ018' metacache= './cache/forecast-NYZ018-0-json-meta.txt' -->
<!-- metaZoneURL='https://api.weather.gov/zones/forecast/NYZ018' -->
<!-- loaded meta info from ./cache/forecast-NYZ018-0-json-meta.txt -->
<!-- META
Array
(
[city] => 3 Miles SSW Brewerton
[state] => NY
[point] => 43.1858,-76.1728
[forecastOfficeURL] => https://api.weather.gov/offices/BGM
[forecastZoneURL] => https://api.weather.gov/zones/forecast/NYZ018
[forecastZone] => NYZ018
[forecastHourlyURL] => https://api.weather.gov/points/43.1858,-76.1728/forecast/hourly
[forecastGridDataURL] => https://api.weather.gov/gridpoints/BGM/50,104
[observationStationsURL] => https://api.weather.gov/points/43.1858,-76.1728/stations
[countyZoneURL] => https://api.weather.gov/zones/county/NYC067
[countyZone] => NYC067
[fireWeatherZoneURL] => https://api.weather.gov/zones/fire/NYZ018
[fireWeatherZone] => NYZ018
[timeZone] => America/New_York
[radarStation] => KTYX
[zoneName] => Onondaga
[WFOname] => Binghamton/Johnson City, NY
)
-->
<!-- point forecast processing -->
Since the issue is transient, and I can't reproduce it, you'll have to capture that data as it is occuring in order for further diagnosis...
A successful Zone forecast load looks like <!-- advforecast2.php (JSON) - V5.04 - 11-Oct-2017 on PHP 7.0.25-->
<!-- RAW NWS URL: http://forecast.weather.gov/MapClick.php?CityName=Clay&state=NY&site=BGM&textField1=43.1858&textField2=-76.1728&e=1&TextType=2 -->
<!-- zone=NYZ018 -->
<!-- corrected NWS API URL='https://api.weather.gov/points/43.1858,-76.1728/forecast' -->
<!-- curl fetching 'https://api.weather.gov/zones/JSON-LD/NYZ018/forecast' -->
<!-- HTTP stats: RC=200 dest=38.68.168.88 port=443 (from sce=50.62.177.13)
Times: dns=0.000 conn=0.010 pxfer=0.043 get=0.012 total=0.055 secs -->
<!-- loaded (Zone forecast) https://api.weather.gov/zones/JSON-LD/NYZ018/forecast - 15762 bytes -->
<!-- wrote cache file ./cache/forecast-NYZ018-0-json.txt -->
<!-- meta info re: point='43.1858,-76.1728' zone='NYZ018' metacache= './cache/forecast-NYZ018-0-json-meta.txt' -->
<!-- metaZoneURL='https://api.weather.gov/zones/forecast/NYZ018' -->
<!-- loaded meta info from ./cache/forecast-NYZ018-0-json-meta.txt -->
<!-- META
Array
(
[city] => 3 Miles SSW Brewerton
[state] => NY
[point] => 43.1858,-76.1728
[forecastOfficeURL] => https://api.weather.gov/offices/BGM
[forecastZoneURL] => https://api.weather.gov/zones/forecast/NYZ018
[forecastZone] => NYZ018
[forecastHourlyURL] => https://api.weather.gov/points/43.1858,-76.1728/forecast/hourly
[forecastGridDataURL] => https://api.weather.gov/gridpoints/BGM/50,104
[observationStationsURL] => https://api.weather.gov/points/43.1858,-76.1728/stations
[countyZoneURL] => https://api.weather.gov/zones/county/NYC067
[countyZone] => NYC067
[fireWeatherZoneURL] => https://api.weather.gov/zones/fire/NYZ018
[fireWeatherZone] => NYZ018
[timeZone] => America/New_York
[radarStation] => KTYX
[zoneName] => Onondaga
[WFOname] => Binghamton/Johnson City, NY
)
-->
<!-- ZONE forecast processing -->
-
Ken:
Sorry about the zone name had a senior moment. :oops:
My default forecast is down again. I have attached the file you wanted to see. The file was captured while the forecast was not working today. This seems to be happening now every day between 8:00am & 5:00pm. I have to wonder what changed.
The default forecast worked again last evening.
Hopefully you see something that can be addressed.
Thanks for your patience and assistance.
-
Hi Tom,
I just ran the following: http://yourlocalweather-clay-ny.com/wxforecast.php?force=1
and the forecast updated based on what I'm seeing.
John
-
John:
Yes it's working again. :???:
It's been crazy over the past week, I don't know if it's the script or the NWS.
I noticed the following two lines when I viewed the page source for http://yourlocalweather-clay-ny.com/wxforecast.php (http://yourlocalweather-clay-ny.com/wxforecast.php) when the Point Forecast was not working and they don't appear in Ken's examples.
<!-- loaded cache file ./cache/forecast-NYZ018-0-json.txt - 15762 bytes -->
<!-- point forecast more than 12hrs old (age h:m:s is 343:59:07) .. use Zone forecast instead -->
Now that it's working the page source matches Ken's example for a successful Point Forecast.
I haven't changed anything on my end so I am guessing it's the NWS.
Hopefully Ken can tell what is happening from the file I sent him.
Thanks
-
Hi Tom,
It's the NWS, I assure you. Very frustrating for all of us. I'm sure Ken is tired of beating his head against the wall on this crap.
-
Ditto... It's the NWS.
If you want to use the older V4.02 advforecast2.php page-scraper script instead, it's attached. When then NWS switches to the new site design, (or to https), it will likely cease to work (and I won't be updating it).
-
Ken:
Thanks for your efforts. It's good to know what the problem is. I learned a little more in the past few days on how the php script works. It's never too late to teach an old dog new tricks. ;)
It's frustrating to see the bad data go on my website when the NWS is messing around with their API. I will stay with the current script and hope the NWS stops changing their API.
Best Regards
-
I have almost daily problems here with NWS messing around. Most of the time ?force=1 fixes it immediately. Nice ammo to have!
-
John / Ken:
How does code: ?force=1 work?
I seen that mentioned before but don't know how it works or where to place the code in the script.
Thanks for your assistance.
-
John / Ken:
How does code: ?force=1 work?
I seen that mentioned before but don't know how it works or where to place the code in the script.
Thanks for your assistance.
Tom, in your case it would simply be http://yourlocalweather-clay-ny.com/wxforecast.php?force=1. I use the link to mine when Ken's error message about the zone/point forecast shows up on the forecast page. Yours is not failing at the time of this writing. Note that it might not work all the time. For me it seems to fail when NWS has their fingers deeper than usual in trying to get the forecasts to work right. Also not all zones will fail at the same time, only those with which they are fussing.
-
The ?force=1 is an optional argument on the URL to the page, not something you configure in the script. Add it to wxforecast.php?force=1 in your browser URL to force a cache reload of the point-forecast. A wxforecast.php?force=2 will force a Zone forecast instead.
So.. don't 'place it in the script' .. just use it (when needed) on the browser URL to cause the cache to reload the point or zone forecast as needed.
-
John & Ken:
Thanks for the feedback. I will keep this info in my notes.
Regards
-
For the past couple of days, I've had issues with the point forecast too. My homepage merely stops midway down the screen. When I look at my detailed forecast page I get the following NWS error message:
Sorry.. the forecast for Minden - Johnson Lane is not available at this time.
NWS server https://api.weather.gov/points/39.0386,-119.7211/forecast (https://api.weather.gov/points/39.0386,-119.7211/forecast) has an error.
View the source of this page for additional information in HTML comments.
If I use the ?force=1, I get the same message but if I use ?force=2, it looks fine except that it says "The detailed point forecast weather is not currently available."
Guess I'll just have to be patient...
-
It's happening down here in south Alabama this morning...hopefully it clears up.
-
Working fine again here in northern Nevada. I just hate all of the emails I get when it appears my website is having problems and it isn't even my problem! I sure wish NWS would get its' act together or at least test new stuff, if that is what they are doing, off-line so it doesn't impact the rest of the world!
-
The old page-scraper version (https://www.wxforum.net/index.php?topic=33332.msg337902#msg337902) still works.. you can regress to that version if you like. There's still no word from the NWS as to likely cutover date, and it's been 1.5 years since they started the V3 version of forecast.weather.gov site. Yes, there are still issues with the API having missing or just spectacularly wrong temps/wind speeds as the load of GFS->api in their back-end gets funky.
-
It's been five days and the forecasts are still missing. I"m going to see if I can remember how to set up the old version that Ken linked to. This is what I've been afraid would happen for me...letting my website run so long that I forget how to do things.<sigh> :neutral:
-
Since you're using the Saratoga USA template, and you have entries in your Settings.php (which you do), you can just replace the existing advforecast2.php with the older version.. no muss, no fuss :)
Don't forget to change Settings.php $SITE['fcstscript'] = 'advforecast-json.php'; // USA-only NWS Forecast script
to $SITE['fcstscript'] = 'advforecast2.php'; // USA-only NWS Forecast script
to enable the older version.
-
Ok, everybody... I think we've still got time to set up a GoFundMe page and still order that Superman cape shipped via Amazon Prime for Ken!!! UU \:D/
Thanks Ken...that worked perfectly!!! At least I can still follow directions (sometimes<grin>). I don't know exactly what you do for a living nor who you work for or if maybe you work for yourself but one thing is for certain they just don't know what a treasure they must have with you 'cause I know the folks here have one!!!!!!!!!! Thanks again!!! =D>
Merry Christmas!!!!
Ed
-
A Merry Christmas to you too, Ed!
Thank you for the kind words! Fortunately, I've been retired since 2004, so my time is not beholden to corporate interests -- it's just my (sometimes all-consuming) hobby.
Best regards,
Ken
-
Ok, everybody... I think we've still got time to set up a GoFundMe page and still order that Superman cape shipped via Amazon Prime for Ken!!! UU \:D/
Thanks Ken...that worked perfectly!!! At least I can still follow directions (sometimes<grin>). I don't know exactly what you do for a living nor who you work for or if maybe you work for yourself but one thing is for certain they just don't know what a treasure they must have with you 'cause I know the folks here have one!!!!!!!!!! Thanks again!!! =D>
Merry Christmas!!!!
Ed
Yep, agree completely =D>
-
Ditto!
There are a lot of knowledgeable and helpful people here on the forum yet it still amazes me how thorough and to the point Ken can get in analyzing and remedying the things that occur in running our setups. He is so helpful, yet great in detailing the issue and resolution so that we can hopefully understand and use it on our own in future issues =D>
Thank you Ken, and Merry and Safe Christmas,
Paul
-
Folks:
I finally gave up on the NWS getting their Forecast API stabilized. :-x The Point Forecast has been continually going up and down on an almost daily basis for several weeks.
I had a user of my website complain to me yesterday that the NWS forecast displayed on my website home page is useless. The best part was he asked what did I do wrong in setting up the website! So that pushed me to revert back to advforecast2.php v.4.02 and now the NWS forecasts are working correctly. I really didn't want to go backwards but the NWS didn't leave me much choice. It will be interesting to see how this all plays out over the coming months.
Never a dull moment with the NWS. :roll:
Best regards
-
I think forecast-v3.weather.gov is not right at the moment.. Its not completely down, but its returning all kinds of invalid data.. Example "Cloudy, with a high near 9.96921E+36. Heat index values as high as 9.96921E+36. Southwest wind around 1.1472352069703E+37 mph. Chance of precipitation is 9.96921E+36%. New ice accumulation of 1.993842E+37 to 1.993842E+37 inches possible." -- from https://forecast-v3.weather.gov/point/40.8936,-73.2412
-
I think the problems are limited to particular locations as I've not had any problems (Boston area) in a while. Although I'm calling BS on the specificity (and it's making me dizzy) :grin: :
Freezing rain likely before 6pm, then snow likely before 7pm, then snow likely and sleet likely between 7pm and 8pm, then freezing rain likely between 8pm and 9pm, then a chance of rain between 9pm and midnight, then freezing rain likely.
-
I think the problems are limited to particular locations as I've not had any problems (Boston area) in a while. Although I'm calling BS on the specificity (and it's making me dizzy) :grin: :
Agreed for Phoenix. The problems continue here on a pretty regular basis but fortunately NWS hasn't killed it nationwide - yet. ?force=1 almost always fixes the problem.
-
I guess I jinxed myself because now I'm getting week-old zone forecasts... :-x
-
I think the problems are limited to particular locations as I've not had any problems (Boston area) in a while. Although I'm calling BS on the specificity (and it's making me dizzy)
I was going to comment, to agree with this statement, with all those folks having problems, I had not noticed any problems at all.
But I didn't want to jinx myself.
I guess I jinxed myself because now I'm getting week-old zone forecasts...
Rats! I have to agree with Jasiu again. Out of the blue it has affected my area, this A.M. my forecast reflected erroneous (possibly old) data.
What I don't understand is that the force=1 option did not appear to do anything to my page. I finally went into the remote server with FTP and deleted the forecast-MIZ049-0-json.txt file and that appears to have corrected (for now) my bad data.
Dennis
-
What I don't understand is that the force=1 option did not appear to do anything to my page.
I have found that sometimes it takes more than one application of the force item to correct errors. When it doesn't work it seems to mean NWS is in there right now messing around.
-
All the ?force=1 does is to try and reload the point-forecast JSON file from api.weather.gov. If it gets a good response (HTTP return code 200), then it stores that in the cache file. The contents of the JSON may be bad (due to internal issues in the api.weather.gov server), and the script does not address that - it uses a HTTP failure code to load the Zone forecast JSON instead (if api.weather.gov responds that way). You won't get an offer of '?force=1' unless both the point and Zone forecasts get HTTP failures from api.weather.gov. The data is not checked (since there's no good way to do a check for good-HTTP-response-but-returns-bad-data).
-
Having the same issue here... Getting "complaints" from the tenants....
Here is the printout from the advforecast2.php results:
<!-- advforecast2.php (JSON) - V5.04 - 11-Oct-2017 on PHP 5.6.32-->
<!-- RAW NWS URL: http://forecast.weather.gov/MapClick.php?lat=38.2604&lon=-76.4459&unit=0&lg=english&FcstType=text&TextType=2 -->
<!-- zone=MDZ017 -->
<!-- corrected NWS API URL='https://api.weather.gov/points/38.2604,-76.4459/forecast' -->
<!-- loaded cache file ./forecast-MDZ017-0-json.txt - 67960 bytes -->
<!-- point forecast more than 12hrs old (age h:m:s is 341:32:43) .. use Zone forecast instead -->
<!-- curl fetching 'https://api.weather.gov/zones/JSON-LD/MDZ017/forecast' -->
<!-- HTTP stats: RC=200 dest=23.218.148.208 port=443 (from sce=192.252.144.35)
Times: dns=0.029 conn=0.043 pxfer=0.114 get=0.037 total=0.151 secs -->
<!-- loaded (Zone forecast) https://api.weather.gov/zones/JSON-LD/MDZ017/forecast - 67835 bytes -->
<!-- wrote cache file ./forecast-MDZ017-0-json.txt -->
<!-- meta info re: point='38.2604,-76.4459' zone='MDZ017' metacache= './forecast-MDZ017-0-json-meta.txt' -->
<!-- metaZoneURL='https://api.weather.gov/zones/forecast/MDZ017' -->
<!-- loaded meta info from ./forecast-MDZ017-0-json-meta.txt -->
<!-- META
Array
(
[city] => Lexington Park
[state] => MD
[point] => 38.2604,-76.4459
[forecastOfficeURL] => https://api.weather.gov/offices/LWX
[forecastZoneURL] => https://api.weather.gov/zones/forecast/MDZ017
[forecastZone] => MDZ017
[forecastHourlyURL] => https://api.weather.gov/gridpoints/LWX/120,45/forecast/hourly
[forecastGridDataURL] => https://api.weather.gov/gridpoints/LWX/120,45
[observationStationsURL] => https://api.weather.gov/gridpoints/LWX/120,45/stations
[countyZoneURL] => https://api.weather.gov/zones/county/MDC037
[countyZone] => MDC037
[fireWeatherZoneURL] => https://api.weather.gov/zones/fire/MDZ017
[fireWeatherZone] => MDZ017
[timeZone] => America/New_York
[radarStation] => KDOX
[zoneName] => St. Marys
[WFOname] => Baltimore/Washington
)
-->
-
Looks like
<!-- point forecast more than 12hrs old (age h:m:s is 341:32:43) .. use Zone forecast instead -->
the Zone forecast was selected due to the age of the point forecast.
The Zone forecast age is NOT checked, so if it's 'old' too, it still gets displayed.
Both issues are caused by NWS problems getting the GFS data to the api.weather.gov server for publishing.
-
Looks like
<!-- point forecast more than 12hrs old (age h:m:s is 341:32:43) .. use Zone forecast instead -->
the Zone forecast was selected due to the age of the point forecast.
The Zone forecast age is NOT checked, so if it's 'old' too, it still gets displayed.
Both issues are caused by NWS problems getting the GFS data to the api.weather.gov server for publishing.
@saratogaWX
Hello Ken,
http://somdweather.com/somd_main_content.php
The advforecast2.php (JSON) - V5.04 - 11-Oct-201 comments say the forecast is over 12 hours old. See attached html txt
But when I load the data in the browser it correctly follows the redirect and a recent forecast is returned (all times are in UTC.)
https://api.weather.gov/points/38.2604,-76.4459/forecast =>23.218.166.94:443 => http code 301 =>
https://api.weather.gov/gridpoints/LWX/120,45/forecast
Returned data contains the date/time entries "properties": {
"updated": "2017-12-27T15:27:32+00:00",
"units": "us",
"forecastGenerator": "BaselineForecastGenerator",
"generatedAt": "2017-12-27T17:59:28+00:00",
"updateTime": "2017-12-27T15:27:32+00:00",
"validTimes": "2017-12-27T09:00:00+00:00/P7DT16H",
What date/time is checked to decide the forecast is to old?
When I use on my test site, the same coordinates 38.2604,-76.4459, the recent forecast is always returned.
Wim
-
Hi Wim,
It uses preg_match('!"updated":\s+"([^"]+)"!is', $html, $matches);
if (isset($matches[1])) {
$age = time() - strtotime($matches[1]);
if ($age > 12 * 60 * 60) {
$agehms = sec2hmsADV($age);
$Status.= "<!-- point forecast more than 12hrs old (age h:m:s is $agehms) .. use Zone forecast instead -->\n";
$Force = 2;
$usingFile = "(Zone forecast)";
$html = fetchUrlWithoutHanging($backupfileName, $cacheName);
$lastURL = $backupfileName; // remember if error encountered
$fSize = strlen($html);
$Status.= "<!-- loaded $usingFile $backupfileName - $fSize bytes -->\n";
to check the 'age' of the forecast v.s. the internal "updated": info in the JSON.
From SoMDWx example, the cache file was loaded, so I can't tell if the 301 was previously followed by code if ($Force == 1 or !file_exists($cacheName) or (file_exists($cacheName) and filemtime($cacheName) + $refreshTime < time())) {
$html = fetchUrlWithoutHanging($fileName, $cacheName);
preg_match('/HTTP\/1.1 (\d+)/', $html, $m);
if (isset($m[1]) and $m[1] >= '400') {
$Force = 2;
$Status.= "<!-- Oops.. point forecast unavailable RC=" . $m[1] . " - using Zone instead -->\n";
}
else {
$lastURL = $fileName; // remember if error encountered
$fSize = strlen($html);
$Status.= "<!-- loaded $fileName - $fSize bytes -->\n";
if (preg_match('/Location: (.*)\r\n/Uis', $html, $matches)) {
$newLoc = $matches[1];
$newurl = APIURL . $newLoc;
$usingFile = "(retry forecast)";
$html = fetchUrlWithoutHanging($newurl, $cacheName);
$fSize = strlen($html);
$Status.= "<!-- loaded $usingFile $newurl - $fSize bytes -->\n";
}
...
-
Just some notes about the api.weather.gov JSON responses that I've found out (over time) since they aren't officially documented.
1) a HTTP return code of 200 yields a complete response, but sometimes, the data in the point-forecast response is bogus with temps/winds in scientific notation 1E37 values). That seems to be a glitch in their internal loading of the GFS to the api servers.
2) a HTTP return code of 400 (or 50x) means not available. Previously, a 503 meant not available, then later codes in the 40x range meant the same thing. When that happens, the script automatically tries the Zone forecast feed.
3) a HTTP return code of 30x is new for the point-forecast, but those are handled by trying the Location: URL in the header automatically. In Wim's example above, the point-forecast URL responded with a gridpoint forecast URL instead (hadn't seen that before) but.. it too was more than 12hrs old <!-- corrected NWS API URL='https://api.weather.gov/points/38.2604,-76.4459/forecast' -->
<!-- curl fetching 'https://api.weather.gov/points/38.2604,-76.4459/forecast' -->
<!-- HTTP stats: RC=301 dest=104.91.32.186 port=443 (from sce=192.252.144.35)
Times: dns=0.041 conn=0.051 pxfer=0.108 get=0.206 total=0.314 secs -->
<!-- headers returned:
HTTP/1.1 301 Moved Permanently
Server: nginx/1.10.2
Content-Type: application/problem+json
Location: /gridpoints/LWX/120,45/forecast
Access-Control-Allow-Origin: *
X-Server-ID: vm-bldr-nids-apiapp45.ncep.noaa.gov
X-Correlation-ID: eabf937d-0cb9-4628-984f-997cb35eaadc
X-Request-ID: eabf937d-0cb9-4628-984f-997cb35eaadc
Pragma: no-cache
Content-Length: 349
Cache-Control: private, must-revalidate, max-age=600
Expires: Wed, 27 Dec 2017 18:20:30 GMT
Date: Wed, 27 Dec 2017 18:10:30 GMT
Connection: keep-alive
Vary: Accept
-->
<!-- loaded https://api.weather.gov/points/38.2604,-76.4459/forecast - 886 bytes -->
<!-- curl fetching 'https://api.weather.gov/gridpoints/LWX/120,45/forecast' -->
<!-- HTTP stats: RC=200 dest=104.91.32.186 port=443 (from sce=192.252.144.35)
Times: dns=0.000 conn=0.009 pxfer=0.069 get=0.124 total=0.193 secs -->
<!-- loaded (retry forecast) https://api.weather.gov/gridpoints/LWX/120,45/forecast - 10333 bytes -->
<!-- wrote cache file ./forecast-MDZ017-0-json.txt -->
<!-- point forecast more than 12hrs old (age h:m:s is 18:42:07) .. use Zone forecast instead -->
<!-- curl fetching 'https://api.weather.gov/zones/JSON-LD/MDZ017/forecast' -->
<!-- HTTP stats: RC=200 dest=104.91.32.186 port=443 (from sce=192.252.144.35)
Times: dns=0.000 conn=0.009 pxfer=0.058 get=0.022 total=0.080 secs -->
<!-- loaded (Zone forecast) https://api.weather.gov/zones/JSON-LD/MDZ017/forecast - 67960 bytes -->
<!-- wrote cache file ./forecast-MDZ017-0-json.txt -->
so the Zone forecast was fetched and used instead.
It is a challenge to keep up with the ongoing development of api.weather.gov as the HTTP responses have changed over time, and they still haven't got reliable data loading for api.weather.gov data from the GFS system.
For those not wanting to experience the issues, reverting to V4.02 of advforecast2.php will work as a page-scraper on the existing weather.gov website, but will break when they finally implement the V3 version of their site.
-
I've been seeing the redirect for some time and just never got around to asking about it (since it seems to take care of itself...).
<!-- curl fetching 'https://api.weather.gov/points/42.4399,-71.2405/forecast' -->
<!-- HTTP stats: RC=301 dest=23.52.160.34 port=443 (from sce=74.208.114.109)
Times: dns=0.253 conn=0.286 pxfer=0.359 get=0.328 total=0.687 secs -->
<!-- headers returned:
HTTP/1.1 301 Moved Permanently
Server: nginx/1.10.2
Content-Type: application/problem+json
Location: /gridpoints/BOX/64,79/forecast
Access-Control-Allow-Origin: *
X-Server-ID: vm-bldr-nids-apiapp20.ncep.noaa.gov
X-Correlation-ID: 1a7bc867-92c0-44eb-90d1-f5b2b29dfc24
X-Request-ID: 1a7bc867-92c0-44eb-90d1-f5b2b29dfc24
Pragma: no-cache
Content-Length: 348
Cache-Control: private, must-revalidate, max-age=600
Expires: Wed, 27 Dec 2017 23:01:11 GMT
Date: Wed, 27 Dec 2017 22:51:11 GMT
Connection: keep-alive
Vary: Accept
-->
<!-- loaded https://api.weather.gov/points/42.4399,-71.2405/forecast - 884 bytes -->
<!-- curl fetching 'https://api.weather.gov/gridpoints/BOX/64,79/forecast' -->
<!-- HTTP stats: RC=200 dest=23.52.160.34 port=443 (from sce=74.208.114.109)
Times: dns=0.000 conn=0.036 pxfer=0.112 get=1.189 total=1.301 secs -->
<!-- loaded (retry forecast) https://api.weather.gov/gridpoints/BOX/64,79/forecast - 10047 bytes -->
<!-- wrote cache file ./cache/forecast-MAZ014-0-json.txt -->
I don't remember when I first saw this but it has been consistent since then.
-
Sigh.. I just checked out https://forecast-v3.weather.gov/documentation and found:
How do I discover weather data using the API?
The API uses linked data to allow applications to discover content. Similar to a web site that provides HTML links to help users navigate to each page; linked data helps applications navigate to each endpoint. The /points/location endpoint is the most common endpoint to discover additional API content given the popularity of weather data based upon a location (latitude and longitude).
For example, to discover the endpoint of the raw forecast, the application would first request:
https://api.weather.gov/points/39.7456,-97.0892
This response tells the application where to find relative information–including office, zone and forecast data–for a given point. The application can then use the linked data in the previous response to locate the raw forecast:
https://api.weather.gov/gridpoints/TOP/31,80
If an application knows the office and grid position for a location (through caching—a similar concept to a bookmark for users), the link data would not be needed to locate the content for raw forecast data.
So this is a new method, (and somewhat ill-conceived which requires TWO accesses instead of ONE) to get the point-forecast data based on a lat/long point. Grrrr... I'll be updating the script to use this as the 'new normal' from them. Based on my quick tests, sometimes the gridpoint URL returns with a 503 (so the Zone failover works). Sometimes it just returns the 'point' data.
<rant>
IMHO, an api query should either return the data requested or fail if the data is not available -- returning a 301-Moved Permanently to a different API URI is just a waste of effort and doubles the required accesses to their servers for a point-forecast request. It would be MUCH better if they just honored the point-forecast request and internally returned the proper gridpoint data as the request.. that would make much more sense from a canonical API standpoint. The redirect should be reserved to direct to a Zone forecast if the point is not available (as with the current weather.gov handling).
</rant>
-
Today I noticed that my system is lacking the font used in DualImage.php. It calls for this font: fradmcn.ttf. I know how to obtain it but I'm wondering first if I am missing something?
-
For the past few days, I'll get reverted to the point cast. I usually just go into my cache folder and delete the 3 JSON files, refresh the page, and it's back. Today, that didn't work. I got this instead.
"Sorry.. the forecast for Bismarck is not available at this time.
NWS server https://api.weather.gov/points/46.7762,-100.7576/forecast has an error.
View the source of this page for additional information in HTML comments."
I changed to the V4 of the forecast and it's back for now.
-
Today I noticed that my system is lacking the font used in DualImage.php. It calls for this font: fradmcn.ttf. I know how to obtain it but I'm wondering first if I am missing something?
The fradmcn.ttf file should be in your ./forecast/icon-templates directory along with the .png icon template images. For the past few days, I'll get reverted to the point cast. I usually just go into my cache folder and delete the 3 JSON files, refresh the page, and it's back. Today, that didn't work. I got this instead.
"Sorry.. the forecast for Bismarck is not available at this time.
NWS server https://api.weather.gov/points/46.7762,-100.7576/forecast has an error.
View the source of this page for additional information in HTML comments."
I changed to the V4 of the forecast and it's back for now.
Yes, there's been a slew of 503-Server error responses from api.weather.gov lately. Given it's coming up to a holiday, I doubt that a webmaster is minding the store there.. Moving back to 4.02 is a good choice for now.
-
Yes it is located there. What drew my attention to it was these warnings thrown by PHP7.2:
[Sat Dec 30 18:46:12.389872 2017] [php7:warn] [pid 10156:tid 780] [client 192.168.10.156:51814] PHP Warning: imagettfbbox(): Invalid font filename in C:\\www\\DualImage.php on line 452, referer: https://mymishawakaweather.com/4cast.php
[Sat Dec 30 18:46:12.389872 2017] [php7:warn] [pid 10156:tid 780] [client 192.168.10.156:51814] PHP Warning: imagettftext(): Invalid font filename in C:\\www\\DualImage.php on line 416, referer: https://mymishawakaweather.com/4cast.php
[Sat Dec 30 18:46:12.389872 2017] [php7:warn] [pid 10156:tid 780] [client 192.168.10.156:51814] PHP Warning: imagettfbbox(): Invalid font filename in C:\\www\\DualImage.php on line 452, referer: https://mymishawakaweather.com/4cast.php
[Sat Dec 30 18:46:12.389872 2017] [php7:warn] [pid 10156:tid 780] [client 192.168.10.156:51814] PHP Warning: imagettftext(): Invalid font filename in C:\\www\\DualImage.php on line 408, referer: https://mymishawakaweather.com/4cast.php
[Sat Dec 30 18:46:12.391872 2017] [php7:warn] [pid 10156:tid 568] [client 192.168.10.156:51815] PHP Warning: imagettfbbox(): Invalid font filename in C:\\www\\DualImage.php on line 452, referer: https://mymishawakaweather.com/4cast.php
[Sat Dec 30 18:46:12.391872 2017] [php7:warn] [pid 10156:tid 568] [client 192.168.10.156:51815] PHP Warning: imagettftext(): Invalid font filename in C:\\www\\DualImage.php on line 416, referer: https://mymishawakaweather.com/4cast.php
[Sat Dec 30 18:46:12.391872 2017] [php7:warn] [pid 10156:tid 568] [client 192.168.10.156:51815] PHP Warning: imagettfbbox(): Invalid font filename in C:\\www\\DualImage.php on line 452, referer: https://mymishawakaweather.com/4cast.php
[Sat Dec 30 18:46:12.391872 2017] [php7:warn] [pid 10156:tid 568] [client 192.168.10.156:51815] PHP Warning: imagettftext(): Invalid font filename in C:\\www\\DualImage.php on line 408, referer: https://mymishawakaweather.com/4cast.php
If I change line 147 to this:
$TTfont = 'c:/www/forecast/icon-templates/fradmcn.ttf'; // location/name of TTF font to write PoP on image
The warning notice stops but whats odd is that there is no difference in the appearance of the page from before the change to after. It makes me wonder if it's a bogus notice?
https://mymishawakaweather.comadvforecast3.php (https://mymishawakaweather.comadvforecast3.php) (My beta test script)
https://mymishawakaweather.com/DualImage.php?sce=view (https://mymishawakaweather.com/DualImage.php?sce=view)
-
You should never have to change the $TTfont to use absolute addressing. It should be left as relative addressing as in the distribution of
$TTfont = $imgBaseDir . 'fradmcn.ttf'; // location/name of TTF font to write PoP on image
with $imgBaseDir = './forecast/icon-templates/'; # directory for png image sources WITH trailing '/'
I'm running PHP 7.1.12 with GD 2.1.0. You're running PHP 7.2.0 with GD 2.1.0 .. it may just be a foible with 7.2.0 -- I have to wait for Plesk to release PHP 7.2.0 so I can do further tests. Also.. you're running Apache on a Windows server (I'm running Apache on a Linux(CentOS) server.
-
I'm told time and time again about how Apache on Windows can be a strange beast.
-
I'm always learning so with that in mind I'm wondering why using relative paths is so strongly prefered over absolute paths.
-
I'm always learning so with that in mind I'm wondering why using relative paths is so strongly prefered over absolute paths.
In particular, with code that is distributed to so many of us, it just can't work any other way because Ken has no way to know what our absolute paths are, much less the capacity to customize versions for each user.
In my own case, I always have a "sandbox" version of my site up where I test things before going live, and the only way that can work is if all paths are relative (there is no one "right" absolute path). For example, my sandbox is using the new forecast API right now (which happens to be working at this moment) while my life site is running the older version.
-
+1 for Jasiu's explanation.
The one word for "why relative paths" is portability. Every installation on a website has a unique set of absolute paths. The only way to make code portable is to use relative paths and insist on a common relative path directory structure. The Saratoga template always uses relative paths, and when unpacked into the document root of a webserver (preserving the directory structure), will work 'out of the box'. It will even work when installed in a subdirectory of the document root if the ajaxXXwx.js files are changed to point to the URI of the realtime file.
So... using relative paths means portability and your website will work even if your hoster changes the absolute path to your document root (by moving you to a different webserver).
-
Thank you both for the explanation!
-
Im not really sure what is going on with my site and the NWS forecast.. Right now when I look at the NWS forecast, I'm seeing what appears to be a cached zone forecast from 12/28... However, if I switch back to the older 4.02 script, I'm getting nothing but a link to force the update -- but still comes back with nothing.. https://www.indigopc.com/wxNWSforecast.php
-
Im not really sure what is going on with my site and the NWS forecast.. Right now when I look at the NWS forecast, I'm seeing what appears to be a cached zone forecast from 12/28... However, if I switch back to the older 4.02 script, I'm getting nothing but a link to force the update -- but still comes back with nothing.. https://www.indigopc.com/wxNWSforecast.php (https://www.indigopc.com/wxNWSforecast.php)
As you'll see in earlier posts in this thread, the symptom you are seeing is most likely NOT an issue with your website but rather, an issue with NWS itself. Most of us see this behavior on a regular basis. We just have to be patient and wait until the NWS forecast is running again.
-
In theory though, 4.02 should be working correctly though, yes?
-
In theory though, 4.02 should be working correctly though, yes?
I've had intermittent issues with both versions. Thus, the reason for the "patience" comment... :?
-
I guess what I'm confused about is if I use the URLs from my Settings.php with the old mapclick URLs, I do get a valid point forecast.. Just not sure why when using the 4.02 version of the script, I come up empty.
-
Have you tried copying that URL from advforecast2.php and pasting it into your browser's address bar to see if it is a good working URL?
-
Alright.. I have 4.02 working again.. Had to change the URL in Settings.php to a regular http link instead of https..
-
Looking at Ken's post (https://www.wxforum.net/index.php?topic=33332.msg339272#msg339272 (https://www.wxforum.net/index.php?topic=33332.msg339272#msg339272)) he mentions that 4.02 will break when NWS goes to the V3 format. Perhaps that is what is happening.
-
Perhaps that is what is happening.
They have not switched. There will be ample notice before they do.
Had to change the URL in Settings.php to a regular http link instead of https..
Both http and https work from here.
-
Yea, the NWS forecast stuff has been messed up for a week now. I just had to do a force=2 to get the zone, all the point shows is an error.
-
In the end -- I just added a 2nd link on my site for the 5.04 version of the script for now and kept 4.02 as the default.. Changed the 5.04 version to advforecast2a.php and made a copy of wxNWSforecast.php called wxNWSforecast2.php and told it to use the advforecast2a.php
That'll do at least until they work everything out. I dont believe there will be any issues with the cache files as the 5.04 files add the json to the filenames from what I can tell.
-
Yes, there are no issues running both scripts -- the cache file names were deliberately set to not interfere with the V4 and V5 versions.
Sadly, I'm getting the issues here in the Western Region.. I'll likely revert too until the NWS can get it together reliably.
-
Just sent a message to our SOO here at the Atlanta NWS on NWSChat asking who to contact at the NWS about the issue.
-
They are looking into it.
-
Was just told the ITO is making a ticket on the issue.
-
OK guys, I need data.
When did the problem start?
How often does it occur?
NWS office or region?
-
Has anyone noticed that the alert link doesn't work? I am under a Wind Chill Advisory and it doesn't show up with the forecast. If you hover over the link, you see the advisory, but when you click the link, it is invalid on the NWS site.
Anyone else?
-
The link seems to go to alerts.weather.gov and not alerts-V2.weather.gov.
Are you running advforecast2.php Version 5.04 - 11-Oct-2017 ?
Is $useProdNWS = false; // =false, use preview V3 sites,set in the script?
If so, then if ($useProdNWS) {
// final production URLs
define('APIURL', "https://api.weather.gov");
define('FCSTURL', "https://forecast.weather.gov");
define('ALERTAPIURL', 'https://api.weather.gov/alerts?active=1&zone=');
define('ALERTURL', 'https://alerts.weather.gov/products/');
}
else {
// pre-production/testing URLs
define('APIURL', "https://api.weather.gov");
define('FCSTURL', "https://forecast-v3.weather.gov");
define('ALERTAPIURL', 'https://api.weather.gov/alerts?active=1&zone=');
define('ALERTURL', 'https://alerts-v2.weather.gov/products/'); // TODO: fix this to final loc.
}
should set the alert URL to use the beta product site.
-
$useProdNWS - true is my setting and it apparently is getting the correct alert (as evidenced by the ability to "see" the alert when you hover) but when I click the link it doesn't work. If I change the link to the "v2" link it works.
-
You should have $useProdNWS = false; for the setting.. the alerts.weather.gov doesn't honor the same URLs as the alerts-V2.weather.gov site does.
-
Ahhhhh... working
-
Looks like NWS issue with api.weather.gov and point-forecast data for some offices today. Including mine out of North Platte. Another one same issue is Goodland.
Get a blank black page with chrome browser and with IE 7 day icons are missing. On website get the strange heat index numbers.
[ You are not allowed to view attachments ]
-
So are there any alternatives when the local office doesn't fix the forecast point and click. Over 9 hours now and still down.
http://www.weather.gov/lbf/ I just get a blank page clicking on map unless I go outside of region.
-
Improvement after contacting office we now have a zone forecast available. Is there a way to push a zone forecast should something like this happen again? The older versions would automatically kick in zone but this latter advforecast2.php doesn't do that unless I missed something in settings. That's why I went back with older version until it completely died the other day. Because of all the API issues.
[ You are not allowed to view attachments ]
-
Fixed finally. Only took 10 hours... #-o
-
You can always 'go back' to the V4.02 advforecast2.php script (download here (https://saratoga-weather.org/advforecast2-V4.02.php?sce=view)) and wait out the final(overdue) fixes to the V3 forecast.weather.gov system. I don't think they're doing a cutover anytime soon.. it's been over a year since they started planning a cutover.
-
Ken I might try again seems like this area has more problems than others. I was running an older version and it failed just the other day so loaded the newer version. I'll keep in mind next time if this continues to happen.
Thanks
-
Email the link to the office and tell them. I have access to NWSChat and I was able to tell them that way, it got fixed pretty quickly.
-
Let me send them a message for you.
-
I had contact with them directly thanks.
Its back up finally.
-
I don't know if this is related, but since I came across it today I thought I'd mention it.
Testing some stuff today, I came across a situation where the following happened:
1) Attempt to fetch with the "points" URL returned at 301 - Moved Permanently. As expected.
2) Attempt to fetch the redirected URL (the one with "gridpoints" in it) returned a 404.
The code didn't handle this well since it wasn't checking for >= 400 after the second fetchUrlWithoutHanging() call.
So... I copied the check from a few lines above and now the code that was:
$usingFile = "(retry forecast)";
$html = fetchUrlWithoutHanging($newurl, $cacheName);
$fSize = strlen($html);
$Status.= "<!-- loaded $usingFile $newurl - $fSize bytes -->\n";
... looks like this:
$usingFile = "(retry forecast)";
$html = fetchUrlWithoutHanging($newurl,$cacheName);
preg_match('/HTTP\/1.1 (\d+)/',$html,$m);
if(isset($m[1]) and $m[1] >= '400') {
$Force =2;
$Status .= "<!-- Oops.. point forecast unavailable RC=".$m[1]. " - using Zone instead -->\n";
}
else
{
$fSize = strlen($html);
$Status .= "<!-- loaded $usingFile $newurl - $fSize bytes -->\n";
}
... and it seems to cover this case, failing over to the Zone forecast.
-
Good idea!
What I find annoying about their point-forecast API is that it now ALWAYS returns a 301 redirect to the gridpoint forecast instead of just returning the data requested. I've never before encountered an API that now requires TWO accesses to get the data .. all the other APIs I've dealt with either return the data or a failure code if data is not available.
This API response is a very poor design (IMHO) .. they should do that lookup internally and just return the data (or not) in one https request. Grrrr.
BTW... I've written to the webmaster about this (and about the long data outages (404 errors)) for points(gridpoints) and Zone forecasts with the api.weather.gov. I see on forecast-v3.weather.gov, the link to the documentation is now not on the page anymore (but is still available via the old link at https://forecast-v3.weather.gov/documentation?redirect=legacy ). I've also asked about the ongoing development/implementation plans for the api.weather.gov and future cutovers.
-
Thanks for all the work as usual, Ken!
I've wondered about just changing the URL in settings to the redirected URL or possibly caching the redirect to avoid the extra jump. But, yeah, we shouldn't have to do that...
-
I've also been experimenting with changing the curl call to follow redirects automatically. That works, but the headers for both the initial point call and the gridpoint call are stacked (with \r\n\r\n separators) in the curl return.. makes finding the last RC a bit more complicated, but doable.
<!-- advforecast2.php (JSON) - V5.05 - 28-Dec-2017 on PHP 7.1.9-->
<!-- RAW NWS URL: http://forecast.weather.gov/MapClick.php?CityName=Saratoga&state=CA&site=MTR&textField1=37.2639&textField2=-122.022&e=1&TextType=2 -->
<!-- zone=CAZ513 -->
<!-- corrected NWS API URL='https://api.weather.gov/points/37.2639,-122.022/forecast' -->
<!-- curl fetching 'https://api.weather.gov/points/37.2639,-122.022/forecast' -->
<!-- note: fetched 'https://api.weather.gov/gridpoints/MTR/97,101/forecast' after redirect was followed. -->
<!-- HTTP stats: RC=200 dest=23.197.96.136 port=443 (from sce=192.168.1.104)
Times: dns=0.000 conn=0.016 pxfer=0.047 get=4295.405 total=4295.452 secs -->
<!-- loaded point-forecast https://api.weather.gov/points/37.2639,-122.022/forecast - 10319 bytes -->
<!-- wrote cache file ./forecast-CAZ513-0-json.txt -->
But.. the timing returned by curl is way off.. Sigh.
They tout that their design wants to be canonical - one request for each class of information needed. That necessitated the current advforecast2.php JSON script to use up to 3 separate requests to get data (point forecast, descriptive info re forecast, alert info for points) instead of one request (like with WU) that has all the info desired. I think a 301 response to a different URL is 'non canonical' and may simplify their internal design, but really (unnecessarily) complicates the design for users of their API.
-
Who is the guy you've been talking to? Possibly Corey Pieper?
Good idea!
What I find annoying about their point-forecast API is that it now ALWAYS returns a 301 redirect to the gridpoint forecast instead of just returning the data requested. I've never before encountered an API that now requires TWO accesses to get the data .. all the other APIs I've dealt with either return the data or a failure code if data is not available.
This API response is a very poor design (IMHO) .. they should do that lookup internally and just return the data (or not) in one https request. Grrrr.
BTW... I've written to the webmaster about this (and about the long data outages (404 errors)) for points(gridpoints) and Zone forecasts with the api.weather.gov. I see on forecast-v3.weather.gov, the link to the documentation is now not on the page anymore (but is still available via the old link at https://forecast-v3.weather.gov/documentation?redirect=legacy ). I've also asked about the ongoing development/implementation plans for the api.weather.gov and future cutovers.
-
Over a year ago, I was talking with Jared Holmberg (NOAA Affiliate) and Robert Bunge (NOAA) about the API. I don't know if they're active in the project any longer, so my message was to the main contact for the forecast-v3.weather.gov site.