WXforum.net

Web Weather => Weather Website PHP/AJAX scripting => Topic started by: mkutche on February 18, 2018, 07:58:12 AM

Title: Point Forecast Down?
Post by: mkutche on February 18, 2018, 07:58:12 AM
Is it just down or is it down for good??

https://prnt.sc/igdtuk (https://prnt.sc/igdtuk)
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 18, 2018, 08:02:08 AM
Older version advforcast2 (2015) went over to zone this morning so switched to newer version (2017). I don't like the newer because new api address is so unreliable its down more than the older.
I just switch back and forth when I catch issues.
Title: Re: Point Forecast Down?
Post by: mkutche on February 18, 2018, 08:18:47 AM
I guess it was on NWS's side it's working right again.
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 18, 2018, 08:35:56 AM
Not sure what version you are running but the older version has less issues for me. The new API address for my region is very unreliable. Ken posted link as text file for older version here.
https://saratoga-weather.org/advforecast2-V4.02.php?sce=view (https://saratoga-weather.org/advforecast2-V4.02.php?sce=view)
I just keep both versions seperate folders and when one goes down switch to the other.
Title: Re: Point Forecast Down?
Post by: mkutche on February 21, 2018, 10:18:22 AM
Older version advforcast2 (2015) went over to zone this morning so switched to newer version (2017). I don't like the newer because new api address is so unreliable its down more than the older.
I just switch back and forth when I catch issues.

I tried both 2015 and 2017 update and still the forecast is down. Ok I did a force update on the 2015 version hopefully it won't break again.
Title: Re: Point Forecast Down?
Post by: miraculon on February 21, 2018, 10:59:50 AM
I am getting some very old forecasts from almost a year ago.

Greg H.
Title: Re: Point Forecast Down?
Post by: CNYWeather on February 21, 2018, 12:20:03 PM
I just went back to 4.02 version. The v5.04 seemed pretty sketchy on mine and everyone else's sites.
Title: Re: Point Forecast Down?
Post by: Bunty on February 22, 2018, 01:34:20 AM
Darn, neither version works for me.  Seldom have I seen it this bad. 
Title: Re: Point Forecast Down?
Post by: gwwilk on February 22, 2018, 08:41:21 AM
My forecasts are also a mess either way!  The Omaha office is the culprit, whereas the Hastings office seems to be OK.
Title: Re: Point Forecast Down?
Post by: satcop on February 22, 2018, 08:45:42 AM
Haven't had problems with getting the cast in the past as much as I have seen others have seemed to.  But it seems that this time it has hit here too (Sterling VA).   No updated forecasts since Tuesday evening, something clearly broke, did they change urls?  Update:  Note forcing it to provide a zone forecast gets one, so this definitely appears to be associated with the point forecast.
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 22, 2018, 08:57:06 AM
My forecasts are also a mess either way!  The Omaha office is the culprit, whereas the Hastings office seems to be OK.

I'm back on the 2015 version starting early yesterday when 2017 had no forecast at all (just missing with a message unavailable). At least older version defaults to zone most times.   
Regional office is North Platte.
Title: Re: Point Forecast Down?
Post by: metzeric on February 22, 2018, 09:04:02 AM
i Went back to the 4.02 version yesterday    however now nothing as updated since yesterday morning and im still stuck on the Wednesday forecast
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 22, 2018, 09:05:39 AM
Stuck here too...
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 22, 2018, 11:56:24 AM
Dacula, you're still running advforecast2.php (JSON) - V5.04 - 11-Oct-2017 on PHP 5.5.38

Maybe it's time to go back to the 4.02 script for advforecast2.php until the NWS gets it together again with api.weather.gov.

Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 22, 2018, 11:59:27 AM
I may do that Ken. I sent them an email today and even pointed them to this thread. I haven't heard back yet but I expect to before the end of the day.

It gets to be a pain to switch back and forth since I end up customizing things so much.  :grin:
Title: Re: Point Forecast Down?
Post by: metzeric on February 22, 2018, 12:08:02 PM
i am now on 4.02  and it didnt change a thing for me :(
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 22, 2018, 12:08:17 PM
I'm going to try (again) contacting the NWS to see what their plans are for api.weather.gov/forecast-v3.weather.gov.

My last missive went unresponded :(
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 22, 2018, 12:12:24 PM
metzeric, your issue seems to be with your web server.. it is getting
Quote
<!-- advforecast2.php (multi) - V4.02 - 07-Nov-2015 -->
<!-- NWS URL: https://forecast.weather.gov/MapClick.php?lat=43.3476&lon=-78.5559&unit=0&lg=english&FcstType=text&TextType=2 -->
<!-- zone=NYZ001 -->
<!-- Network error: php_network_getaddresses: getaddrinfo failed: Name or service not known (0) -->
<!-- loading https://forecast.weather.gov/MapClick.php?lat=43.3476&lon=-78.5559&unit=0&lg=english&FcstType=text&TextType=2 - 7895 bytes -->

It appears the name lookup for forecast.weather.gov is failing, and that is a web hoster issue.. call tech support for them and ask them to fix the DNS lookup issue.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 22, 2018, 12:14:31 PM
I was getting these errors earlier.
 [ You are not allowed to view attachments ]
Title: Re: Point Forecast Down?
Post by: miraculon on February 22, 2018, 12:17:35 PM
Seems to working now.

Greg H.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 22, 2018, 12:17:53 PM
And now that same forecast is working... go figure.
 [ You are not allowed to view attachments ]
Title: Re: Point Forecast Down?
Post by: satcop on February 22, 2018, 12:19:42 PM
Working here again as well (out of offices lwx akq fcx), which were not before, so maybe someone fed the hamsters powering the computers at nws. ](*,)
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 22, 2018, 12:21:10 PM
Just got this message back
------------------------
Hi Steve,

Thanks!  I've opened up a ticket with the appropriate folks to get it looked it.  Seems kind of odd - see the errors with points in FFC's and GSP's area, but not BMX's, even though point-and-click shows things are up to date.

Steve L.
NWS Peachtree City, GA
Title: Re: Point Forecast Down?
Post by: metzeric on February 22, 2018, 01:07:52 PM
ken what is the you are using that shows the dns error  so  i can tell godaddy??

metzeric, your issue seems to be with your web server.. it is getting
Quote
<!-- advforecast2.php (multi) - V4.02 - 07-Nov-2015 -->
<!-- NWS URL: https://forecast.weather.gov/MapClick.php?lat=43.3476&lon=-78.5559&unit=0&lg=english&FcstType=text&TextType=2 -->
<!-- zone=NYZ001 -->
<!-- Network error: php_network_getaddresses: getaddrinfo failed: Name or service not known (0) -->
<!-- loading https://forecast.weather.gov/MapClick.php?lat=43.3476&lon=-78.5559&unit=0&lg=english&FcstType=text&TextType=2 - 7895 bytes -->

It appears the name lookup for forecast.weather.gov is failing, and that is a web hoster issue.. call tech support for them and ask them to fix the DNS lookup issue.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 22, 2018, 01:09:04 PM
Right click on the page with the error and view source.
Title: Re: Point Forecast Down?
Post by: metzeric on February 22, 2018, 01:16:18 PM
Now i show it saying the following    it doesnt show an error anymore    but the forecast is still wrong    still a go daddy problem?
      <!-- advforecast2.php (multi) - V4.02 - 07-Nov-2015 -->
<!-- NWS URL: https://forecast.weather.gov/MapClick.php?lat=43.13564&lon=-78.69149&unit=0&lg=english&FcstType=text&TextType=2 -->
<!-- zone=NYZ001 -->
<!-- loading ./cache/forecast-NYZ001-0.txt - 65102 bytes -->
<!-- isZone='1' -->
<!-- start=22630 finish=26793 length=4163 -->
<!-- text zone forecast
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 22, 2018, 01:18:29 PM
I clicked on the link and it worked and has the correct date and time

 [ You are not allowed to view attachments ]
Title: Re: Point Forecast Down?
Post by: metzeric on February 22, 2018, 01:26:10 PM
I clicked on the link and it worked and has the correct date and time

 [ You are not allowed to view attachments ]

the hard link   works fine  but the website isnt  updating it  still says National Weather Service Forecast for: Niagara County
Issued by: NWS Buffalo, NY
Updated: 929 AM EST Wed Feb 21 2018
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 22, 2018, 01:26:30 PM
Yes, the link is good.. if his webserver could get the IP address for forecast.weather.gov, the script would work also.

Something in the upstream DNS resolvers for his webserver is broken and that's for the hoster's tech support to fix.
Title: Re: Point Forecast Down?
Post by: metzeric on February 22, 2018, 02:10:34 PM
I switched back to the 5.02 version  form the 4.02 and everything is fine again   strange


Yes, the link is good.. if his webserver could get the IP address for forecast.weather.gov, the script would work also.

Something in the upstream DNS resolvers for his webserver is broken and that's for the hoster's tech support to fix.
Title: Re: Point Forecast Down?
Post by: gwwilk on February 22, 2018, 03:23:07 PM
Reverted back to 5.04 and now everything is working normally again.
Title: Re: Point Forecast Down?
Post by: daybreak on February 27, 2018, 11:59:16 AM
Hi Ken,

Mine is not working today.  http://www.daybreakimagery.com/weather/ or http://www.daybreakimagery.com/weather/forecast.php.  On the forecast.php page it says this-
Forecast blank? Force Update.  Do I need to change scripts?

Thanks,

Richard
Title: Re: Point Forecast Down?
Post by: Jasiu on February 27, 2018, 12:04:45 PM
Hi Ken,

Mine is not working today.  http://www.daybreakimagery.com/weather/ or http://www.daybreakimagery.com/weather/forecast.php.  On the forecast.php page it says this-
Forecast blank? Force Update.  Do I need to change scripts?

Thanks,

Richard

Was just looking at this. FWIW the JSON version of the script is working for me if you want to try that temporarily.

Anyway, it's getting 301 redirected because the URL is http rather than https - at least that's what I'm seeing.  I thought that was fixed in the code recently, but I'm looking at code based on version 4.02...

Will look at a quick patch but also await Ken's reply.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 27, 2018, 12:07:17 PM
I think it has as much to do with the forecast office as much as anything. My 5.04 script is working fine today
Title: Re: Point Forecast Down?
Post by: CNYWeather on February 27, 2018, 12:10:54 PM
Must be a general NWS issue today. No forecasts including forecast discussion or hazardous outlooks for me.
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 27, 2018, 12:17:27 PM
4.02 is down here also force update not working either. I tried the 5.04 and it gave me a forecast for Feb 20th, 7 days old. Yikes! #-o
Title: Re: Point Forecast Down?
Post by: Jasiu on February 27, 2018, 12:27:15 PM
OK it isn't the HTTPS thing. Fixed that and I'm seeing this...

Code: [Select]
!-- loading https://forecast.weather.gov/MapClick.php?lat=42.4399&lon=-71.2405&unit=0&lg=english&FcstType=text&TextType=2 - 251 bytes -->
<!-- redirect found in
HTTP/1.1 301 Moved Permanently
Server: AkamaiGHost
Content-Length: 0
Location: https://forecast.weather.gov/MapClick.php?lat=42.4399&amp;lon=-71.2405&amp;unit=0&amp;lg=english&amp;FcstType=text&amp;TextType=2
Date: Tue, 27 Feb 2018 17:22:26 GMT
Connection: close

 -->
<!-- loading https://forecast.weather.gov/MapClick.php?zoneid=MAZ014&zflg=1 - 204 bytes -->
<!-- loading ./cache/forecast-MAZ014-0.txt - 204 bytes -->
<!-- loading https://forecast.weather.gov/MapClick.php?zoneid=MAZ014&zflg=1 - 204 bytes -->
<!-- isZone='' -->
<!-- HTML:
HTTP/1.1 301 Moved Permanently
Server: AkamaiGHost
Content-Length: 0
Location: https://forecast.weather.gov/MapClick.php?zoneid=MAZ014&zflg=1
Date: Tue, 27 Feb 2018 17:22:27 GMT
Connection: close


-->

Unless I'm missing something, the redirects are to the exact same URL.

Plugging URL into a browser works for me... Have other things to do at the moment but will come back to this later if no one has figured it out.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 27, 2018, 12:32:43 PM
Yea, the AFD stuff is not working here, yet my forecast is. Go figure. That's why you never want to rely on the government if you want things done correctly.  :roll:
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 27, 2018, 12:38:59 PM
This just in....

NOUS42 KWNO 271712
ADMNFD

SENIOR DUTY METEOROLOGIST NWS ADMINISTRATIVE MESSAGE
NWS NCEP CENTRAL OPERATIONS COLLEGE PARK MD
1712Z TUE FEB 27 2018


WEB SERVICE CHANGES FOR NCO STARTING AT 28/1500Z...


Removal of Weak Ciphers from NCEP Web Sites

As required by US Department of Homeland Security Binding
Operational Directive 18-01,  NCEP/NCO will be making changes to
the underlying protocols that are used to access our public and
internal web sites. For web sites, the directive addresses
enforcement of secure HTTPS protocols for all access.


These changes will include:

    Enabling HTTPS for sites that were previously HTTP-only

    Redirecting HTTP traffic for all sites to the respective
HTTPS counterpart

    Restricting HTTPS handshakes to Cipher suites that are known
to be secure.

    Enabling a configuration that marks the affected sites for
HTTPS Strict Transport Security (HSTS)


The potential impacts that have been identified include:

    Scripts that are attempting to connect to the HTTP service
that do not honor the redirect to HTTPS will fail.

    Scripts or applications that do not support HTTPS protocols
will be unable to connect.

    Browsers that do not support secure ciphers may be unable to
connect (we believe that this would only affect browsers that are
already obsolete, but that does not mean that they are not
possibly still in-use. Example: IE8 on Windows XP)

    Users who have restrictive firewall rules at their side that
do not permit access to the HTTPS service may be unable to
connect.


Please be aware that all of the changes above are, in addition to
being mandated by DHS, considered ā??best practiceā?? in network
security and privacy.  For more information on the transition to
HTTPS-only, please see https://https.cio.gov/
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 27, 2018, 12:43:48 PM
So today with no announcement they decided to make the changes.

My 5.04 starting working again. 4.02 is dead.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 27, 2018, 12:53:09 PM
The AFD is working for me now
Title: Re: Point Forecast Down?
Post by: Jasiu on February 27, 2018, 12:54:07 PM
OK, I spent a few more minutes on this and got 4.02 working. These are the changes I made to 4.02 advforecast2.php and settings.php - not sure if all were necessary but....

1) The one change I'm sure about: Take the fetchUrlWithoutHanging() function from 5.04 and forklift/replace it into the 4.02 code.

2) Change $backupfileName to use HTTPS:

Code: [Select]
//$backupfileName = "http://forecast.weather.gov/MapClick.php?zoneid=$NOAAZone&zflg=1";
$backupfileName = "https://forecast.weather.gov/MapClick.php?zoneid=$NOAAZone&zflg=1";

3) Change $newurl to use HTTPS:

Code: [Select]
//  $newurl = 'http://forecast.weather.gov/MapClick.php?'.$newlatlong.'&unit=0&lg=english&FcstType=text&TextType=2';
  $newurl = 'https://forecast.weather.gov/MapClick.php?'.$newlatlong.'&unit=0&lg=english&FcstType=text&TextType=2';

4) I don't think this one is critical, but I changed $forecastwarnings to use HTTPS also.

Code: [Select]
//    $forecastwarnings .= '<a href="http://forecast.weather.gov/' . $warns[1][$i] . '" target="_blank">' .
//          '<strong><span style="color: red">'. $warns[2][$i] . "</span></strong></a><br/>\n";
    $forecastwarnings .= '<a href="https://forecast.weather.gov/' . $warns[1][$i] . '" target="_blank">' .
         '<strong><span style="color: red">'. $warns[2][$i] . "</span></strong></a><br/>\n";

5) Finally, in settings.php, similar change to $SITE['fcsturlNWS'],

Code: [Select]
//$SITE['fcsturlNWS'] = "http://forecast.weather.gov/MapClick.php?CityName=" .
$SITE['fcsturlNWS'] = "https://forecast.weather.gov/MapClick.php?CityName=" .

I think those are all of the changes and I now get the "old" style point forecast.
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 27, 2018, 01:12:08 PM
I tried the https changes on the 4.02 and still doesn't work here. At least 5.04 is working now but don't like not having backup 4.02 because 5.04 was so unreliable.
Title: Re: Point Forecast Down?
Post by: swohiowx on February 27, 2018, 01:15:47 PM
OK, I spent a few more minutes on this and got 4.02 working. These are the changes I made to 4.02 advforecast2.php and settings.php - not sure if all were necessary but....

......

I think those are all of the changes and I now get the "old" style point forecast.

those changes fixed me...thanks!
Title: Re: Point Forecast Down?
Post by: daybreak on February 27, 2018, 01:24:40 PM
I changed my script from 4.02 to 5.04 and mine is working now.  This is from the St. Louis office.

Richard
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 27, 2018, 02:02:59 PM
I wonder if this was brought on by hackers. Yesterday malwarebytes was warning the NWS sites are malicious URL and asked if I wanted to continue.  I found it odd and now this.

This was the report:


-Blocked Website Details-
Malicious Website: 1
, , Blocked, [-1], [-1],0.0.0

-Website Data-
Category: Unspecified
Domain: zn9tslis2mj3kopgp-cfigroup.siteintercept.qualtrics.com
IP Address: 173.222.34.16
Port: [56386]
Type: Outbound
File: C:\Program Files (x86)\Google\Chrome\Application\chrome.exe



(end)
Title: Re: Point Forecast Down?
Post by: 92merc on February 27, 2018, 03:01:25 PM
I saw my V5 active script was reverting.  I flushed my JSON files from the cache, refreshed the page and it all came back.  I had to do the same thing yesterday morning.
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 27, 2018, 03:12:21 PM
Grrr.  They changed the forecast.weather.gov site to force HTTPS.

So.. I've updated the OLD advforecast2.php to V4.03 and switched to curl for doing the access.

Use:  https://saratoga-weather.org/advforecast2-V4.03.php?sce=view to have the OLD script work again.

Dang 'em
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 27, 2018, 03:19:40 PM
Thanks Ken 4.03 is working here.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 27, 2018, 03:49:02 PM
Knock on wood.... but I'm glad I didn't switch back. I've really had VERY few issues with 5.04 other than those times when the NWS screws up.
Title: Re: Point Forecast Down?
Post by: AWL on February 27, 2018, 04:13:30 PM
Is this NWS change also stopping the"forecast discussion" from working?
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 27, 2018, 04:15:26 PM
On mine it did, I added the "s" and all is good.
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 27, 2018, 04:37:27 PM
Also not working are.
wxnoaadiscuss.php
wxnoaaproducts.php

I changed all the http to https in.... inc-show-wfo-discuss.php..... and made no difference.
Title: Re: Point Forecast Down?
Post by: chief-david on February 27, 2018, 04:39:59 PM
good here after the new change.
Looks like I had not updated since 4.01

usually when things like that happen it is the schools fault
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 27, 2018, 05:02:00 PM
Is this NWS change also stopping the"forecast discussion" from working?
Yes.. forecast-discussion.php V1.06 is now working with https

Download from the script page at https://saratoga-weather.org/scripts-NWSmisc.php

Busy day.  Never thought the NWS would just engage https only w/o a note to that effect on the NWS notification list.
Sigh. 
Title: Re: Point Forecast Down?
Post by: lddaly on February 27, 2018, 05:23:21 PM
Thank you, Ken!
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 27, 2018, 05:28:46 PM
Never mind its working now. Must of been something in cache still. Thanks

Ken I was going to try forecast-discussion.php but for some reason I'm getting SF office even though I changed line 48 to
Code: [Select]
$myNWS = 'LBF'; So what am I doing wrong here? I did a ctr-f5 to clear browser cache and no change.
[/s]
https://www.valentinenebraska.net/forecast-discussion.php
Title: Re: Point Forecast Down?
Post by: tshattuck on February 27, 2018, 05:48:56 PM
Ken:
I just updated advforecast2 v.4.02 to v.4.03 and the NWS forecasts are all working again.

Just wanted to say thanks for all your efforts. Your the best!  =D&gt; =D&gt; =D&gt;

Tom
Title: Re: Point Forecast Down?
Post by: DaculaWeather on February 27, 2018, 06:08:52 PM
It was a  pretty sudden change, this is NWS Birmingham:
----------------
Just an FYI to all customers, our web server now defaults to an HTTPS address by default when you load it. Therefore, there may be some links and other items that are currently not working due to this global change. We apologize for the inconvenience, and will do our best to get things back working - such as the obs table on our frontpage - which is the most notable item not currently working. You can still click on the city name to get an obs table for that location in the meantime.
Title: Re: Point Forecast Down?
Post by: ValentineWeather on February 27, 2018, 06:20:47 PM
It was a  pretty sudden change, this is NWS Birmingham:
----------------
Just an FYI to all customers, our web server now defaults to an HTTPS address by default when you load it. Therefore, there may be some links and other items that are currently not working due to this global change. We apologize for the inconvenience, and will do our best to get things back working - such as the obs table on our frontpage - which is the most notable item not currently working. You can still click on the city name to get an obs table for that location in the meantime.

Wouldn't be surprised from the suddenness of going to secure addresses it had something to do with what malwarbytes was seeing yesterday. Might as well blame the Russians, it was probably China but we don't want to upset our trade allies.
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 27, 2018, 06:28:40 PM
Or.. it could be due to a Symantec cert that is to expire soon, and this https://blog.qualys.com/ssllabs/2017/09/26/google-and-mozilla-deprecating-existing-symantec-certificates
Title: Re: Point Forecast Down?
Post by: Weather Goat on February 27, 2018, 06:31:52 PM
Wow!  Thanks, Ken!
I was racking my brain this morning why the AFD pages worked but not in a script.  Great fix.
Roger
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 27, 2018, 06:35:11 PM
Never mind its working now. Must of been something in cache still. Thanks

Ken I was going to try forecast-discussion.php but for some reason I'm getting SF office even though I changed line 48 to
Code: [Select]
$myNWS = 'LBF'; So what am I doing wrong here? I did a ctr-f5 to clear browser cache and no change.
[/s]
https://www.valentinenebraska.net/forecast-discussion.php
It's working now.. you probably needed to wait for the bad cache to expire on your website.  You can force it to reload by forecast-discussion.php?cache and it will reload from the URL
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 27, 2018, 06:36:24 PM
It was a  pretty sudden change, this is NWS Birmingham:
----------------
Just an FYI to all customers, our web server now defaults to an HTTPS address by default when you load it. Therefore, there may be some links and other items that are currently not working due to this global change. We apologize for the inconvenience, and will do our best to get things back working - such as the obs table on our frontpage - which is the most notable item not currently working. You can still click on the city name to get an obs table for that location in the meantime.
Ahh.. at least one office commented about it.  The elves supporting the main forecast.weather.gov site just had a poorly-communicated http->https conversion.
Title: Re: Point Forecast Down?
Post by: SLOweather on February 27, 2018, 08:43:32 PM
Thanks, Ken.... :)

Grrr.  They changed the forecast.weather.gov site to force HTTPS.

So.. I've updated the OLD advforecast2.php to V4.03 and switched to curl for doing the access.

Use:  https://saratoga-weather.org/advforecast2-V4.03.php?sce=view to have the OLD script work again.

Dang 'em
Title: Re: Point Forecast Down?
Post by: metzeric on February 27, 2018, 08:58:04 PM
I got my forecast discussion page working using the 1.06 version  however   my main page has a forecast summary box   and that still shows  "forecast discussion unavailable, please try later."      that box uses a file called  "forecast-summary.php"  version 1.03
i am not sure what the fix for that is though

thanks
Eric

Is this NWS change also stopping the"forecast discussion" from working?
Yes.. forecast-discussion.php V1.06 is now working with https

Download from the script page at https://saratoga-weather.org/scripts-NWSmisc.php

Busy day.  Never thought the NWS would just engage https only w/o a note to that effect on the NWS notification list.
Sigh.
Title: Re: Point Forecast Down?
Post by: saratogaWX on February 27, 2018, 09:23:48 PM
More busy... I've updated the advforecast2.php (JSON Version) to V5.05 and it has extra code to better handle redirects and failover to zone forecasts when needed.

It really irritates me that they've set the API for a point forecast to ALWAYS return a 301 to a gridpoint forecast instead of just returning the JSON requested.  Grrrr...  I've enabled curl to follow one redirect, set the timeout longer (6 seconds) and handle the muddled headers returning from a curl-fetch-with-redirect.  Also set the point forecast life to be 18 hrs (instead of 12) to be 'stale' and fail over to the zone forecast.

Hopefully these mods will make our sites more stable while the api.weather.gov is still having flaky responses.

I've also decided to dual post the V4.04 and V5.05 versions of the script on the script page (https://saratoga-weather.org/scripts-carterlake.php) so there will be a durable link to the old (page scraper) script while the cutover to the new forecast.weather.gov is still pending.

Template users can pick up the updates at update tool page (https://saratoga-weather.org/wxtemplates/updates.php) with a query of Base-USA, Plugin-*, 27-Feb-2018

I think I'll rest up a bit.. it's been a busy day for NWS-related changes.
Title: Re: Point Forecast Down?
Post by: SoMDWx on February 28, 2018, 07:42:59 AM
Thanks Ken again for the updates! Forecast discussion and advforecast went south on me as probably with everyone else... Back to working this morning...

Awesome job keeping up with these changes....   =D&gt;

Jim
Title: Re: Point Forecast Down?
Post by: Jasiu on February 28, 2018, 08:53:06 AM
Thank you for the hard work, Ken. I think it's time for me to pony up some $$ in appreciation again. (Anyone can do so at http://saratoga-weather.org/wxtemplates/).

Wondering as I was looking at the code yesterday... might it be time for one common "fetchURL" routine to include rather than the half-dozen or so versions that are in the various files? Would probably need additional parameters for things like accepted data type, timeouts, follow redirects, etc. But at least a fix in the common routine wouldn't have to be applied in so many places. Just a thought...

Title: Re: Point Forecast Down?
Post by: cospringswx on February 28, 2018, 09:38:50 PM
Thanks for the update Ken. Working here. =D&gt;

http://cospringsweather.com/wxforecast.php
Title: Re: Point Forecast Down?
Post by: Jasiu on March 01, 2018, 07:46:02 AM
Well, this is..... interesting??

As of this morning, 4.03 is working. 5.05 is acting rather weird on NWS's side.

Here is the comment stream from the code:

Code: [Select]
<!-- File Name:  https://forecast.weather.gov/MapClick.php?CityName=Lexington&state=MA&site=MTR&textField1=42.4399&textField2=-71.2405&e=1&TextType=2 -->
<!-- advforecast2.php (JSON) - V5.05 - 27-Feb-2018 on PHP 7.2.1-->
<!-- RAW NWS URL: https://forecast.weather.gov/MapClick.php?CityName=Lexington&state=MA&site=MTR&textField1=42.4399&textField2=-71.2405&e=1&TextType=2 -->
<!-- zone=MAZ014 -->
<!-- corrected NWS API URL='https://api.weather.gov/points/42.4399,-71.2405/forecast' -->
<!-- curl fetching 'https://api.weather.gov/points/42.4399,-71.2405/forecast' -->
<!-- note: fetched 'https://api.weather.gov/gridpoints/BOX/64,79/forecast' after redirect was followed. -->
<!-- HTTP stats:  RC=404 dest=104.72.62.193 port=443 (from sce=74.208.114.109)
      Times: dns=0.000 conn=0.031 pxfer=0.063 get=0.775 total=0.837 secs -->
<!-- headers returned:
HTTP/1.1 404 Not Found
Server: nginx/1.10.2
Content-Type: application/problem+json
Access-Control-Allow-Origin: *
Pragma: no-cache
X-Server-ID: vm-bldr-nids-apiapp22.ncep.noaa.gov
X-Correlation-ID: 99795aeb-e0cb-4d16-af5e-254f3f8ff3a8
X-Request-ID: 99795aeb-e0cb-4d16-af5e-254f3f8ff3a8
Cache-Control: private, must-revalidate, max-age=600
Expires: Thu, 01 Mar 2018 12:47:06 GMT
Date: Thu, 01 Mar 2018 12:37:06 GMT
Content-Length: 364
Connection: keep-alive
Vary: Accept
Strict-Transport-Security: max-age=15768000 ; includeSubDomains
 -->
<!-- Oops.. point forecast unavailable RC=404 - using Zone instead -->

After this, the zone forecast loads fine.  So... Why the 404?  I tried loading the redirected URL in a browser and got this:

Code: [Select]
{
    "correlationId": "2442b844-33f9-4e85-a3b4-f0b8316473c4",
    "title": "Data Unavailable For Requested Point",
    "type": "https://api.weather.gov/problems/InvalidPoint",
    "status": 404,
    "detail": "Unable to provide data for requested point 90,120.55176052112",
    "instance": "https://api.weather.gov/requests/2442b844-33f9-4e85-a3b4-f0b8316473c4"
}

So the API thinks I'm trying to get the forecast for the North Pole? Hmmm.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on March 01, 2018, 10:12:49 AM
Jasiu, I sent your code on to the NWS to have them look at it.
Title: Re: Point Forecast Down?
Post by: saratogaWX on March 01, 2018, 11:19:54 AM
The API acts a bit strange (and has for months).  A point query now always results in a 301 redirect to a gridpoint query.

The gridpoint query either returns the data (RC=200) or if the data is not available, a 404-Not Found.  That is usually because (I think) the underlying GFS data for that gridpoint is being refreshed.  Their site (and the V5 script) both try a Zone forecast instead and display that data (less accurate) until the gridpoint data is again available.
Title: Re: Point Forecast Down?
Post by: Jasiu on March 09, 2018, 10:47:51 AM
Quick note to those who are in the Boston NWS office area: They know that point forecasts are down and are working on it...

Zone loads fine.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on March 09, 2018, 10:56:42 AM
Had a problem here yesterday with the Atlanta NWS and they told me this:
"Working an issue now where our grid files aren't making it up to the server."

Once they fixed that mine started working.
Title: Re: Point Forecast Down?
Post by: 92merc on March 09, 2018, 11:04:05 AM
Yesterday afternoon, both of my V4 and V5 versions wouldn't work.  This morning, I was finally able to flip back to V4.  But V5 still shows as down.
Title: Re: Point Forecast Down?
Post by: DaculaWeather on March 09, 2018, 11:17:57 AM
v5 is working with no problems here today. I wonder how much of these problems are local NWS office related issues?

I've got the email address of the Atlanta NWS tech guy and he is usually on it pretty quickly. But again, they shouldn't be having issues either...
Title: Re: Point Forecast Down?
Post by: 92merc on March 09, 2018, 02:00:18 PM
Well, I complain on the forum and poof!  It's magically is working again!
Title: Re: Point Forecast Down?
Post by: ValentineWeather on March 10, 2018, 06:27:52 AM
This may be old news, saw this today after only zone forecast was showing even on NWS site.
https://forecast-v3.weather.gov/documentation?redirect=legacy
Title: Re: Point Forecast Down?
Post by: tshattuck on March 19, 2018, 09:19:59 PM
Ken:
I am running the new advforecast2 v.5.05 php script and have an issue. The Tonight Forecast is "Mostly Clear" but the icon displayed is the Cold.png image. This is the second night in a row this has occurred - same forecast.
Is there an image missing or did the NWS throw another curve ball?

Your assistance is appreciated
Thanks
Title: Re: Point Forecast Down?
Post by: saratogaWX on March 19, 2018, 09:30:50 PM
It's the NWS.. they specify the icon to use.  I think that when the forecast temperature is low ( like in the teens or lower), they choose to use 'cold' as the icon.  The script does not switch icons (except to local copies), so it's what they specify.
Title: Re: Point Forecast Down?
Post by: tshattuck on March 19, 2018, 10:42:16 PM
Ken:
Thanks for the clarification.
Title: Re: Point Forecast Down?
Post by: daybreak on March 20, 2018, 10:03:57 AM
Hello Everyone,

Well my forecast quit today, it's been working fine up to this point.  I was running version 5.04 and upgraded to 5.05 after it stopped working this morning but that didn't help.  Should I go back to Version 4?  Or is this a temporary thing?  My forecast is at the bottom of this page: http://www.daybreakimagery.com/weather/ (http://www.daybreakimagery.com/weather/) vertically and also on this page the normal way.  http://www.daybreakimagery.com/weather/forecast.php (http://www.daybreakimagery.com/weather/forecast.php) 

Thank you,

Richard
Title: Re: Point Forecast Down?
Post by: saratogaWX on March 20, 2018, 10:42:37 AM
The issue may be related to a cache file issue
Quote
!-- advforecast2.php (JSON) - V5.05 - 27-Feb-2018 on PHP 5.3.3-->
<!-- RAW NWS URL: http://forecast.weather.gov/MapClick.php?CityName=Alma&state=IL&site=LSX&textField1=38.7229&textField2=-88.9101&e=0&TextType=2 -->
<!-- zone=ILZ070 -->
<!-- corrected NWS API URL='https://api.weather.gov/points/38.7229,-88.9101/forecast' -->
<!-- curl fetching 'https://api.weather.gov/points/38.7229,-88.9101/forecast' -->
<!-- HTTP stats:  RC=301
      Times: dns=0.006 conn=0.008 pxfer=0.022 get=0.003 total=0.025 secs -->
<!-- headers returned:
HTTP/1.1 301 Moved Permanently
Server: nginx/1.10.2
Content-Type: application/problem+json
Location: /gridpoints/LSX/139,80/forecast
Access-Control-Allow-Origin: *
Pragma: no-cache
X-Server-ID: vm-bldr-nids-apiapp35.ncep.noaa.gov
X-Correlation-ID: 715d440b-0cf7-44b0-9017-44274074f8de
X-Request-ID: 715d440b-0cf7-44b0-9017-44274074f8de
Content-Length: 349
Cache-Control: private, must-revalidate, max-age=600
Expires: Tue, 20 Mar 2018 14:49:11 GMT
Date: Tue, 20 Mar 2018 14:39:11 GMT
Connection: keep-alive
Vary: Accept
Strict-Transport-Security: max-age=31536000 ; includeSubDomains ; preload
 -->
<!-- loaded point-forecast https://api.weather.gov/points/38.7229,-88.9101/forecast - 961 bytes -->
<!-- meta info re: point='38.7229,-88.9101' zone='ILZ070' metacache= './forecast-ILZ070-0-json-meta.txt' -->
<!-- metaZoneURL='https://api.weather.gov/zones/forecast/ILZ070' -->
<!-- Meta cache file not found .. reloading data from NWS site -->
<!-- getting metadata for point from https://api.weather.gov/points/38.7229,-88.9101 -->
<!-- curl fetching 'https://api.weather.gov/points/38.7229,-88.9101' -->
<!-- HTTP stats:  RC=200
      Times: dns=0.000 conn=0.001 pxfer=0.016 get=0.003 total=0.019 secs -->
<!-- distance=0 from Alma -->
<!-- getting metadata for forecast zone from https://api.weather.gov/zones/forecast/ILZ070 -->
<!-- curl fetching 'https://api.weather.gov/zones/forecast/ILZ070' -->
<!-- HTTP stats:  RC=200
      Times: dns=0.000 conn=0.001 pxfer=0.015 get=0.003 total=0.018 secs -->
<!-- getting metadata for forecast office from https://api.weather.gov/offices/LSX -->
<!-- curl fetching 'https://api.weather.gov/offices/LSX' -->
<!-- HTTP stats:  RC=200
      Times: dns=0.000 conn=0.001 pxfer=0.015 get=0.005 total=0.019 secs -->
<!-- unable to write meta cache file ./forecast-ILZ070-0-json-meta.txt -->

You may want to create a ./cache/ directory, allow writing to that directory, and change the script to use ./cache/ instead of ./ to use that directory.  If there are issues writing the cache files, the forecast may not appear even if the fetching was successful.
Title: Re: Point Forecast Down?
Post by: daybreak on March 20, 2018, 10:50:05 AM
Thanks Ken!

I'm php code illiterate but I'll try to figure this out.  I'm not sure I totally understand but I'll give it my best shot!

Thanks,

Richard
Title: Re: Point Forecast Down?
Post by: saratogaWX on March 20, 2018, 10:57:35 AM
The line in advforecast2.php to change is
Code: [Select]
$cacheFileDir = './'; // default cache file directory
.. change it to
Code: [Select]
$cacheFileDir = './cache/'; // default cache file directory
after you create a ./cache/ directory on your website.
Title: Re: Point Forecast Down?
Post by: daybreak on March 20, 2018, 02:58:56 PM
Hi Ken,

I created the cache directory and changed the line of code but still nothing.  Unless I'm doing something wrong, which is very possible.

Richard
Title: Re: Point Forecast Down?
Post by: saratogaWX on March 20, 2018, 03:23:27 PM
You created the cache directory below the weather directory?  (that is weather/cache/ from the document root of your website)
Make sure the cache directory has permissions of 766 so files can be written there.
Title: Re: Point Forecast Down?
Post by: dasman on March 20, 2018, 05:43:28 PM
Something is up with mine today. Notice all of the low temps on the attatched. [ You are not allowed to view attachments ]
Title: Re: Point Forecast Down?
Post by: saratogaWX on March 20, 2018, 09:12:52 PM
Ahhh... you have the API occasional issue where extreme range numbers get populated .. before you start a fusion plant with those numbers, just call your local NWS office and say the forecast-v3.weather.gov current gridpoint forecast is wonky for your area. 
Title: Re: Point Forecast Down?
Post by: daybreak on March 20, 2018, 09:28:46 PM
I created the directory in the correct place and still not working.    :sad:
Title: Re: Point Forecast Down?
Post by: saratogaWX on March 20, 2018, 10:01:11 PM
Hmmm... you are running a really old PHP 5.3.3 on your site.  Is it possible to upgrade to PHP 5.5 5.6 or 7.1?  The old curl on 5.3.3 had a few https foibles and maybe you're having the issue there.  I tried your URL+zone in the current JSON script (which you are running that version) and it worked fine.  Your site is also uses ngnix for web services and not apache, but I don't think that would make a difference (but the old PHP might).

Contact the tech support for your hoster and ask them what permissions to use on the weather/cache/ directory in order to allow a PHP script to write a file there, and ask about upgrading the PHP version.  Since you're not running a Saratoga template site, I don't have access to my customary diagnostic tool, so I can only use the HTML comments in the view-source of the script running to see what issues there are .. the 'unable to write' the cache file(s) is the current big issue.
Title: Re: Point Forecast Down?
Post by: saratogaWX on March 20, 2018, 10:20:42 PM
I ran advforecast2.php?force=2 on your site.. and the zone forecast was displayed (and the cache files are saved).  The Point forecast is still not working, but there is progress :)
Title: Re: Point Forecast Down?
Post by: daybreak on March 21, 2018, 08:07:41 AM
Ok Ken thank you so much for your help.  I will contact them today and see about upgrading.

Richard
Title: Re: Point Forecast Down?
Post by: daybreak on March 25, 2018, 06:44:44 PM
Hi Ken,

I thought I should give you an update.  My web guy had been on a camping trip and returned this weekend and updated my site to PHP 7.  That did the trick, so thank you very much for all your help and suggestions.

Richard