Author Topic: advforecast2 issue  (Read 3197 times)

0 Members and 1 Guest are viewing this topic.

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
advforecast2 issue
« on: September 23, 2017, 12:44:41 PM »
I am having issues with my forecast pages not being updated. Some of them work fine while others do not. I assume this is a NWS issue but that may be a bad assumption. I'm running the latest version of advforecast2. I've attached a screen shot of one of the forecasts and the discrepancy in time shown.


Any thoughts on whether this is just a NWS issue or do I have something else that may be causing the problem. I'm getting lots of comments from my users regarding the "out of date" forecasts they are seeing. This has been going on for the past few weeks.
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: advforecast2 issue
« Reply #1 on: September 23, 2017, 02:51:41 PM »
I suspect it's the NWS.  I had a similar issue first thing this morning, but it's now resolved.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: advforecast2 issue
« Reply #2 on: September 23, 2017, 03:02:01 PM »
Thanks Jerry. I too suspect this is a NWS issue since I can select forecasts for other out of state/non-local areas and everything looks good. Guess I am just perplexed that NWS has been having this problem for quite some time now. I hate getting emails and messages from my users telling me I have a problem with my forecasts on my website. Perhaps I should add some text telling them to check the update time and if it's not relatively current, that the issue is due to data issues with the NWS and not my website...  :(
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #3 on: September 23, 2017, 07:38:21 PM »
Try reloading the page with ?force=1 to force a cache reload.  It may result in a Zone forecast (because a new point-forecast wasn't available), but that will do a reload for the 'freshest'.

Looking at the page with view-source (and adding the ?force=1 to it) will show in HTML comments exactly what is happening.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: advforecast2 issue
« Reply #4 on: September 23, 2017, 11:44:04 PM »
Try reloading the page with ?force=1 to force a cache reload.  It may result in a Zone forecast (because a new point-forecast wasn't available), but that will do a reload for the 'freshest'.

Looking at the page with view-source (and adding the ?force=1 to it) will show in HTML comments exactly what is happening.
Do I add the ?force=1 to the advforecast2.php like this: http://carsonvalleyweather.com/advforecast.php?force=1 or some other way? I tried with the ?force=1 with the advforecast.php and wxforecast.php and get essentially the same results when I view-source of the resulting webpage. I do see a similar indication of a failure in that the forecast updated info is always as follows:
  • <!-- forecast updated '2017-09-18T21:09:50+00:00' (Mon, 18-Sep-2017 2:09pm PDT) for lat,long or zone 'POINT (-119.7211 39.0386)' -->
That translates to 09-18-2017 2:09pm PDT which seems to be the last update I've seen for my primary forecast location.
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #5 on: September 24, 2017, 11:15:05 AM »
You can add ?force=1 to either the wxforecast.php or advforecast2.php pages.. the effect is still the same: try reloading from the api.weather.gov site.

Did the HTML comments above that line indicate RC=200 (successfully got data) on the curl request?
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: advforecast2 issue
« Reply #6 on: September 24, 2017, 10:44:18 PM »
Hi Ken,

I have attached a copy of both of my view sources. The first is for WXFORECAST.php and the second is for ADVFORECAST2.PHP. Both are txt files.

I do see RC=200 in both files. And, tonight, all of the forecasts have current date/time stamps on them. So, I am once again guessing that the problem was with the NWS servers not responding with the correct information.

Thanks for the help but it looks like there really isn't anything wrong other than NWS issues.

Best regards,
Don


Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #7 on: September 25, 2017, 12:40:05 PM »
Hi Don,
Yes, it's an NWS issue...
Quote
<!-- advforecast2.php (JSON) - V5.03 - 16-Jul-2017 on PHP 5.6.31-->
<!-- RAW NWS URL: http://forecast.weather.gov/MapClick.php?site=mtr&smap=1&textField1=39.038556&textField2=-119.721083&TextType=2 -->
<!-- zone=NVZ003 -->
<!-- corrected NWS API URL='https://api.weather.gov/points/39.0386,-119.7211/forecast' -->
<!-- curl fetching 'https://api.weather.gov/points/39.0386,-119.7211/forecast' -->
<!-- HTTP stats:  RC=503 dest=23.217.44.49 port=443 (from sce=198.71.62.63)
      Times: dns=0.060 conn=0.096 pxfer=0.172 get=0.270 total=0.442 secs -->
<!-- headers returned:
HTTP/1.1 503 Service Unavailable
Server: nginx
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Content-Type: application/problem+json
X-Powered-By: PHP/5.6.25
Retry-After: 120
Access-Control-Allow-Origin: *
X-Server-ID: vm-bldr-nids-apiapp115.ncep.noaa.gov
X-Correlation-ID: 47e13ce0-3a3e-4506-8150-a5f964c0794c
X-Request-ID: 47e13ce0-3a3e-4506-8150-a5f964c0794c
Cache-Control: max-age=600
Expires: Sun, 24 Sep 2017 03:38:20 GMT
Date: Sun, 24 Sep 2017 03:28:20 GMT
Connection: close
Vary: Accept
 -->
<!-- Oops.. point forecast unavailable RC=503 - using Zone instead -->
<!-- curl fetching 'https://api.weather.gov/zones/JSON-LD/NVZ003/forecast' -->
<!-- HTTP stats:  RC=200 dest=23.217.44.49 port=443 (from sce=198.71.62.63)
      Times: dns=0.000 conn=0.034 pxfer=0.106 get=0.037 total=0.142 secs -->
<!-- loaded (Zone forecast) https://api.weather.gov/zones/JSON-LD/NVZ003/forecast - 10060 bytes -->
<!-- wrote cache file ./cache/forecast-NVZ003-0-json.txt -->

At least the point request received a 503-Server Error so the script fetched the Zone (successfully) instead and parsed that JSON for data.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: advforecast2 issue
« Reply #8 on: October 11, 2017, 12:27:27 PM »
Okay, it looks like something else is going on now. I am seeing no updates for my local forecast and I do see a RC=200 in the view source. I have attached a portion of that info to this post. And yes, I did try the ?force=1 but without any changes seen.

Additionally, I notice that the forecast section of the index.php page is not showing current forecast data either. Looking at the view source for that page also shows the same date of 10-08-2017 so it appears that the NWS servers aren't giving me good data - again.

Any thoughts on what may be going on?
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #9 on: October 11, 2017, 12:45:45 PM »
Funny... Bob over at Racers.net sent me an inquiry about this too..

The odd thing is the stale forecast is giving a RC=200 (instead of the usual 4xx or 5xx return code).

I've done a bit of fiddling with advforecast2.php (attached) to try and eliminate 'old' point forecasts with a Zone forecast, but can't fully test it since my request for the point-forecast API URL gets a 503 (normal) fail and does the Zone automatically.

Try this and see if it fixes the current error...
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: advforecast2 issue
« Reply #10 on: October 11, 2017, 01:04:54 PM »
Installed and now I get the substitute forecast as I would expect.

Will watch it for a while to see what happens.

Thanks for a quick solution!!!
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: advforecast2 issue
« Reply #11 on: October 11, 2017, 01:27:29 PM »

Hi Ken,

Something else doesn't look right when I view the forecast data on my index.php page. What I see is "tonight's forecast low and tomorrow's forecast high. During the day, I would normally expect to see today's forecast high and tonight's forecast low. Not sure if this has anything to do with NWS or not but thought I would bring it up while we are talking about forecast data.
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #12 on: October 11, 2017, 01:35:01 PM »
The 'Outlook' in the dashboard always shows the $forecasticon[1] (next period) while the detailed Outlook below the dashboard shows the $forecasticon[0] and $forecasticon[1] values.  What you're seeing may be due to the Zone forecast being used instead of the point forecast.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #13 on: October 11, 2017, 05:29:30 PM »
I've released V5.04 advforecast2.php via the usual channels standalone and Saratoga Template Update Tool (Base-USA, Plugin-*, 11-Oct-2017)

It includes the handling of a point-forecast API query successfully returning a 'stale' (> 12hr old) forecast to cause a failover to the Zone forecast.  It also has a bit of cleanup in the Zone forecast parsing so High/Low temperatures are better extracted.  Turns out the NWS includes extra \n (new-line) literals in the zone text.

Be aware that updates in the Base-* also include V1.16 of get-metar-conditions-inc.php (fixes issue with fractional visibility display in km, change to use curl for data fetch) and V3.00 of get-USNO-sunmoon.php (use curl fetch, add optional calculations instead of USNO fetch).
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline jgillett

  • Forecaster
  • *****
  • Posts: 1187
  • Boltek, Win7 Pro, ToA
    • TiggrWeather Phoenix
Re: advforecast2 issue
« Reply #14 on: October 12, 2017, 11:31:51 AM »
Having trouble getting to your server, Ken. The updates page does not respond at all - just the little blue loading circle in FF. Also noticed your banner here took well over 30 seconds to pop into place.

Just FYI.
John
W7JKG

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #15 on: October 12, 2017, 11:48:02 AM »
Thanks .. The https access seemed to be stuck in TLS negotiations.  I've restarted the Apache services.. seems better now.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline jgillett

  • Forecaster
  • *****
  • Posts: 1187
  • Boltek, Win7 Pro, ToA
    • TiggrWeather Phoenix
Re: advforecast2 issue
« Reply #16 on: October 12, 2017, 11:52:49 AM »
Still stuck, Ken. The blue connect circle for both pages. Had one on this page as well but it finally cleared after one minute but the updates page still has not loaded after >3 minutes. Also your banner is still missing here.
John
W7JKG

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #17 on: October 12, 2017, 11:57:15 AM »
Hmmm... I'll dig a bit deeper with Tech Support.  Thanks again!
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: advforecast2 issue
« Reply #18 on: October 12, 2017, 12:04:38 PM »
Yup, having the same problems. I get a "This site can't be reached" message on https://saratoga-weather.org/.

Had a similar issue last night but after several tries it worked okay but not this morning...
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #19 on: October 12, 2017, 12:34:54 PM »
I'm working with 1and1 server support now.  Try using the http version instead of https while the issue is being addressed.

Sorry for the inconvenience...
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline jgillett

  • Forecaster
  • *****
  • Posts: 1187
  • Boltek, Win7 Pro, ToA
    • TiggrWeather Phoenix
Re: advforecast2 issue
« Reply #20 on: October 12, 2017, 01:30:24 PM »
Not you, it's the other guys :!:  Anyway https now working but still a bit slow.

Thanks.
John
W7JKG

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #21 on: October 12, 2017, 02:00:58 PM »
I seem to be getting more traffic than usual and running up against some VPS limits on IP traffic.. grrr.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline 92merc

  • BismarckWeather.net
  • Forecaster
  • *****
  • Posts: 1308
  • BismarckWeather.net
    • BismarckWeather.net
Re: advforecast2 issue
« Reply #22 on: October 12, 2017, 03:52:33 PM »
Weird.  I accessed the site at 8:40 am this morning with no issues.  Put the updated code on my site and everything worked great.
https://www.BismarckWeather.net
Davis VP2, Cumulus, WeatherDisplay, Blitzortung, Saratoga Scripts, NOAA Stream via PI

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2 issue
« Reply #23 on: October 12, 2017, 04:10:20 PM »
So the issue was likely caused by having too many SSH attempts by miscreants and having fail2ban 'jail' them, leading to a clog in the max iptables allowed.  Sigh.  I've changed to use basic firewall rules instead (limiting FTP, SSH access) to avoid the iptables clog.

Now it seems to be running much more responsively.  Knock wood...
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

 

anything