Author Topic: NWS Forecast Map - Not Updated Since December 24th  (Read 897 times)

0 Members and 1 Guest are viewing this topic.

Offline CNYWeather

  • Forecaster
  • *****
  • Posts: 2295
    • CNYWeather
NWS Forecast Map - Not Updated Since December 24th
« on: January 05, 2022, 07:45:37 AM »
I noticed today that my Forecast Map http://www.cnyweather.com/wxforecastmap.php hasn't updated since December 24th.
I'm surprised non of my website visitors hadn't emailed me about it. Anyway, I have advforecast2 version 5.17 installed.

The map isn't current on page load, but if you double click on it, it will give you the current forecast.
Where in the Settings.php file do I change it from Westmoreland which doesnt have a point forecast available to Utica, NY which does?
I don't see anyone having any issues like this, so something must have changed somewhere along the way
and the script for that page hasn't been updated either.

THANKS!!
« Last Edit: January 05, 2022, 07:49:40 AM by CNYWeather »
Tony




Offline rrrick8

  • Senior Contributor
  • ****
  • Posts: 224
    • Vermilion weather
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #1 on: January 05, 2022, 08:43:04 AM »
Change the coordinates to match your desired location...
Code: [Select]
#  set to station latitude/longitude (decimal degrees)
  $myLat = 40.130;    //North=positive, South=negative decimal degrees
  $myLong = -87.624;   //East=positive, West=negative decimal degrees

Mine still hangs up after a few days so I manually adjust by a few degrees to fetch a slightly different "pinpoint" but still within my city.

Seems to be a cache problem.
Severe Weather Manager-Vermilion County EMA
CWOP-CW9931 KILDANVI5

Offline CNYWeather

  • Forecaster
  • *****
  • Posts: 2295
    • CNYWeather
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #2 on: January 05, 2022, 11:45:38 AM »
There's another issue somewhere i'll have to find. I did change the lat/long

Code: [Select]
#  set to station latitude/longitude (decimal degrees)
  $myLat = 43.2128;    //North=positive, South=negative decimal degrees
  $myLong = -75.4557;   //East=positive, West=negative decimal degrees

It was still showing the old coordinates. I deleted the file off the server, page came up blank,
uploaded a fresh version and still on page load its the old coordinates and Dec 24th.
Tony




Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #3 on: January 05, 2022, 12:14:13 PM »
You seem to have two issues:
1) your site is using Cloudflare caching, and external caching like that is never a good idea for a weather website (which needs up-to-date information).
2) the api.weather.gov gridpoint with data seems to be 'stale' (Updated Fri, 24-Dec-2021 3:49am EST) and that's a NWS cache issue (been going on now for over a year with sporadic 'stale' data available by the API.  They use Akamai caching for their data, and the automatic updates/refreshes for that data seem to be very fragile and break often.)

My advforecast2.php V5.17 spots that and handles that issue by displaying the Zone forecast instead.  The forecast map script uses NO caching and just does 3 pulls from api.weather.gov for display (and all the icons come from there also).
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline CNYWeather

  • Forecaster
  • *****
  • Posts: 2295
    • CNYWeather
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #4 on: January 05, 2022, 04:52:10 PM »
I have GoDaddy as a host. Where is the Cloudflare caching coming from or is that something with GoDaddy?
Tony




Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #5 on: January 05, 2022, 05:10:41 PM »
Cloudflare caching is likely a cPanel setting with GoDaddy.  If you want to continue using caching, then disable it for .php and .txt files. Then images and JavaScripts would be cached (would be bad news if you're frequently updating a webcam image, however).

As I said, personal weather websites don't really need caching, and having it enabled means that 'current' weather might actually be 'stale' weather depending on the cache lifetime.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline CNYWeather

  • Forecaster
  • *****
  • Posts: 2295
    • CNYWeather
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #6 on: January 06, 2022, 10:08:41 AM »
Cloudflare caching is likely a cPanel setting with GoDaddy.  If you want to continue using caching, then disable it for .php and .txt files. Then images and JavaScripts would be cached (would be bad news if you're frequently updating a webcam image, however).

As I said, personal weather websites don't really need caching, and having it enabled means that 'current' weather might actually be 'stale' weather depending on the cache lifetime.

After looking at the NWS-forecast-map code, I see that Cloudflare is where the java script map library is. Line 213:

Code: [Select]
<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.5.1/leaflet.js" type="text/javascript"></script>
Tony




Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #7 on: January 06, 2022, 11:03:55 AM »
That is just the source for the Leaflet script (happens to be stored on a Cloudflare CDN) .. it has nothing to do with Cloudflare being enabled for caching of pages/etc on your website -- it looks like pages/etc are not being cached on your website at this time (based on the HTTP response headers missing any of the usual cache control headers).

The problem with the forecast map script is due entirely to problems with caching BY THE NWS of api.weather.gov responses with stale data.
As I said before, this is a sporadic but long-standing problem with the NWS and data refresh to the Akamai cache servers that the NWS will have to fix.

Right now, I have a more serious data problem I've reported to the NWS API team -- the point->gridpoint lookups for MTR (Monterey/SF Bay) office seem to have a +0.5 degree latitude bias, so the forecast is coming back from the wrong place.  My own page at https://saratoga-weather.org/forecast-map.php is showing the forecast area for Saratoga lat/long as somewhere between Oakland and San Leandro, about 45miles north of me.  Grrrr...   Since the advforecast2.php V5.17 uses the same API, it, too, is showing a forecast that's not for my actual area.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline CNYWeather

  • Forecaster
  • *****
  • Posts: 2295
    • CNYWeather
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #8 on: January 07, 2022, 12:17:28 PM »
Thanks Ken.

Wonder also if there is some Chrome Browser cache deal too. I went on my phone to the same page last night
and it had the latest forecast time on the map. I checked on my desktop and the page shows updated December 26th
now instead of the 24th.

I'm moving soon to Ionos for my host. Hopefully the issue will go away.

Tony




Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #9 on: January 07, 2022, 12:25:08 PM »
Alas, the issue is not going away anytime soon .. it's a fix the NWS has to make with their data dissemination structure from their core servers to the Akamai servers that we access via api.weather.gov.  It's not something we can fix.  I've been reporting on and following the problem for over a year now.  Sigh.

On my other issue (MTR point->gridpoint lookups returning forecasts +0.5 degree North latitude) seems to be mostly fixed.. now it's only off by one gridpoint, so the forecast is now back in Saratoga area.
« Last Edit: January 07, 2022, 12:27:29 PM by saratogaWX »
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline SteveFitz1

  • Forecaster
  • *****
  • Posts: 519
    • Tyler Texas Weather
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #10 on: January 17, 2022, 02:42:50 PM »
Ken,

I'm having a similar problem getting old data (12/30/2021) from the forecast script https://theindiescondos.com/NWS-forecast-map-v220.php. I understand the NWS issues as you've explained them. I'm finding one thing that doesn't make sense to me. When I run the script in debug mode, it shows me the link for fcstURL as https://api.weather.gov/gridpoints/MOB/54,46/forecast, which is correct. And it shows an RC=200. I can enter that URL into my browser and get today's current data. However, the script keeps returning the 12/30/21 data. I assumed since the script is going to the correct URL and giving a return code of 200, it would be getting the correct data. However, when I print the array $rawgridpointJSON, it is still 12/30/21 data.

Since I'm getting RC=200 back from the correct URL, should I not expect to see current data as I see when just entering that same URL in my browser?

Thanks very much.

Steve

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #11 on: January 17, 2022, 03:05:05 PM »
The disparity between the responses shown by the fetches from your webserver v.s. you doing a direct view of the URL from your browser is easy to explain.  Your webserver's DNS lookup returns an IP address for a Akamai cache server that is likely different from your browser's DNS lookup returning a different IP address (and a different Akamai server) to respond.

The underlying problem that the NWS has yet to solve is that not ALL the Akamai cache servers are getting updated regularly.  If your DNS lookup happens to get the address of an Akamai server that's not updated, then you get the stale data.  It won't get 'unstale' until either the DNS round-robin for Akamai server addresses rotates to a server with current data, or the stale server gets a successful refresh of data.

Your website DNS lookups may return different results than your home browser DNS lookups, and that's expected.

Hope this helps...
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline SteveFitz1

  • Forecaster
  • *****
  • Posts: 519
    • Tyler Texas Weather
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #12 on: January 17, 2022, 03:47:21 PM »
Ken,

Thanks very much for that explanation. It makes sense. Bet you wish the NWS would get their act together on this!

Steve

Offline CNYWeather

  • Forecaster
  • *****
  • Posts: 2295
    • CNYWeather
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #13 on: January 17, 2022, 04:35:05 PM »
My page on the desktop is stuck on Dec. 24 still.

Once odd thing, if I hit the NWS-forecast-map.php page, it is updated to the latest forecast this afternoon.
I've done that a few times and it's always up to date. The wxforecast.php page that contains that, is on December 24th.

Weird  :?
Tony




Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Forecast Map - Not Updated Since December 24th
« Reply #14 on: January 17, 2022, 05:05:13 PM »
It's the difference for DNS resolution of api.weather.gov that your website server gets an address that is a stale cache server while your desktop gets an ip address of a non-stale server.

Been reporting this issue to the NWS for over a year now .. still not resolved fully, but likely to be resolved sometime in the first half of this year.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP