Author Topic: NWS Detailed Point Forecast Problem  (Read 10661 times)

0 Members and 1 Guest are viewing this topic.

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
Re: NWS Detailed Point Forecast Problem
« Reply #75 on: January 28, 2018, 04:57:40 PM »
Improvement after contacting office we now have a zone forecast available. Is there a way to push a zone forecast should something like this happen again? The older versions would automatically kick in zone but this latter advforecast2.php doesn't do that unless I missed something in settings. That's why I went back with older version until it completely died the other day. Because of all the API issues.

 [ You are not allowed to view attachments ]
Randy

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
Re: NWS Detailed Point Forecast Problem
« Reply #76 on: January 28, 2018, 05:55:23 PM »
Fixed finally. Only took 10 hours... #-o
Randy

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Detailed Point Forecast Problem
« Reply #77 on: January 28, 2018, 06:05:47 PM »
You can always 'go back' to the V4.02 advforecast2.php script (download here) and wait out the final(overdue) fixes to the V3 forecast.weather.gov system.  I don't think they're doing a cutover anytime soon.. it's been over a year since they started planning a cutover.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
Re: NWS Detailed Point Forecast Problem
« Reply #78 on: January 28, 2018, 06:16:31 PM »
Ken I might try again seems like this area has more problems than others. I was running an older version and it failed just the other day so loaded the newer version. I'll keep in mind next time if this continues to happen.
Thanks
« Last Edit: January 28, 2018, 06:21:24 PM by ValentineWeather »
Randy

Offline DaculaWeather

  • WxElement panel
  • Forecaster
  • *****
  • Posts: 3206
    • North Georgia Weather
Re: NWS Detailed Point Forecast Problem
« Reply #79 on: January 28, 2018, 07:13:12 PM »
Email the link to the office and tell them. I have access to NWSChat and I was able to tell them that way, it got fixed pretty quickly.

Offline DaculaWeather

  • WxElement panel
  • Forecaster
  • *****
  • Posts: 3206
    • North Georgia Weather
Re: NWS Detailed Point Forecast Problem
« Reply #80 on: January 28, 2018, 07:14:19 PM »
Let me send them a message for you.

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
Re: NWS Detailed Point Forecast Problem
« Reply #81 on: January 28, 2018, 07:15:18 PM »
 I had contact with them directly thanks.

Its back up finally.
Randy

Offline Jasiu

  • Forecaster
  • *****
  • Posts: 947
    • LexMAWeather
Re: NWS Detailed Point Forecast Problem
« Reply #82 on: January 28, 2018, 08:53:19 PM »
I don't know if this is related, but since I came across it today I thought I'd mention it.

Testing some stuff today, I came across a situation where the following happened:

1) Attempt to fetch with the "points" URL returned at 301 - Moved Permanently.  As expected.

2) Attempt to fetch the redirected URL (the one with "gridpoints" in it) returned a 404.

The code didn't handle this well since it wasn't checking for >= 400 after the second fetchUrlWithoutHanging() call.

So...  I copied the check from a few lines above and now the code that was:

Code: [Select]
      $usingFile = "(retry forecast)";
      $html = fetchUrlWithoutHanging($newurl, $cacheName);
      $fSize = strlen($html);
      $Status.= "<!-- loaded $usingFile $newurl - $fSize bytes -->\n";

... looks like this:

Code: [Select]
      $usingFile = "(retry forecast)";
      $html = fetchUrlWithoutHanging($newurl,$cacheName);

      preg_match('/HTTP\/1.1 (\d+)/',$html,$m);
      if(isset($m[1]) and $m[1] >= '400') {
        $Force =2;
        $Status .= "<!-- Oops.. point forecast unavailable RC=".$m[1]. " - using Zone instead -->\n";
      }
      else
      {
        $fSize = strlen($html);
        $Status .= "<!-- loaded $usingFile $newurl - $fSize bytes -->\n";
      }
 

... and it seems to cover this case, failing over to the Zone forecast.
https://lexmaweather.info
On Mastodon: @LexMAWeather@toot.community

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Detailed Point Forecast Problem
« Reply #83 on: January 29, 2018, 11:11:27 AM »
Good idea!

What I find annoying about their point-forecast API is that it now ALWAYS returns a 301 redirect to the gridpoint forecast instead of just returning the data requested.  I've never before encountered an API that now requires TWO accesses to get the data .. all the other APIs I've dealt with either return the data or a failure code if data is not available.

This API response is a very poor design (IMHO) .. they should do that lookup internally and just return the data (or not) in one https request.  Grrrr.

BTW... I've written to the webmaster about this (and about the long data outages (404 errors)) for points(gridpoints) and Zone forecasts with the api.weather.gov.  I see on forecast-v3.weather.gov, the link to the documentation is now not on the page anymore (but is still available via the old link at https://forecast-v3.weather.gov/documentation?redirect=legacy ).  I've also asked about the ongoing development/implementation plans for the api.weather.gov and future cutovers.   
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline Jasiu

  • Forecaster
  • *****
  • Posts: 947
    • LexMAWeather
Re: NWS Detailed Point Forecast Problem
« Reply #84 on: January 29, 2018, 11:18:11 AM »
Thanks for all the work as usual, Ken!

I've wondered about just changing the URL in settings to the redirected URL or possibly caching the redirect to avoid the extra jump. But, yeah, we shouldn't have to do that...
https://lexmaweather.info
On Mastodon: @LexMAWeather@toot.community

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Detailed Point Forecast Problem
« Reply #85 on: January 29, 2018, 11:26:12 AM »
I've also been experimenting with changing the curl call to follow redirects automatically.  That works, but the headers for both the initial point call and the gridpoint call are stacked (with \r\n\r\n separators) in the curl return.. makes finding the last RC a bit more complicated, but doable.
Quote
<!-- advforecast2.php (JSON) - V5.05 - 28-Dec-2017 on PHP 7.1.9-->
<!-- RAW NWS URL: http://forecast.weather.gov/MapClick.php?CityName=Saratoga&state=CA&site=MTR&textField1=37.2639&textField2=-122.022&e=1&TextType=2 -->
<!-- zone=CAZ513 -->
<!-- corrected NWS API URL='https://api.weather.gov/points/37.2639,-122.022/forecast' -->
<!-- curl fetching 'https://api.weather.gov/points/37.2639,-122.022/forecast' -->
<!-- note: fetched 'https://api.weather.gov/gridpoints/MTR/97,101/forecast' after redirect was followed. -->
<!-- HTTP stats:  RC=200 dest=23.197.96.136 port=443 (from sce=192.168.1.104)
      Times: dns=0.000 conn=0.016 pxfer=0.047 get=4295.405 total=4295.452 secs -->
<!-- loaded point-forecast https://api.weather.gov/points/37.2639,-122.022/forecast - 10319 bytes -->
<!-- wrote cache file ./forecast-CAZ513-0-json.txt -->
But.. the timing returned by curl is way off.. Sigh.

They tout that their design wants to be canonical - one request for each class of information needed.  That necessitated the current advforecast2.php JSON script to use up to 3 separate requests to get data (point forecast, descriptive info re forecast, alert info for points) instead of one request (like with WU) that has all the info desired.  I think a 301 response to a different URL is 'non canonical' and may simplify their internal design, but really (unnecessarily) complicates the design for users of their API.
« Last Edit: January 29, 2018, 11:28:08 AM by saratogaWX »
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline DaculaWeather

  • WxElement panel
  • Forecaster
  • *****
  • Posts: 3206
    • North Georgia Weather
Re: NWS Detailed Point Forecast Problem
« Reply #86 on: January 29, 2018, 11:40:00 AM »
Who is the guy you've been talking to? Possibly Corey Pieper?
Good idea!

What I find annoying about their point-forecast API is that it now ALWAYS returns a 301 redirect to the gridpoint forecast instead of just returning the data requested.  I've never before encountered an API that now requires TWO accesses to get the data .. all the other APIs I've dealt with either return the data or a failure code if data is not available.

This API response is a very poor design (IMHO) .. they should do that lookup internally and just return the data (or not) in one https request.  Grrrr.

BTW... I've written to the webmaster about this (and about the long data outages (404 errors)) for points(gridpoints) and Zone forecasts with the api.weather.gov.  I see on forecast-v3.weather.gov, the link to the documentation is now not on the page anymore (but is still available via the old link at https://forecast-v3.weather.gov/documentation?redirect=legacy ).  I've also asked about the ongoing development/implementation plans for the api.weather.gov and future cutovers.

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NWS Detailed Point Forecast Problem
« Reply #87 on: January 29, 2018, 12:03:10 PM »
Over a year ago, I was talking with Jared Holmberg (NOAA Affiliate) and Robert Bunge (NOAA) about the API.  I don't know if they're active in the project any longer, so my message was to the main contact for the forecast-v3.weather.gov site. 
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

 

anything