Topic: Analysis tool for advforecast2.php V5.17 (JSON) API failures logs

[saratogaWX]

I'm still seeing the stale gridpoint forecasts (every day in the mornings) from api.weather.gov Akamai cache servers.

Since the V5.16 script included optional logging of failures to ./cache/advforecast-log-YYYY-MM-DD.[txt|csv] files, I've written a script set to display the combined logs by day with sortable columns and search capability.

Note this script ONLY works with the V5.16+ version.  It will not display anything for the V4.x version of the script.

My stats at https://saratoga-weather.org/NWS-analyze/

Script set at https://saratoga-weather.org/NWS-analyze/NWS-analyze.zip .. if you install, I suggest you unpack it in the ./NWS-analyze/ directory on your website for tidiness.

I've posted about it on the API Discussion and hopefully with get the API developers to follow up on the stale stuff.

[gwwilk]

This tool is marvelous and very detailed.  Ken shared it with me pre-production and Saturday, the 22^nd, I sent the links (4 for each in all) for my site(s) with explanations regarding my difficulties over the past several months in getting a current point forecast from api.weather.gov.

The point forecast success rate improved later that day, and it has gotten even better since then.  Now, fingers-crossed, I rarely have advforecast2.php's output fall over to my area's zone forecast.

For me, it worked extremely well.  I just checked, and there have only been two failures today.  This is incredible considering there were up to a hundred a day when I first installed the analysis tool!

Thanks, Ken!

[Jasiu]

One bug fix: the array_search call in gen-csv.php bombs out if there are no matching files in the cache because $dList isn't set. I just declared it right after the glob() call.  Did the same thing in index.php.

Will add to my production code and turn logging back on (and report in on the NWS board).

Thanks for this!!

[saratogaWX]

Thanks for that, Jasiu!

I've updated the .zip with those changes to get-csv.php and index.php.  Also added an automatic link for a detail HTTP log display (in new page), and set a <meta> to have robots not index nor follow the links.  Re-download the NWS-analyze.zip for the updates.

Oh, and forgot to say that you'll only have logs to analyze if advforecast2.php V5.16+ has

Code: [Select]

# V5.16 logging capability for curl accesses to api.weather.gov
$doLogging = true;       // =true to enable summary logging, =false for no summary log file
# log saved to $cacheFileDir/advforecast2-log-YYYY-MM-DD.csv  (tab-delimited CSV file)
#
$doLoggingDetail = true; // =true to eanable detail logging, =false for no detail file
# log saved to $cacheFileDir/advforecast2-log-YYYY-MM-DD.txt
(which is the default).  Both the .txt and .csv logs are needed for get-csv.php to produce the display results.

[Jasiu]

Another weirdness I'm seeing in my sandbox, Ken.

I have failures for yesterday (25th) but none for today (because I haven't run the fetch many times). So I get a display for today (26th) that is empty and and pulldown/select shows the 25th. But there is no way to make it go to the 25th. If this isn't clear let me know. I'll leave the sandbox in its current state in case you need an example.

https://jjwebsites.com/weather/next/NWS-analyze/

[saratogaWX]

Jasiu, I love it that you easily find the edge cases

Does the cache file have both .txt and .csv for advforecast2-log-2021-05-25 and none for 2021-05-26?

The select options should only list dates if there are files available.. puzzled...

[Jasiu]

Jasiu, I love it that you easily find the edge cases

Does the cache file have both .txt and .csv for advforecast2-log-2021-05-25 and none for 2021-05-26?

The select options should only list dates if there are files available.. puzzled...

Yes, files just for 5-25.  The select bit is correct. It's just that the page display default is for the current date (i.e., now it's for the 27th; the headline says "NWS advforecast2 V5 Failure Log for 2021-05-27").

I'll look at how the page date is selected but I'm also dealing with a puppy right now 

[Jasiu]

OK, it's just a matter of making sure $theDate gets set to the last available date if the current date has no data. Something like this:

Code: [Select]

// $theDate = $nowDate;

if(isset($_REQUEST['date']) and
  preg_match('!^\d{4}-\d{2}-\d{2}$!',$_REQUEST['date'])) {
  $tryDate = $_REQUEST['date'];
  $i = array_search($tryDate,$dList);
  if($i !== false) {
    $theDate = $dList[$i];
  } // else {
//    $theDate = $nowDate;
//  }
}

if (!isset($theDate) )
{
  if (count($dList) > 0)
    $theDate = end($dList);
  else
    $theDate = $nowDate;
}

Propagated to gen-csv.php and details.php.

[saratogaWX]

Thanks for fixing the issue!  I've added the code to my site and the .zip file.    

[Jasiu]

Doh. My bug this time...

Replace

Code: [Select]

$theDate = end($dList);
with

Code: [Select]

$theDate = $dList[0];
Didn't pay attention to the use of rsort() for $flist.

[saratogaWX]

Thanks again, I noticed that this morning and did the same fix before seeing your message.  I've updated the .zip distribution too

[92merc]

I have the page running.  https://www.bismarckweather.net/NWS-analyze/

But my forecast has been pulling blank pages.  https://www.bismarckweather.net/advforecast2V5.php

[saratogaWX]

Oops... a leftover syntax error on my part.
Change

Code: [Select]

if (jason_last_error() !== JSON_ERROR_NONE) { to

Code: [Select]

if (json_last_error() !== JSON_ERROR_NONE) {and it won't fatal error due to the missing jason_last_error function.  (my typo in the source)

[Jasiu]

Hey Ken,

Have you let the folks at nco.ops@noaa.gov know about the presence of this feature? I'm planning to send an email soon to let them know about my sites.

[gwwilk]

I'm almost afraid to say this, but for a couple of weeks now my 'advforecast2.php' has been returning point forecasts, not the zone forecasts!

The only exceptions are very brief, probably when the source is updating.

Ken, thanks very much for spearheading what appears to be a genuine fix for a very vexing problem!

(And as I posted this, my zone forecast temporarily appeared, but was flushed with '?force=1'.)

[saratogaWX]

Glad your issue has abated.. mine has resurfaced particularly with Akamai server a104-70-164-87.deploy.akamaitechnologies.com returning stale info.

I've sent a new message to nco.ops@noaa.gov about it and pointed them to the analysis tool on my site.

Meanwhile, in the long thread about this issue, I posted

The issue still exists, and (for me) appears to be the same Akamai server at 184.87.164.14 that is returning stale (>18hrs old) gridpoint forecasts. I've implemented a browser for the error logs at https://saratoga-weather.org/NWS-analyze/ which shows all the failures (by date) with all the additional headers in the requests needed to diagnose. Since this 'stale' issue has gone on for a year now, I'd hoped to see some progress at fixing the NWS->Akamai cache issues, but so far, no joy.

to which one of the API folks replied

The good news is we kicked off the sprint to refactor the service that @StephenClouse mentioned which directly impacts anything related to the netcdf grids. The bad news is that it immediately got derailed supporting the team completing the QA process for the other project he mentioned. On the plus side, the other effort puts the entire program on better footing. We're not responsible for deploy, but I estimate the other project should be released in a few months, and then the refactor for the grid service a few months after that.

[BigOkie]

Glad your issue has abated.. mine has resurfaced particularly with Akamai server a104-70-164-87.deploy.akamaitechnologies.com returning stale info.

I've sent a new message to nco.ops@noaa.gov about it and pointed them to the analysis tool on my site.

Meanwhile, in the long thread about this issue, I posted

The issue still exists, and (for me) appears to be the same Akamai server at 184.87.164.14 that is returning stale (>18hrs old) gridpoint forecasts. I've implemented a browser for the error logs at https://saratoga-weather.org/NWS-analyze/ which shows all the failures (by date) with all the additional headers in the requests needed to diagnose. Since this 'stale' issue has gone on for a year now, I'd hoped to see some progress at fixing the NWS->Akamai cache issues, but so far, no joy.

to which one of the API folks replied

The good news is we kicked off the sprint to refactor the service that @StephenClouse mentioned which directly impacts anything related to the netcdf grids. The bad news is that it immediately got derailed supporting the team completing the QA process for the other project he mentioned. On the plus side, the other effort puts the entire program on better footing. We're not responsible for deploy, but I estimate the other project should be released in a few months, and then the refactor for the grid service a few months after that.

As someone who is a software QA analyst by profession I can relate the frustration that both end-user and developers have.  Developers want stuff to work right, QA testers want to try and break it.  What often gets in the way is product managers/owners who think they have projects that have higher priority (and in many cases they do) but I've worked with MANY PMs that are clueless.  It's a maddening cycle.  If NWS is use Agile methodology, sprints can get derailed easily if there are hotfixes needed.  If they don't deem this a hotfix, it will always get reverted to the backlog unfortunately.

[Jasiu]

Hey Ken and all,

I made a mod to this code so that I wouldn't have to do a "view-source" to get to the HTTP comments when looking at the details log. It is in details.php.

Code: [Select]

<?php 
  echo "<pre>\n";
// readfile($cacheDir."advforecast2-log-$theDate.txt");
$output = file_get_contents($cacheDir."advforecast2-log-$theDate.txt");
$output = str_replace("<", "&lt", $output);
$output = str_replace(">", "&gt", $output);
echo $output;
echo "</pre>\n";
?>

Pretty simple - just changing the < and > to their HTTP escape characters. Now I get everything I want to look at on the details page.

[saratogaWX]

Nice improvement!  Thanks, Jasiu!