WXforum.net

Web Weather => Weather Website PHP/AJAX scripting => Topic started by: PSJohn on February 06, 2019, 07:23:03 PM

Title: NWS CPC USA Extremes Script Update...
Post by: PSJohn on February 06, 2019, 07:23:03 PM
FYI: the NWS CPC USA Extremes script needs an update.

Out dated URL: $url2 = "http://www.cpc.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt";
NEW URL:        $url2 = "https://www.cpc.ncep.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt";

Just update file or wait until Ken updates it!  :grin: ;)
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on February 06, 2019, 07:38:28 PM
Ahh.. takes one more step
Code: [Select]
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER,0); has to be added to the curl_fetch since the cert used on www.cpc.noaa.gov does NOT include that site.  Sigh.

usaextremes.php download from https://saratoga-weather.org/usaextremes.php?sce=view
stateextremes.php download from https://saratoga-weather.org/stateextremes.php?sce=view

Thanks John for the heads-up.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on February 07, 2019, 04:33:36 PM
Download again this afternoon if you're still having issues.  I had to add
Code: [Select]
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,0);
also to bypass the bad cert on www.cpc.noaa.gov.  Some server installations check/some don't (my main server doesn't check, my local XAMPP did).  This allows it to work while they still have a cert that doesn't include the specific site.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: SoMDWx on February 08, 2019, 08:19:40 PM
Still getting N/A for the entries......

Jim
Title: Re: NWS CPC USA Extremes Script Update...
Post by: PSJohn on February 09, 2019, 02:19:59 AM
Still getting N/A for the entries......
I had to use this source url: "https://www.cpc.ncep.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt" in order for it to work for me.

Ken's file above uses "https://www.cpc.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt".

Not sure if that's a mistake. Ken?  :-)
Title: Re: NWS CPC USA Extremes Script Update...
Post by: SoMDWx on February 09, 2019, 08:24:09 AM
Great catch! That worked for me! Thanks!

Jim
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on February 09, 2019, 01:51:42 PM
 #-o   :oops: D'Oh! 

You're correct.. I'd not noticed that the host had changed from www.cpc.noaa.gov to www.cpc.ncep.noaa.gov.

Corrected now in 4.03e of the scripts.

Thanks!
Title: Re: NWS CPC USA Extremes Script Update...
Post by: jgillett on February 09, 2019, 02:50:48 PM
Thanks, Ken!
Title: Re: NWS CPC USA Extremes Script Update...
Post by: SoMDWx on February 09, 2019, 03:14:03 PM
Yes, Thanks Ken for all that you do and continue to do!   =D>

Jim
Title: Re: NWS CPC USA Extremes Script Update...
Post by: Otis on February 09, 2019, 07:09:17 PM
Agreed - Thanks Ken for everything you do to support this bunch  =D>
Title: Re: NWS CPC USA Extremes Script Update...
Post by: lstrip on February 10, 2019, 04:05:50 PM
Just updated the usaextremes and stateextremes scripts.
Shows cache updating but when i run either script, I get nothing.
https://vancamweather.com/usaextremes.php (https://vancamweather.com/usaextremes.php)
File permission is set to 666.
What am I missing?
Thanks
Larry
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on February 10, 2019, 09:07:44 PM
The script is currently returning
Quote
500 Error. Internal Server Error.

Setting the cache file (and the script) to 755 should be enough to have it operate correctly.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: lstrip on February 10, 2019, 09:52:08 PM
Ken
I have set permissions on both the script and cache file to 755.
Still nothing when I run
https://vancamweather.com/stateextremes.php (https://vancamweather.com/stateextremes.php)
or
https://vancamweather.com/usaextremes.php (https://vancamweather.com/usaextremes.php)
Thanks
Larry
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on February 10, 2019, 10:04:10 PM
Larry, the scripts do not produce output directly.. just load some PHP variables with the data so you can use PHP print statements to put that data where you'd like to see it.

Running view-source:https://vancamweather.com/stateextremes.php and view-source:https://vancamweather.com/usaextremes.php
shows they are running correctly and setting the PHP variables with data.

You need to do something like
Code: [Select]
<?php
include_once("usaextremes.php");

print 
"<p>USA Extremes for $reportDate</p>\n";
print 
"<p>High Temperature<br/><br/>$usahigh</p>\n";
print 
"<p>Low Temperature<br/><br/>$usalow</p>\n";
print 
"<p>High Precipatation<br/><br/>$usaprecip</p>\n";
print 
"<p><small>Data courtsey of <a href=\"";
print 
"https://www.cpc.ncep.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/";
print 
"\">NWS-CPC</a></small></p>\n";

?>
for the USA version.. similarly for the stateextremes version.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: lstrip on February 10, 2019, 10:30:38 PM
Ken
Thanks for replying.
I have run the usa and stateextreme scripts for quite awhile.
The data always was displayed on the menubar below the alerts
in a box with small font and color.
Did the updated script change this?
Larry
Title: Re: NWS CPC USA Extremes Script Update...
Post by: lstrip on February 10, 2019, 10:37:31 PM
Ken
This is an example of what I would get when I ran the scripts.
https://somdweather.com/somd_usaextremes.php (https://somdweather.com/somd_usaextremes.php)

Larry
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on February 10, 2019, 10:48:19 PM
The script I distributed has always only returned data in $variables .. never printed anything but HTML comments.
Looks like the somd_usaextremes.php script was using an earlier source.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: lstrip on February 11, 2019, 03:58:22 PM
Thanks Ken
Title: Re: NWS CPC USA Extremes Script Update...
Post by: AZmonsooncats on February 12, 2019, 07:08:46 PM
As always Ken thanks for the update. Got this script working again now all I have to do is figure out  ](*,) how migrate my weather software to the new computer.

Jeanette
Title: Re: NWS CPC USA Extremes Script Update...
Post by: gwwilk on February 26, 2020, 07:20:55 AM
The USA and State Extremes scripts haven't displayed the latest data since Sunday, February 23rd (https://www.gwwilkins.org), Ken.  Breezy's home page displays the same problem (https://indiantrailweather.com/), so I'm guessing the NWS has changed something again... :-(
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on February 26, 2020, 11:07:58 AM
My version 4.03e uses

$url2 = "https://www.cpc.ncep.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt";

for data, and both state/USA scripts are working fine.  That was updated in Feb, 2019.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: gwwilk on February 26, 2020, 01:22:00 PM
Thanks, Ken.  That's the version and $url2 I'm using.  The obviously NWS-related problem has somehow been resolved, because the USA and State Extremes are now both populating correctly.

Appreciate your prompt attention, Ken, to this evanescent glitch.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on February 26, 2020, 01:41:01 PM
Next time it happens, run view-source:http://your.site.com/usaextremes.php?force=1 to force a cache reload and see the results of the fetch process.
It should show what the underlying issue is with fetching the URL.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: SteveFitz1 on March 31, 2020, 06:44:44 PM
Has anyone else noticed the USA Extremes temperatures haven't been updated since last Wednesday, 3/25/2020?

https://www.cpc.ncep.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt (https://www.cpc.ncep.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt)

Looks like the location has changed and is now in csv format

https://www.cpc.ncep.noaa.gov/products/cadb/ (https://www.cpc.ncep.noaa.gov/products/cadb/)

Steve
Title: Re: NWS CPC USA Extremes Script Update...
Post by: gwwilk on March 31, 2020, 07:25:07 PM
Ive also seen this, but have been otherwise preoccupied...
Title: Re: NWS CPC USA Extremes Script Update...
Post by: gwwilk on April 01, 2020, 06:13:39 AM
This script is Ken True's, and I'm not capable of fixing it.  Therefore I'm going to abandon it for now.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: mth on April 01, 2020, 09:17:30 AM
Potential source?  https://www.wpc.ncep.noaa.gov/discussions/hpcdiscussions.php?disc=nathilo&version=0&fmt=reg
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on April 01, 2020, 11:58:16 AM
I'll take a look at alternative sourcing for the data for the usaextremes and stateextremes scripts since I seem to have adopted them...

Looks like the https://www.cpc.ncep.noaa.gov/products/cadb/ will be the new source, but I have to write some shim code to get the new file and use the .CSV format instead of fixed-width text for the scripts.  The raw .csv files are available via https, but.. they have year/month/day as part of the directory structure and filenames, so just a bit more tricky than the old fixed URL.

I'm on it :)
Title: Re: NWS CPC USA Extremes Script Update...
Post by: SoMDWx on April 02, 2020, 10:59:01 AM
I have re-written the existing script to grab those numbers for now. you can see it on my page... I'm sure Ken will have something alot nicer then what i have done but at least it is a temporary fix...

https://somdweather.com/somd_usaextremes3.php

Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on April 04, 2020, 01:52:39 PM
I've done a rewrite of usaextremes.php (including the function of stateextremes.php) as a new worldextremes.php (V5.00 - 04-Apr-2020).

The expanded script now uses the CSV files from https://www.cpc.ncep.noaa.gov/products/cadb/ which includes worldwide data.
A small caveat however, the data is NOT WELL PROOFED so you can get some max/min temperatures for a station showing >25C differences.
The script already filters out >40C differences (configurable).

Now the following variables are returned:
Quote
$omittedCountry (text list of countries excluded from $world high/low/precip scans)
Note: the setting $ignoreCountrys is the array of country names to to exclude
$worldhigh
$worldlow
$worldprecip

$selectedCountry (setting: country name for selected country high/low/precip)
Note: run http://your.site.com/worldextremes.php?list to see the list of country names to use
$countryhigh
$countrylow
$countryprecip

$usahigh (Note: for lower-48 USA states)
$usalow (Note: for lower-48 USA states)
$usaprecip (Note: for lower-48 USA states)

$selectedState (setting: USA state 2-character name abbreviation in settings area)
$selectState (a copy of $selectedState for compatibility with old stateextremes.php)
$statehigh
$statelow
$stateprecip

$reportDate (nicely formatted date of the report)
$stateReportDate (a copy of $reportDate for compatibility with old stateextremes.php)

To replace the usaextremes.php (and stateextremes.php) just use include_once("worldextremes.php") and comment out/remove the include for stateextremes.php.

See the docs at https://saratoga-weather.org/scripts-NWSmisc.php#NWSEXTR which has the download link for the script.

Many thanks to Jim (SoMDWx) for his assistance with this script!  =D&gt;  \:D/

Best regards,
Ken
Title: Re: NWS CPC USA Extremes Script Update...
Post by: SoMDWx on April 04, 2020, 02:05:57 PM
Thanks Ken! Awesome work as usual...:)
Title: Re: NWS CPC USA Extremes Script Update...
Post by: gwwilk on April 04, 2020, 06:33:39 PM
 [tup]
Much appreciated, Ken!
 =D&gt;
Title: Re: NWS CPC USA Extremes Script Update...
Post by: colonieweather on April 05, 2020, 09:58:22 AM
Thank you Ken.  The work you do is much appreciated!
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on April 05, 2020, 10:57:03 AM
You are all welcome!  I did a slight update to V5.01-05-Apr-2020 today.  Iddaly noted that the array_key_first() function I'd used is a PHP7+ only function.  The V5.01 script replaces that with PHP5+ calls to array_keys()/array_pop() so the script should work on all versions of PHP from 5.x up.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: gwwilk on April 05, 2020, 11:31:50 AM
Thanks, Ken!  My testing site bombed out while my web site and my localhost worked.  Now my testing site works with v5.1!  I obviously haven't updated my Debian server to a PHP 7.x version.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: lddaly on April 05, 2020, 09:16:55 PM
You are all welcome!  I did a slight update to V5.01-05-Apr-2020 today.  Iddaly noted that the array_key_first() function I'd used is a PHP7+ only function.  The V5.01 script replaces that with PHP5+ calls to array_keys()/array_pop() so the script should work on all versions of PHP from 5.x up.

Thank you, Ken! Works perfectly. [tup] https://www.planoweather.com/wxextremes.php
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on June 25, 2020, 08:43:59 PM
A slight update to V5.02 - 25-Jun-2020.  The CPC added a station_id column to the CSV file (drat).

Download from the script page (https://saratoga-weather.org/scripts-NWSmisc.php#NWSEXTR) and recustomize as needed.
Title: Re: NWS CPC USA Extremes Script Update...
Post by: PSJohn on June 26, 2020, 12:25:12 AM
Thanks for the update Ken.
Has anyone notice the frequent data error from this one site:

   High Temp:   124F at Le Mars Municipal Airport, IA


Anyway to exclude this?
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on June 26, 2020, 11:17:35 AM
Just updated to 5.03 -- it now has a $ignoreStations array to select a stn_id to ignore for bogus data.  I've added in the Le Mars one as the initial one to ignore.
Download 5.03 from the script page (https://saratoga-weather.org/scripts-NWSmisc.php#NWSEXTR) and customize as needed.

I do wish the CPC would screen/sanity check those reports.  50C/124F for a High in Iowa for several days running???
Title: Re: NWS CPC USA Extremes Script Update...
Post by: SteveFitz1 on June 26, 2020, 12:02:31 PM
I do wish the CPC would screen/sanity check those reports.  50C/124F for a High in Iowa for several days running???

And -17C/1F for a low at Camp David, MD (99KRSP) this morning is quite a stretch as well.

Steve
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on June 26, 2020, 12:07:11 PM
Yes, I just saw (and fixed that) .. they were reporting the same temp for min/max (now the script rejects it).
Also removed the Alaska entries from the Lower 48 Min/Max display.
Download again for the update (version hasn't changed).
Title: Re: NWS CPC USA Extremes Script Update...
Post by: saratogaWX on June 26, 2020, 12:10:49 PM
I thought y'all might be interested in the code I use on my wxextremes.php page in the <div id="main-copy">...</div> on a copy of wxnewpage.php put
Code: [Select]
<?php include_once('worldextremes.php');
print 
"<h1>Weather Extremes for $reportDate</h1>\n";
print 
"<p>&nbsp;</p>\n";
gen_display('World'." (omitting $omittedCountry)",$worldhigh,$worldlow,$worldprecip);

gen_display('Canada',$countryhigh,$countrylow,$countryprecip);

gen_display('USA (lower 48)',$usahigh,$usalow,$usaprecip);

list(
$glhigh,$gllow,$glprecip) = scan_for($worldStations,'United States','AK',$tUOM,$rUOM);
gen_display('Alaska'."",$glhigh,$gllow,$glprecip);

gen_display('California',$statehigh,$statelow,$stateprecip);

list(
$glhigh,$gllow,$glprecip) = scan_for($worldStations,'United States','HI',$tUOM,$rUOM);
gen_display('Hawaii'."",$glhigh,$gllow,$glprecip);

print 
'<p><small><strong>Data from <a href="https://www.cpc.ncep.noaa.gov/products/cadb/">NOAA Climate Prediction Center</a>.  ';
print 
'worldextremes.php script by <a href="/scripts-NWSmisc.php#NWSEXTR">Saratoga-weather.org</a></strong></small><br/>'."\n";
print 
"Raw-data CSV file <a href=\"".gen_url($utcUse)."\">Download</a></p>\n";

function 
gen_display($legend,$hightemp,$lowtemp,$precip) {
print "<h2>$legend</h2>\n";
print "<table style=\"border: none;\">\n";
print "<tr><td><span style=\"color: red\"><strong>High Temperature:</strong></span></td><td>$hightemp</td></tr>\n";
print "<tr><td><span style=\"color: blue\"><strong><br/>Low Temperature:</strong></span></td><td><br/>$lowtemp</td></tr>\n";
print "<tr><td><span style=\"color: green\"><strong><br/>Precipitation:</strong></span></td><td><br/>$precip</td></tr>\n";
print "</table>\n";
print "<p>&nbsp;</p>\n";
}
?>


Title: Re: NWS CPC USA Extremes Script Update...
Post by: PSJohn on July 30, 2020, 05:23:45 PM
Seems that they don't make thermometers that go beyond 118 in the Middle East!  :grin: I see that often as the World Extreme.

 [ You are not allowed to view attachments ]