Topic: USA Extremes Showing 'N/A'

[swohiowx]

Fitzweather,

Here's mine, with some additional tags for World extremes [if you want them]......should be usable as-is by just replacing your original version.

 [ You are not allowed to view attachments ]

[saratogaWX]

And now, WU has discontinued that page...

Try this using a NWS site instead

Code: [Select]

<?php
############################################################################
#
#   Project:    USA Extremes
#   Module:     usaextremes.php
#   Purpose:    Provides USA Extremes for Web page display
#   Authors:    Michael (michael@relayweather.com)
#               Ken True (webmaster@saratoga-weather.org) V2.01, V2.02
############################################################################
#      Usage:  Place the following on your webpage
#      CHMOD 666 for cacheFile2.php
#      include_once('usaextremes.php');
#      
#      Then call the following tags within the page where you would like them displayed:
#      $usahigh
#      $usalow
#      $usaprecip (note: $usaprecip will be 'N/A' in V2.02
#
#  Note: allow_url_fopen = on 
#    in your php.ini is required to run this script.
############################################################################
// version
// Version 2.00 - 14-Sep-2010 - initial release
// Version 2.01 - 21-Sep-2012 - adapted for WeatherUnderground use - K. True
// Version 2.02 - 23-Aug-2016 - adapted for www.wpc.ncep.noaa.gov - K. True
$usaextremesverion = "2.02";
echo "<!-- USA Extremes Script Version $usaextremesverion. -->\n";
/////////////////////////////////////////////////////////////////////////////
//SETTINGS START HERE////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////
// Site to Parse
$url2 = "http://www.wpc.ncep.noaa.gov/discussions/hpcdiscussions.php?disc=nathilo";
// Set the time zone
$ourTZ = 'America/New_York';
// Name of cache file  --  This file must be set to CHMOD-666
$cacheFile2 = "usaextremesCache3.txt";  
// Age of cache file before re-fetch caching time, in seconds (3600 = 1 hour)
$cache_life = '3600'; 
/////////////////////////////////////////////////////////////////////////////
//END SETTINGS///////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////// 
if (file_exists($cacheFile2)) {
    $filemtime = filemtime($cacheFile2);
    $filesize = filesize($cacheFile2);
    if (0 == $filesize){
        $filemtime = 0;
    }
} else {
    $filemtime = 0;
}   
//   open the cache file and write the new data and then close the file.
$forceRefresh = (isset($_REQUEST['force']))?true:false;
$current_time = time();
$cache_age = $current_time - $filemtime;
if ($forceRefresh or $cache_age >= $cache_life){
   print "<!-- fetching from '$url2' -->\n";
   $html2 = curl_get_contents($url2); 
   $fp2 = fopen($cacheFile2, 'w');
   fwrite($fp2, $html2);
   fclose($fp2);
   echo "<!-- The cache life HAS expired and fresh data re-wrote the cache file -->\n";
} else {
   echo "<!-- The cache life HAS NOT expired and fresh data was not written to the cache file -->\n";
// Open the cache file, read it, then close it   
    $handle2 = fopen($cacheFile2, "r");
    $filesize = filesize($cacheFile2);
    $html2 = fread($handle2, $filesize);
    fclose($handle2);
}

function curl_get_contents($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0");
$data = curl_exec($ch);
curl_close($ch);
return $data;
}

/* looking for:

<hr width="100%"> 
  <div id="printarea">
<pre><font face="arial,helvetica">National High and Low Temperature (for the contiguous United States)
NWS Weather Prediction Center, College Park, MD
Issued 8 pm EDT Tuesday, August 23, 2016

High Temperature for Tuesday, August 23, 2016
<span style="font-size:.9em;">(as received by 8 pm EDT August 23)</span>
<b>107 at Imperial, CA</b>

Low Temperature for Tuesday, August 23, 2016
<span style="font-size:.9em;">(as received by 8 pm EDT August 23)</span>
<b>19 at Stanley, ID</b>
</font></pre>
  </div>
</div>

or

<hr width="100%"> 
  <div id="printarea">
<pre><font face="arial,helvetica">National High and Low Temperature (for the contiguous United States)
NWS Weather Prediction Center, College Park, MD
Issued 2 am EDT Tuesday, August 23, 2016

High Temperature for Monday, August 22, 2016
<span style="font-size:.9em;">(as received by 2 am EDT August 23)</span>
<b>113 at Death Valley, CA</b>

Low Temperature for Monday, August 22, 2016
<span style="font-size:.9em;">(as received by 2 am EDT August 23)</span>
<b>27 at Bondurant, WY</b>
<b>27 at Stanley, ID</b>
<b>27 at Yellowstone National Park South Entrance, WY</b>
</font></pre>
  </div>
</div>

*/

//Finds USA High
preg_match_all('|High Temperature for .*</span>(.*)Low Temperature|Uis', $html2, $matches);
//  print "<pre>\n".print_r($matches,true)."</pre>\n";

if(isset($matches[1][0])) { 
//  print "<!-- high matches \n".print_r($matches,true)."-->\n";

  $usahigh = trim($matches[1][0]);
  $usahigh = str_replace('</b>','</b><br/>',$usahigh);
  $usahigh = strip_tags($usahigh,'<br>');  
  print "<!-- usahigh='$usahigh' -->\n";
} else {
   $usahigh = 'N/A';
}

//Finds USA Low
preg_match_all('|Low Temperature for .*</span>(.*)</font>|Uis', $html2, $matches);
//  print "<!-- low matches \n".print_r($matches,true)."-->\n";

if(isset($matches[1][0])) { 
  
  $usalow = trim($matches[1][0]);
  $usalow = str_replace('</b>','</b><br/>',$usalow);
  $usalow = strip_tags($usalow,'<br>');
    
  //echo $usalow;
  print "<!-- usalow='$usalow'-->\n";
} else {
   $usalow = 'N/A';
}

/* -- note: precip not included in www.wpc.ncep.noaa.gov data - K. True

//Finds USA High Precip
preg_match_all('|<div id="precip"[^>]*>(.*)</div>|Uis', $html2, $matches);
//  print "<pre>\n".print_r($matches,true)."</pre>\n";

if(isset($matches[1][0])) { 
  $raintable = $matches[1][0]; 
  preg_match_all('|<td>(.*)</td>|Uis',$raintable,$matches);
  
//  print "<pre>\n".print_r($matches,true)."</pre>\n";

  $usaprecip = trim(preg_replace("|\n|is",'',strip_tags($matches[1][2]).'<br/>'.strip_tags($matches[1][0])));  
  print "<!-- usaprecip='$usaprecip' -->\n";
} else {
   $usaprecip = 'N/A';
}
*/
$usaprecip = 'N/A';
print "<!-- usaprecip='$usaprecip' -->\n";
print '<!-- $filemtime = '.$filemtime.' $cache_age = '.$cache_age.' seconds.' . " -->\n";
print "<!-- Cache refresh rate = $cache_life seconds.  Cache age = $cache_age seconds. -->\n";
?>
Note.. the NWS site only provides temperature high and low, and not precip info, so $usaprecip will always return 'N/A' .. you should change your presentation code to omit Precipitation in the display.

Best regards,
Ken

[tbrasel]

Thank you, Ken, & appreciate your time. I did not realize how quickly a person could get use to glancing at those parameters. Hopefully the NWS might add precipitation at some point down the road.

Thanks again,
tbrasel

[gwwilk]

Thanks, Ken for resuscitating this script!  At least now the USA high and low can easily be seen at a glance.

[SteveFitz1]

Ken,

I came across this file on the NOAA site containing all of yesterday's reported precip. I don't know if you might want to grab yesterday's highest precip value from it.
http://www.cpc.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt

Steve

[Jáchym]

LOL,
that is exactly site that I was looking at 2 days ago thinking it could be useful. The only "problem" is that it covers only the U.S.

[Jáchym]

And the format it is in is relatively easy to use

[Silversword]

Hi All,

Finally got mine to work on my menu/nav bar.  Good job Ken and others.

Wonder if it can be embellished by showing a degree symbol and F and/or C after the numbers.  It would be a nice touch.

Thanks for the work that has been done to resolve some of this issue.  Something is better than nothing.

Regards,

--Stan Y.
   Maui, Hawaii

[Jáchym]

Can you post the script here?

[tmabell]

I noticed that Relayweather.com shows Max, Min & Precip.  I am wondering how there are getting the precip field to populate.

[saratogaWX]

He's likely scraping accuweather.com for the data (against their T&C of website usage.. not recommended).

[saratogaWX]

Ok, I've changed the script to use the new source (pretty much rewrote the bulk of the script).

Now it returns $usahigh, $usalow, and $usaprecip(if >0.00).  A few new options are available in the settings in the script, and it does allow units or no-units display.

Code: [Select]

<?php
############################################################################
#
#   Project:    USA Extremes
#   Module:     usaextremes.php
#   Purpose:    Provides USA Extremes for Web page display
#   Authors:    Michael (michael@relayweather.com)
#               Ken True (webmaster@saratoga-weather.org) V2.01, V2.02
############################################################################
#      Usage:  Place the following on your webpage
#      CHMOD 666 for cacheFile2.php
#      include_once('usaextremes.php');
#      
#      Then call the following tags within the page where you would like them displayed:
#      $usahigh
#      $usalow
#      $usaprecip (note: $usaprecip will be 'N/A' in V2.02
#
#  Note: allow_url_fopen = on 
#    in your php.ini is required to run this script.
############################################################################
// version
// Version 2.00 - 14-Sep-2010 - initial release
// Version 2.01 - 21-Sep-2012 - adapted for WeatherUnderground use - K. True
// Version 2.02 - 23-Aug-2016 - adapted for www.wpc.ncep.noaa.gov - K. True
//
// Version 4.00 - 11-Sep-2016 - rewrite to use new URL:
//       http://www.cpc.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt

$usaextremesverion = "4.00";
echo "<!-- USA Extremes Script Version $usaextremesverion saratoga-weather.org -->\n";
/////////////////////////////////////////////////////////////////////////////
//SETTINGS START HERE////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////
// Site to Parse
$url2 = "http://www.cpc.noaa.gov/products/analysis_monitoring/cdus/prcp_temp_tables/dly_glob1.txt";
// Set the time zone
$ourTZ = 'America/New_York';
// Name of cache file  --  This file must be set to CHMOD-666
$cacheFileDir = './'; // directory to store cache file in.
$cacheFile2 = "usaextremesCache4.txt";  
// Age of cache file before re-fetch caching time, in seconds (3600 = 1 hour)
$cache_life = '3600';
$reportDateFormat = "l, F j, Y"; // Day, Month d, yyyy 
$tUOM = '&deg;F'; // or ='' for no temperature unit display
$rUOM = 'in';     // or ='' for no rain unit display
/////////////////////////////////////////////////////////////////////////////
//END SETTINGS///////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////// 
global $SITE;
if (isset($SITE['cacheFileDir']))   {$cacheFileDir = $SITE['cacheFileDir']; }

if (file_exists($cacheFile2)) {
    $filemtime = filemtime($cacheFile2);
    $filesize = filesize($cacheFile2);
    if (0 == $filesize){
        $filemtime = 0;
    }
} else {
    $filemtime = 0;
}   
//   open the cache file and write the new data and then close the file.
$forceRefresh = (isset($_REQUEST['force']))?true:false;
$current_time = time();
$cache_age = $current_time - $filemtime;
if ($forceRefresh or $cache_age >= $cache_life){
   print "<!-- fetching from '$url2' -->\n";
   $html2 = curl_get_contents($url2); 
   $fp2 = fopen($cacheFile2, 'w');
   fwrite($fp2, $html2);
   fclose($fp2);
   echo "<!-- The cache life HAS expired and fresh data re-wrote the cache file -->\n";
} else {
   echo "<!-- The cache life HAS NOT expired and fresh data was not written to the cache file -->\n";
// Open the cache file, read it, then close it   
    $handle2 = fopen($cacheFile2, "r");
    $filesize = filesize($cacheFile2);
    $html2 = fread($handle2, $filesize);
    fclose($handle2);
}

function curl_get_contents($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0");
$data = curl_exec($ch);
curl_close($ch);
return $data;
}

/* looking for:

*****NOTE***************NOTE*********************NOTE************************
FILE FORMAT:  FIRST RECORD IS DATE IN THE FORM YYYYMMDD. THE REMAINING RECORDS
 CONTAIN MAXT(F),MINT(F),REPORTED AND ESTIMATED PRECIP IN HUNDRETHS OF INCHES,
 9 WEATHER CHARACTERS,ID AND CITY NAMES WHERE AVAILABLE. 
*****NOTE***************NOTE*********************NOTE********************
                         20160910
---  0---|--- 10---|--- 20---|--- 30---|--- 40---|--- 50---|--- 60---|--- 70---|--- 80---|--- 90---|
123456789|123456789|123456789|123456789|123456789|123456789|123456789|123456789|123456789|123456789|
  37  32    1    1 9DSSS////70026 AK BARROW/W.POST W.ROGERS
  41  34   20   20 9R/RRSSR/70086 AK BARTER ISLAND/DEW STN 
  46  39    2    2 9MMM/RRRM70104 AK CAPE_LISBURNE(AWOS)   
  44  41    1    1 9/D/RMDDD70117 AK TIN_CITY_AFS_(AWOS)   
  51  46    0    0 9////////70133 AK KOTZEBUE/RALPH WIEN ME
  55  33    0    0 9----////70173 AK INDIAN_MTN_AFS_AWOS   
  90  75    0    0 8/T-//TTT78514 PR RAMEY AFB             
  90  78   15   15 8/////RR/78526 PR SAN JUAN              
  90  76    0    0 8///R/RRR78535 PU ROOSEVELT_ROADS_NAS   
  89  81  122  122 8-/-/-/--78543 IS CHARLOTTE AMALIE VIRGI
  84  74    1    1 8////////91066 HI MIDWAY ISLAND NAVAL AI
  88  73    0    0 8////////91162 HI KEKAHA, PACIFIC MISSIL
  87  78    0    0 8//R/////91165 HI LIHUE                 
  79  68    8    8 8///RR//R91170 HI WHEELER               
  83  75   12   12 8/////RRR91176 HI KANEOHE               
  88  73    0    0 8////////91178 HI BARBERS_PT_NAS/OAHU   
  86  73    4    4 8///R////91182 HI HONOLULU              
  87  73    0    0 8///////R91186 HI MOLOKUI               
  88  73    1    1 8///R////91190 HI KAHULUI               
  84  70    5    5 8/R/RRR//91285 HI HILO/LYMAN_FIELD      
  91  73    0    0 8//MS////9911R TX BRENHAM, BRENHAM MUNIC


*/

$usahighStation = '';
$usahighValue = -999;
$usalowStation = '';
$usalowValue = 999;
$usaprecipStation = '';
$usaprecipValue = 0;
$usaStations = array();
$reportDate = '';

$rawrecs = explode("\n",$html2); // process the file
$idx=0;
foreach ($rawrecs as $n => $rec) {
if($reportDate == '' and preg_match('/^\s+(\d+)$/',$rec,$matches)) {
$reportDate = gmdate("l, F j, Y",strtotime($matches[1].'T1200'));
print "<!-- reportDate='$reportDate' -->\n";
continue;
}
$recmarker = substr($rec,19,1);
if($recmarker == '9' or $recmarker == '8') {
list($tHigh,$tLow,$tPrecip,$tPrecipEstim,$tData,$tState,$tStation) = 
  sscanf($rec,' %d %d %d %d %s %s %[^\n]s');
$tStation = strtolower( trim($tStation) );
$tStation = str_replace('/','| ',$tStation);

$tStation = ucwords( str_replace('_',' ', $tStation) );
$tStation = str_replace('| ','/',$tStation);
if(isset($_REQUEST['debug'])) {
  print "<!-- rec='$rec' -->\n";
          print "<!-- tHigh='$tHigh' tLow=$tLow, tPrecip='$tPrecip', tPrecipEstim='$tPrecipEstim',";
  print " tData='$tData', tState='$tState', tStation='$tStation' -->\n";
}
$usaStations[$idx] = join("\t",array($tHigh,$tLow,$tPrecip,$tPrecipEstim,$tData,$tState,$tStation));
$idx++;
}

}

// now scan for the highs/lows 
if (count($usaStations) > 0) {
  $usahigh = '';
  $usalow = '';
  $usaprecip = '';
} else {
  $usahigh = 'N/A';
  $usalow = 'N/A';
  $usaprecip = 'N/A';
}

foreach ($usaStations as $n => $vals) {
list($tHigh,$tLow,$tPrecip,$tPrecipEstim,$tData,$tState,$tStation) = explode("\t",$vals);
if($tHigh < -90 or $tLow < -90 ) { continue; }

if($tHigh > $usahighValue) {
$usahighValue = $tHigh;
$usahighStation = $n; // remember for later
}
if($tLow < $usalowValue) {
$usalowValue = $tLow;
$usalowStation = $n; // remember for later
}
if($tPrecip > $usaprecipValue) {
$usaprecipValue = $tPrecip;
$usaprecipStation = $n;

}

}
if(isset($_REQUEST['debug'])) {

  print "<!-- usahighValue=$usahighValue station=$usahighStation ".$usaStations[$usahighStation]." -->\n";
  print "<!-- usalowValue=$usalowValue station=$usalowStation ".$usaStations[$usalowStation]." -->\n";
  print "<!-- usaprecipValue=$usaprecipValue station=$usaprecipStation ".$usaStations[$usaprecipStation]." -->\n";
}

// now pass through to see if any duplicate high, low, high precip exists and format the strings
foreach ($usaStations as $n => $vals) {
list($tHigh,$tLow,$tPrecip,$tPrecipEstim,$tData,$tState,$tStation) = explode("\t",$vals);
if($tHigh < -90 or $tLow < -90 ) { continue; }

if($tHigh == $usahighValue) {
$usahigh .= "<b>$tHigh$tUOM at $tStation, $tState</b>\n";
}
if($tLow == $usalowValue) {
$usalow .= "<b>$tLow$tUOM at $tStation, $tState</b>\n";
}
if($tPrecip == $usaprecipValue and $tPrecip > 0) {
$tPrecipFmt = sprintf("%01.2f",$tPrecip/100);
$usaprecip .= "<b>$tPrecipFmt$rUOM at $tStation, $tState</b>\n";
}
}
// Make results HTML pretty

$usahigh   = str_replace("\n","<br/>\n",$usahigh);
$usalow    = str_replace("\n","<br/>\n",$usalow);
$usaprecip = str_replace("\n","<br/>\n",$usaprecip);

print "<!-- usahigh  ='$usahigh' -->\n";
print "<!-- usalow   ='$usalow' -->\n";
print "<!-- usaprecip='$usaprecip' -->\n";
print '<!-- $filemtime = '.$filemtime.' $cache_age = '.$cache_age.' seconds.' . " -->\n";
print "<!-- Cache refresh rate = $cache_life seconds.  Cache age = $cache_age seconds. -->\n";
?>

[Otis]

Thanks Ken, as always nice job. 

[tmabell]

Thank you Ken, it works very well.

[gwwilk]

Nice, Ken! 

Thanks very much.

[saratogaWX]

Since I seem to have been the one to update this script, I've decided to adopt it (awwww...) and give it a home in my script page for NWS miscellaneous scripts.  The code has been updated to add a self-downloader and a little comment cleanup, so you don't need to replace it if you're using the above version posted in a codebox.  I've started numbering this version at 4.00 (so as not to confuse it with Michael's V3.x version which he his not distributing).

Best regards,
Ken

[W3DRM]

Looks great to me. Thanks for doing this Ken! 

[Silversword]

Hi All and Ken,

Got mine updated and seems to be working well.  Thank you Ken for making the embellishments and adding the precip data.

Regards,

--Stan Y.
   Maui, Hawaii
 

[saratogaWX]

Have a small update to the script V4.01 14-Sep-2016 to correct an issue with a malformed data record.  Please replace your usaextremes.php script with the update.

[SteveFitz1]

Ken,

I display the extremes values on my home page. Every time this page is loaded, the data is retrieved from the NWS site (<!-- The cache life HAS expired and fresh data re-wrote the cache file -->). However, if I just run usaextremes.php by itself, the data is received from cache (<!-- The cache life HAS NOT expired and fresh data was not written to the cache file -->). I have file permissions set to 666 for the cache file. In looking at the code, it appears to be an issue with "If $force$Refresh" that's forcing the reload.

You can see this at www.tylertexasweather.com.

Thanks,

Steve

[saratogaWX]

Is there something in your main page code that sets $_REQUEST['force'] to a value?  If so, that will force the refresh of the cache.
It's using usaextremesCache4.txt as the cache file and it seems to be writable (has current data).

You can change the script from

Code: [Select]

$forceRefresh = (isset($_REQUEST['force']))?true:false; to

Code: [Select]

$forceRefresh = (isset($_REQUEST['force']))?true:false;
$forceRefresh = false; to disable this feature.

[SteveFitz1]

Is there something in your main page code that sets $_REQUEST['force'] to a value?  If so, that will force the refresh of the cache.

Ken,

I'm using your UV Index Forecast script. It looks like this one may be the culprit that sets $_REQUEST['force'] to a value. I'll take your suggestion and just turn off the 'Force' code.

Thanks,

Steve

[Breezy]

Thank you Ken,

Your time and efforts are very much appreciated !!! It's back on the page ... 

Jeff

[SteveFitz1]

Ken,

There's a couple of issues showing up in today's USA Extremes values with the usaextremes.php script. 1) There is no Low value being generated. It only displays "°F at , ". 2) The location of the Maximum Precip value is "Station Name Not Found". While this is, in fact, the station name shown next to today's max value in the dly_glob1.txt file, would it be possible to throw any occurrence of this "station" out and go to the next highest 'valid' station instead?

Thanks,

Steve

[ALITTLEweird1]

Mine is showing that too, but it's probably an issue with the NWS. Lets wait till tomorrow to see if it corrects itself before changes to the script.