WXforum.net
Web Weather => Weather Website PHP/AJAX scripting => Topic started by: gwwilk on March 11, 2018, 09:20:28 AM
-
There has been no new IOD posted by the NOAA ENVVL (https://www.nnvl.noaa.gov/) since February 5, 2018.
Has this been abandoned? Maybe the person or persons responsible for this feature were promoted/transferred and those who remain aren't aware of it? Or something???
-
I gather it's not a very popular feature since it's taken over a month before someone complained about it.
-
I gather it's not a very popular feature since it's taken over a month before someone complained about it.
Or I’m just very patient. :shock:
I look at it every day.
-
Image of the Day is over here:
https://www.nesdis.noaa.gov/satellites-image-of-the-day
-
Image of the Day is over here:
https://www.nesdis.noaa.gov/satellites-image-of-the-day
Thanks!
However, looking at that site, it's more like image-of-some-days-maybe-today-maybe-not. I couldn't find a way to easily scrape the most recent image. The image directory isn't accessible for reading, so it's going to require a retrieval of the latest image page's html output from the main page, and then scraping the latest image URL from there. Not a 3 minute project.
Maybe Ken can figure it out. I think it's his script.
-
I'll remove the Image of the Day from my front page menu for now.
-
I'll remove the Image of the Day from my front page menu for now.
As I have done.
-
The new page doesn't seem to have a real 'image of the day', so I'll likely deprecate the script as the source has dried up. :(
The main page now says The NOAA Environmental Visualization Laboratory site will no longer be updated. Please visit https://www.nesdis.noaa.gov for the latest information on the Nation's environmental satellites and data. The staff wishes to thank you for your support since 1999.
and the new main page says NESDIS is proposing to phase out dissemination of our data to non-governmental users via the PRODUCT DISTRIBUTION AND ACCESS system (PDA). We welcome comments through March 23, 2018.
So it looks like so-long to this feature.
-
Removed here too. We just checked our stats and the page had received 434 unique pageviews during 2017.
-
I've now removed wxnoaaiod.php/get-nnvl-iod.php from the Base-USA template distribution and marked the script as discontinued in the scripts page. Sorry folks.. no source data, ergo no script needed...
-
The new page doesn't seem to have a real 'image of the day', so I'll likely deprecate the script as the source has dried up. :(
The main page now says The NOAA Environmental Visualization Laboratory site will no longer be updated. Please visit https://www.nesdis.noaa.gov for the latest information on the Nation's environmental satellites and data. The staff wishes to thank you for your support since 1999.
and the new main page says NESDIS is proposing to phase out dissemination of our data to non-governmental users via the PRODUCT DISTRIBUTION AND ACCESS system (PDA). We welcome comments through March 23, 2018.
So it looks like so-long to this feature.
I understand, Ken, but I was hoping you would be able to figure out how to get from here (https://www.nesdis.noaa.gov/satellites-image-of-the-day) to here (https://www.nesdis.noaa.gov/content/third-noreaster-10-days-impacts-new-england) to here (https://www.nnvl.noaa.gov/NESDIS/site/images/20180313-noreaster.png).
In reading the relevant verbiage (https://www.nesdis.noaa.gov/content/partnership-policy) I'm not sure if they mean images by 'Data'. Who is to know? Oh well, I liked the IOD very much. :-(
-
What a shame, I had hoped that the script could be linked to the new site. :-(
-
I've given little thought to scraping the new NOAA IOD primarily because they've restricted access to the image directory, i.e. no directory listing is available to outsiders, but individual images are available when directly referenced. So for now rather than not provide any way to access the IOD I've placed a link to the Satellite Image of the Day Gallery (https://www.nesdis.noaa.gov/satellites-image-of-the-day) at the bottom of my Saratoga Menu's Almanac section.
This will have to do for now. These are great images that shouldn't go without being seen!
-
Great images. Thanks for that link, Jerry.
-
Been working on this, only a novice at php so I expect better ways to do it
Amend the settings to your website, displays the newest image, working on the text
Hopefully somebody can can improve this.
Might get error first time run as no files fetched/saved
Working here http://www.newquayweather.com/wxnoaaIOD.php (http://www.newquayweather.com/wxnoaaIOD.php) click on image for full size
<?php
// PHP script by newquayweather.com
//
// Version iotd.php V1.0 - 24-April-2018
//
$Version = 'iotd.php V1.0 - 24-April-2018';
//
// cacheName is name of file used to store cached webpage
$cacheName = "/home/xxxx/public_html/xxxx/iotd/data/iotdcache.txt";
// filename is url of webpage to get
$fileName = "https://www.nesdis.noaa.gov/satellites-image-of-the-day";
// file to save processed data to
$record = "/home/xxx/public_html/xxxx/iotd/data/iotd.html"; // used to store the file
$refetchSeconds = 14; // refetch every nnnn seconds 3600 = 1 hour, 7200 = 2 hrs, 14400 = 4 hrs etc
// end of settings
//
// ------ start of code -------
if (file_exists($cacheName) and filemtime($cacheName) + $refetchSeconds > time()) {
$html = implode('', file($cacheName));
} else {
$html = fetchUrlWithoutHanging($fileName);
if (strlen($html) > 0 ) {
$fp = fopen($cacheName, "w");
if ($fp) {
$write = fputs($fp, $html);
fclose($fp);
} else {
print "error";
}
}
}
// get image name
preg_match('|https://www.nesdis.noaa.gov/sites/default/files/styles/large/public/(.*)jpg|Uis',$html,$pic);
preg_match('|<div class="field-satellite-image-thumbnail">(.*)</div>|Uis',$html,$matches);
$pattern = '|<div class="field-satellite-image-thumbnail">|'; // new
$replacement=''; // new
$matches=preg_replace($pattern, $replacement, $matches);
$pattern = '/typeof="foaf:Image" class="img-responsive"/';
$replacement=''; // new
$matches=preg_replace($pattern, $replacement, $matches);
$pattern = '|</div>|'; // new
$replacement=''; // new
$matches=preg_replace($pattern, $replacement, $matches);
// update url
$pattern = '/\/content\/(.*)">/';
$replacement="https://www.nnvl.noaa.gov/NESDIS/site/images/" . $pic[1]. "png\" target=\"_blank\" />"; // new url for large image
$matches=preg_replace($pattern, $replacement, $matches);
$data=$matches[0];
file_put_contents($record, $data);
print $data;
// ----------------------------functions -----------------------------------
function fetchUrlWithoutHanging($url)
{
$numberOfSeconds=5;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url); // set url
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); // set browser/user agent
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $numberOfSeconds);
curl_setopt($ch, CURLOPT_TIMEOUT, $numberOfSeconds);
$data=curl_exec($ch);
return($data);
}
?>
-
Looks great, smokie! The big stumbling block for me is the REGEX statements, and you've nailed them. =D> \:D/ =D>
Looking forward to seeing how the text shapes up. :grin:
-
Thanks gwwilk
With some small changes, ie change preg_match to preg_match_all, and some looping code you could show any number of past images as well, this was what I started with but like you the REGEX got me so I went with get newest image only first.
I will have a go at multiple images next
-
I know this is not the same but instead of the NOAA, I would like to use NASA IOD example: https://www.nasa.gov/multimedia/imagegallery/iotd.html
Would it be possible to change something like that? Those of you who know me, know that I know nothing about this stuff, but rather than give up on the NOAA pic of the day I would like to have something.
So if it is possible, please let me know -- I can't really screw up something that doesn't work any more, but even I wouldn't bet on that.
Thanks for any input.
yamiacaveman