Author Topic: NOAA IOD No Longer Active?  (Read 1190 times)

0 Members and 1 Guest are viewing this topic.

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2002
    • SouthEast Lincoln, NE Weather
NOAA IOD No Longer Active?
« on: March 11, 2018, 09:20:28 AM »
There has been no new IOD posted by the NOAA ENVVL since February 5, 2018.

Has this been abandoned?  Maybe the person or persons responsible for this feature were promoted/transferred and those who remain aren't aware of it?  Or something???
Regards, Jerry Wilkins
gwwilk@inebraska.com

Offline Bunty

  • Forecaster
  • *****
  • Posts: 1957
  • Home of Oklahoma State University
    • Welcome to Stillwater Weather
Re: NOAA IOD No Longer Active?
« Reply #1 on: March 11, 2018, 04:11:51 PM »
I gather it's not a very popular feature since it's taken over a month before someone complained about it.

Also the 2nd home page using modified AltDashboard 6.81 at http://stillwaterweather.com/wx/2ndhome.php

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2002
    • SouthEast Lincoln, NE Weather
Re: NOAA IOD No Longer Active?
« Reply #2 on: March 11, 2018, 05:18:42 PM »
I gather it's not a very popular feature since it's taken over a month before someone complained about it.
Or Iím just very patient. :shock:

I look at it every day.
Regards, Jerry Wilkins
gwwilk@inebraska.com

Offline dwhitemv

  • KORJUNCT3
  • Member
  • *
  • Posts: 24
    • My PWS on Weather Underground
Re: NOAA IOD No Longer Active?
« Reply #3 on: March 12, 2018, 02:02:39 PM »

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2002
    • SouthEast Lincoln, NE Weather
Re: NOAA IOD No Longer Active?
« Reply #4 on: March 12, 2018, 03:23:31 PM »
Image of the Day is over here:

https://www.nesdis.noaa.gov/satellites-image-of-the-day
Thanks!

However, looking at that site, it's more like image-of-some-days-maybe-today-maybe-not.  I couldn't find a way to easily scrape the most recent image.  The image directory isn't accessible for reading, so it's going to require a retrieval of the latest image page's html output from the main page, and then scraping the latest image URL from there.  Not a 3 minute project.

Maybe Ken can figure it out.  I think it's his script.
Regards, Jerry Wilkins
gwwilk@inebraska.com

Offline Bunty

  • Forecaster
  • *****
  • Posts: 1957
  • Home of Oklahoma State University
    • Welcome to Stillwater Weather
Re: NOAA IOD No Longer Active?
« Reply #5 on: March 13, 2018, 03:14:41 PM »
I'll remove the Image of the Day from my front page menu for now.

Also the 2nd home page using modified AltDashboard 6.81 at http://stillwaterweather.com/wx/2ndhome.php

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2002
    • SouthEast Lincoln, NE Weather
Re: NOAA IOD No Longer Active?
« Reply #6 on: March 13, 2018, 03:51:50 PM »
I'll remove the Image of the Day from my front page menu for now.
As I have done.
Regards, Jerry Wilkins
gwwilk@inebraska.com

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 6738
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NOAA IOD No Longer Active?
« Reply #7 on: March 13, 2018, 04:48:51 PM »
The new page doesn't seem to have a real 'image of the day', so I'll likely deprecate the script as the source has dried up.  :(

The main page now says
Quote
The NOAA Environmental Visualization Laboratory site will no longer be updated. Please visit https://www.nesdis.noaa.gov for the latest information on the Nation's environmental satellites and data. The staff wishes to thank you for your support since 1999.

and the new main page says
Quote
NESDIS is proposing to phase out dissemination of our data to non-governmental users via the PRODUCT DISTRIBUTION AND ACCESS system (PDA). We welcome comments through March 23, 2018.

So it looks like so-long to this feature.
« Last Edit: March 13, 2018, 04:52:08 PM by saratogaWX »
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline lddaly

  • Forecaster
  • *****
  • Posts: 444
Re: NOAA IOD No Longer Active?
« Reply #8 on: March 13, 2018, 04:57:30 PM »
Removed here too. We just checked our stats and the page had received 434 unique pageviews during 2017.

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 6738
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NOAA IOD No Longer Active?
« Reply #9 on: March 13, 2018, 05:40:50 PM »
I've now removed wxnoaaiod.php/get-nnvl-iod.php from the Base-USA template distribution and marked the script as discontinued in the scripts page.  Sorry folks.. no source data, ergo no script needed...
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2002
    • SouthEast Lincoln, NE Weather
Re: NOAA IOD No Longer Active?
« Reply #10 on: March 13, 2018, 06:46:49 PM »
The new page doesn't seem to have a real 'image of the day', so I'll likely deprecate the script as the source has dried up.  :(

The main page now says
Quote
The NOAA Environmental Visualization Laboratory site will no longer be updated. Please visit https://www.nesdis.noaa.gov for the latest information on the Nation's environmental satellites and data. The staff wishes to thank you for your support since 1999.

and the new main page says
Quote
NESDIS is proposing to phase out dissemination of our data to non-governmental users via the PRODUCT DISTRIBUTION AND ACCESS system (PDA). We welcome comments through March 23, 2018.

So it looks like so-long to this feature.
I understand, Ken, but I was hoping you would be able to figure out how to get from here to here to here.

In reading the relevant verbiage I'm not sure if they mean images by 'Data'.  Who is to know?  Oh well, I liked the IOD very much. :-(
Regards, Jerry Wilkins
gwwilk@inebraska.com

Offline Forever

  • Contributor
  • ***
  • Posts: 126
Re: NOAA IOD No Longer Active?
« Reply #11 on: March 13, 2018, 11:01:44 PM »
What a shame, I had hoped that the script could be linked to the new site.  :-(

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2002
    • SouthEast Lincoln, NE Weather
Re: NOAA IOD No Longer Active?
« Reply #12 on: April 22, 2018, 06:47:14 PM »
I've given little thought to scraping the new NOAA IOD primarily because they've restricted access to the image directory, i.e. no directory listing is available to outsiders, but individual images are available when directly referenced.  So for now rather than not provide any way to access the IOD I've placed a link to the Satellite Image of the Day Gallery at the bottom of my Saratoga Menu's Almanac section.

This will have to do for now.  These are great images that shouldn't go without being seen!
« Last Edit: April 22, 2018, 06:49:15 PM by gwwilk »
Regards, Jerry Wilkins
gwwilk@inebraska.com

Offline jgillett

  • Forecaster
  • *****
  • Posts: 1153
  • unnamed weather station, Boltek, Win7 Pro, ToA
    • TiggrWeather Phoenix
Re: NOAA IOD No Longer Active?
« Reply #13 on: April 23, 2018, 11:20:57 AM »
Great images. Thanks for that link, Jerry.
John
W7JKG
BUY FROM AMAZON  HERE... Thank You! :-)

Offline smokie

  • Member
  • *
  • Posts: 30
    • Newquay Cornwall. UK
Re: NOAA IOD No Longer Active?
« Reply #14 on: April 25, 2018, 10:40:54 AM »
Been working on this, only a novice at php so I  expect better ways to do it

Amend the settings to your website, displays the newest image, working on the text

Hopefully somebody can can improve this.
Might get error first time run as no files fetched/saved

Working here http://www.newquayweather.com/wxnoaaIOD.php  click on image for full size


Code: [Select]
<?php
// PHP script by newquayweather.com
// 
// Version iotd.php  V1.0 - 24-April-2018
// 
  
$Version 'iotd.php V1.0 - 24-April-2018';
//
// cacheName is name of file used to store cached webpage
  
$cacheName "/home/xxxx/public_html/xxxx/iotd/data/iotdcache.txt";
// filename is url of webpage to get
 
$fileName "https://www.nesdis.noaa.gov/satellites-image-of-the-day";
// file to save processed data to
 
$record "/home/xxx/public_html/xxxx/iotd/data/iotd.html";  // used to store the file 
 
 
$refetchSeconds 14;   // refetch every nnnn seconds 3600 = 1 hour, 7200 = 2 hrs, 14400 = 4 hrs etc

// end of settings
//
// ------ start of code -------

if (file_exists($cacheName) and filemtime($cacheName) + $refetchSeconds time()) {
      
$html implode(''file($cacheName));
    } else {
        
$html fetchUrlWithoutHanging($fileName);
      if (
strlen($html) > ) {
      
$fp fopen($cacheName"w");
      if (
$fp) {
        
$write fputs($fp$html);
        
fclose($fp);
      } else {
            print 
"error";
      }
}
}

// get image name
 
preg_match('|https://www.nesdis.noaa.gov/sites/default/files/styles/large/public/(.*)jpg|Uis',$html,$pic);

   
preg_match('|<div class="field-satellite-image-thumbnail">(.*)</div>|Uis',$html,$matches);

  
$pattern '|<div class="field-satellite-image-thumbnail">|'// new
  
$replacement='';  // new
  
$matches=preg_replace($pattern$replacement$matches);
  
  
$pattern '/typeof="foaf:Image" class="img-responsive"/';
  
$replacement='';  // new
  
$matches=preg_replace($pattern$replacement$matches);
  
  
$pattern '|</div>|'// new
  
$replacement='';  // new
  
$matches=preg_replace($pattern$replacement$matches);
// update url  
  
$pattern '/\/content\/(.*)">/';
  
$replacement="https://www.nnvl.noaa.gov/NESDIS/site/images/" $pic[1]. "png\"  target=\"_blank\" />";  // new url for large image
  
$matches=preg_replace($pattern$replacement$matches);
  
$data=$matches[0];

file_put_contents($record$data);

print 
$data;

// ----------------------------functions ----------------------------------- 
 
 
function fetchUrlWithoutHanging($url
   {
$numberOfSeconds=5;  
$ch curl_init();    
curl_setopt($chCURLOPT_URL$url); // set url
curl_setopt($chCURLOPT_SSL_VERIFYPEERfalse);  
curl_setopt($chCURLOPT_USERAGENT"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); // set browser/user agent    
curl_setopt($chCURLOPT_RETURNTRANSFERtrue);
curl_setopt($chCURLOPT_CONNECTTIMEOUT$numberOfSeconds);
curl_setopt($chCURLOPT_TIMEOUT$numberOfSeconds);
$data=curl_exec($ch);
   return(
$data);

   }    

?>

« Last Edit: April 25, 2018, 10:44:28 AM by smokie »
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2002
    • SouthEast Lincoln, NE Weather
Re: NOAA IOD No Longer Active?
« Reply #15 on: April 25, 2018, 11:16:50 AM »
Looks great, smokie!  The big stumbling block for me is the REGEX statements, and you've nailed them. =D&gt; \:D/ =D&gt;

Looking forward to seeing how the text shapes up. :grin:
Regards, Jerry Wilkins
gwwilk@inebraska.com

Offline smokie

  • Member
  • *
  • Posts: 30
    • Newquay Cornwall. UK
Re: NOAA IOD No Longer Active?
« Reply #16 on: April 25, 2018, 12:21:14 PM »
Thanks gwwilk

With some small changes, ie change preg_match to preg_match_all, and some looping code you could show any number of past images as well, this was what I started with but like you the REGEX  got me so I went with get newest image only first.

I will have a go at multiple images next
« Last Edit: April 25, 2018, 12:36:35 PM by smokie »
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline yamiacaveman

  • Senior Contributor
  • ****
  • Posts: 284
    • Penn Lake Weather
Re: NOAA IOD No Longer Active?
« Reply #17 on: April 27, 2018, 08:06:34 AM »
I know this is not the same but instead of the NOAA, I would like to use NASA IOD example: https://www.nasa.gov/multimedia/imagegallery/iotd.html

Would it be possible to change something like that? Those of you who know me, know that I know nothing about this stuff, but rather than give up on the NOAA pic of the day I would like to have something.

So if it is possible, please let me know -- I can't really screw up something that doesn't work any more, but even I wouldn't bet on that.

Thanks for any input.

yamiacaveman