Author Topic: PHP Space Weather Script  (Read 14122 times)

0 Members and 1 Guest are viewing this topic.

Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #25 on: April 10, 2017, 02:09:50 AM »
It appears you are still getting the error, just tried your wxspace page.

I should of mentioned the cache file needed to be deleted so as to force refresh data, there has to be something different with your set up as it shouldnt take more than a couple of seconds.

These are the only settings needed to be changed to match your set up, assuming jpgraphs installed

path to cache file
$cacheName = "/home/XXXXX/public_html/xxxxx/sunspots/".date("Y")."/sunspotscache30d.txt";

Create dir for year
if (!is_dir('/home/XXXXX/public_html/xxxx/sunspots/'.date("Y"))) {
    mkdir('/home/XXXXX/public_html/xxxx/sunspots/'.date("Y"));
}

The script goes in wxgraphs

Hope this helps,
phil
« Last Edit: April 10, 2017, 02:11:39 AM by smokie »
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #26 on: April 10, 2017, 08:07:48 AM »
I've narrowed the problem to an inability of 'sunspots30day.php' to reliably fetch and save the 'sunspotscache30d.txt' data, for reasons unknown to me.  Many times it ends up with an empty file which trashes the space script completely.  I've disabled the sunspot graph in my 'wxspacewx.php' until this problem can be sorted out.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #27 on: April 10, 2017, 09:50:24 AM »
Just a thought do you have slow broadband ?

You could try increasing the time out from 30 to a higher figure in this line, as this was an issue when I
first wrote the script for some users.

$ctx = stream_context_create(array('http' => array('timeout' => 30)));

With todays speeds now it could be reduced

phil
« Last Edit: April 10, 2017, 10:00:00 AM by smokie »
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #28 on: April 10, 2017, 10:36:51 AM »
Just a thought do you have slow broadband ?

You could try increasing the time out from 30 to a higher figure in this line, as this was an issue when I
first wrote the script for some users.

$ctx = stream_context_create(array('http' => array('timeout' => 30)));

With todays speeds now it could be reduced

phil
I have a 300+ Mb fiber optic connection, so there should be no issue with broadband.  I already tried increasing the timeout to 120 earlier today, but that didn't help either.

I'm on a GoDaddy shared Linux web server, and constantly upload my weather data and files in addition to running chron jobs using cURL quite often in order to sidestep their meager curl restrictions.  I suspect that an overloaded web server is the problem.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #29 on: April 10, 2017, 03:17:03 PM »
I to have fibre to the house, its excellent

Does the script produce any errors apart from the header one ?

I've attached a cached file see if it produces a graph
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #30 on: April 10, 2017, 06:08:30 PM »
I to have fibre to the house, its excellent

Does the script produce any errors apart from the header one ?

I've attached a cached file see if it produces a graph
The script occasionally produces a graph when the 'sunspots30day.php' script manages to scrounge the data.  When it doesn't I've modified (last night) the cache refresh logic to
Code: [Select]
if ((file_exists($cacheName) and filesize($cacheName)>30) and filemtime($cacheName) + $refetchSeconds > time()) { because the dead filesize is 30.  This still doesn't reliably produce a data file.

I just tried running 'sunspots30day.php' alone and after deleting the bad 'sunspotscache30d.txt' it ran once.  Then it refused to run again.  I'll get back at it probably tomorrow.  I played around with the directory structure a little, but didn't get anywhere yet.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #31 on: April 11, 2017, 01:18:51 AM »
I'm puzzled by this, what php version do you have I'm on 5.6, I wonder if any other users are having the same issue
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #32 on: April 11, 2017, 08:09:13 AM »
I'm puzzled by this, what php version do you have I'm on 5.6, I wonder if any other users are having the same issue
PHP 5.4
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #33 on: April 11, 2017, 06:50:26 PM »
After trying multiple options including
Code: [Select]
$ctx = stream_context_create(array('ftp' => array('overwrite' => true)));rather than
Code: [Select]
$ctx = stream_context_create(array('http' => array('timeout' => 30)));
I adopted a defensive strategy for this recalcitrant download.  I defined the following
Code: [Select]
'$testCache = (my path)/sunspots/sunspots30day-test.txt' and then I added, in context,
Code: [Select]
  $text=file_get_contents($ftp_server, 0, $ctx);
  if (strlen($text)>30) { // If we have data
file_put_contents($cacheName, ""); // only then zero file
  } else { // Otherwise!
file_put_contents($testCache, (date("M j, y: g ia",time()).' - strlen($text): '.strlen($text)."\n"), FILE_APPEND);
  exit; // Don't trash the last good data!
}
  while ($line < $lineend) {
    $split = preg_split("/[\n]/", $text, -1, 0 );
in order to avoid trashing a good data file.

I reactivated my cURL job to run 'sunspots30day.php' at 9 minutes after every hour in order to  give me some idea of how often the script is falling over by looking at my '$testCache'.  After 24 hours I might have a better idea of how often it fails.  I've also added the sunspots graph back to my 'wxspacewx.php' script.

This might not totally cure my problem, but it will hopefully at least minimize the symptoms.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #34 on: April 12, 2017, 02:24:33 AM »
I just tried your page, no main content was displayed

try entering the data url and image url in your browser

ftp://ftp.swpc.noaa.gov/pub/latest/DSD.txt

and

image
https://sohowww.nascom.nasa.gov/data/realtime/hmi_igr/512/latest.jpg


takes about a second here to display both image and data

Update
Just tried your page again and its working

in the script found error
$bgimageurl="https://sohowww.nascom.nasa.gov//data/realtime/hmi_igr/512/latest.jpg";

change to
$bgimageurl="https://sohowww.nascom.nasa.gov/data/realtime/hmi_igr/512/latest.jpg";

extra /  got in (gov//data) take one out
« Last Edit: April 12, 2017, 03:16:47 AM by smokie »
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #35 on: April 12, 2017, 08:16:06 AM »
Good eye, Phil!  Unfortunately correcting the munged image url didn't improve the retrieval of the data file.  I see, and have seen, that the data file is retrieved quickly when its url is entered alone on the address bar, but when the url is called by 'sunspots30day.php' it somehow frequently falls over.

The php 'exit()' command I placed in 'sunspots30day.php' wasn't performing as I intended by returning to the calling script but as it should according to the documentation by exiting all scripts.  Replacing it with 'return;' should do the trick (Dreamweaver correctly forbids potentially using any values with 'return()' in this instance but rather insists it only be 'return' when not used in a function).  I'm leaving my cURL of 'sunspots30day.php' in place for now but running it every half hour now, so at most the graph should be just an hour or two old.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #36 on: April 12, 2017, 10:04:30 AM »
Its a puzzle, i wonder if its anything to do with php version ?

The data file needs only to be updated once a day, I update mine around 04:30 GMT, this was done by trial and error

The DSD url will list when the file was updated, todays was

:Product: Daily Solar Data            DSD.txt
:Issued: 0825 UT 12 Apr 2017


The refresh part was for the background image
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #37 on: April 12, 2017, 11:06:58 AM »
Its a puzzle, i wonder if its anything to do with php version ?

The data file needs only to be updated once a day, I update mine around 04:30 GMT, this was done by trial and error

The DSD url will list when the file was updated, todays was

:Product: Daily Solar Data            DSD.txt
:Issued: 0825 UT 12 Apr 2017


The refresh part was for the background image
Good to know!  I'll work with that in mind.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #38 on: April 12, 2017, 07:47:33 PM »
I've backed off my cURL of 'sunspots30day.php' to run every hour and have 'sunspots30day.php' checking for an expired data file only every 24 hours while still checking for a background image no older than 1 hour.  As insurance I still have 'wxspacewx.php' 'include_once('sunspots30day.php')', but it should the script shouldn't hang and should always load promptly unless there has been a real problem with the data file.

Still a mystery why my GoDaddy server has such a problem. :?: :shock: :?:  It almost makes me want to host my website myself.  Almost.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #39 on: April 13, 2017, 05:08:34 AM »
Jerry, been having a think, I've tried using ftp_get, works for me on the rough script to download dsd.txt

<?php

// define some variables
$local_file = 'DSD.txt';
$server_file = '/pub/latest/DSD.txt';
$ftp_server = 'ftp.swpc.noaa.gov';

$ftp_user_name = "anonymous";
$ftp_user_pass = "";
// set up basic connection
$conn_id = ftp_connect($ftp_server);

// login with username and password
$login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);

// try to download $server_file and save to $local_file
if (ftp_get($conn_id, $local_file, $server_file, FTP_ASCII)) {
    echo "Successfully written to $local_file\n";
} else {
    echo "There was a problem\n";
}

// close the connection
ftp_close($conn_id);

?>
« Last Edit: April 13, 2017, 05:19:36 AM by smokie »
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #40 on: April 13, 2017, 08:00:41 PM »
My server still stalled on the 'ftp_get', so I'm sticking with what I've got for now.  If it falls over after 24 hours, the cURL running every hour will keep trying to update the  stale data until it's successful, then it will stop updating for another 24 hours.  Meanwhile my graph is still present on the server and the users see no delay displaying wxspacewx.php.  This graph is also used in my 'wxastro5.php' script, which had been running the '30G' script but it was cheating by running on your site without the delay.  The graph also displays when I use 'http://www.newquayweather.com/wxgraphs/sunspots30day.php', and that would probably have been the simplest, albeit somewhat underhanded, solution.  These astro-wx scripts aren't much used by my viewers, but I like them.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #41 on: April 14, 2017, 02:13:40 AM »
I'm at a loss now,  I'm on bluehost have been for years, do any of your other scripts use ftp_get or file_get_contents ?
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #42 on: April 14, 2017, 08:25:58 AM »
I'm at a loss now,  I'm on bluehost have been for years, do any of your other scripts use ftp_get or file_get_contents ?
Here's one, but I didn't write it.  I only use it, and it at times is recalcitrant but likewise eventually comes through, although sometimes only after repeated <CTRL><F5>'s in Chrome or FF when the cache is defeated with '?cache=r'.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #43 on: April 16, 2017, 09:34:09 AM »
I'm at a loss now, this has to be a GoDaddy issue, perhaps another GoDaddy user can shed some light on this, there seems to be a lot of pages if you search google

Try this, its using cURL, it seems to work even with an ftp url for me

<?php
$curl = curl_init();
curl_setopt ($curl, CURLOPT_URL, "ftp://ftp.swpc.noaa.gov/pub/latest/DSD.txt");
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
    $text = curl_exec ($curl);
    echo $text;
    curl_close ($curl);
?>
« Last Edit: April 16, 2017, 09:43:58 AM by smokie »
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: PHP Space Weather Script
« Reply #44 on: April 16, 2017, 11:51:01 PM »
Jerry,

Almost a week ago you answered a question regarding what verion of PHP you were running. Your answer was 5.4. Is there anyone else successfully running the code on that version of PHP? I never saw any response to your reply and we have seen some code not run correctly on certain versions of PHP. Is there anyway you can upgrade to a later version of PHP?
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #45 on: April 17, 2017, 08:19:10 AM »
Jerry,

Almost a week ago you answered a question regarding what verion of PHP you were running. Your answer was 5.4. Is there anyone else successfully running the code on that version of PHP? I never saw any response to your reply and we have seen some code not run correctly on certain versions of PHP. Is there anyway you can upgrade to a later version of PHP?
Nope.  I'm apparently stuck at 5.4.19 with as far as I can see no ability to upgrade.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline gwwilk

  • Southeast Lincoln Weather
  • Forecaster
  • *****
  • Posts: 2578
    • SouthEast Lincoln, NE Weather
Re: PHP Space Weather Script
« Reply #46 on: April 17, 2017, 08:28:13 AM »
I'm at a loss now, this has to be a GoDaddy issue, perhaps another GoDaddy user can shed some light on this, there seems to be a lot of pages if you search google

Try this, its using cURL, it seems to work even with an ftp url for me

<?php
$curl = curl_init();
curl_setopt ($curl, CURLOPT_URL, "ftp://ftp.swpc.noaa.gov/pub/latest/DSD.txt");
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
    $text = curl_exec ($curl);
    echo $text;
    curl_close ($curl);
?>
The data file was retrieved by this simple script, but only after a 20-30 second delay.  I'm wondering if there isn't some sort of GoDaddy-wide malicious script filter the results have to pass through before they're given the 'OK'.
Regards, Jerry Wilkins
gwwilk@gmail.com

Offline ConligWX

  • Forecaster
  • *****
  • Posts: 836
  • #conligwx
    • conligwx.org
Re: PHP Space Weather Script
« Reply #47 on: April 17, 2017, 01:43:12 PM »
Quote from: gwwilk
Nope.  I'm apparently stuck at 5.4.19 with as far as I can see no ability to upgrade.

best to contact your provider. 5.4 was discontinued in 2015 and is now out of support.  minimum they should be using is 5.6

http://php.net/supported-versions.php
Regards Simon
Davis Vantage Pro2 Plus (6162UK) • Daytime FARS • WeatherLink Live • AirLink • PurpleAir PA-II-SD • CumulusMX •


Offline smokie

  • Senior Member
  • **
  • Posts: 51
    • Newquay Cornwall. UK
Re: PHP Space Weather Script
« Reply #48 on: April 17, 2017, 03:15:51 PM »
I'm at a loss now, this has to be a GoDaddy issue, perhaps another GoDaddy user can shed some light on this, there seems to be a lot of pages if you search google

Try this, its using cURL, it seems to work even with an ftp url for me

<?php
$curl = curl_init();
curl_setopt ($curl, CURLOPT_URL, "ftp://ftp.swpc.noaa.gov/pub/latest/DSD.txt");
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
    $text = curl_exec ($curl);
    echo $text;
    curl_close ($curl);
?>
The data file was retrieved by this simple script, but only after a 20-30 second delay.  I'm wondering if there isn't some sort of GoDaddy-wide malicious script filter the results have to pass through before they're given the 'OK'.

20 secs is way to long as well, as Toxic says perhaps you can upgrade php
Newquay, Cornwall.UK  > WS2300 Operational since May 2007. New home from 6/4/08 www.newquayweather.com

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: PHP Space Weather Script
« Reply #49 on: April 19, 2017, 03:40:50 PM »
Jerry,

I just finished a chat session with GoDaddy support. Here is what they say they provide in the way of PHP support:
  • Native/Basic PHP coverage is with 5.4
  • You can upgrade FREE to either 5.5.38 or 5.6.30
All you have to do is contact their support group to get the upgrade (according the chat I just had).
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849