Author Topic: Slow loading main index (Update usno.mil applications server is down fix below )  (Read 10026 times)

0 Members and 1 Guest are viewing this topic.

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8605
    • Meteotemplate
For me the site is completely down, just times out after a while

Offline lrosenman

  • Contributor
  • ***
  • Posts: 148
    • Siena Weather
the DNS disappeared again.....

LER-MAC-3301:~ lrosenma$ host api.usno.navy.mil
Host api.usno.navy.mil not found: 3(NXDOMAIN)
LER-MAC-3301:~ lrosenma$ host usno.navy.mil
Host usno.navy.mil not found: 3(NXDOMAIN)
LER-MAC-3301:~ lrosenma$ host www.usno.navy.mil
Host www.usno.navy.mil not found: 3(NXDOMAIN)
LER-MAC-3301:~ lrosenma$

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8605
    • Meteotemplate
What information does your page get from the USNO exactly?

Offline lrosenman

  • Contributor
  • ***
  • Posts: 148
    • Siena Weather
this is the saratoga template sunrise/sunset, et al.  Not a biggie, I just renamed the script again.


Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
Or, if you really want to use this page then what I would do is set the timeout for it to something really low, like 5 seconds maximum. If the site does not respond within 5 seconds (which it should under normal circumstances) then it will just skip that part, so that would prevent these 30+ second load times.

Maybe someone understands this code. Would changing false to true prevent the hangup. I see there is also a 5 second timer that could be lowered to 1 second but I'm hanging 34 seconds so don't think it would help. Never mind I tried using true it didn't work

Code: [Select]
// get contents from one URL and return as string
 function get_sunmoon_fetchUrlWithoutHanging($url,$useFopen=false) {
// thanks to Tom at Carterlake.org for this script fragment
  global $Debug, $doDebug, $needCookie,$timeStamp,$TOTALtime,$Version;
  $overall_start = time();
  if (! $useFopen) {
   // Set maximum number of seconds (can have floating-point) to wait for feed before displaying page without feed
   $numberOfSeconds=5;   

   // Suppress error reporting so Web site visitors are unaware if the feed fails
   error_reporting(0);

   // Extract resource path and domain from URL ready for fsockopen
   $FullUrl = $url;
   $urlParts = parse_url($url);
   
« Last Edit: November 26, 2016, 12:59:17 PM by ValentineWeather »
Randy

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8605
    • Meteotemplate
To be quite honest, I would get rid of USNO altogether, I still dont see the point of using it if you dont need it. Or maybe you do, but no-one told me yet what exactly the script extracts from that site. Then we could talk about rewriting it so that it calculates it all locally (if possible of course).

Offline lrosenman

  • Contributor
  • ***
  • Posts: 148
    • Siena Weather
@ValentineWeather: The issue is the DNS is taking too long to do the resolution of api.usno.navy.mil.  I'm not sure how to
avoid that.


Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
What information does your page get from the USNO exactly?
It gets the precise sun/moon data in the form:
Quote
<!-- get-USNO-sunmoon.php - Version 2.04 - 11-Nov-2015 -->
<!-- refetch seconds=3600 -->
<!-- using Cached version of ./cache/USNO-moondata.txt -->
<!-- unchunking response -->
<!-- in=854 out=854 bytes -->
<!-- processing JSON entries for Moon data -->
<!-- json_decode returns  - No errors -->
<!-- USNOdata
Array
(
    [beginciviltwilight] => 06:31
    [beginciviltwilightdate] => 11/26/2016
    [sunrise] => 06:59
    [sunrisedate] => 11/26/2016
    [suntransit] => 11:56
    [suntransitdate] => 11/26/2016
    [sunset] => 16:52
    [sunsetdate] => 11/26/2016
    [endciviltwilight] => 17:20
    [endciviltwilightdate] => 11/26/2016
    [moonrise] => 04:16
    [moonrisedate] => 11/26/2016
    [moontransit] => 10:00
    [moontransitdate] => 11/26/2016
    [moonset] => 15:38
    [moonsetdate] => 11/26/2016
    [moonphase] => Waning Crescent
    [illumination] => 6%
    [hoursofpossibledaylight] => 09:53
)
 -->

The USNO data is used for weather software that DON'T return the data needed (like WeatherLink, older Meteobridge, Meteohub, etc) so they could display the dashboard and wxastronomy data just like the weather software programs that do provide the data.

That was done using the IP address (instead of the DNS hostname).

You can change get-USNO-sunmoon.php
Code: [Select]
$USNOUrl = "http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
to
Code: [Select]
#$USNOUrl = "http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
$USNOUrl = "http://199.211.133.93/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
to temporarily use the IP address instead of the hostname.. it's working fine.
« Last Edit: November 26, 2016, 01:03:45 PM by saratogaWX »
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
Thanks Ken that works. Why not just leave it as IP permanently?
Randy

Offline lrosenman

  • Contributor
  • ***
  • Posts: 148
    • Siena Weather
Thanks Ken that works. Why not just leave it as IP permanently?

because it MIGHT change IP / add IP's without notice :)

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8605
    • Meteotemplate
What information does your page get from the USNO exactly?
It gets the precise sun/moon data in the form:
Quote
<!-- get-USNO-sunmoon.php - Version 2.04 - 11-Nov-2015 -->
<!-- refetch seconds=3600 -->
<!-- using Cached version of ./cache/USNO-moondata.txt -->
<!-- unchunking response -->
<!-- in=854 out=854 bytes -->
<!-- processing JSON entries for Moon data -->
<!-- json_decode returns  - No errors -->
<!-- USNOdata
Array
(
    [beginciviltwilight] => 06:31
    [beginciviltwilightdate] => 11/26/2016
    [sunrise] => 06:59
    [sunrisedate] => 11/26/2016
    [suntransit] => 11:56
    [suntransitdate] => 11/26/2016
    [sunset] => 16:52
    [sunsetdate] => 11/26/2016
    [endciviltwilight] => 17:20
    [endciviltwilightdate] => 11/26/2016
    [moonrise] => 04:16
    [moonrisedate] => 11/26/2016
    [moontransit] => 10:00
    [moontransitdate] => 11/26/2016
    [moonset] => 15:38
    [moonsetdate] => 11/26/2016
    [moonphase] => Waning Crescent
    [illumination] => 6%
    [hoursofpossibledaylight] => 09:53
)
 -->

The USNO data is used for weather software that DON'T return the data needed (like WeatherLink, older Meteobridge, Meteohub, etc) so they could display the dashboard and wxastronomy data just like the weather software programs that do provide the data.

That was done using the IP address (instead of the DNS hostname).

You can change get-USNO-sunmoon.php
Code: [Select]
$USNOUrl = "http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
to
Code: [Select]
#$USNOUrl = "http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
$USNOUrl = "http://199.211.133.93/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
to temporarily use the IP address instead of the hostname.. it's working fine.

OK I see. All that can be done purely by a PHP script, just not sure about the Moon transit, but I think it should not be a problem as well

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8605
    • Meteotemplate
I could see if I can make a standalone version of the scripts used in Meteotemplate by extracting the dependencies, of course only if Ken agreed and then if it worked it could replace this, because that way you would have it 100% reliable and loaded practically immediately

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
Thanks Ken that works. Why not just leave it as IP permanently?

because it MIGHT change IP / add IP's without notice :)

Shouldn't change IP, I use IP address for other servers where name doesn't work, most have a static.
Randy

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
I could see if I can make a standalone version of the scripts used in Meteotemplate by extracting the dependencies, of course only if Ken agreed and then if it worked it could replace this, because that way you would have it 100% reliable and loaded practically immediately

I say go for it. But with IP working I'll keep as is. I don't think IP will change anytime soon maybe never.
Randy

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8605
    • Meteotemplate
OK, I will try. In case it works I then see absolutely no reason for keep on using USNO, because even if it worked, calculating it directly on your page means it will always work and it will always be faster.

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
It's always best to use DNS names for external sites as the IP address may change in the future if they change hosts/hoster/add external caching services to the site.  Using IP addresses should be considered a stop-gap measure to bypass a short-term issue with DNS.

I've sent another email to hostmaster at navy.mil regarding the problem.  It's very strange that an entire DNS zone for a prominent website could go offline -- perhaps their authorative DNS  holding the zone is being DDoSed or the server is just having internal issues with the DNS daemon.

 
I could see if I can make a standalone version of the scripts used in Meteotemplate by extracting the dependencies, of course only if Ken agreed and then if it worked it could replace this, because that way you would have it 100% reliable and loaded practically immediately

Jachym, if you have code that does this in PHP, please send it and I'll add it to the template set .. thanks for the offer!
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline lrosenman

  • Contributor
  • ***
  • Posts: 148
    • Siena Weather
Thanks Ken that works. Why not just leave it as IP permanently?

because it MIGHT change IP / add IP's without notice :)

Shouldn't change IP, I use IP address for other servers where name doesn't work, most have a static.
it probably WON"T However, USNO might move it, change servers, etc, and just change the DNS.  It's just good practice.

(I'm a sysadmin by trade, and these things happen, and they won't tell you).

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8605
    • Meteotemplate
It's always best to use DNS names for external sites as the IP address may change in the future if they change hosts/hoster/add external caching services to the site.  Using IP addresses should be considered a stop-gap measure to bypass a short-term issue with DNS.

I've sent another email to hostmaster at navy.mil regarding the problem.  It's very strange that an entire DNS zone for a prominent website could go offline -- perhaps their authorative DNS  holding the zone is being DDoSed or the server is just having internal issues with the DNS daemon.

 
I could see if I can make a standalone version of the scripts used in Meteotemplate by extracting the dependencies, of course only if Ken agreed and then if it worked it could replace this, because that way you would have it 100% reliable and loaded practically immediately

Jachym, if you have code that does this in PHP, please send it and I'll add it to the template set .. thanks for the offer!

Hi Ken,
I do have scripts for basically calculating all the parameters you are using (just not sure about Moon transit). They are used in various parts of Meteotemplate so I will put them all together and also extract the relevant parts from other functions they rely on, which are normally part of MT.
I will probably just prepare an associative array that has all the values and I would then send it to you to add it inside the script because obviously you know best how to do it. I will try either today or tomorrow (most likely tomorrow, Im at work now).

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
Thanks Ken that works. Why not just leave it as IP permanently?

because it MIGHT change IP / add IP's without notice :)

Shouldn't change IP, I use IP address for other servers where name doesn't work, most have a static.
it probably WON"T However, USNO might move it, change servers, etc, and just change the DNS.  It's just good practice.

(I'm a sysadmin by trade, and these things happen, and they won't tell you).

Okay stand corrected. Hopefully the DNS issue gets corrected or a fix. A time out fix written into code would be something else useful.
Randy

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
As one of the many non-programmer members of the WXforum, I just want to say THANK YOU! to both Jachym and Ken for working together to give us a better way of making these routines work. It's efforts like this that make me proud to be a member of this forum. I'm sure the outcome will be something all of can use and appreciate.
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8605
    • Meteotemplate
Thanks, I should come home tonight from work around midnight - so plenty of time to work on it :-) I cannot guarantee but I will see what can be done.

Offline lrosenman

  • Contributor
  • ***
  • Posts: 148
    • Siena Weather
The DNS is back again....

LER-MAC-3301:~ lrosenma$ host api.usno.navy.mil
api.usno.navy.mil has address 199.211.133.93
LER-MAC-3301:~ lrosenma$ host usno.navy.mil
usno.navy.mil mail is handled by 30 guardian.usno.navy.mil.
usno.navy.mil mail is handled by 10 pri-jeemsg.eemsg.mail.mil.
usno.navy.mil mail is handled by 20 sec-jeemsg.eemsg.mail.mil.
LER-MAC-3301:~ lrosenma$ host www.usno.navy.mil
www.usno.navy.mil has address 199.211.133.90
LER-MAC-3301:~ lrosenma$

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8605
    • Meteotemplate
well.... question is how long it will last... :D
Im still in my office, but I will definitely look at this tonight and tomorrow

Offline lrosenman

  • Contributor
  • ***
  • Posts: 148
    • Siena Weather
well.... question is how long it will last... :D
Im still in my office, but I will definitely look at this tonight and tomorrow

Of course.  But I figured I'd let folks know.   

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
When I went to website they are saying address is changing to aa.
So I changed the api to aa and it seems to be working.
Web version: http://aa.usno.navy.mil/rstt/onedaytable?ID=AA&year=2016&month=11&day=26&state=NE&place=valentine

Code change line 116: from
Code: [Select]
$USNOUrl = "http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";to
Code: [Select]
$USNOUrl = "http://aa.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
Don't know if this takes care of disappearing DNS but working now.
Randy

 

anything