Author Topic: Local METAR reports Not loading (Read 11646 times)

ed_heaton · « **Reply #175 on:** November 06, 2019, 05:20:27 PM »

Thanks Ken and I'll approach Hostgator tomorrow and see what happens.

ed_heaton · « **Reply #176 on:** November 06, 2019, 07:59:55 PM »

Odd how this seems to be a reoccurring issue across web hosting. Is there an alternative to posting the metar data?

Jasiu · « **Reply #177 on:** November 06, 2019, 08:31:56 PM »

Couple of options...

1) https://aviationweather.gov/metar/data

This would involve an HTML scrape - pulling everything within the <code> tag.

Example:

https://aviationweather.gov/metar/data?ids=KBOS&format=raw&hours=0&taf=off&layout=off

returns (look for "data starts here"):

Code: [Select]

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<base href="aviationweather.gov" /><title>AWC - METeorological Aerodrome Reports (METARs)</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" >
<meta http-equiv="content-type" content="text/html; charset=utf-8" >
<meta http-equiv="Refresh" content="900" >
<meta name="DC.language" content="en-us" scheme="DCTERMS.RFC1766" >
<meta name="description" content="Aviation Weather Center Homepage provides comprehensive user-friendly aviation weather Text products and graphics." >
<meta name="keywords" content="aviation, weather, icing, turbulence, convection, pirep, metar, taf, airmet, sigmet, satellite, radar, surface, wind, temperature, aloft, airplane, NEXRAD, GOES, WSR-88D, precipitation, rain, snow, sleet, thunderstorm, en-route, prognosis, chart" >
<meta name="DC.title" content="AWC - Aviation Weather Center" >
<meta name="DC.description" content="Aviation Weather Center Home Page ... METARs Page" >
<meta name="DC.creator" content="NOAA's National Weather Service - Aviation Weather Center Homepage" >
<meta name="DC.date.created" content="2013-03-15" scheme="ISO8601" >
<meta name="DC.date.reviewed" content="2013-03-15" scheme="ISO8601" >

<link href="//code.jquery.com/ui/1.10.4/themes/smoothness/jquery-ui.css" media="screen" rel="stylesheet" type="text/css" >
<link href="/images/favicon.ico" rel="shortcut icon" >
<link href="/images/noaa_logo.png" rel="apple-touch-icon" >
<link href="/css/layout.css" media="" rel="stylesheet" type="text/css" >
<script type="text/javascript" src="//code.jquery.com/jquery-1.9.1.js"></script>
<script type="text/javascript" src="//code.jquery.com/ui/1.10.3/jquery-ui.js"></script>
<script type="text/javascript" src="/lib/OpenLayers-2.13.1/OpenLayers.js"></script>
<script type="text/javascript" src="/javascript/awc_openlayers.js"></script>
<script type="text/javascript" async="true" id="_fed_an_ua_tag" src="https://dap.digitalgov.gov/Universal-Federated-Analytics-Min.js?agency=DOC&subagency=AWC&pua=ua-33523145-2"></script>

</head>
<body >
<div id="awc_pagewrapper">
<!-- Main content -->
<div id="awc_main">
<div id="awc_main_content">
<script type="text/javascript">
  function goto(format,hours, taf, layout){
    var id_elem = document.getElementById("ids");
    var url = "/metar/data?ids="+id_elem.value+"&format="+format+"&hours="+hours+"&taf="+taf+"&layout="+layout;
    window.location = url;
  }
</script>
<div id="title">METAR Data<div id="title_linkbar">
<div class='title_link'><a href='/metar?date=' title = 'METAR Home'>METAR Home</a></div>
<div class='title_link'><a href='/metar/plot?date=' title = 'METAR Plots'>Plot</a></div>
<div class='title_link on_page'>Data</div>
<div class='title_link'><a href='/metar/board?date=' title = 'METAR Board'>Board</a></div>
<div class='title_link'><a href='/metar/help?page=text&date=' title = 'METAR Information'>Info</a></div>
</div>
</div>
<div id="awc_main_content_wrap">
  <p clear="both">
<strong>Data at: 0123 UTC 07 Nov 2019</strong></p>
<!-- Data starts here -->
<code>KBOS 070054Z 30006KT 10SM CLR 09/M07 A3035 RMK AO2 SLP276 T00891072</code><br/>
<hr width='65%' />
<!-- Data ends here -->
</p>
</div>
</div> <!-- awc_main_content -->
</div> <!-- awc_main -->
</div> <!-- awc_pagewrapper -->
</body>
</html>

2) Probably a cleaner way to get the same info is to use the NWS API "latest observation" endpoint. But this would essentially be a rewrite.

Example:

https://api.weather.gov/stations/KBOS/observations/latest

This returns JSON and therefore doesn't require a METAR decoder.

ed_heaton · « **Reply #178 on:** November 07, 2019, 06:21:08 AM »

Here's what I got this morning from tech support at Hostgator:

I have checked the server settings and website logs. I could not find any issues from our end. Since NWS is blocking the IP address, I suggest that you contact them to remove the block. Unfortunately, we do not have an access to the blacklist on NWS.

So it looks like I'm stuck

ed_heaton · « **Reply #179 on:** November 07, 2019, 06:30:59 AM »

Jasiu - what would a rewrite involve and a cost of a re-write? According to the template web list on Ken's page I'm not the only one having this issue.

Thanks

P.S. On a side note I noticed that some of the metar (Ken's site for example) say "Night time" then the conditions. Mine never did say the time of day... ex - night time, dawn

Jasiu · « **Reply #180 on:** November 07, 2019, 09:54:16 AM »

Quote from: ed_heaton on November 07, 2019, 06:30:59 AM

Jasiu - what would a rewrite involve and a cost of a re-write? According to the template web list on Ken's page I'm not the only one having this issue.

It's essentially a replacement of get-metar-conditions-inc.php. The code would be a bit simpler because the values would be taken straight from the JSON returned rather than having to decode a METAR string.

Let's let Ken chime in as this is his code. I could be recruited to to do the initial code (I don't want to be on the hook for maintenance, although I'm not going anywhere soon - I hope!).

Quote

P.S. On a side note I noticed that some of the metar (Ken's site for example) say "Night time" then the conditions. Mine never did say the time of day... ex - night time, dawn

Can't say I ever saw that. The icon choice is dependent upon the time of day, so that e.g. clear sky at night gives you a dark image with a moon instead of blue sky and a sun. It may have something to do with that.

ed_heaton · « **Reply #181 on:** November 07, 2019, 11:00:45 AM »

Thank You for the feedback and it would be greatly appreciated

I'd be happy to help with time and knowledge compensation.

saratogaWX · « **Reply #182 on:** November 07, 2019, 12:32:45 PM »

Quote from: Jasiu on November 07, 2019, 09:54:16 AM

Quote from: ed_heaton on November 07, 2019, 06:30:59 AM
Jasiu - what would a rewrite involve and a cost of a re-write? According to the template web list on Ken's page I'm not the only one having this issue.

It's essentially a replacement of get-metar-conditions-inc.php. The code would be a bit simpler because the values would be taken straight from the JSON returned rather than having to decode a METAR string.

Let's let Ken chime in as this is his code. I could be recruited to to do the initial code (I don't want to be on the hook for maintenance, although I'm not going anywhere soon - I hope!).

Quote
P.S. On a side note I noticed that some of the metar (Ken's site for example) say "Night time" then the conditions. Mine never did say the time of day... ex - night time, dawn

Can't say I ever saw that. The icon choice is dependent upon the time of day, so that e.g. clear sky at night gives you a dark image with a moon instead of blue sky and a sun. It may have something to do with that.

The tgftp.noaa.nws.gov server has been accumulating world-wide METAR reports for years now and was the only public-available source for that raw data (AFAIK). Several aviation based sites now do the same, and ogimet.net does also.

I just checked out the api.weather.gov/stations/{ICAO}/observations/latest feeds for several US and international sites, and it appears that it also has international data.

I'll work up a get-metar-conditions-inc.php script to use it as an option. Thanks for the suggestion.

Jasiu · « **Reply #183 on:** November 07, 2019, 12:58:35 PM »

Quote from: saratogaWX on November 07, 2019, 12:32:45 PM

I'll work up a get-metar-conditions-inc.php script to use it as an option. Thanks for the suggestion.

Ken being awesome, as usual!

ed_heaton · « **Reply #184 on:** November 07, 2019, 01:02:32 PM »

We would expect no less from Ken... He's always willing to lend me a hand when I'm in need!

wvdkuil · « **Reply #185 on:** November 07, 2019, 01:03:06 PM »

Quote from: saratogaWX on November 07, 2019, 12:32:45 PM

The tgftp.noaa.nws.gov server has been accumulating world-wide METAR reports for years now and was the only public-available source for that raw data (AFAIK). Several aviation based sites now do the same, and ogimet.net does also.

I just checked out the api.weather.gov/stations/{ICAO}/observations/latest feeds for several US and international sites, and it appears that it also has international data.

I'll work up a get-metar-conditions-inc.php script to use it as an option. Thanks for the suggestion.

Hello Ken,
The api.weather.gov/stations/{ICAO}/observations/latest returns often older data compared to the direct feeds

https://tgftp.nws.noaa.gov/data/observations/metar/stations/KBOS.TXT
https://tgftp.bldr.ncep.noaa.gov/data/observations/metar/stations/KBOS.TXT

2019/11/07 17:54
KBOS 071754Z 22012KT 10SM FEW055 BKN110 OVC140 14/04 A3004 RMK AO2 SLP173 T01440039 10150 20083 58035

https://api.weather.gov/stations/KBOS/observations/latest
"rawMessage": "KBOS 071654Z 21014KT 10SM BKN110 OVC140 14/04 A3008 RMK AO2 SLP186 T01390039",

Sometimes they are almost the same, sometimes 30 minutes or more difference.

===

https://api.weather.gov/stations/EBBR/observations/latest
"rawMessage": "EBBR 071720Z 21010KT CAVOK 07/04 Q0999 NOSIG",

The other two https://tgftp.bldr.ncep.noaa.gov/data/observations/metar/stations/EBBR.TXT
2019/11/07 17:50
EBBR 071750Z 21009KT CAVOK 07/04 Q0999 NOSIG
Wim

saratogaWX · « **Reply #186 on:** November 07, 2019, 01:41:58 PM »

Thanks for the heads-up about timeliness of the reports on api.weather.gov.

I've done a quick mod to get-metar-conditions-inc.php V1.18 to optionally use api.weather.gov instead of tgftp.nws.noaa.gov.
The 'hack' simply gets the JSON, and uses only the rawdata and timestamp to let the rest of the script do the decoding of the METAR as normal. I didn't have time to use the decoded conditions in the JSON with this code .. this retains compatibility with the other scripts that may use this too.

To test it out:
1) replace get-metar-conditions-inc.php with the attached version 1.18.
2) in Settings.php, add

Code: [Select]

$SITE['useMetarAPI'] = true; to enable use of api.weather.gov.

Let me know (Ed) how it works for you.

saratogaWX · « **Reply #187 on:** November 07, 2019, 01:46:43 PM »

Quote from: ed_heaton on November 07, 2019, 06:30:59 AM

P.S. On a side note I noticed that some of the metar (Ken's site for example) say "Night time" then the conditions. Mine never did say the time of day... ex - night time, dawn

Ed, the conditions icon/text string come from Weather-Display webtags .. it's based on a combination of Solar sensor(day) and nearby Metar (night) with overrides for wind/rain. Since you're not using Weather-Display, your template conditions come from the nearby METAR you selected, with overrides for local rain events.

ed_heaton · « **Reply #188 on:** November 07, 2019, 01:50:38 PM »

Makes sense Ken on the weather display... I'll load the new file here shortly! Thank you!

ed_heaton · « **Reply #189 on:** November 07, 2019, 01:54:06 PM »

New Script seems to be working! Thanks Ken and I'll monitor it closely over the next couple of day!

ed_heaton · « **Reply #190 on:** November 07, 2019, 02:04:49 PM »

Any special place in settings.php to insert the $SITE['userMetarAPI'] = true; ?

Thanks

saratogaWX · « **Reply #191 on:** November 07, 2019, 02:28:56 PM »

No special place for it in Settings.php .. anywhere within the configurable settings area is fine.

And it’s

Code: [Select]

$SITE['useMetarAPI'] = true;

ed_heaton · « **Reply #192 on:** November 07, 2019, 03:03:25 PM »

Got it... Thanks Ken

ed_heaton · « **Reply #193 on:** November 07, 2019, 03:12:44 PM »

Oops something's off -

Jasiu · « **Reply #194 on:** November 07, 2019, 03:33:02 PM »

Ed, what you are seeing is what Wim described above. I also get old data for some locations.

But script-wise, it's working in both modes (true/false).

for Ken's quick work!

Ken, you think one of us should open a ticket w/ NWS on the stale data issue?

ed_heaton · « **Reply #195 on:** November 07, 2019, 03:39:10 PM »

Jasiu - it is showing data (sometime out of date data but data without errors) However I am missing data for my main metar (KMQS - Coatesville, Pennsylvania, USA Distance from station: ENE 6mi, 10km) I'm not sure what's going on with that...

saratogaWX · « **Reply #196 on:** November 07, 2019, 03:44:21 PM »

That's what Wim was talking about.. the data on api.weather.gov is not as 'fresh' as the data on tgftp.nws.noaa.gov site. Since you're blocked from using the primary site, some 'stale' data is likely better than no data, I think.

saratogaWX · « **Reply #197 on:** November 07, 2019, 03:49:36 PM »

I just looked at https://api.weather.gov/stations/KMQS/observations/latest and the issue is no rawMessage data:

Code: [Select]

        "timestamp": "2019-11-07T20:29:00+00:00",
        "rawMessage": null,

the rawMessage part of the JSON is supposed to have the raw METAR observation. Since it's empty, there's nothing to decode

ed_heaton · « **Reply #198 on:** November 07, 2019, 03:51:01 PM »

Thanks Ken!

Jasiu · « **Reply #199 on:** November 07, 2019, 04:07:37 PM »

Quote

the rawMessage part of the JSON is supposed to have the raw METAR observation. Since it's empty, there's nothing to decode

Sounds like another issue to report.

News:

Author Topic: Local METAR reports Not loading (Read 11646 times)