Author Topic: NOAA server access problems (Read 3011 times)

Kpalivec · « **on:** November 17, 2008, 03:10:23 PM »

Greetings.

I run a couple of the php scripts on my WX website's main page.
I use the 5 day forecast and advisory-rss scripts.
Up until a day or so again they were working fine then suddenly I started getting XML parse errors with the rss script and access errors in my server's httpd logs with the forecast script.

here's the strange part. If I paste the urls for the NOAA sites the scripts use to gather data from into a webbrowser on my local machine I get the NOAA pages just fine. If I do the same from my web server I get 400 or 404 errors. I'm beginning to suspect that my server has been blacklisted or blocked from the NOAA servers. But seeing as my website probably only hits their site for this data maby 5 times aday. I dont think it could be for accessive accesses. Has anyone else encountered this problem?

In the mean time I'm going to continue to explore other possible causes for the errors.

Thanks to anyone who might have some insight as to what might be causing this.

Kevin Palivec
http://www.bigcountrywx.net

Kpalivec · « **Reply #1 on:** November 17, 2008, 03:18:36 PM »

UPDATE. The url to the point forecast had changed.
from

to

http://forecast.weather.gov/MapClick.php?site=sjt&map.x=220&map.y=74
which is odd because both urls produce a forecast page when the url is entered into a browser.
I suspect some kind of a browser identity script for the introduced error I was seeing.

Still havent figured out the rss-advisories.phph error I'm getting.
This is what gets written to the cache txt file when the script attempts to get the feed.
from this url...

http://www.weather.gov/alerts/wwarssget.php?zone=TXZ113

<HTML><HEAD><TITLE>Error 400</TITLE></HEAD>
<body bgcolor="#ffffff">
<h3>
Error 400 - Bad Request
</h3>

</body>
</html>


to recap, this error happens if the php script attempts to go to this url for data.
If the same url is used in a web browser, the advisory page is displayed.

Any insight anyone could share would be great!!
Kevin

saratogaWX · « **Reply #2 on:** November 17, 2008, 04:55:59 PM »

I tried running http://dragonneo.net/~bigcountrywx/rss-advisory.php?cache=refresh to see if the cache file was writable, and it was (view source shows)

Quote

<p>Advisory Information Unavailable, error reading RSS Data saved from the <a href="http://www.weather.gov/alerts/wwarssget.php?zone=TXZ113">NOAA advisories server.</a>

</p>

and the http://dragonneo.net/~bigcountrywx/cache/rss-advisory-TXZ113.txt cache file has

Quote

<HTML><HEAD><TITLE>Error 400</TITLE></HEAD>
<body bgcolor="#ffffff">
<h3>
Error 400 - Bad Request
</h3>

</body>
</html>

Try changing rss-advisory.php code from

Code: [Select]

       fputs($socketConnection, "GET $resourcePath HTTP/1.0\r\nHost: $domain\r\nUser-agent: $UA\r\nConnection: close\r\n\r\n");

to

Code: [Select]

       fputs($socketConnection, "GET $resourcePath HTTP/1.1\r\nHost: $domain\r\nUser-agent: $UA\r\nConnection: close\r\n\r\n");

and see if that helps.

Best regards,
Ken

Kpalivec · « **Reply #3 on:** November 17, 2008, 05:16:54 PM »

howdy,

first off let me thank you fore the quick reply!

I made the change but it's still getting the same error and since then the 5 day forecast has stopped working again.

I'm checking my server's logs but all the errors seem to be the same.

I'm beginning to wonder if there's some kind of a browser agent block or config problem?

Kevin

Kpalivec · « **Reply #4 on:** November 17, 2008, 06:21:22 PM »

Ok I found out what was going on... I think!

first off the isp who hosts my server seems to have had or has a proxy issue or something.

all requests to www.weather.gov were being sent to yahoo for some bizzaro reason. the ip's the dns was resolving were all weird. so I just pinned the correct ones into the server's hosts file.

Second issue (with the 5 day forecast script) was an apperent change in the html source for the point forecast page. the php script line...

preg_match('|<tr valign ="top" align="center">(.*)<table width="670"|s', $html, $betweenspan);
$forecastop = $betweenspan[1];

searched to through the html for and ending of ">(.*)<table width="670" but the page's source now reads ">(.*)<table width="100%"

I don't know if this is just the 5day point forcast format for my area or if it's an across the board change. but when I edited the php to note the change. the forcast started working again.

Kevin Palivec

saratogaWX · « **Reply #5 on:** November 17, 2008, 06:54:52 PM »

Having a bad address in the proxy for weather.gov sure sounds like the root cause of the issue.. nice sleuthing!

Thanks for the heads-up about the format change in the point-forecast page.

Best regards,
Ken

News:

Author Topic: NOAA server access problems (Read 3011 times)

Kpalivec

NOAA server access problems

Kpalivec

Re: NOAA server access problems-update-

saratogaWX

Re: NOAA server access problems

Kpalivec

Re: NOAA server access problems

Kpalivec

Re: NOAA server access problems

saratogaWX

Re: NOAA server access problems