WXforum.net

Administration => WXForum Bugs/Suggestions => Topic started by: SlowModem on April 22, 2015, 08:53:28 AM

Title: Offline Last Night?
Post by: SlowModem on April 22, 2015, 08:53:28 AM
I tried several different times with several different browsers to get on the forum last night and kept getting "failure to connect" messages.  I think it was a 501 error (or maybe a 500).  Was something going on?
Title: Re: Offline Last Night?
Post by: Maumelle Weather on April 22, 2015, 08:54:35 AM
The host 1 and 1 had some serious issues yesterday afternoon/evening. Ken had posted about it on the Weather-Watch Forum.
Title: Re: Offline Last Night?
Post by: Farmtalk on April 22, 2015, 09:13:58 AM
That was a pretty big down-time for someone as big as 1&1! But I suppose everybody has their problems sometime. And this was the error messaged I received when it was down. What was that, say about 8 hours?
Title: Re: Offline Last Night?
Post by: miraculon on April 22, 2015, 09:14:18 AM
I checked with DOJ.me (Down or Just Me) and confirmed that it was down. It happened to be the first time I was using Comodo Ice Dragon, so at first I thought that there was a problem on my end.

http://doj.me/ (http://doj.me/)

Greg H.
Title: Re: Offline Last Night?
Post by: virusdunil on April 22, 2015, 09:16:05 AM
The proof that even Super Heroes goes down sometimes hehehe
Title: Re: Offline Last Night?
Post by: Jáchym on April 22, 2015, 09:30:16 AM
http://www.weather-watch.com/smf/index.php/topic,63250.0.html

Ken wrote about it here
Title: Re: Offline Last Night?
Post by: ValentineWeather on April 22, 2015, 11:44:24 AM
I checked with DOJ.me (Down or Just Me) and confirmed that it was down. It happened to be the first time I was using Comodo Ice Dragon, so at first I thought that there was a problem on my end.

http://doj.me/ (http://doj.me/)

Greg H.

Thanks for that link.
Title: Re: Offline Last Night?
Post by: nincehelser on April 22, 2015, 01:59:01 PM
There seemed to be several outages yesterday in major data centers across the US.  I've no idea if they were linked or just coincidences. 
Title: Re: Offline Last Night?
Post by: saratogaWX on April 22, 2015, 02:36:24 PM
Yep, down due to hoster issues.

The logs show the last successful access was 21/Apr/2015:18:21:16 -0400

The first 'back online' access was at 22/Apr/2015:02:29:43 -0400

so just a bit more than 9hrs outage.

I could SSH to the site, but oddities prevailed.  Load factor was in the 20s and filesystem commands like ls all failed with 'ls: cannot open directory .: Input/output error'

I'd called the tech support, waited for 16 minutes for pickup, then was dropped because the operator couldn't hear my responses.
There was 'audio pixelation' on the on-hold music, and the auto-responder couldn't decipher my touch-tone entries.

The webserver hasn't been rebooted, so I'm suspecting some kind of back-end SAN failure.

Both WXForum.net and the regional network hub (northamericanweather.net) are hosted on the same server, so both were down for the count.  My other sites (saratoga-weather.org and a group of regional networks) were not impacted by the outage.

It's really bad when the Phone system also goes out during an outage.  I tried Twitter and email.. finally got responses after the problem was fixed.

Ken
Title: Re: Offline Last Night?
Post by: cospringswx on April 22, 2015, 02:48:03 PM
Ken I want to commend your hard work and keeping this forum running. Keeping members updated on what went wrong is above and beyond service. The list goes on. Again Thank you sir.
Title: Re: Offline Last Night?
Post by: WeatherHost on April 22, 2015, 03:16:15 PM
It was weird because everything worked -- you could ping, do a traceroute, the DNS was good; just no page loads.  Nothing but a blank screen.  No error screen, nada.

Title: Re: Offline Last Night?
Post by: SlowModem on April 22, 2015, 03:46:46 PM
Ken I want to commend your hard work and keeping this forum running. Keeping members updated on what went wrong is above and beyond service. The list goes on. Again Thank you sir.

Very well put! 
Title: Re: Offline Last Night?
Post by: SlowModem on April 22, 2015, 03:52:17 PM
http://www.weather-watch.com/smf/index.php/topic,63250.0.html

Ken wrote about it here

I'll have to try to remember this.  I'm not a WD user, so it doesn't usually come to mind.
Title: Re: Offline Last Night?
Post by: Jáchym on April 23, 2015, 12:21:07 PM
Hmmm, so is it the case that now its the WD forum that is down? I cant connect to it...
Title: Re: Offline Last Night?
Post by: CNYWeather on April 23, 2015, 12:23:07 PM
I was just going to ask that also. I get ERR_CONNECTION_REFUSED
Title: Re: Offline Last Night?
Post by: saratogaWX on April 23, 2015, 01:56:17 PM
Odd, weather-watch.com is hosted in Germany by (guess who) 1and1 (Hertzner.com).  Maybe they're having the same issue that pulled down a bunch of USA servers yesterday.
Title: Re: Offline Last Night?
Post by: Jáchym on April 23, 2015, 06:57:10 PM
Or somebody doesnt like PWS fans.... reminds me of what happened about a month ago when worldweatheronline.com was under such a strong DDoS they had to completely relocate servers after a few days and just at the same time openweathermap was also down for a few days due to DDoS
Title: Re: Offline Last Night?
Post by: ALITTLEweird1 on April 23, 2015, 08:20:29 PM
Odd, weather-watch.com is hosted in Germany by (guess who) 1and1 (Hertzner.com).  Maybe they're having the same issue that pulled down a bunch of USA servers yesterday.

Hopefully Chris knows about this. Besides being on the forum yesterday, it had been over 2 months since he has been there.
Title: Re: Offline Last Night?
Post by: Jáchym on April 24, 2015, 05:24:44 AM
Wow, so I just tried it and it is still down? Now I dont even get a connection error, just a blank page.
Title: Re: Offline Last Night?
Post by: wvdkuil on April 24, 2015, 06:15:49 AM
This is the last message on the announcements part of the ww forum:

http://www.weather-watch.com/smf/index.php/topic,63272.msg505083.html#new

Quote
One disk has been replaced and the RAID array is currently rebuilding. When the first disk has been fully integrated the second failing disk will be replaced. The server will run slower than normal whilst the array rebuilds...there's a lot of data to sync onto the new disk!

They are "on the air"  again.
Title: Re: Offline Last Night?
Post by: Farmtalk on April 24, 2015, 11:40:25 AM
Yep, working good here now! Good to see everything back to normal!  8-)
Title: Re: Offline Last Night?
Post by: Jáchym on April 28, 2015, 11:32:32 AM
I think the WD forum is down again.... blitzortung was down too.... this forum was down couple of days ago.... is this just a coincidence or is actually somebody trying to hack all sites of weather enthusiasts? :D
Title: Re: Offline Last Night?
Post by: saratogaWX on April 28, 2015, 12:28:24 PM
Yes, I'm getting
Quote
Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.
at Weather-Watch.com.

Chris may be doing some rebuilding after the disk went bad last week.  I've sent him a message about it.
Title: Re: Offline Last Night?
Post by: Jáchym on April 28, 2015, 12:54:16 PM
Thanks Ken
Title: Re: Offline Last Night?
Post by: tbrasel on April 28, 2015, 01:06:37 PM
Thanks Ken
Title: Re: Offline Last Night?
Post by: saratogaWX on April 28, 2015, 02:57:37 PM
For this Weather-DIsplay.com outage, I've replied to another thread with Chris(Administrator) responses.

http://www.wxforum.net/index.php?topic=25915.msg250146#new

It looks like it may be an extended outage due to hardware failure, but he has a good backup from 24 April to rebuild.

WXSIM users should probably switch to using the Bohler GFS source in WXSIMATE until Chris is back in service.