Author Topic: Weather-Watch down (Read 5104 times)

ALITTLEweird1 · « **on:** April 23, 2015, 06:31:07 PM »

Been down for most of the day today. Anyone heard anything from the admin or the mods?

Jáchym · « **Reply #1 on:** April 23, 2015, 06:43:44 PM »

http://www.wxforum.net/index.php?topic=25896.0

gwwilk · « **Reply #2 on:** April 28, 2015, 10:42:05 AM »

Weather-Watch is down again this morning with this message:
'Connection Problems

Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.'

I wonder if Chris and Brian know?

saratogaWX · « **Reply #3 on:** April 28, 2015, 12:30:31 PM »

I've sent Chris a message about it.

saratogaWX · « **Reply #4 on:** April 28, 2015, 02:39:20 PM »

I've just heard from Chris(administrator)

Quote

Looks like the RAID system has died. I was hoping to get one good disk back but it looks like the rebuild has failed.

Please let them know I'm investigating. It may need a complete rebuild. I have backups from 24th but it will take some time to do the build.

I'll keep you posted as info is available.

Best regards,
Ken

saratogaWX · « **Reply #5 on:** April 28, 2015, 04:14:41 PM »

A bit more news:

Quote

Maybe install the old good disk in new hdwe?

Sorry it's the two disks in the RAID mirror that have died nearly at the save time. SMART errors on one and the second disk suffered a mechanical failure. As far as I know the RAID card and server are fine

Ouch. New disks and a long rebuild slog from backup I guess...

Yes
Wondering if the mechanical failure put more pressure on the other disk that drove it over the edge. Tried to rebuild the RAID from the working disk but it didn't work

That's possible, especially if the original RAID config did load spreading amongst the disks for primary activity and background catchup for the redundant copies.
Does the hoster have an ETA for a repaired platform so you can start the rebuilds ?

2-4 hours for disk replacement

Gonna be a long night for you.

saratogaWX · « **Reply #6 on:** April 28, 2015, 06:31:51 PM »

Just in from Chris

Quote

Thanks. If I can get the disks replaced and a new OS building on it before I fall asleep I'll be happy

New disk installed. Mirror is rebuilding from the first new disk. It will need interesting to see if there's a recoverable image on it when it's finished.

Hopefully, the RAID array will rebuild successfully. If not, the reformat/rebuild process will have to start.

saratogaWX · « **Reply #7 on:** April 28, 2015, 07:48:10 PM »

Last status for today

Quote

Looks like too much is damaged. Lots of I/O errors when I access the root position

Ouch again!! Guess it is reformat/rebuild/reload from backup time. Maybe best to get some sleep and tackle afresh in the morning.

Going to get the OS installing then it's ready to do what I can tomorrow. Got a busy day at real work so will have to do what I can when I can

New OS installed but it's minimal so got loads of software to install. Backups are coming back across as I type. The good thing is that I now have a shiny new Centos 7.1 server instead of old Centos 5
Time for bed now though

ALITTLEweird1 · « **Reply #8 on:** April 28, 2015, 08:32:32 PM »

Thanks for the updates Ken.

Jáchym · « **Reply #9 on:** April 29, 2015, 06:27:49 AM »

The forum is up again....

unfortunately EVERYTHING (all posts, member registrations etc.) since January 2014 are not recoverable and permanently lost. I just had to make a new registration.... Real shame so much stuff from the last 15 months regarding so many scripts, issues with WD and general comments is gone.

Jáchym · « **Reply #10 on:** April 29, 2015, 06:34:25 AM »

I also posted it there, but I am going to ask here as well in case someone has experience with this, maybe Ken...

Do you think it would be possible to somehow extract the lost data from:
http://web.archive.org/web/20150318014021/http://www.weather-watch.com/smf/index.php

last image of the web is from 18 March 2015, which is still much better than January 2014...

wvdkuil · « **Reply #11 on:** May 07, 2015, 03:00:48 AM »

Thursday May 7 07:00 UTC Servers are down again.

The forum servers http://www.weather-watch.com/smf/index.php are unreachable , browser message "connection refused"
or give a "404 Not found" message

The directory structure root <-> smf can be displayed but the folders are empty.

Jáchym · « **Reply #12 on:** May 07, 2015, 03:03:04 AM »

Yes I can confirm that, I get the same message. I wonder what happened

broadstairs · « **Reply #13 on:** May 07, 2015, 03:28:18 AM »

Yes it's still down and for WXSim users the GFS/Advection data is getting old if using McMahon so switch to Bohler.

Stuart

ALITTLEweird1 · « **Reply #14 on:** June 19, 2015, 09:32:25 PM »

Down again?

saratogaWX · « **Reply #15 on:** June 19, 2015, 09:37:00 PM »

Yes, with a SMF settings error... since it's 2:30am in the UK, likely it won't awaken Chris (administrator), but he'll get it online in the AM.

broadstairs · « **Reply #16 on:** June 20, 2015, 02:39:25 AM »

Well either Chris got up early or it fixed itself

anyway it is up now at least for me....

Stuart

weatherc · « **Reply #17 on:** June 20, 2015, 09:33:57 AM »

http://www.weather-watch.com/smf/index.php gives 404 Not found now...

mikeym2m · « **Reply #18 on:** June 20, 2015, 09:58:41 AM »

Same here "404 not found"

MikeyM

broadstairs · « **Reply #19 on:** June 20, 2015, 10:14:45 AM »

Yes I get a 404 now as well. However it was up earlier and I also got my GFS OK for 20th 00hrs.

Stuart

saratogaWX · « **Reply #20 on:** June 20, 2015, 10:18:49 AM »

I'd sent Chris (administrator at weather-watch.com) a message last night, and he'd responded with "the server needed a reboot". So it was up for a while. I just heard from Niko

Quote

Looks like it has crashed again this morning, per Chris he will not be able to look at it until late this evening UK, so some time this afternoon Pacific.

so the issue is a bit deeper than a simple reboot and will need Chris' expert care to fix.

Patience all... internet happens

ed2kayak · « **Reply #21 on:** June 21, 2015, 09:08:51 AM »

Is that a disadvantage of not using a hosted server? Not to say that they don't have issues at times.

Unfortunately the http://www.weather-watch.com site, with program files is also affected.

weatherc · « **Reply #22 on:** June 21, 2015, 09:23:35 AM »

Quote

Is that a disadvantage of not using a hosted server?

From what i know it is a hosted server, at one of Europe's biggests serverhosts. With dedicated server are there usually 2 service-levels, unmanaged and managed. I have no glue what level the ww-server use but a guess is unmanaged (like my own dedi at the same host).

The difference here is that with unmanaged is it, exactly, unmanaged, and you as admin are on your own to keep it online software-wise, and the host fix hardware-issues.

I don't see it as an disadvantage to use unmanaged as theese wx-related stuffs (like GFS-downloads/processing) are pretty custom things and the support at the host should not have any glue about it anyway.

Joel · « **Reply #23 on:** June 21, 2015, 05:29:20 PM »

Hi, some news from Facebook :

Quote

HI all.Just to update re the WD forum: Chris the admin has been busy commisioning new hardware, as the existing hardware has been too unreliable...all on the weekend too.So the forum should be up again soon.Thanks for all your hard work again Chris!

saratogaWX · « **Reply #24 on:** June 21, 2015, 05:53:44 PM »

Chris responded with

Quote

Busy building a brand new server. Don't like the old one, too unreliable

-- it may take some time to get the site fully functioning again.

WXSIM users should switch to the Bohler feed for GFS data.. the McMahon feed is inoperative right now.

News:

Author Topic: Weather-Watch down (Read 5104 times)