kc4kqe
Member
 Offline
Posts: 32
KC4KQE
|
 |
« on: June 20, 2012, 12:48:24 AM » |
|
Noticed my automatic image uploads stalled at 11:38am local (AZ) time today, and logs showed wunderground wasn't allowing incoming ftp. Tried by hand:
ftp webcam.wunderground.com
Connected to webcam.wunderground.com.
421 Service not available, remote server has closed connection
ftp>
Thaaaaaaaat's not happy. Anyone else having trouble? (I would assume so).
I'm resolving webcam.wunderground.com as 38.102.137.106. Logs show that address hasn't changed since I started uploading a few weeks ago.
Putting in a ticket with wunderground support, but I usually don't hear from them for a couple of days. If anyone has any support contacts at wunderground, might be worth hitting them up.
|
|
|
|
|
Logged
|
|
|
|
kc4kqe
Member
Offline
Posts: 32
KC4KQE
|
 |
« Reply #1 on: June 20, 2012, 04:26:42 AM » |
|
Just to follow-up with some more information... This definitely appears to be a widespread, but not total, outage.
It looks like they are still processing a much smaller number of incoming files (by reviewing webcam status by region), but what appears to be 90%+ of active cameras listed have not updated since the issue started (presently 13 hours ago, mid-day Tuesday in the US).
From the client side, I believe I can rule out DNS or routing issues -- ip hasn't changed nor routing. I've tried connecting from my own data centers on the West coast, East coast and central US -- same behavior.
I have no visibility or knowledge of the actual back-end at wunderground -- I would certainly hope/expect that address (38.102.137.106) is just a front-end to a large cluster of load-balancing networks and/or systems. An educated guess would be they are operating at a small fraction of their normal capacity (some kind of internal network or redundancy failure), but that is complete speculation.
Digging a little deeper, network traces don't show much from my end -- the ftp connection is set up normally (send SYN/ECN/CWR, receive SYN/ACK with port assignment, send ACK to assigned port) -- but then I immediately receive a FIN/ACK and the connection is closed normally (ACK, RST/ACK). I doubt it's a firewall issue or similar, since I do see the port assignment and even a FIN from the assigned port, etc. Without visibility into the wunderground side, I'm really stabbing in the dark...
Opened "critical" ticket RNP-481449 with wunderground support a while ago, but (surprise), no reply yet.
|
|
|
|
|
Logged
|
|
|
|
|
DanS
|
|
« Reply #2 on: June 20, 2012, 05:35:00 AM » |
|
Yes my webcam's last upload was at 06:39 GMT on the 19th. I see many others with the same time drop out. I manually send one and get a "unable to connect with FTP server" message back. All other wx data makes it to WU just the webcam server at their end is having issues.
|
|
|
|
« Last Edit: June 20, 2012, 05:36:47 AM by DanS »
|
Logged
|
|
|
|
|
BigOkie
|
|
« Reply #3 on: June 20, 2012, 09:08:33 AM » |
|
Keep in mind also that wunderground offers to do a web fetch of your camera image...I would imagine those are processing. Probably FTP server is down.
|
|
|
|
|
Logged
|
 Current setup: Honeywell TE923W, DIY aspirated temp/hygro unit in Ambient Shield housing with a DIY FARs daytime. |
|
|
|
d_l
|
|
« Reply #4 on: June 20, 2012, 09:36:27 AM » |
|
The web fetch option is working fine. Those are the web cams that you see updating. A few months ago it was the web fetch cameras that were having problems being updated. This lasted for for a few days.
|
|
|
|
|
Logged
|
--Dave--Wireless VP2 w/ solar, 24hr FARS, Heater, (Envoy-WLIP)*2-Meteohub, WL 6.0.0, WU & W4U= KNVRENO37 NetcamXLPeople always talk about the weather, but they never do anything about it. Not me. I'm gonna measure it. www.tceweather.com
|
|
|
|
gwwilk
|
|
« Reply #5 on: June 20, 2012, 10:25:58 AM » |
|
My webcam images reappeared again on WU as of an hour or so ago. It looks like they're getting it sorted out.
|
|
|
|
|
Logged
|
|
|
|
blaforest
Member
Offline
Posts: 12
|
|
« Reply #6 on: June 20, 2012, 11:10:43 AM » |
|
Mine is back up as well. Went down around 11:40am PST yesterday.
|
|
|
|
|
Logged
|
|
|
|
dmp
Member
Offline
Posts: 1
|
|
« Reply #7 on: June 20, 2012, 12:08:14 PM » |
|
I'm glad you posted this as the posting got me to join the forum.  And, like the others, my cam is working once again, too. |
|
|
|
|
Logged
|
|
|
|
|
Skywatch
|
|
« Reply #8 on: June 20, 2012, 01:29:23 PM » |
|
Same here. I reacted off of instinct and assumed WD wasn't uploading. Hit the test button in the WU cam uploader to fix the issue before it occured to me to see what everyone else was going. Glad it's fixed.
|
|
|
|
|
Logged
|
Davis VP2+ & leaf/soil stn. WMR200-UVN800-2xTHGR810-THWR800. Acu-rite 00639W. WeatherDisplay V10.37 B45 WU KTXMCKIN27 PWS MCKTXAWS1 CWOP DW4088 WXbug p14388
HAKUNA MATATA |
|
|
kc4kqe
Member
Offline
Posts: 32
KC4KQE
|
|
« Reply #9 on: June 20, 2012, 03:42:37 PM » |
|
dmp - Welcome to the forum!
BigOkie and d_l - Thanks for the reminder about the web remote-retrieve option - I had completely forgotten about that since I don't use it myself. Definitely makes sense -- ftp was down, but web polling was still working, resulting in the active/inactive camera distribution I saw.
I also confirmed ftp began working again this morning around 1350Z (0650 MST), but I'm still seeing quite a few upload failures throughout the day (as recent as 20 minutes ago), so I'm not sure it's safe to call restoration of service quite yet. Certainly better than it was last night, though.
Got an email reply to my ticket with a generic "we are aware of the issue and working to fix it".
|
|
|
|
|
Logged
|
|
|
|
|
