Author Topic: Connection Issues  (Read 5748 times)

0 Members and 1 Guest are viewing this topic.

Offline JohnN

  • Forecaster
  • *****
  • Posts: 348
    • Monmouth County Weather
Connection Issues
« on: June 08, 2014, 03:03:35 PM »
I just briefly got a message saying "Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later." Yesterday, while trying to view a topic, I got another error (I believe it was a 500-something error). The forum seems to be fine right now, but I just want to bring this to your attention incase it is an attack on the forum or something.
www.monmouthcountyweather.comUK Met Office: 45560463
Twitter: @John_N_WxCoCoRaHS: NJ-MN-64
CWOP: E1682Davis Vantage Vue, Pro2Plus w/24 hr FARS, Ambient Weather WS-1000
PWS/WFY: SOUTHARDNJCumulusMX
Wunderground: KNJHOWEL11
                       KNJHOWEL15
LiveATC: KWRI/KNEL/New York Center
             KMJX/KBLM/3N6/N12

Offline corwyyn

  • Forecaster
  • *****
  • Posts: 328
  • Far, far east Mesa, AZ
    • Sleeping Saguaros Weather  Mesa, AZ
Re: Connection Issues
« Reply #1 on: June 09, 2014, 04:17:15 AM »
I just briefly got a message saying "Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later." Yesterday, while trying to view a topic, I got another error (I believe it was a 500-something error). The forum seems to be fine right now, but I just want to bring this to your attention incase it is an attack on the forum or something.
I've gotten the '500 Internal server error' message a couple of times over the past two days, most recently about five minutes before posting this.  Gave the forum a couple of minutes to recover and got back in with no problems.  I'll keep track and post up if I get more.
Kevin
AD5VG
Blitzortung station 1042

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Connection Issues
« Reply #2 on: June 09, 2014, 10:44:45 AM »
I've been researching the logs.. and adding new entries to the .htaccess block list :)
Today's outages caused by server farms in Ukraine and Afghanistan -- now on block list.

Best regards,
Ken
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Online AWL

  • Forecaster
  • *****
  • Posts: 727
    • Ardmore Weather Live
Re: Connection Issues
« Reply #3 on: June 09, 2014, 10:46:21 AM »
I get this a few times a day:

"Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request."

-Doug

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Connection Issues
« Reply #4 on: June 09, 2014, 10:50:27 AM »
That happens when too many concurrent connections are present and no more PHP processes are available.
It is usually caused by a rogue robot/spider crawling lots of pages from several IP address at the same time.
Google, Yahoo! and MSN/Bing are well behaved and slowly crawl the site.  Rogue search engines are not
as 'website friendly' and cause congestion.  I've blocked the biggest offenders (Baidu, Yandex, etc), but
new ones crop up from time to time.  It's a never-ending battle :)
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Online AWL

  • Forecaster
  • *****
  • Posts: 727
    • Ardmore Weather Live
Re: Connection Issues
« Reply #5 on: June 09, 2014, 11:12:45 AM »
.  It's a never-ending battle :)

I know you work hard to keep this forum up and running. Many thanks for all your time and effort!

-Doug

Offline cospringswx

  • Forecaster
  • *****
  • Posts: 4136
    • Colorado Springs Weather
Re: Connection Issues
« Reply #6 on: June 09, 2014, 11:35:08 AM »
.  It's a never-ending battle :)

I know you work hard to keep this forum up and running. Many thanks for all your time and effort!

-Doug

+1




Ryan 

Colorado Springs, CO
www.cospringsweather.com
Davis Vantage Vue
Weather Display Software
Amcrest HD IP Camera

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Connection Issues
« Reply #7 on: June 09, 2014, 01:43:12 PM »
Looks like most of the beating-up of the site was from unruly search bot from multiple IP addresses in UA/AF.  Got them blocked now :)
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: Connection Issues
« Reply #8 on: June 10, 2014, 04:36:23 PM »
I get this a few times a day:

"Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request."

-Doug

I got several of those messages earlier today. And now, one of the

"Connection Problems

Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.
"

At the moment there are "97 Guests, 29 Users (0 Buddies, 5 Spiders)" showing as being on-line.

Just takes patience and retrying to get back on again.

Ken - thank for all you do!!!  =D>
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Connection Issues
« Reply #9 on: June 10, 2014, 04:45:51 PM »
Found some more silly-business going on.. robot registration attempts from China, Russia and Ukraine.  They're blocked now with 403s and shouldn't even launch a PHP process.

Our hosting offers 15 concurrent PHP processes.. the 16th and above get a 500-error until the concurrent number drops to 15 or below.

The valid (Google, Yahoo!, and Bing) bots all are well behaved and don't clog up the queue much.  It's the miscreants that try robo-registration or rogue webcrawlers that don't behave and can use up our PHP threads.

The server hosting WXForum.net also hosts northamericanweather.net (and the automation behind the Global Google map data collection), but that only uses a couple of PHP processes every 5 minutes or so.

I did implement custom error pages for 400, 403, 404 and 500 errors as I too, got tired of seeing the default ones from Apache, and saved a '404' on those default pages when looking for the error page to offer).

I'm watching and analyzing the Apache logs multiple times per day, and adjusting the .htaccess block list accordingly.

Thanks for your patience in getting this issue down to a 'only sometimes' problem soon...

Best regards,
Ken
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline corwyyn

  • Forecaster
  • *****
  • Posts: 328
  • Far, far east Mesa, AZ
    • Sleeping Saguaros Weather  Mesa, AZ
Re: Connection Issues
« Reply #10 on: June 11, 2014, 08:05:15 AM »
Ken we appreciate everything you do to keep the site up and functional - I know it can sometimes be a steep uphill battle and all the ID10Ts you have to fight on a daily basis are just more annoyance.  I did want to let you know I still occasionally see the 'Sorry, SMF was unable to connect to the database' message at various times of the day and the past few nights page loads in the wee hours have been slow, I'm sure because of the continued shenanigans of the above-mentioned parties.  Hopefully it will all get sorted in short order...
Kevin
AD5VG
Blitzortung station 1042

Offline moehoward4

  • Forecaster
  • *****
  • Posts: 689
  • Jack in SE MA.
Re: Connection Issues
« Reply #11 on: June 11, 2014, 08:19:17 AM »
Real bad for me....since 4:15 am (et) today, better than 60% of the time I've been getting the messages....3 times I had to close my browser and start over.
3 Davis set-ups...which one ya wanna talk about? And I got ALL my manuals....

Offline George Richardson

  • WxElement panel
  • Forecaster
  • *****
  • Posts: 1391
    • Smith Mountain Lake Weather
Re: Connection Issues
« Reply #12 on: June 11, 2014, 08:59:46 AM »
In as much as I know absolutely nothing about running a forum, I have no suggestion as to how to improve the present situation. I will observe that if I were a new lurker, I would not come back. Hope you smart guys are able to fix this.

George

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: Connection Issues
« Reply #13 on: June 11, 2014, 12:17:55 PM »
The problem is continuing this morning. Lots of connection failures but, most annoying is the length of time it takes to get to a page, when it finally comes up. I am seeing load times as long as 45 seconds just to get to a desired post. Many times that results in a connection error while I'm waiting...

All of my other websites I go to load quickly as is normal. Only WXFORUM.NET is problematic.

Ken - I'm not complaining about you, just venting my frustrations...
« Last Edit: June 11, 2014, 12:20:31 PM by W3DRM »
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline JohnN

  • Forecaster
  • *****
  • Posts: 348
    • Monmouth County Weather
Re: Connection Issues
« Reply #14 on: June 11, 2014, 12:25:06 PM »
I don't have any suggestions, but I just want to say thanks for all you do to keep this site up and running. I can't imagine what it must be like to have to keep up with the bots and spammers. Hopefully this quiets down soon.
www.monmouthcountyweather.comUK Met Office: 45560463
Twitter: @John_N_WxCoCoRaHS: NJ-MN-64
CWOP: E1682Davis Vantage Vue, Pro2Plus w/24 hr FARS, Ambient Weather WS-1000
PWS/WFY: SOUTHARDNJCumulusMX
Wunderground: KNJHOWEL11
                       KNJHOWEL15
LiveATC: KWRI/KNEL/New York Center
             KMJX/KBLM/3N6/N12

Offline Cutty Sark Sailor

  • WxElement panel
  • Forecaster
  • *****
  • Posts: 3393
    • Frankfort Weather - TwinHollies WeatherCenter
Re: Connection Issues
« Reply #15 on: June 11, 2014, 04:54:28 PM »
It ain't all WxForum:
Problems again with some Time-Warner RR tbones in/out of Chicago, etc... I know it's affecting me again especially on some sites, especially HTTPS. Same blankety blank timeouts and long delays as 2 wks ago.  Usually takes 'em 2-3 days to remedy. Sucks.
Advised my local ISP, and they were just overjoyed, of course. Most times locals haven't been aware, since not everybody is directly affected or notices anything.... at least Blitzotung routing hasn't gone down, completely, like before.  Also some problems in/out of San Jose...
 


Offline cospringswx

  • Forecaster
  • *****
  • Posts: 4136
    • Colorado Springs Weather
Re: Connection Issues
« Reply #16 on: June 11, 2014, 06:22:03 PM »
I'm having trouble here as well and I have Comcast out of CO Springs. I got a message saying too many bots/folks trying to access the site.




Ryan 

Colorado Springs, CO
www.cospringsweather.com
Davis Vantage Vue
Weather Display Software
Amcrest HD IP Camera

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Connection Issues
« Reply #17 on: June 11, 2014, 06:41:53 PM »
We're still having issues.  Today's seem to be a slow website-to-backend-SQL issue, and not a concurrent PHP issue (since I'd ejected the abuser IP addresses yesterday).  My own site (on a different 1and1 server) is experiencing connection/timeout issues too.. likely something overloaded within the 1and1 datacenter.  The WXForum.net server is running a load average < 1.0 (while my other site is running about 3... grrr)

Still working on getting some stability in the accesses.. they should take 1 to 2 seconds for most page views.

Best regards,
Ken
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline SlowModem

  • Weather at the speed of dialup!
  • Forecaster
  • *****
  • Posts: 6641
  • WX @ 26.4 kbs
    • Watts Bar Weather
Re: Connection Issues
« Reply #18 on: June 12, 2014, 07:58:00 AM »
I couldn't connect at all yesterday.  Today is ok so far.
Greg Whitehead
Ten Mile, TN USA

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6364
    • Valentine Nebraska's Real-Time Weather
Re: Connection Issues
« Reply #19 on: June 12, 2014, 08:35:14 AM »
Looking much better so far today, but it is early morning in California. 
Randy

Offline wxtech

  • Weather Equipment Technician
  • Forecaster
  • *****
  • Posts: 1698
  • USAF Weather Equipment Maintenance Tech (retired)
    • Lexington, Ga. Weather
Re: Connection Issues
« Reply #20 on: June 12, 2014, 08:40:34 AM »
Yesterday 6/11 couldn't connect using http://www.wxforum.net/index.php that was in my favorites.  But could connect using http://www.wxforum.net   
Couldn't post anything 6/11.  Working OK now, 6/12 @ 8:30am.
Al Washington, Lexington, Ga.,  NWS Coop station=LXTG1, Fischer Porter, SRG, MMTS. 
CoCoRaHS=GA-OG-1. CWOP=CW2074.  Davis VP2+ WLIP 5.9.2, VP(original) serial, VWS v15.00 p02. ImageSalsa, Win7 & Win8 all-in-one.

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Connection Issues
« Reply #21 on: June 12, 2014, 10:34:17 AM »
Yes, it does seem better this morning.  The backend webserver<>SQLserver connection is much more responsive (very slow yesterday).  Also, the pileup of PHP processes (awaiting backend fulfillment of SQL requests) has abated too.  Avg server load factor fairly steady at < 1.0.

I've also added more IP address ranges to the .htaccess based on frequent appearances in the Ban log and/or the httpBL Spammer log, so that should free up additional resources (those requesters get a 403-Forbidden from Apache versus having the forum PHP software decide to give them that.. less PHP overhead).

For grins, here's a summary of the top 20 blocked this morning
Quote
Count    IP            Name
    158 37.58.100.234   Ahrefs Pte Ltd (Afghanistan)(htaccess)
    128 37.58.100.228   Ahrefs Pte Ltd (Afghanistan)(htaccess)
     81 110.89.37.229   CHINANET FUJIAN (Blocked)(htaccess)
     27 111.79.21.248   CHINANET JIANGXI (Blocked)(htaccess)
     15 95.108.244.252   Yandex LLC(RU) (Blocked)(htaccess)
     15 120.40.145.184   CHINANET FUJIAN (Blocked)(htaccess)
     13 134.249.55.30   Kyivstar GSM (Ukraine) (Blocked)(htaccess)
      9 183.207.228.12   China Mobile Comm Beijing (Blocked)(htaccess)
      7 62.210.146.43   Iliad-Entreprises Business Hosting (FR)(htaccess)
      6 5.248.86.251   Kyivstar GSM (Ukraine)(htaccess)
      6 31.41.219.180   Besthosting (UA)(htaccess)
      5 63.141.239.2   DataShack Hosting (Kansas City, MO)(htaccess)
      5 111.242.206.146   Chunghwa Telecom Taiwan (Blocked)(htaccess)
      4 183.207.228.14   China Mobile Comm Beijing (Blocked)(htaccess)
      4 175.44.59.157   China Unicom Fujian Province (CN)(htaccess)
      4 110.89.35.180   CHINANET FUJIAN (Blocked)(htaccess)
      3 31.6.71.235   LIVENET (PL)(htaccess)
      3 198.50.227.128   OVH Hosting (Blocked)(htaccess)
      3 180.76.5.19   Baidu Beijing (Blocked)(htaccess)
      3 175.44.45.172   China Unicom Fujian Province (CN)(htaccess)
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline jgillett

  • Forecaster
  • *****
  • Posts: 1187
  • Boltek, Win7 Pro, ToA
    • TiggrWeather Phoenix
Re: Connection Issues
« Reply #22 on: June 12, 2014, 03:38:50 PM »
Rogue search engines are not as 'website friendly' and cause congestion.  I've blocked the biggest offenders (Baidu, Yandex, etc), but new ones crop up from time to time.  It's a never-ending battle :)
Have the same problems here with those and many others. Biggest problem is they don't bother looking at robots.txt, and the few that do don't follow the instructions there. That forces a block by IP, and most of them use numerous IPs. Very annoying...
John
W7JKG

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: Connection Issues
« Reply #23 on: June 12, 2014, 11:42:25 PM »
Thankfully, everything seems to be back to the normal fast access and updates on the WXFORUM, we have become accustomed to, once again!

Thanks for all of your hard work Ken!  =D&gt;
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Connection Issues
« Reply #24 on: June 13, 2014, 11:52:59 AM »
It's been a somewhat uphill battle, but the major miscreants are consigned to get 403-Forbidden by Apache instead of tying up PHP threads in the httpBL check (anti-spammer) or the Ban list and that has cut back the load quite a bit.  I also think that 1and1 cleared up a webserver-mySQLserver connection/load issue and that has REALLY helped with responses now back into the normal range (knock wood).

I'm continuing to do log analysis several times a day to look for new 'players' to be smacked down :)

Best regards,
Ken
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP