Author Topic: Grrr... 1and1 'auto-block' has blocked my home IP address  (Read 18946 times)

0 Members and 1 Guest are viewing this topic.

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 6746
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Grrr... 1and1 'auto-block' has blocked my home IP address
« on: March 27, 2012, 12:54:22 PM »
In the topic 'a little automation is a dangerous thing', I file this under major annoyances:

On 26-Mar-2012 after 11:28pm, apparently the 1and1 traffic watchdog script(s) deemed that accesses from my Comcast IP address were worthy of blacklisting, so all access to my sites (HTTP, FTP, SSH) were blocked.  So.. no weather station uploads for conditions have happened since that time.

I called tech support at 9:20 describing the issue (no HTTP/FTP/SSH access from my IP address to any of my websites at 1and1).
At 9:40, they confirmed that the IP address was blocked with no exact description of the cause, just allusions to 'too much traffic, invalid attempts, etc.'   I explained that this Comcast IP address has been used for weather station upload purposes for many years with the same traffic profile (which is well within the non-limits of 'Unlimited' service) and requested the address be whitelisted instead so I could again have access from my home to my websites.   'Should be restored in 4hrs or less' was the response.

Grrr....   :twisted: ](*,)
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 6746
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #1 on: March 27, 2012, 02:37:55 PM »
And... my home IP address was 'whitelisted' at 11:10am on 27-Mar-2012 and data from my station began flowing to the website again.  What a pain!

Note for connectivity debuggers -- if your site is inaccessable from your normal (home) IP address, try using your cell-phone (WiFi OFF) or another IP address to access your site .. if that works, then it's likely a block has been put on your home IP address, and it's time to call hoster tech support for resolution.

BTW.. my FTP internet traffic hasn't changed dramatically.  I'm still doing about 1.7 to 2.4Gb/day in FTP uploads to the site.
Larger than most, I think, due to having 6 weather software, 1 radar, 2 WASP2 instances all uploading.  I do use Fling to handle WeatherLink, Cumulus, VWS, Meteohub, WeatherCat, WeatherSnoop uploads, with only GRLevel3, WASP2 (2 instances), Weather-Display doing uploads.  Hey... I've got templates to test :)

Ken
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline neondesert

  • Forecaster
  • *****
  • Posts: 628
    • http://www.neondesertweather.com
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #2 on: March 27, 2012, 03:02:01 PM »
Ouch, that's a long time to be down!

Did you have any FTP clients uploading an incorrect password when this happened?  I know I've done this myself while configuring
FTP software only to find out that I was hammering the server with invalid credentials and was subsequently blocked. 
Fortunately, a quick ticket sent to support resolves the issue in about 30min.

Glad to see you're up and running now.  8-)
Larry
"But it's a DRY Heat!"


Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3314
    • Carson Valley Weather
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #3 on: March 27, 2012, 03:15:11 PM »
Ken, I can sympathize with you. I've had the same thing happen to me twice with 1&1 suddenly, and without any notice at all, shutting me down. Both times it took almost a full day to get everything back up and running again. I too had been blacklisted due to "too frequent uploads" for my weather data. Since then I dropped the upload frequency and haven't had any further problems. I also had the same problem with ICDSOFT when I was using them for hosting so it's not just 1&1.

The support folks I talked with said they had no contact with the group who initiates the blacklist activity and could only send them an email to request resetting the block. Thus, the reason for the delay in getting up and running again.

My biggest complaint is that they don't let you know they blocked you. It's the least they could do for their customers.

Don - W3DRM - Minden, Nevada --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KRNO2
Davis Wireless VP2, WD 10.37s80,
StartWatch, VirtualVP, VPLive, , Win10 Pro
--- Logitech HD Pro C920 webcam
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 6746
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #4 on: March 27, 2012, 03:28:56 PM »
Ouch, that's a long time to be down!

Did you have any FTP clients uploading an incorrect password when this happened?  I know I've done this myself while configuring
FTP software only to find out that I was hammering the server with invalid credentials and was subsequently blocked. 
Fortunately, a quick ticket sent to support resolves the issue in about 30min.

Glad to see you're up and running now.  8-)
Hi Larry,
No, AFAIK, no changes to the software with different(invalid) credentials during the time.  I'm downloading and analyzing the current FTP log for credentials errors .. no real difference in the traffic patterns of upload of 1.7 to 2.3Gb/day for the last month. I think they may have tried a new filter on the auto-block and it just didn't like the daily volume so blacklisted it.  But.. with 1and1, you never really get a definitive answer from the security folks.  :(

Ken, I can sympathize with you. I've had the same thing happen to me twice with 1&1 suddenly, and without any notice at all, shutting me down. Both times it took almost a full day to get everything back up and running again. I too had been blacklisted due to "too frequent uploads" for my weather data. Since then I dropped the upload frequency and haven't had any further problems. I also had the same problem with ICDSOFT when I was using them for hosting so it's not just 1&1.

The support folks I talked with said they had no contact with the group who initiates the blacklist activity and could only send them an email to request resetting the block. Thus, the reason for the delay in getting up and running again.

My biggest complaint is that they don't let you know they blocked you. It's the least they could do for their customers.


Hi Don,

Yes, very annoying!  If only their actions would take place after examination of the history, then flag it if very different (instead of just 'volume').  I can understand why they don't notify you as there's nothing to connect my home IP address with my account email on their servers .. they could (maybe) send email's to the email address (contact form) on the three sites I host at 1and1, but that's asking for 'human' intervention in what is likely a script run by automation for security.

Best regards,
Ken
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3314
    • Carson Valley Weather
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #5 on: March 27, 2012, 03:44:44 PM »
Ken, they should hire us to write their specs. That would solve the whole problem :lol: \:D/

I agree, it's probably just a new algorithm or a perhaps new person that has caused the problem. I have always wondered why the support techs don't have direct access to the other groups who activated the block. Maybe they're in a different country...

Don - W3DRM - Minden, Nevada --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KRNO2
Davis Wireless VP2, WD 10.37s80,
StartWatch, VirtualVP, VPLive, , Win10 Pro
--- Logitech HD Pro C920 webcam
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 6746
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #6 on: March 27, 2012, 04:05:26 PM »
I think they have 'internal firewalls' to prevent the 1st level responders from directly communicating with the server admins and or security admins except through passing data via internal trouble tickets.  I strongly suspect (based on the accents), the 1st level tech support is based offshore and the server admins/security admins are likely based at the actual datacenters (but don't respond except to internal trouble tickets).
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline Cienega32

  • Forecaster
  • *****
  • Posts: 2571
    • East Mesa Weather
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #7 on: March 30, 2012, 09:51:48 PM »
1&1 with ComCast ISP providing the data and I don't think I noticed anything "stalled" on my site in that time frame. I'm on the road but I usually check the data times on the site to see if I still have a presence. Didn't notice anything odd but I don't send as much data per day.


Pat ~ Davis VP2 6153-Weatherlink-Weather Display-StartWatch-VirtualVP-Win7 Pro-64bit
www.LasCruces-Weather.com   www.EastMesaWeather.com

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 6746
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #8 on: March 30, 2012, 10:27:18 PM »
AFAIK, it was only my home IP address that was blocked.  I did get a (mostly nonspecific) reply from tech support saying
Quote
Dear Ken True, (Customer ID: ********)
 
Thank you for contacting us.
 
We have verified that your IP address was blocked by our firewall.  This is due
to one or several reasons listed below.
 
1. Too many failed authentication attempts on the server.
 
2. Too many attempts to log in without providing the password information.
 
3. Too many connections to the server.
 
Please try to reconnect 4 hours after your last connection attempt. To prevent
this from reoccurring in the future, we strongly recommend to adjust your
settings to less than 5 concurrent connections.
 
If you have any further questions please do not hesitate to contact us.
Well, (1) and (2) were not caused from my home systems (since no change in FTP setup/credentials was done and they had been working fine).

I suspect (3) may have been the cause .. they've not said how many concurrent FTP connections you can have, but here are the programs with direct FTP usage:

Weather-Display (1 constant connection for clientrawrealtimeftp with 5 second uploads, 1 connection every 5 minutes for main FTP upload)
Cumulus (1 connection every 5 minutes for main FTP, 1 realtime.txt update every 10 seconds)
GRLevel3 (1 connection every 10 minutes)
WASP2   (1 connection every 5 minutes)
Fling (1 connection constantly - handles VWS, Meteohub, WeatherLink, WeatherCat, WeatherSnoop file uploads)

so I see how I could have 8 connections from weather software, and maybe one from me via Dreamweaver to update the site.  Doesn't sound to onerous on their FTP server.

I'm setting up a local FTP server to be the target for WASP2 uploads, then add the local folders to the Fling list of sites.  That would reduce concurrent connections by 2.  I tried it with GRLevel3, but it really slowed down the Fling processing, so best to let GRLevel3 handle the FTP upload directly.
« Last Edit: March 30, 2012, 10:36:18 PM by saratogaWX »
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline Cienega32

  • Forecaster
  • *****
  • Posts: 2571
    • East Mesa Weather
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #9 on: March 31, 2012, 02:34:46 AM »
I thought I remember seeing 5 concurrent FTP connects but that may have been with my original basic Home plan. Or it may have been something I overheard in a dream about milkshakes - I really can't remember but it (5) sounds familiar from somewhere.

I sure hope you get it squared away without any major issues.

Pat ~ Davis VP2 6153-Weatherlink-Weather Display-StartWatch-VirtualVP-Win7 Pro-64bit
www.LasCruces-Weather.com   www.EastMesaWeather.com

Offline Johnmac

  • Contributor
  • ***
  • Posts: 127
    • Westminster Massachusetts Weather
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #10 on: May 19, 2012, 10:58:28 AM »
I realize that this is an older thread, but I am having the same issue with justhost.com after they moved me over to a new server. I am using Fling to upload my data at 5 minute intervals. The error message that I get is that the password is no good, but I know it is correct as the data has been uploading just fine.

They whitelisted my IP, but that does not do any good, if you have another (what they call an attack from the IP) it blocks your IP again. After they reset it, Fling will work for 2 or 3 days then it is denied again. I have changed my upload times from 5 minutes to 10 minutes after this latest block. Will see if that helps.

Is it possible that Fling could send the wrong password after a few days of operation? I rebooted today, lets see if that helps. I did read elsewhere that Fling had some leaks or handle issues. This problem may have started after updating Fling, but I am not sure.

John
Westminster, MA USA

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 6746
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #11 on: May 19, 2012, 12:55:31 PM »
I think that more hosters are now checking for concurrent connection limits like 1and1 is doing, so this may become a more common problem, especially for enthusiasts who run multiple weather software packages (like me).

I use Weather-Display (with some stuff from WeatherLink) to run my main site, but also have multiple weather software running (mostly for testing with the template sets).  So my main site has:

Weather-Display (1 connection for clientrawrealtime uploads, 1 connection for main WD ftp)
GRLevel3 (1 connection)
WASP2 (two instances, 1 connection each)
Cumulus (1 connection for realtime FTP, 1 for main files FTP)

Fling supports (with 1 connection) uploads from WeatherLink, VWS (including wflash realtimes), MeteoHub, WeatherSnoop, WeatherCat as needed.

The issue with Fling was a steady consumption of non-pagable memory which used to cause a system crash every 3 days or so.  I couldn't find a fix from the vendor, so I did a small Perl script to run via Windows Scheduler to check the memory for the Fling task every 5 minutes, and kill the Fling system task when it exceeded 16Mb.  It auto restarts after the 'kill' so no data is lost, but the non-pagable memory is freed up by the kill :)   Now my system stays up until reboot after the Microsoft 'black tuesday' patches are installed each month.

Don't know if this will help your issue, but I offer the script I'm using for Fling kill.

You'll need ActiveState's Perl (http://www.activestate.com/activeperl/downloads ) installed,
Microsoft's pslist and pskill commands from the PsTools set (http://technet.microsoft.com/en-us/sysinternals/bb896649 )
And to set up a Scheduled Task to run the zap-fling.pl at your desired interval.

zap-fling.pl:
Code: [Select]
#!/usr/bin/perl
#
# use the pstools to find the current NP memory for fling, and optionally
# use pskill to zap it to release the held memory
#
# K. True - 30-Oct-2011
#
# Run via Windows Scheduler or Linux cron .. all output is to the log file(s)
#
# ---- configurable settings ----
$maxMem = 16384;  # amount in KB that we'll tolerate

$logsDir = "./fling-logs/"; # place to store the YYYYMMDD.txt

# ---- end of configurable settings --
#
$|=1; # no buffering of output

 $cur_time = time();
 @months = ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec");

 $log_filename = ISO_datestamp($cur_time) . ".txt";
 $nice_date = nice_datestamp($cur_time);

 open (OUT,">>${logsDir}${log_filename}");

open(PS,"pslist -m fling |") || die "..unable to run pslist $!\n";

while (<PS>)
{
  #print $_;
  $rec = $_;
  ($name,$PID,$VM,$WS,$Priv,$PrivPK,$Faults,$NonP,$Page) = split(/\s+/);
  next unless $Page =~ m/\d+/;
  #print STDOUT "$name\t$PID\t$NonP\n";
  print OUT "$nice_date\t$rec";
  print STDOUT "$nice_date\t$rec";
 
  if($NonP >= $maxMem) {
    print OUT "$nice_date\t$name PID=$PID pskill for $NonP > $maxMem KB\n";
    print STDOUT "$nice_date\t$name PID=$PID pskill for $NonP > $maxMem KB\n";
open(PK,"pskill $PID |") || die "..unable to run pslist $!\n";
while (<PK>) {
   print OUT "$nice_date\t$_";
   print STDOUT "$nice_date\t$_";
 
}
    print OUT "$nice_date\t-------------------------------------------\n";
    print STDOUT "$nice_date\t-------------------------------------------\n";

  }
}

# ------  end of main program -----

sub nice_datestamp {
    my $d = shift;
   
    my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($d);
    my $nicedate = sprintf(
        "%02d\-%s\-%04d %02d\:%02d\:%02d",
        $mday,$months[$mon],$year+1900,$hour,$min,$sec);
    return("$nicedate");

}

sub ISO_datestamp {
    my $d = shift;
   
    my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($d);
    my $nicedate = sprintf("%04d%02d%02d",$year+1900,$mon+1,$mday);
    return("$nicedate");

}

You'll need a ./fling-logs/ directory to store the YYYYMMDD.txt files created by the script which contains the log of the activities so you'll see when fling gets zapped.  I've attached a sample from my system.

Hope this helps...

Best regards,
Ken
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Boltek-PCI/NexStorm, microSferics ToA, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge/hub
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline Johnmac

  • Contributor
  • ***
  • Posts: 127
    • Westminster Massachusetts Weather
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #12 on: May 20, 2012, 10:07:13 PM »
Ken,

Thanks for the suggestions. For now I have added the Fling program to my "Startwatch" program that I have been using for some other programs. It gives the option of killing the Fling.exe file if the memory gets too high. Based on your comments, I set that level at 16mb, but feel that may be too high based on the usage the Startwatch program is reporting. I will watch it and see if the memory starts creeping up. Currently it is less than 2mb.

Thanks for the comments to help solve my problem.
John
Westminster, MA USA

Offline weatherc

  • Senior Contributor
  • ****
  • Posts: 277
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #13 on: June 18, 2012, 05:30:26 PM »
Quote
I think that more hosters are now checking for concurrent connection limits like 1and1 is doing, so this may become a more common problem, especially for enthusiasts who run multiple weather software packages (like me).

It is. I have seen more and more wx-sites kicked out due to this and are slowly being almost a problem for us wx-sites. Not actually for me as i have own server but for those on webhotels. Seems webhosts trys to get more sites to fit on the boxes with tighter limits or something.

When i installed cPanel on the dedicated server i use i started with default settings for FTP. On same server are a handful wxsites. It tookn't long time before the FTP-limit was reached, and my FTP'ers are only WD and NSLog + a scheduled batscript every now and then. Plus then my "own connections" sometimes Filezilla,SSH and browser.
I don't remember what the deafult setting was but below 10 it was and only after set it to near 20 the problem went away.

I think there may also sometimes be some connections what maybe not are closed properly and hanging around for some time what increase the amount of connections.

So, yes, its pretty easy to reach the roof of those if not webhoster has increased them....

Henkka

Offline DaculaWeather

  • It's a Jeep thing... you wouldn't understand.
  • WxElement panel
  • Forecaster
  • *****
  • Posts: 3159
  • SCCA EM #156
    • North Georgia and US Weather
Re: Grrr... 1and1 'auto-block' has blocked my home IP address
« Reply #14 on: June 19, 2012, 06:53:58 AM »
You know I had the problem not long ago when they told me I had 186,000 connections on my server!!! IXWebHosting shut me down for about 6 hours. I had to raise enough hell to get them to turn me back on. And of course when I asked them what I could have done about it, they said nothing. Thanks. So shut me down and not tell me, for something I have no control over. Makes perfect sense.