Web Weather > Weather Web Site Help

How to stop a spider to crawl my site

(1/2) > >>

Axelvold:
How could i get this spider to stop crawling my site?

Agent String: Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/spider.html;) Gecko/2008032620

It does not respect the robots.txt file, and it comes from a different IP every time (15 times today)  :evil:

I have also been in contact with them twice but it still crawls my site.

Bushman:
http://www.kloth.net/internet/bottrap.php

Axelvold:
Thanks Bushman, there was a lot of helpfull information on that site.

I did use this, and it looks that it did work.


--- Code: ---SetEnvIfNoCase User-Agent "^Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/spider.html;) Gecko/2008032620" bad_bot
Order Allow,Deny
Allow from all
Deny from env=bad_bot
--- End code ---

Bushman:
Bad bot!! No.  bad bot!!!  :)  Glad it work for you.

info:
Axelvold

very nice site you have

Navigation

[0] Message Index

[#] Next page