How to block a bot by User Agent Sting

How to block a bot by User Agent Sting

Do you have those bandwidth hogging bots as much as Phil and I do? Did you know you can block them in your .htaccess file?

Block the BOT

Let block the most annoying bot on the internet - Baidu spider

vim .htaccess

Now Add the following to block

RewriteCond %{HTTP_USER_AGENT} Baiduspider [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} Baidu [NC] 
RewriteRule ^.*$ - [F,L]

Thats it save the file, and you are now blocking the Baidu spider

Testing to see if its blocked

One way to do this is to use curl

curl -I http://www.briansnelson.com -A "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"

Now you will get a 403 Forbidden

HTTP/1.1 403 Forbidden
Date: Mon, 06 Jan 2014 19:18:11 GMT
 
 
 
 
  • 0 Users Found This Useful
Was this answer helpful?

Related Articles

CentOS One Liner Commands

Centos Oneliners kill all pts for x in $(ps aux | grep pts| awk '{print $2}'); do kill...

How do I reset my root password?

To reset your VPS root password, enter the VPS management panel at...

How to block Bittorent traffic with IPTables

How to Block Bittorrent Traffic with IPtables IPTABLES is a user-space application program...

APF Firewall -Advanced Policy Firewall - Basics

APF Firewall -Advanced Policy Firewall - Basics Advanced Policy Firewall, or APF, is a...

How to extract a tar.gz file

So you have went to that website and downloaded the latest version of your files. But they are in...