How to block a bot by User Agent Sting

How to block a bot by User Agent Sting

Do you have those bandwidth hogging bots as much as Phil and I do? Did you know you can block them in your .htaccess file?

Block the BOT

Let block the most annoying bot on the internet - Baidu spider

vim .htaccess

Now Add the following to block

RewriteCond %{HTTP_USER_AGENT} Baiduspider [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} Baidu [NC] 
RewriteRule ^.*$ - [F,L]

Thats it save the file, and you are now blocking the Baidu spider

Testing to see if its blocked

One way to do this is to use curl

curl -I http://www.briansnelson.com -A "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"

Now you will get a 403 Forbidden

HTTP/1.1 403 Forbidden
Date: Mon, 06 Jan 2014 19:18:11 GMT
 
 
 
 
  • 0 Users Found This Useful
Was this answer helpful?

Related Articles

OpenVPN Tutorial For Debian/Ubuntu on OpenVZ

What is OpenVPN?OpenVPN is a reliable and well tested VPN solution that is able to run over a...

How do I SSH into my VPS? (OS X/Terminal)

If you are on a Mac, SSH-ing into your VPS is very simple. Simply launch "Terminal" by going to...

How do I SSH into my VPS? (Windows/Putty)

This article will teach you how to access your server via SSH using the PuTTY client on Windows....

Install Remi and EPEL yum repo Centos 7

Install Yum EPEL and Remi repositories on Centos 7 Adding additional useful repo's on Centos 6...

Install PHP/PHP-FPM 5.4 Centos 7

Install PHP/PHP-FPM 5.4 on Centos 7 PHP is a server-side scripting language designed for web...