Question

I need help in .httaccess rewrite rule. One of my client says “A website abc.com should send traffic (with 301 redirect) from google, bing and yahoo search engigne bots only to cba.com otherwise show index.html (its a white page) i.e if Google robot is crawling abc.com it should see a redirect to bcd.com instead of the real content”.

is this possible like this?

RewriteEngine On 
RewriteCond %{HTTP_USER_AGENT} Googlebot [OR]
RewriteCond %{HTTP_USER_AGENT} msnbot [OR]
RewriteCond %{HTTP_USER_AGENT} Slurp
RewriteRule ^(.*)$ http://bcd.com/$1 [L,R=301]

Thanks

Answer 1

You can get a list of user agents here: http://www.user-agents.org/

But what you have should work. You could add an additional check for the hostname:

RewriteEngine On 
RewriteCond %{HTTP_HOST} abc.com$ [NC]
RewriteCond %{HTTP_USER_AGENT} Googlebot [OR]
RewriteCond %{HTTP_USER_AGENT} msnbot [OR]
RewriteCond %{HTTP_USER_AGENT} Slurp
RewriteRule ^(.*)$ http://bcd.com/$1 [L,R=301]

Those rules work for me in a blank htaccess file. This is my request:

GET /something HTTP/1.1
Host: abc.com
User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)  

And this is apache’s response:

HTTP/1.1 301 Moved Permanently
Date: Mon, 15 Oct 2012 22:10:55 GMT
Server: Apache
Location: http://bcd.com/something
Content-Length: 289
Content-Type: text/html; charset=iso-8859-1