I would like to automatically detect Google and other Crawlers and log them into my ASP.NET website. Has anyone found a reliable way to do this? The Login part is easy, however to reliably detect them is the real issue.
Regards.
Share
Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I would like to automatically detect Google and other Crawlers and log them into my ASP.NET website. Has anyone found a reliable way to do this? The Login part is easy, however to reliably detect them is the real issue.
Regards.
Look at the user-agent. It should be googlebot. A more reliable way is to do a reverse lookup of the IP adddress. It will tell you if it is a google bot or not. I’d use both methods. BUT NOTE: You will slow your site since you will be doing a reverse lookup for every visitor.
http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=80553