Question

I am trying to let GoogleBot login to my website and bypass the authentication.
The web site uses devise for its authentication and I was thinking about creating a username for google and then let GoogleBot automatically login to that user.

The way I want to identify GoogleBot is based on its IP addresses.

Is there anyway to link a list of IP addresses to a username and bypass the login process?

Thanks,

Answer 1

Just a warning: Google has a policy that it will not index content that cannot be accessed without a login. There’s a special policy in effect that gets around this limitation called “First Click Free” (see http://googlewebmastercentral.blogspot.com/2008/10/first-click-free-for-web-search.html) used by sites like ExpertExchange. If you violate this policy, google may penalize your site effectively removing it from most searches. Google webmaster central is also a good resource for this stuff, see http://support.google.com/webmasters/bin/answer.py?hl=en&answer=40349, as a starting point.