As spiders will generally not execute javascript i am thinking of taking one of the options below in order to successfully get them to index the content of a website that requires age verification.
My preferred solution:
Checking for a cookie ‘ageverification‘. If it does not exist, add some javascript to
redirect the user to ~/verifyage.aspx which will add the required cookie and redirect the user to their previous page.
Another solution:
As above, but do not redirect the user. Instead, if the cookie doesnt exist, draw the age verification form ‘over the top’ of the existing page.
Another solution:
Add a ‘Yes I am over 18’ anchor link that a crawler can follow. I am slightly skeptical over the legality of this.
Any insight or ideas greatly appreciated.
What I do – I store age verification in session data. If the session variable does not exist, the server appends a div to the end of the body (after the footer) with the click to verify or click to exit. I use CSS to have it cover the content.
For the css – I use:
That causes the div the cover all other content in a graphical browser even though it is placed at the very end of the body.
For users without JS enabled, the “Enter” link points to a web page that sets the session variable and returns the user to the page they requested. That results in two page loads of the browser for them to get to the content they want which is not ideal, but it is the only way to do it for non JS enabled browsers.
For JS enabled browsers, a small JavaScript is attached to the page that will change the “Enter” link href link to # and attach a very basic function to the click event, so that clicking on Enter triggers the use XMLHttpRequest to tell the server the person clicked “Enter”. The server then updates the session and responds to the XMLHttpRequest with a 200 OK response, triggering the JavaScript to hide the age verification div covering the content. Thus the session is updated so the server knows the user verified the age and the user gets to see the content they wanted with no page reloading in the browser, a much better user experience.
The age verification thus works without JavaScript by sending the user to the verify page the stateless way or in a much friendlier way with JavaScript.
When a search spider crawls the site, it gets the age verification div on every page because a spider will not have the necessary session variable set, but since the div is at the very end of the html body the spider still indexes the real content first.