I have some content that i want to prevent Google Indexing because its duplicated across a few sites.
I understand I can use a robots.txt file to do this but the content doesn’t exist on its on URL.
Let me explain the content:
When a user clicks on a button jquery.load() will load a DIV with this content that is brought to the front of the page (z-index) kind of like a popup. The main 2 parts are a Join and Help section.
Is there a way for me to prevent indexing?
Its kind of like preventing indexing of a DIV or prevent indexing on a button click.
The content is not on the page to start with…
Its in a seperate PHP file which jquery requests if the user clicks on a link/button.
Once that button/link is clicked its loaded into the DIV that resides in the DOM.
Most search engines do not execute javascript while indexing. So your lazy loaded content will not be indexed. It’s a different story if your div is present in the html code and will become visible later.
As Stan pointed out (see comments) the Google search indexer can execute javascript. I was however unable to find proper documentation or an official statement by Google (Please post a comment if someone should find one). Because it is not properly documented and Stan’s sources say that some js / AJAX content will be indexed i can only recommend not to rely on the indexing of dynamic content. There is no way to predict or verify the outcome.
If you want your content to be properly indexed the right (compatible) way to do this would be:
By having a separate pages for each content that you want to index you have very granular control over what is indexed and what is not.