I use jquery templates, and I’ve recently got reports of exceptions as the google bot has arrived. (I know I could just handle the exceptions, but that is not the reason for my question.)
In my app I’ve got links like this:
http://www.example.com/?widget=${widget.id}
- client side that is parsed as a Jquery template at run time and obviously ${widget.id} is rendered.
The google bot is hitting my source though, where the link does look like the above, attempting to follow it, and that is not correct.
What is the best mechanism to solve this? If it is a href is nofollow sufficient? Or is it something like robots.txt? I don’t really need any indexing beyond / –> it’s a single page app. Or is there some cleverness I don’t know about to allow google bot to fetch these links accurately?
If you’re only talking about the google bot you can change the behavior of the parser regarding parameters in your urls via the google webmaster tools.
To prevent all bots from indexing the url you can block it in the robots.txt (the wrong one)
And to enable bots to parse it correctly you can add it to a sitemap.xml (the correct ones)