I am currently optimizing my site for search engines. It is mainly a database driven site. I am using C# on the back end but database content is loaded via jQuery ajax and a web service. Therefore, my database content is not in html at the point that the bots will crawl it. My site is kind of like an online supermarket format in that there are thousands of items in my database, users can load a single one of these or more onto the web page at a time and the page does not change significantly once items are loaded.
My question is, how (if at all) can I get my database contents indexed? I was thinking of having an anchor that links to an aspx page (eg called mydatabase) which loads all of my database items as a big html list. Then, using jQuery, I would make the anchor invisible to users. The data would still be accessible to users but not by this link, it would be accessed by using the jQuery interface I have created.
The thing is, I don’t really want users to see this big, messy list – would google results show this page eg www.mysite.com/mydatabase.aspx as a search result? Also would google see this as “keyword rich” spam page? I have done quite a lot of research but found nothing on this. only instructions for php. Please help I’m not sure what to do and need to know the best way to go about this.
It’s a shame you haven’t taken the progressive enhancement approach as it would mean you would have started with a standard HTML output that’s crawlable, and then adding the layering behaviour (AJAX) on top for the user experience.
Providing a single file (e.g. mydatabase.aspx) that lists all of your products in a list format provides no real value for the reason you gave – it would just be a big useless list. No editorial content relevance for each link etc.
You’re much better off taking another look at your information architecture and trying ensure that each product is accessibile by it’s own unique URL, then classifying the products into groups (result pages), being careful to think about pagination.
You can still make this act like a single-page application using AJAX, but you’d want to look into HTML5’s History API to achieve this in a search engine friendly way.