I have a website on which I implement infinite scroll: when a user reaches the end of a page, an AJAX call is made and new content is attached to the bottom of the page. This, however, means that all content after the first “page break” is unattainable by search crawlers. For example, I have a page that lists all items with the “infographic” tag. There are actually several dozens of such items, but crawlers are able to see only the first 10, because other items are loaded based on the position of the content relative to browser window. Since crawlers don’t have browser windows, new items are not loaded at all.
What is the proper way, then, to let search crawlers access the full content of web pages with infinite scroll, while also allowing users to enjoy the infinite scroll and the lack of pagination?
Make a View All page
Make another page, with everything listed on it and linking to the items normally that are contained on the page with the infinite scroll. Then place a small link (maybe at the very bottom) of the infinite scroll page called
all whatever. In other words if your page is listing products the link should sayShow All Productsor similar. If the page is blog articles then the link should be something likeAll Articles. Yes humans might not like the long load but for Google it doesn’t matter that it’s large. It will download it and follow the links in it normally.Furthermore if your pagination is iterating through tens of thousands of items then you can break down your ‘View All Page’ in sections similar to how a blog archive works, or a product catalog works. The point is that you are providing an alternative means for humans without javascript and those that really want to see everything and at the same time also for Google and even other search engines to crawl your inventory of pages.
Finally as a secondary measure add a
/sitemap.xmlfile which has an index of every article/product/inventory or whatever. See http://www.sitemaps.org/You can watch an official Google Webmaster video titled, Pagination and SEO, about the view all concept, pagination, canonical urls and Google’s rel=next and rel=prev attributes.
http://www.youtube.com/watch?v=njn8uXTWiGg