I am working on a project in which i have to search for terms on a search engine and then cluster the results on their contextual sense. So i have to treat each result as a document. unfortunately, the data present along with each result on the result page is too little for clustering. Hence, I wanted to know from where the search engines get the abstract for each result that they show. If i could get that entire abstract then i can cluster the results by treating them as separate documents.
From where does google get the abstract ?
For eg: If you search for “1000 Mile” on google, the second result shows the following abstract:
“The women’s 1000 Mile Collection is based on classic designs and reflects Wolverine’s long heritage of crafting quality footwear. Complementing these classics …”
This abstract is not present in the Meta tags of the page.
From where does Google find this data.
Thanks
That site is Flash-based, and Google can index Flash content, so given that the snippet isn’t in the HTML source of the page as you point out, nor is it in the cached version of the page, I’m guessing that it’s somewhere in the Flash movie.
It’s kind of arbitrary that the snippet mentions ‘The women’s 1000 Mile Collection’ while the site link itself is to the parent category of 1000 mile, not just women’s, so I’m guessing here that gathering snippet-friendly metadata from a Flash site is an imprecise science. That’s my best guess.
In this Google Webmaster blog post, they explain how they use external text or HTML files loaded into the Flash movie, and in one of the comments Jonathan Simon says (sorry):
“We try our best to crawl Flash content but the results can sometimes be less than ideal. You are only seeing a title in the search results for your site because that’s the only bit of HTML text that you have outside of your Flash content. You could add a Meta description element to offer more information in HTML. You could also add some other text that’s not a part of your Flash content. Just doing this should improve the snippet you see associated with your site in the search results.”