Question

I’m trying to build something akin to Facebook’s “Share” functionality for my website.

I’ve gotten to the point where I can accept a URL, scrape it for meta keywords and suitably get titles/descriptions, but I’m a bit stuck as to the best way to determine ‘likely’ photos the user may want to share.

I currently use the SimpleXMLElement to turn the page into a traversable DOM, and find all the tags, turning them into absolute URLs. After that, I’m not sure how I can go about finding a suitable thumbnail.

Do I download them all, and go by file size? Do I use some sort of heuristic like, “was encountered in the middle of the page”?

Does anyone else have any recommendations, suggestions, or tips?

Answer 1

I wrote something similar a while ago to get images from scraped blog posts. My criteria for choosing an image was something along the lines of getting a list of all images on the page then assigning ‘priority points’:

• Ignore images hosted from a blacklist taken from AdBlocker’s list
• Ignore indirect images, eg linked to from stylesheets or in an IFRAME
• Ignore images under 50 pixels wide or high
• Ignore images which repeat more than once
• Assign priority points to images hosted from a whitelist of hosts (eg photobucket, imageshack.us)
• Assign priority points to the largest 3 images on the page
• Assign priority points to images on the same host
• Assign priority points to images with an ALT tag defined
• Assign priority points to images appearing in a P tag

Then pick the one with the most priority points. It certainly wasn’t foolproof or overly scientific but it got something useful far more often than not.