I’m developing an ecommerce search engine that allows you to search for products in a lot of ecommerce websites.
How do I approach the matter?
I need an application that will be able to scan websites, parse their HTML and determine which of the images in the website are product images, which are product descriptions, which are product prices.
Would be happy to hear any idea, example.
Thanks in advance.
edit: My question is not how to get the HTML from the websites(which is called screen scraping) but more on how to parse that information and understand which of the html contains the actual data i am looking for, and which is not.
You may find this thread helpful in your quest. I had outlined the basic steps there. Here’s the link to all questions tagged as ‘Screen-scraping‘ on SO. Also, lots of material on the web – Google.