Question

So lets say I’m scraping multiple pages (lets say a 1000) on a website. I want to know which language is best to use to scrape those pages with – javascript or python.

Further, I’ve heard about javascript scrapers being faster (due to multiple get requests), but am unsure how to implement this – can anyone enlighten me?

Thanks!

Answer 1

This is just my opinion but I would rank them like this

• javascript might be the best choice but only if you have a node
  environment already set up. The advantage of javascript scrapers is
  they can interpret the js in the pages you’re scraping.
• next is a three way tie between perl python and ruby. They all have a mechanize library and do xpath and regex in a sensible way.
• Down at the bottom is php. It’s lack of a cookie handling library like mechanize (curl isn’t great) and it’s clumsy dom and regex functions make it a poor choice for scraping.