Home/ Questions/Q 6009985
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-23T02:03:02+00:00

I need to do lot html parsing / scraping /search engine /crawling. There are

  • 0

I need to do lot html parsing / scraping /search engine /crawling.

There are many libraries currently like Scrapy, Beautiful Soup, lxml , lxml2 requests, pyquery.

Now i don’t want to try each of these and then decide. basically i want to follow on one and then study in detail and then use that most often.

So which library should i go for which can perform all function mentioned above. Even though there may be diff solutions for diff problems. But i want onelibrary which could do all things even though it takes time to code but should be possible

Is it possible to do indexing in lxml? Is PyQuery same as lxml or its different?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I’m using Beautiful Soup and am very happy with it. So far it answered all my scraping needs. Two main benefits:

    • It’s pretty good at handling non-perfect HTML. Since browsers are quite lax, many HTML documents aren’t 100% well-formed
    • In addition to high-level access APIs, it has low-level APIs which make it extendible if some specific scraping need isn’t directly provided
    • 0