Home/ Questions/Q 410037
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-12T17:50:57+00:00

I have approx. 40k+ html documents where I need to extract information from. I

  • 0

I have approx. 40k+ html documents where I need to extract information from. I have tried to do so using PHP+Tidy(because most files are not well-formed)+DOMDocument+XPath but it is extremely slow…. I am advised to use regexp but the html files are not marked up semantically (table based layout, with meaning-less tag/classes used everywhere) and I don’t know where i should start…

Just being curious, is using regexp (PHP/Python) faster than using Python’s XPath library? Is Xpath library for Python generally faster than PHP’s counterpart?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    If speed is a requirement have a look at lxml. lxml is a pythonic binding for the libxml2 and libxslt C libraries. Using the C libraries is much faster than any pure php or python version.

    There are some impressive benchmarks from Ian Bicking:

    In Conclusion

    I knew lxml was fast before I started these benchmarks, but I didn’t expect it to be quite this fast.

    Parsing Results:

    Parsing Resutls http://1.2.3.9/bmi/blog.ianbicking.org/wp-content/uploads/images/parsing-results.png

    • 0