Home/ Questions/Q 7851865
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-02T19:16:54+00:00

I want to scrape data from a page which loads DOM elements using Ajax

  • 0

I want to scrape data from a page which loads DOM elements using Ajax call.

I have tried with the old solution line PyQt4-based scraping, which loads the DOM after it’s fully loaded, but the problem is that I need to do a POST request and it’s only available for GET.

The new Python module ghost.py has time out issues: when it fetches a large DOM tree it raises a time out exception.

If anyone knows any specific way or tools that can help me to do a POST request and grab the data after fully loaded DOM, that will help me a lot.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You can use Selenium to automate browser and access dom. Selenium has python driver hence you can write code in python to navigate to the page. click buttons and wait for ajax call to complete before you start scrapping.

    • 0