basically, I am trying to scrape webpages with php but I want to do so after the initial javascript on a page executes – I want access to the DOM after initial ajax requests, etc… is there any way to do this?
Share
Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
basically, I am trying to scrape webpages with php but I want to do so after the initial javascript on a page executes – I want access to the DOM after initial ajax requests, etc… is there any way to do this?
Short answer: no.
Scraping a site gives you whatever the server responds with to the HTTP request that you make (from which the “initial” state of the DOM tree is derived, if that content is HTML). It cannot take into account the “current” state of the DOM after it has been modified by Javascript.