Question

I need a javascript function to automatically login and then scrape some detail from a website. I have the login details saved on a server side database. I need to use these login details and login to a website and then scrape some basic information from there.

I heard that we can do this by dynamic iframing of the urls… and get things done… but I need to know the exact details on how to get this done.

Answer 1

This sounds like a job for a headless browser such as PhantomJS rather than attempting to use CURL and PHP with a javascript front end. It would require you to install some software onto your server though, which is easy, but requires command line access.

PhantomJS is a headless WebKit with JavaScript API. It has fast and
native support for various web standards: DOM handling, CSS selector,
JSON, Canvas, and SVG.

PhantomJS is an optimal solution for fast headless testing, site
scraping, pages capture, SVG renderer, network monitoring and many
other use cases.

I have used it for this very purpose myself before. You can even inject your favourite javascript framework (such as jQuery) into the DOM of the page to make it easier for you to navigate around the elements.