Question

I know how to load a page from another website and analyze it but the website I’m trying to load some pages from, doesn’t let unregistered users to visit those pages. I do have a username and a password to load those pages normally in my browser, but I’m wondering if I could do it in PHP or not? :/
I’m not sure what information I should give you about the website but if what I already told is incomplete just ask what information I should give.

Thanks.

Answer 1

Most websites use cookies to store information related to your authentication status.
To get past this programatically, you’ll have to send this information everytime you make a request. Here is how you can get it –

• Using a tool like firebug, inspect the cookies the site sends when you login manually.
• Write code to capture this information and send this cookie with subsequent requests

Note: Do check the ToS of the site you are trying to scrape. Some sites do not permit you to scrape or use their content without prior permission