Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I am using Nokogiri to screen scrape a few websites.
My website is hosted on US servers, and so when it fetches the website, the website responses as if the user was a US user. I need the website to responds as if I was an Australian user, even though the server is located in the US.
When running locally it works fine because locally it responds as if it was Australian.
How can I read the site using Nokogiri as if I was from another country?
You have to run your requests through a proxy in Australia.
This doesn’t have anything to do with Nokogiri – it applies no matter how you’re trying to scrape a page. HTTP travels over TCP, which is a bi-directional protocol so you can’t spoof its IP address. If you tried to spoof the IP address of a TCP packet, you would never get your response back.