Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I am using RCurl in R to try and download data from a website, but I’m having trouble finding out what URL to use. Here is the site:
http://www.invescopowershares.com/products/holdings.aspx?ticker=PGX
See how in the upper right, above the displayed sheet, there’s a link to download the data as a .csv file? I was wondering if there was a way to find a regular HTTP address for that .csv file, because RCurl can’t handle the Javascript commands.
Clicking on the Download link executes this piece of JavaScript:
That
__doPostBackfunction appears to simply fill in a couple of hidden form fields on that page then submit a POST request.A quick googling shows that RCurl is capable of submitting a POST request. So, what you would need to do is look in the source of that page, find the form with name “aspnetForm”, take all the fields from that form, and create your own POST request that submits the fields to the action URL (http://www.invescopowershares.com/products/holdings.aspx?ticker=PGX).
Can’t guarantee this will work, though. There appears to be a hidden form field named
__VIEWSTATEthat appears to encode some information, and I don’t know how this factors in.