Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I have a small webcrawler that sometimes has to crawl twitter and pull out URL’s. I use a modified version of the Webclient class provided in the .net framework.
Normally this works fine, even with shortened URL’s from sites.
However, when following a shortened URL, the webclient times out.
You think they’re filtering certain clients?
How I can fix this or why its happening?
Are you sure you can hit that URL from your network, without going through a proxy?
Does your webclient control follow redirects? You could test this by creating a TinyURL and see if your webclient can browse to it.
If you are going through a proxy in your browser, you’ll need to set it up in the WebClient control.
It should be easy to test if they are filtering clients – set the UserAgent on the Request object to match that of FireFox for example.