Question

My code goes like this:

URL url;
URLConnection uc;
StringBuilder parsedContentFromUrl = new StringBuilder();
String urlString="http://www.example.com/content/w2e4dhy3kxya1v0d/";
System.out.println("Getting content for URl : " + urlString);
url = new URL(urlString);
uc = url.openConnection();
uc.connect();
uc.getInputStream();
BufferedInputStream in = new BufferedInputStream(uc.getInputStream());
int ch;
while ((ch = in.read()) != -1) {
    parsedContentFromUrl.append((char) ch);
}
System.out.println(parsedContentFromUrl);

However when I am trying to access the URL through browser there is no problem , but when I try to access it through a java program, it throws expection:

java.io.IOException: Server returned HTTP response code: 403 for URL

What is the solution?

Answer 1

Add the code below in between uc.connect(); and uc.getInputStream();:

uc = url.openConnection();
uc.addRequestProperty("User-Agent", 
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)");

However, it a nice idea to just allow certain types of user agents. This will keep your website safe and bandwidth usage low.

Some possible bad ‘User Agents’ you might want to block from your server depending if you don’t want people leeching your content and bandwidth. But, user agent can be spoofed as you can see in my example above.