Question

I’m trying to download an HTML file from a site. I’m using the following simple method:

 URL url = new URL("here goes the link to the html file");
 BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream()));
 String htmlfile = "";
 String temp;
 while ((temp = br.readLine()) != null) {
       htmlfile+= temp;
 }

The problem is that I get the following String in the htmlfile variable:

The installation of ... requires the acceptance of a cookie by your browser    
software.    The cookie is used to ensure that you and only you are 
able to access information ....

In other words, I need to somewhat enable cookies when opening a stream from the url. Is it possible to achieve this by using URL or do I need a different method? Thanks in advance

Answer 1

You can use addRequestProperty() to set a cookie on a URLConnection object, e.g.

URL url = new URL("here goes the link to the html file");
URLConnection connection = url.openConnection();
connection.addRequestProperty("Cookie", "here goes the cookie");
BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));

If the cookie isn’t known beforehand, but is set by a reply to a previous HTTP request, then one can use getHeaderFields() etc on the URLConnection instance representing the previous HTTP exchange to retrieve the cookie.