Question

i’m trying to get an entire WebPage through a URLConnection.

What’s the most efficient way to do this?

I’m doing this already:

URL url = new URL("http://www.google.com/");
URLConnection connection;
connection = url.openConnection();
InputStream in = connection.getInputStream();        
BufferedReader bf = new BufferedReader(new InputStreamReader(in));
StringBuffer html = new StringBuffer();
String line = bf.readLine();
while(line!=null){
    html.append(line);
    line = bf.readLine();
}
bf.close();

html has the entire HTML page.

Answer 1

Your approach looks pretty good, however you can make it somewhat more efficient by avoiding the creation of intermediate String objects for each line.

The way to do this is to read directly into a temporary char[] buffer.

Here is a slightly modified version of your code that does this (minus all the error checking, exception handling etc. for clarity):

        URL url = new URL("http://www.google.com/");
        URLConnection connection;
        connection = url.openConnection();
        InputStream in = connection.getInputStream();        
        BufferedReader bf = new BufferedReader(new InputStreamReader(in));
        StringBuffer html = new StringBuffer();

        char[] charBuffer = new char[4096];
        int count=0;

        do {
            count=bf.read(charBuffer, 0, 4096);
            if (count>=0) html.append(charBuffer,0,count);
        } while (count>0);
        bf.close();

For even more performance, you can of course do little extra things like pre-allocating the character array and StringBuffer if this code is going to be called frequently.