Question

When I execute the following java program, sometimes I get an empty response, sometimes I get the real (redirected) content.

ByteArrayOutputStream output = new ByteArrayOutputStream();
URL url = new URL( "https://stackoverflow.com/questions/84629" );
IOUtils.copy( url.openStream(), output );
System.out.println( output.toString() );

The URL https://stackoverflow.com/questions/84629 is a redirect to What's your favorite "programmer" cartoon?.

I looked through other SO questions and tried to use the suggested HttpUrlConnection, but the result was the same. The response code is always 200, but sometimes there is the correct html output, sometimes it’s just an empty string.

Can you explain what is happening here?

---

EDIT

Here’s the code without Apache commons:

ByteArrayOutputStream output = new ByteArrayOutputStream();
URL url = new URI( "https://stackoverflow.com/questions/84629" ).toURL();
InputStream openStream = url.openStream();
byte[] buffer = new byte[ 1024 ];
int size = 0;
while( (size = openStream.read( buffer ) ) != -1 ) {
    output.write( buffer, 0, size );
}
System.out.println( output.toString() );

I’m using Windows XP and Java 1.6.0_17.

---

I captured the traffic using wireshark:

GET /questions/84629 HTTP/1.1
User-Agent: Java/1.6.0_17
Host: stackoverflow.com
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive

HTTP/1.1 200 OK
Cache-Control: private
Server: Microsoft-IIS/7.0
Date: Tue, 10 Nov 2009 22:42:42 GMT
Content-Length: 0

Answer 1

I think it’s like @carl-smotricz said: “You’re probably running up against a spam filter or something.”