Question

I’m sending a HTTP request over a TCP socket and I’m getting the headers as response, though the content only contains a question mark. What is that about?

Here is my code:

Socket sock = null;
OutputStream out = null;
InputStream in = null;

try {
    // open socket
    sock = new Socket(this.addr, this.port);

    // get output stream
    out = sock.getOutputStream();

    // create request
    StringBuffer request = new StringBuffer();
    request.append("GET " + this.uri + " HTTP/1.1").append(this.CRLF);
    request.append("Host: " + this.host).append(this.CRLF);
    request.append("Cache-Control: no-cache").append(this.CRLF);
    request.append("Connection: keep-alive").append(this.CRLF);
    request.append("User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11").append(this.CRLF);
    request.append("Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8").append(this.CRLF);
    request.append("Accept-Encoding: gzip,deflate,sdch").append(this.CRLF);
    request.append("Accept-Language: en-GB").append(this.CRLF);
    request.append("Accept-Language: ISO-8859-1,utf-8;q=0.7,*;q=0.3").append(this.CRLF);
    request.append("Pragma: no-cache").append(this.CRLF);
    request.append(this.CRLF);

    // write request per byte for the lulz
    for(int i = 0; i < request.length(); i++) {
        out.write(request.toString().getBytes()[i]);
        System.out.print((char) request.toString().getBytes()[i]);
    }

    out.flush();

    // open inputstream
    in = sock.getInputStream();

    int inbyte;

    // read response per byte for the lulz
    while((inbyte = in.read()) > 0) {
        System.out.print((char) inbyte);
    }

    // close out, in and socket
    out.close();
    in.close();
    sock.close();
} catch (IOException e) {
    e.printStackTrace();
}

You can see my request headers, though here’s the actual output:

GET / HTTP/1.1
Host: www.timseverien.nl
Cache-Control: no-cache
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-GB
Accept-Language: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Pragma: no-cache

And finally, the response:

HTTP/1.1 200 OK
Date: Fri, 13 Jul 2012 07:47:25 GMT
Server: Apache/2
X-Pingback: http://www.timseverien.nl/xmlrpc.php
Vary: Accept-Encoding,User-Agent
Content-Encoding: gzip
Content-Length: 2758
Keep-Alive: timeout=1, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8

?

Why am I getting this question mark instead of the source code?

Answer 1

Content-Encoding: gzip – your response is compressed and cannot be reliably printed to the screen.

Remove Accept-Encoding from your headers and you should receive plain text.

If you want to play with http, start with http/1.0. It’s much easier to handle.