Question

I’m using Apache Commons HttpClient to grab some data from a server. My problem is that the returned XML data is always truncated to the first 64k. I was hoping this might just be a case of setting a limit on the relevant object, but apparently not – or at least, I can’t find any information about such a method. I’m assuming it’s a client issue as the server belongs to another company and presumably serves the data fine to everyone else.

Any ideas?

My code btw is super simple:

protected String send(Server server, String query) throws Exception {
    PostMethod post = new PostMethod(server.getUrl());
    post.setParameter("XMLString", query);
    try {
        client.executeMethod(post);
       return post.getResponseBodyAsString();
    } finally {
        post.releaseConnection();
    }
}

fyi, same thing happens with the following code using InputStreams rather than the getResponseBodyAsString() method.

protected String send(Server server, String query) throws Exception {
    PostMethod post = new PostMethod(server.getUrl());
    post.setParameter("XMLString", query);
    try {
        client.executeMethod(post);
        InputStream stream = post.getResponseBodyAsStream();
        BufferedReader reader = new BufferedReader(new InputStreamReader(stream));
        String line = null;
        StringBuilder sb = new StringBuilder();
        while ( (line=reader.readLine()) != null ) {
            sb.append(line);
        }
        return sb.toString();
    } finally {
        post.releaseConnection();
    }
}

So, do you know if there is a limit set somewhere? The fact that the String is response is always 64k does seem to suggest there must be. But where?!

Thanks

Alastair

Answer 1

ERRATUM: the 64k limit is false. It as in artefact of the eclipse debugger which won’t show appears the be the truncating culprit.

SOLUTION: I have now got the code working as follows:

PostMethod post = new PostMethod(server.getUrl());
    post.setParameter("XMLString", query);
    post.setRequestContentLength(PostMethod.CONTENT_LENGTH_AUTO);
    post.setStrictMode(true);
    try {
        client.executeMethod(post);
        return post.getResponseBodyAsString();
    } finally {
        post.releaseConnection();
    }

I’ll admit I don’t know why that works, but speculate it’s something to do with the content length settings.

Separately, be aware that these methods are deprecated and I’m only using them because I’m working on a legacy system. If you are working on a more adaptable system I’d suggest following @HellGhost’s advice.