Question

This is a pretty esoteric issue that I can’t produce a small test case for, so sorry in advance. But maybe someone has run into something like it previously.

I have code like this (using restify):

server.put("/whatever", function (serverRequest, serverResponse, next) {
    serverRequest.pause();

    serverRequest.on("data", function (chunk) {
        console.log("from outside", chunk);
    });

    doSomeAsyncStuff(function (err) {
        serverRequest.on("data", function (chunk) {
            console.log("from inside", chunk);
        });
        serverRequest.on("end", function () {
            next();
        });

        serverRequest.resume();
    });
});

When I hit this server using CURL, this works great. But when I hit it with XMLHttpRequest, I get one less "from inside" log line than I do "from outside" log lines. It seems one of the data events is getting lost, despite my best efforts to pause ASAP.

---

Here is the CURL command I am using:

curl -X PUT -T file.pdf http://localhost:7070/whatever -v

And here is the XMLHttpRequest code (works in recent versions of Chrome):

var arrayBuffer = fromElsewhere();
var xhr = new XMLHttpRequest();

xhr.open("PUT", "http://localhost:7070/whatever");
xhr.setRequestHeader("Content-Length", arrayBuffer.byteLength);
xhr.setRequestHeader("Content-Type", "application/pdf");
xhr.send(arrayBuffer);

One notable difference is that CURL seems to send Expect: 100-continue before uploading, whereas XMLHttpRequest does not. I tried adding that header manually but of course it didn’t actually do much (i.e. Chrome did not wait for a response, it just sent up all the PDF data along with the original request). Even so, I don’t know why this would effect things.

Answer 1

Somewhat predictably, this didn’t have anything to do with curl vs. XMLHttpRequest, but instead with the fact that serverRequest.pause is only advisory; it doesn’t actually pause right away. That is, it’s pretty much useless.

So presumably in the CURL case the timing was nice enough that pause actually worked as expected, whereas in the XMLHttpRequest case the timing was off, and one of the "from outside" data events managed to slip through the “advisory” pause.

There are apparently various fixes for this, discussed in the thread, but I’m still pretty shaky on this whole streams/buffers universe so I won’t try to recommend one in this answer.

I’ve added a documentation pull request in the hopes nobody else tries to use pause assuming that it actually works.