Question

I’m having trouble receiving “large” files from a web server using C sockets; namely when these files (or so I suspect) are larger than the size of the buffer I’m using to receive them. If I attempt to ask (through a GET request) for a simple index.html that’s not bigger than a few bytes, I get it fine, but anything else fails. I’m assuming that my lack of knowledge on select() or recv() is what’s failing me. See here:

fd_set read_fd_set;
FD_ZERO(&read_fd_set);
FD_SET((unsigned int)socketId, &read_fd_set);

/* Initialize the timeout data structure. */
struct timeval timeout;
timeout.tv_sec = 2;
timeout.tv_usec = 0;

// Receives reply from the server
int headerReceived = 0;
do {
    select(socketId+1, &read_fd_set, NULL, NULL, &timeout);

    if (!(FD_ISSET(socketId, &read_fd_set))) {
       break;
    }

    byteSize = recv(socketId, buffer, sizeof buffer, 0);

    if (byteSize == 0 || (byteSize < BUFFER_SIZE && headerReceived)) {
       break;
    }

    headerReceived = 1;

} while(1);

That’s right, after sending the GET request to the web server, which I’m pretty sure the server is getting just fine, and GET requests from any other client (like any web browser) are working as intended.

Thanks in advance, any help is greatly appreciated.

Answer 1

One thing that I spot is that you don’t reinitialize the selection during the loop. This is probably why you get small files successfully; they are received in one go and the loop doesn’t have to be iterated.

I suggest you put the:

FD_ZERO(&read_fd_set);
FD_SET((unsigned int)socketId, &read_fd_set);
timeout.tv_sec = 2;
timeout.tv_usec = 0;

inside the loop (before you invoke select), and it might just work.