Question

Is there a C/C++ way to read data from a socket using read() and having the receiving buffer be a file (ofstream) or a similar self-extending object (vector e.g.)?

EDIT: The question arose while I contemplated how to read a stream socket that may receive the contents of a, say 10000+ byte file. I just never did like putting 20000 or 50000 bytes (large enough for now) on the stack as a buffer where the file could be stored temporarily till I could stick in into a file. Why not just stream it directly into the file to star with.

Much like you can get at the char* inside a std:string, I thought of something like

read( int fd, outFile.front(), std::npos );  // npos = INT_MAX

or something like that.

end edit

Thanks.

Answer 1

This is simplistic, and off the top of my fingers, but I think something along these lines would work out:

template <unsigned BUF_SIZE>
struct Buffer {
    char buf_[BUF_SIZE];
    int len_;
    Buffer () : buf_(), len_(0) {}
    int read (int fd) {
        int r = read(fd, buf_ + len_, BUF_SIZE - len_);
        if (r > 0) len_ += r;
        return r;
    }
    int capacity () const { return BUF_SIZE - len_; }
}

template <unsigned BUF_SIZE>
struct BufferStream {
    typedef std::unique_ptr< Buffer<BUF_SIZE> > BufferPtr;
    std::vector<BufferPtr> stream_;
    BufferStream () : stream_(1, BufferPtr(new Buffer<BUF_SIZE>)) {}
    int read (int fd) {
        if ((*stream_.rbegin())->capacity() == 0)
            stream_.push_back(BufferPtr(new Buffer<BUF_SIZE>));
        return (*stream_.rbegin())->read(fd);
    }
};

In a comment, you mentioned you wanted to avoid creating a big char buffer. When using the read system call, it is generally more efficient to perform a few large reads rather than many small ones. So most implementations will opt for large input buffers to gain that efficiency. You could implement something like:

std::vector<char> input;
char in;
int r;
while ((r = read(fd, &in, 1)) == 1) input.push_back(in);

But that would involve a system call and at least one byte copied for every byte of input. In contrast, the code I put forth avoids extra data copies.

I don’t really expect the code I put out to be the solution you would adopt. I just wanted to provide you with an illustration of how to create a self-extending object that was fairly space and time efficient. Depending on your purposes, you may want to extend it, or write your own. Off the top of my head, some improvements may be:

• use std::list instead, to avoid vector resizing
• allow API a parameter to specify how many bytes to read
• use readv to always allow at least BUF_SIZE bytes (or more than BUF_SIZE bytes) to be read at a time