Question

How can I combine a open(Filename, {delayed_write, Size, Delay}) with an index on where to write this data to?

I want to wait until I receive a certain amount of data and then write it to a position in the file.

Also is {read_ahead, Size} the opposite of {delayed_write, Size, Delay}? I would like to read a certain amount of data to send it.

Thanks

Answer 1

read_ahead is kind of the opposite of delayed_write in the sense that reading is the opposite of writing.

If you want to read and send bigger chunks of memory you don’t need read_ahead, just read big chunks and send them (not many os calls to save here).

From the file:open/2 manpage on read_ahead:

If read/2 calls are for sizes not significantly less than, or even greater than Size
bytes, no performance gain can be expected.

You don’t need to specify and index when opening. Just use pwrite/3 or a combination of position/2 and write/2.

But writing in different positions of the file might just reduce the gain of delayed_write since (also manpage of file:open/2):

The buffered
data is also flushed before some other file
operation than write/2 is executed.

If you have chunks of data for several positions collect them in a list of {Location, Bytes} and from time to time write them with file:pwrite/2 all in one go. This can map to very efficient writev(2) system call that writes several chunks in one go.