I am trying to understand what would be the difference between calling FileChannel.write in short successions with a 16k buffer AND mapping multiple ByteBuffers with an append size of 16k as described here: https://stackoverflow.com/a/7367952/962872
I would think that the mapped bytebuffer approach produces a lot of garbage as you discard the MappedByteBuffers after each append. And I am not sure it is faster either. And you still have to do a bunch of mapping operations… (one per append).
Or perhaps you should map a huge ByteBuffer (as big as possible) and just keep writing to this MappedByteBuffer?
I am using the FileChannel.write approach with a Java-side 16kb buffer as a “fast” way to write a file but I want to make sure I am not missing something faster/better.
Can anyone shed a light?
It would if you created 16 KB buffers. If you created 1 GB buffers would won’t need as many and there is little penalty for doing so (assuming you have a 64-bit JVM)
I would check you are not already writing as fast or faster than you drive can write anyway. The main advantage Memory Mapped Files give you is much lower typical latency. The throughput you can write will be limited by the speed of your drive. If you have a typical SSD, your CPU can still write out data faster than it can consume and if you use HDD it probably doesn’t matter what you do as the drive is so much slower.
Typical write throughput of
Often when CPU is not the bottleneck, what you do in software has little impact on the performance of the application.
In terms of latency, the typical latency of