Question

Using log4j on Unix, which Appender would perform the best to write 1000Meg :

1) Using RollingFileAppender writing 10 file of 100 Meg

or

2) Using a FileAppender and writing a single 1000Meg file

In other words, using java on unix, does the size matter?

Thank you

Answer 1

There no Java-side performance difference between writing to a small file or writing to a large file. There might be a small difference at the OS level when a file gets big enough that an extra level of index blocks is required (FS dependent), but it is probably not worth worrying about.

There will be a performance cost in implementing the file rolling behavior. The appender has to:

• test / remember how big the file is,
• close the current one,
• rename it,
• open a new file.

My gut feeling is that this is not likely to be significant. (However, it would be worth measuring to see if the performance impact should be a concern. Also, you should probably ask yourself if you are not doing too much logging.)

You have to compare all of the above against the advantages of file rolling:

• Having a bounded size on log files means that your logging won’t fill the disk, causing problems for the application and potentially others on the same machine.
• Smaller log files can make it easier / quicker to do searches for events at specific times. (Running less on a 1000Mb file can be painful …)