Question

What’s the best way and best tool for logging in multi-threaded environment, so that each thread has it’s own logger instance and separate file. is this even possible?

Answer 1

You may try using a custom Log4J appender, which takes thread id as a parameter and filters messages based on what thread calls it. Create it on the fly, attach it to the logger.

There are multiple problems with this approach though:

1. Too many appenders would slow down the logging
2. AppServers usually have a thread pool. That means that the same thread over time would take part in executing totally unrelated requests, which will end up in the same log file.

I suggest you consider a simpler approach: log thread id into the same log file. It’s fast and simple, log4j has a % flag for doing it. Later you can grep/split the log file per thread id if required.

Update:

Actually, you may have a single custom appender, which will open log files on demand when a new thread logs a record (the appender is executed within that thread context, just call Thread.currentThread().getName()). But you’ll have to re-implement all usual log file tasks (rotation) or delegate it to standard appender for each file.