Question

In python if a logger is configured in the parent process, then will the child process also get that logger? To be more clear, in my application I configure root logger for parent process by doing logger = logging.getlogger() and adding handlers to it. Now when a child process is forked and it does

logger = logging.getlogger()
logger.info("dfsdf")

then all the logs are processed according to the parent’s root logger. I didn’t configure the root logger for the child. How is it possible? They are two different processes then how can they have same logger?

Answer 1

When you fork a process, it ‘inherits’ the parent process memory, including the logger configuration.

From the Fork Wikipedia page:

The fork operation creates a separate address space for the child. The child process has an exact copy of all the memory segments of the parent process, though if copy-on-write semantics are implemented actual physical memory may not be assigned (i.e., both processes may share the same physical memory segments for a while). Both the parent and child processes possess the same code segments, but execute independently of each other.

This is not unique to Python; this happens for any UNIX process that is forked, be it implemented in C, Perl or Python.

The multiprocessing module uses this (on platforms that support it) to spin up new processes quickly.

Note that inheriting a logger may lead to race conditions; the logging module only knows about thread-safety; it uses a thread lock to serialize access to handlers, but that lock is not shared across processes (everything in the child process is a copy, and not the same object).

This means that when you log messages from both the parent and child at the same time, the log entries can end up mixed together as the OS switches between the two processes while writing the log entries to the file.