Question

I am trying to add a module to a project that uses the standard Python (2.7) logging libraries. What I want to achieve is the following:

• Anything equal or worse than WARNING goes to wherever the main program (outside of my control) has configured the logging to go (usually stderr)
• Everything else (debug messages mainly) goes to a log file

Please note that the key issue here is that I do not want to modify the global logging settings, since those are controlled by the main app. Otherwise I would use the example in here:

http://docs.python.org/2/howto/logging-cookbook.html#logging-to-multiple-destinations

I’ve tried this:

logger = logging.getLogger('my_module')
logger.setLevel(logging.WARNING)
fh = logging.FileHandler('my_module.log')
fh.setLevel(logging.DEBUG)
logger.addHandler(fh)

but then I only get warning messages in my file.

If I replace the above line with:

logger.setLevel(logging.DEBUG)

then I get debug messages in both stderr and my file.

Then I tried this:

logger = logging.getLogger('my_module')
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.NullHandler())
console = logging.StreamHandler()
console.setLevel(logging.WARNING)
logger.addHandler(console)
fh = logging.FileHandler('my_module.log')
fh.setLevel(logging.DEBUG)
logger.addHandler(fh)

But again no luck.

So can I achieve this without having to call basicConfig() or anything else that affects the global logging infrastructure? I only want to modify my own module here if possible.

Thanks!

Answer 1

• Anything equal or worse than WARNING goes to wherever the main program (outside of my control) has configured the logging to go
  (usually stderr)
• Everything else (debug messages mainly) go to a log file

You can’t do it with a single logger without setting WARNING level on ancestor’s handlers. From the docs:

Logger.propagate

If this evaluates to true, logging messages are passed by this logger and by its child loggers to the handlers of higher level
(ancestor) loggers. Messages are passed directly to the ancestor
loggers’ handlers – neither the level nor filters of the ancestor
loggers in question are considered.

---

what do you suggest a module can do in this situation?

In general other than the setting of WARNING level for the logger and adding NullHandler(), library code shouldn’t configure logging at all.

If you want an easy way for the main application to provide you with a debug info then define a function that configures DEBUG level logging for your library.

---

I don’t want the library to be DEBUG except for the file, where I would like to see everything.

If you can’t control the main program then you shouldn’t create debug files as a side-effect of using your_module unless explicitly asked to do so:

import your_module # in the main application

your_module.get_logger().log_to_file(filename) # without this line your module
                                               # shouldn't create debug files
...
your_module.some_function()

---

I am not sure how this logger hierarchy works: the logger I created is a child of the root logger that the app configured?

'' is a root logger. 'a.b' is a child of 'a' logger.