Question

I’m thinking it may be beneficial to sprinkle some debug logging throughout my app (a painting-type program), and to have this info written to a file. My current debugging strategy is to hook up a custom exception listener (sys.excepthook) and to allow the user to e-mail me a copy of the stack trace that caused the crash.

This has been pretty handy at seeing what the user’s done to cause the program to crash, but I feel that a log file could certainly help.
I’m wondering what the best way to do this is. I’m thinking of enabling logging via a command-line switch and to create a log per “run” of the program, and to e-mail myself a copy of the log when a crash occurs. However, the log’s won’t help if the app’s not in debug mode!

I’m a little worried of the log filling up too fast – if I place logging inside some mouse motion event handlers, then it’ll create many entries. Also, a log file may grow pretty large, and just be filled with irrelevant information to me when examining a bug report.

How do you guys deal with this? I’m interested in the frequency of logging – since my app responds to many events (e.g. mouse movement; depending on user input) I don’t want to create excess logging.

Answer 1

As said before you can use the logger module. You can set a default level of logging (for example Warning) and put all kind of messages in you code, since debug until critical. As humble explanation, if you set your logging level as Warning, even if your code log debug messages they will not appear in the file (or stdout). That’s because the logging module will log only messages with priority higher or equal to Warning (Warning, Error and critical).

As a simple explanation look at this code:

import logging
import logging.handlers as handlers

logger = logging.getLogger('myapp')
hdlr = logging.FileHandler('/tmp/myapp.log')
formatter = logging.Formatter('%(asctime)s %(levelname)s: %(message)s')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr)
logger.setLevel(logging.WARNING)

logger.debug('debug!')
logger.info('info!')
logger.warning('warning!')
logger.error('error!')
logger.critical('critical!')

It creates a file called myapp.log:

magun@~: cat /tmp/myapp.log
2010-11-05 12:27:25,359 WARNING: warning!
2010-11-05 12:27:25,362 ERROR: error!
2010-11-05 12:27:26,071 CRITICAL: critical!
magun@~:

If you’re worried about the file size you can use a rotating log, witch will discard the oldest logs based in your criteria:

import logging
import logging.handlers as handlers

logger = logging.getLogger('myapp')
hdlr = handlers.RotatingFileHandler('/tmp/log/myapp.log', maxBytes=100, backupCount=5)
formatter = logging.Formatter('%(asctime)s %(levelname)s: %(message)s')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr)
logger.setLevel(logging.WARNING)

for i in range(20):
    logger.debug('debug%i!'%i)
    logger.info('info%i!'%i)
    logger.warning('warning%i!'%i)
    logger.error('error%i!'%i)
    logger.critical('critical%i!'%i)

In this case I used a log file with a max size of 100 bytes (pretty small, you should raise it) and a backup count of 5. This means when a log reaches 100bytes it will be “rotated”: a new (and empty) myapp.log will created and the old one will become myapp.log.1. In the next rotation myapp.log.1 will become myapp.log.2, the myapp.log will be come the new myapp.log.1. It will repeat until we have myapp.log, myapp.log.1, myapp.log.2, … myapp.log.n (in this example the limit will be myapp.log.5). When we hit this the, and log need to be rotated, the myapp.log.5 file will be discarded. So, the size limit if 5*100bytes.

Take a look of what happened:

magun@~: ls /tmp/log/
myapp.log  myapp.log.1  myapp.log.2  myapp.log.3  myapp.log.4  myapp.log.5
magun@~: cat /tmp/log/myapp.log
2010-11-05 12:33:52,369 ERROR: error19!
2010-11-05 12:33:52,376 CRITICAL: critical19!
magun@~: cat /tmp/log/myapp.log.1
2010-11-05 12:33:52,362 CRITICAL: critical18!
2010-11-05 12:33:52,369 WARNING: warning19!
magun@~: cat /tmp/log/myapp.log.2
2010-11-05 12:33:52,355 WARNING: warning18!
2010-11-05 12:33:52,362 ERROR: error18!
magun@~: cat /tmp/log/myapp.log.3
2010-11-05 12:33:52,348 ERROR: error17!
2010-11-05 12:33:52,355 CRITICAL: critical17!
magun@~: cat /tmp/log/myapp.log.4
2010-11-05 12:33:52,340 CRITICAL: critical16!
2010-11-05 12:33:52,348 WARNING: warning17!
magun@~: cat /tmp/log/myapp.log.5
2010-11-05 12:33:52,333 WARNING: warning16!
2010-11-05 12:33:52,340 ERROR: error16!
magun@~:

As you can see we lost many logs (0-15), but the most recent are there, preserving the user free space. Don’t forget to read the log from bottom up:)