Question

I’m attempting to add some contextual information to some logs via the logging module. I need to be able to view a projectid next to each line in the logs, with over 20,000 projects created daily this data is really helpful. To do this I’ve created a derivative of the logging.Filter module.

import logging

class MTFilter(logging.Filter):

def __init__(self, projectid=0):
    self.projectid = projectid

def filter(self, record):
    record.projectid = self.projectid
    return True

Here’s my LOGGING variable in settings.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'filters': {
        'project': {
            '()':  'app.proj.logging.mtfilter.MTFilter',
        },  
    },
    'formatters': { 
        'projectformat': {
            'format': '%(asctime)s %(levelname)8s PID[%(projectid)d] %(name)s[%(funcName)s]: %(message)s',
        },
    },
    'handlers': {
        'null': {
            'level': 'DEBUG',
            'class': 'django.utils.log.NullHandler',
        },
        'project-log': {
            'level': 'DEBUG',
            'class': 'logging.handlers.RotatingFileHandler',
            'formatter': 'projectformat',
            'filename': os.path.join(SITE_ROOT, '../logs/django.log'),
            'filters':  ['project'],
            'maxBytes': 1024*1024*16, #16Mb
        },
    },
    'loggers': {
        '': {
            'handlers':     ['null'],
            'level':        'DEBUG',
            'propagate':    True,
        },
        'proj': {
             'handlers':    ['project-log'],
             'level':       'DEBUG',
         },
    }
}

And in my view I use the following

logger = logging.getLogger('proj')
logger.info('Log Message')

By not putting any value for ‘projectid’ in Logging.filters.project I get the default format value, which is ‘0’. The log result looks like the following:

2011-07-26 02:41:44,488     INFO PID[0] proj[view]: Log Message

What I’d like to do is grab the ‘projectid’ value dynamically somehow, eg. when creating a logger object, or using some Middleware but I just don’t know how to do it. Does anyone have an suggestions?

Answer 1

What I’d like to do is grab the ‘projectid’ value dynamically somehow,
eg. when creating a logger object, or using some Middleware but I just
don’t know how to do it.

It’s not appropriate to do this when creating a logger, as that’s a one-time operation. What you probably need to do is to find some way of getting the current project ID into the filter. For example, instead of passing the record into the filter at construction time, you could use a design with a callable that the filter calls to get the project ID. For example:

class ProjectFilter(logging.Filter):
    def __init__(self, func):
        self.func = func

    def filter(self, record):
        record.projectid = self.func()
        return True

The callable can be anything that knows how to get the project ID from the current context (e.g. a thread local – though that’s not endorsed by the Django core team, it’s what thread locals were invented for, and used quite successfully by other frameworks such as Flask). I don’t know enough about your application to suggest how the callable might work.