Question

My plan is whenever a script is run, to take the current time and create a folder for the log files I generate. Each step of the script makes a new log file in this folder.

The issue is, I’m trying to figure out how to use the same folder for the duration of the script. What’s happening now is each time a logger is created it gets the current time then, making a new folder each time.

Here’s what the module looks like:

logger.py

...

CURRENT_TIME = # get current time
LOG_FOLDER = "logs/%s/" % CURRENT_TIME

...

def get_logger(name):
    # Create a log file with the given name in the LOG_FOLDER

What happens is each time I import logger.py it recalculates CURRENT_TIME. I figured the way to avoid this is to do from logger.py import * which executes the code once for all the modules, making them have the same log folder.

The main issue is the script calls other python scripts, spawning new processes and the like. When these new processes import logger, they haven’t imported it yet so they will regenerate CURRENT_TIME.

So what’s a good way to fix this? One solution I thought of is have the logger use a constant folder called temp and when the main script finishes rename it to the current time (or store the current time at the beginning of the script). I don’t really like this solution as if there is an exception or error and the script dies, the folder will remain temp when I want it in that case to be the time of the script so I know when it failed.

Answer 1

The classic solution would be to use an environment variable which will be shared between your process and all children.

Demo (save as x.py, warning: will call itself over and over again):

import os
import random
from subprocess import call

def get_logger():
    return os.environ.setdefault("LOG_DIR", logger_name())

def logger_name(): return "log" + str(random.random())

print "Process " + str(os.getpid()) + " uses " + get_logger()

call(["python",  "x.py"])

References: os.environ docs, the concept of environment variables