Question

We have an application based on Excel 2003 and Python 2.4 on Windows XP 32bit. The application consists of a large collection of Python functions which can be called from a number of excel worksheets.

We’ve notcied an anomolous behavior which is that sometimes in the middle of one of these calls the python interpreter will start hunting around for modules which almost certainly are already loaded and in memory.

We know this because we were able to hook-up Sysinternal’s Process Monitor to the process and observe that from time to time the process (when called) starts hunting around a bunch of directories and eggs for certain .py files.

The obvious thing to try is to see if the python search-path had become modified, however we found this not to be the case. It’s exactly what we’d expect. The odd thing is that:

• The occasions on which this searching behavior was triggered appears to be random, i.e. it did not happen every time or with any noticable pattern.

• The behavior did not affect the result of the function. It returned the same value irrespective of whether this file searching behavior was triggered.

• The folders that were being scanned were non-existant (e.g. J:/python-eggs ) on a machine where J-drive contained no-such folder. Naturally procmon reports that this generated a file-not found error.

It’s all very mysterious so I dont expect anybody to be able to provide a definitive answer as to what might be going wrong. I would appreciate any suggestions about how this problem might be debugged.

Thanks!

Answers to comments

1. All the things that are being searched for are actual, known python files which exist in the main project .egg file. The odd thing is that at the time they are being searched-for those particuar modules have already been imported. They must be in memory in order for the process to work.

2. Yes, this affects performance because sometimes this searching behavior tries to hit network drives. Also by searching eggs which couldnt possibly contain certain modules it the process gets interrupted by the corporate mandated virus-scanner. That slows down what would normally be a harmless and instant interruption.

3. This is stock python 2.4.4. No modifications.

Answer 1

Python programs can import modules at any time, not just during program load. Try searching the modules you are using for import.

If this doesn’t work, you can write an import hook to catch and report all attempted imports before they occur. For example, if you run this before everything else, you will get a dump of every attempted import and its source:

import sys, traceback

class ImportDebugger:
    def find_module(self, fullname, path=None):
        print "Attempting to import %s:" % fullname
        traceback.print_stack()

sys.meta_path.insert(0, ImportDebugger())