Question

OK, so I’m writing a framework that looks for python files in sub directories that are named task.py and then look for classes that are derived from the base class Task and collect them.
I decided that I needed to add a meta class to Task, but then the issubclass() started to behave in a weird way.
Here is how the directory layout looks:

start.py                
tasks/__init__.py
tasks/base.py
tasks/sub/__init__.py   # empty
tasks/sub/task.py

start.py:

#!/usr/bin/env python
from tasks.base import Task1, Task2
from tasks.sub.task import SubTask1, SubTask2

print "Normal import:"
print "is SubTask1 sub class of Task1? %s" % issubclass(SubTask1, Task1)
print "is SubTask2 sub class of Task2? %s" % issubclass(SubTask2, Task2)

from tasks import setup
print "Imp import:"
setup()

tasks/init.py

import os.path, imp, types
from tasks.base import Task1, Task2

# Find all task definitions
ALL_TASK1 = { }
ALL_TASK2 = { }
def _append_class(d, task, base):
    if (type(task) == types.TypeType) and issubclass(task, base):
        if task == base:
            return
        if not task.name in d:
            d[task.name] = task

this_dir = os.path.dirname(os.path.abspath(__file__))
for root, dirs, files in os.walk(this_dir):
    if not "task.py" in files:
        continue
    mod_name = "task"
    f, pathname, description = imp.find_module(mod_name, [ root ])
    m = imp.load_module(mod_name, f, pathname, description)
    f.close()

    for task in m.__dict__.itervalues():
        _append_class(ALL_TASK1, task, Task1)
        _append_class(ALL_TASK2, task, Task2)

def setup():
    print "All Task1: %s" % ALL_TASK1
    print "All Task2: %s" % ALL_TASK2

tasks/base.py

class MetaClass (type):
    def __init__(cls, name, bases, attrs):
        pass

class Base (object): 
    __metaclass__ = MetaClass

    def method_using_metaclass_stuff(self):
        pass

class Task1 (Base):
    pass

class Task2 (object):
    pass 

tasks/sub/task.py

from tasks.base import Task1, Task2

class SubTask1 (Task1): # Derived from the __metaclass__ class
    name = "subtask1"

class SubTask2 (Task2):
    name = "subtask2"

When I run setup.py I got the following output (ALL_TASK1 dict is empty!):

Normal import:
is SubTask1 sub class of Task1? True
is SubTask2 sub class of Task2? True
Imp import:
All Task1: {}
All Task2: {'subtask2': <class 'task.SubTask2'>}

But when I comment out the __metaclass__ line in the class Base (Task1‘s base class) I got the output I expected (ALL_TASK1 dict isn’t empty):

Normal import:
is SubTask1 sub class of Task1? True
is SubTask2 sub class of Task2? True
Imp import:
All Task1: {'subtask1': <class 'task.SubTask1'>}
All Task2: {'subtask2': <class 'task.SubTask2'>}

I don’t understand how the meta class can affect issubclass() when the module is imported via the imp functions, but not when the module is imported with the normal import.

Can someone explain this to me (I’m using python 2.6.1)?

Answer 1

Your code doesn’t work via normal import either.

>>> from tasks.base import Task1
>>> type(Task1)
<class 'tasks.base.MetaClass'>
>>> from types import TypeType
>>> type(Task1) == TypeType
False
>>> issubclass(type(Task1), TypeType)
True
>>> 

When you instantiate the metaclass (as a class), the instance doesn’t have type TypeType, it has type tasks.base.MetaClass. if you test for that in addition to TypeType, then your code works as expected.

def _append_class(d, task, base):
    if (issubclass(type(task), types.TypeType) and issubclass(task, base):

This yields the same output as when you commented out the metaclass line and has an advantage over your origninal code in that it allows your users to define their own metaclasses and have them function as tasks. If you want to explicitly dissallow this (and please don’t, I might want to use your framework someday) you can just check for both your metaclass or TypeType specifically.