Question

Sorry, badly worded title. I hope a simple example will make it clear. Here’s the easiest way to do what I want to do:

class Lemon(object):

    headers = ['ripeness', 'colour', 'juiciness', 'seeds?']

    def to_row(self):
        return [self.ripeness, self.colour, self.juiciness, self.seeds > 0]

def save_lemons(lemonset):
    f = open('lemons.csv', 'w')
    out = csv.writer(f)
    out.write(Lemon.headers)
    for lemon in lemonset:
        out.writerow(lemon.to_row())

This works alright for this small example, but I feel like I’m “repeating myself” in the Lemon class. And in the actual code I’m trying to write (where the number of variables I’m exporting is ~50 rather than 4, and where to_row calls a number of private methods that do a bunch of weird calculations), it becomes awkward.

As I write the code to generate a row, I need to constantly refer to the “headers” variable to make sure I’m building my list in the correct order. If I want to change the variables being outputted, I need to make sure to_row and headers are being changed in parallel (exactly the kind of thing that DRY is meant to prevent, right?).

Is there a better way I could design this code? I’ve been playing with function decorators, but nothing has stuck. Ideally I should still be able to get at the headers without having a particular lemon instance (i.e. it should be a class variable or class method), and I don’t want to have a separate method for each variable.

Answer 1

We could use some metaclass shenanegans to do this…

In python 2, attributes are passed to the metaclass in a dict, without
preserving order, we’ll also want a base class to work with so we can
distinguish class attributes that should be mapped into the row. In python3, we could dispense with just about all of this base descriptor class.

import itertools
import functools
@functools.total_ordering
class DryDescriptor(object):
    _order_gen = itertools.count()
    def __init__(self, alias=None):
        self.alias = alias
        self.order = next(self._order_gen)

    def __lt__(self, other):
        return self.order < other.order

We will want a python descriptor for every attribute we wish to map into the
row. slots are a nice way to get data descriptors without much work. One
caveat, though, we’ll have to manually remove the helper instance to make the
real slot descriptor visible.

class slot(DryDescriptor):
    def annotate(self, attr, attrs):
        del attrs[attr]
        self.attr = attr
        slots = attrs.setdefault('__slots__', []).append(attr)

    def annotate_class(self, cls):
        if self.alias is not None:
            setattr(cls, self.alias, getattr(self.attr))

For computed fields, we can memoize results. Memoizing off of the annotated
instance is tricky without a memory leak, we need weakref. alternatively, we
could have arranged for another slot just to store the cached value. This also isn’t quite thread safe, but pretty close.

import weakref
class memo(DryDescriptor):
    _memo = None
    def __call__(self, method):
        self.getter = method
        return self

    def annotate(self, attr, attrs):
        if self.alias is not None:
            attrs[self.alias] = self

    def annotate_class(self, cls): pass

    def __get__(self, instance, owner):
        if instance is None:
            return self
        if self._memo is None:
            self._memo = weakref.WeakKeyDictionary()
        try:
            return self._memo[instance]
        except KeyError:
            return self._memo.setdefault(instance, self.getter(instance))

On the metaclass, all of the descriptors we created above are found, sorted by
creation order, and instructed to annotate the new, created class. This does
not correctly treat derived classes and could use some other conveniences like
an __init__ for all the slots.

class DryMeta(type):
    def __new__(mcls, name, bases, attrs):
        descriptors = sorted((value, key) 
                             for key, value 
                             in attrs.iteritems() 
                             if isinstance(value, DryDescriptor))

        for descriptor, attr in descriptors:
            descriptor.annotate(attr, attrs)

        cls = type.__new__(mcls, name, bases, attrs)
        for descriptor, attr in descriptors:
            descriptor.annotate_class(cls)

        cls._header_descriptors = [getattr(cls, attr) for descriptor, attr in descriptors]
        return cls

Finally, we want a base class to inherit from so that we can have a to_row
method. this just invokes all of the __get__s for all of the respective
descriptors, in order.

class DryBase(object):
    __metaclass__ = DryMeta

    def to_row(self):
        cls = type(self)
        return [desc.__get__(self, cls) for desc in cls._header_descriptors]

Assuming all of that is tucked away, out of sight, the definition of a class
that uses this feature is mostly free of repitition. The only short coming is
that to be practical, every field needs a python friendly name, thus we had the
alias key to associate 'seeds?' to has_seeds

class ADryRow(DryBase):
    __slots__ = ['seeds']

    ripeness = slot()
    colour = slot()
    juiciness = slot()

    @memo(alias='seeds?')
    def has_seeds(self):
        print "Expensive!!!"
        return self.seeds > 0

>>> my_row = ADryRow()
>>> my_row.ripeness = "tart"
>>> my_row.colour = "#8C2"
>>> my_row.juiciness = 0.3479
>>> my_row.seeds = 19
>>>
>>> print my_row.to_row()
Expensive!!!
['tart', '#8C2', 0.3479, True]
>>> print my_row.to_row()
['tart', '#8C2', 0.3479, True]