Question

So say I have some classes X, Y and Z using SQLAlchemy declarative syntax to define some simple columns and relationships

Requirements:

1. At the class level, (X|Y|Z).primary_keys returns a collection of
   the respective class’ primary keys’ (InstrumentedAttribute
   objects) I also want (X|Y|Z).relations to reference the class’
   relations in the same way

2. At the instance level, I would like the same attributes to reference
   those attributes’ instantiated values, whether they’ve been
   populated using my own constructors, individual attributes
   setters, or whatever SQLAlchemy does when it retrieves rows from
   the db.

So far I have the following.

import collections 
import sqlalchemy
import sqlalchemy.ext.declarative
from sqlalchemy import MetaData, Column, Table, ForeignKey, Integer, String, Date, Text
from sqlalchemy.orm import relationship, backref

class IndexedMeta(sqlalchemy.ext.declarative.DeclarativeMeta):
        """Metaclass to initialize some class-level collections on models"""
    def __new__(cls, name, bases, defaultdict):
        cls.pk_columns = set()
        cls.relations = collections.namedtuple('RelationshipItem', 'one many')( set(), set())
        return super().__new__(cls, name, bases, defaultdict)

Base = sqlalchemy.ext.declarative.declarative_base(metaclass=IndexedMeta)


def build_class_lens(cls, key, inst):
    """Populates the 'indexes' of primary key and relationship attributes with the attributes' names. Additionally, separates "x to many" relationships from "x to one" relationships and associates "x to one" relathionships with the local-side foreign key column"""
    if isinstance(inst.property, sqlalchemy.orm.properties.ColumnProperty):
        if inst.property.columns[0].primary_key:
            cls.pk_columns.add(inst.key)

    elif isinstance(inst.property, sqlalchemy.orm.properties.RelationshipProperty):
        if inst.property.direction.name == ('MANYTOONE' or 'ONETOONE'):
            local_column = cls.__mapper__.get_property_by_column(inst.property.local_side[0]).key
            cls.relations.one.add( (local_column, inst.key) )
        else:
            cls.relations.many.add(inst.key)


sqlalchemy.event.listen(Base, 'attribute_instrument', build_class_lens)

class Meeting(Base):
    __tablename__ = 'meetings'
    def __init__(self, memo):
        self.memo = memo
    id = Column(Integer, primary_key=True)
    date = Column(Date)
    memo = Column('note', String(60), nullable=True)
    category_name = Column('category', String(60), ForeignKey('categories.name'))
    category = relationship("Category", backref=backref('meetings'))
    topics = relationship("Topic",
        secondary=meetings_topics,
        backref="meetings")

...
...

Ok, so that gets me by on the class level, though I feel like I am doing silly things with metaclasses, and I get some strange intermittent errors where the ‘sqlalchemy’ module allegedly isn’t recognized in build_class_lens and evals to Nonetype.

I am not quite sure how I should proceed at the instance level.
I’ve looked into the events interface. I see the ORM event init, but it seems to run prior to the __init__ function defined on my models, meaning the instance attributes haven’t yet been populated at that time, so I can’t build my ‘lens’ on them.
I also wonder if the Attribute event set might be of help. That is my next try, though i still wonder if it is the most appropriate way.

All in all I really wonder if I am missing some really elegant way to approach this problem.

Answer 1

I think the metaclass thing with declarative goes by the old XML saying, “if you have a problem, and use XML, now you have two problems”. The metaclass in Python is useful pretty much as a hook to detect the construction of new classes, and that’s about it. We now have enough events that there shouldn’t be any need to use a metaclass beyond what declarative already does.

In this case I’d go a little further and say that the approach of trying to actively build up these collections is not really worth it – it’s much easier to generate them lazily, as below:

from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
import collections
from sqlalchemy.orm.properties import RelationshipProperty

class memoized_classproperty(object):
    """A decorator that evaluates once at the class level, 
       assigns the new value to the class.
    """

    def __init__(self, fget, doc=None):
        self.fget = fget
        self.__doc__ = doc or fget.__doc__
        self.__name__ = fget.__name__

    def __get__(desc, self, cls):
        result = desc.fget(cls)
        setattr(cls, desc.__name__, result)
        return result

class Lens(object):
    @memoized_classproperty
    def pk_columns(cls):
        return class_mapper(cls).primary_key

    @memoized_classproperty
    def relations(cls):
        props = collections.namedtuple('RelationshipItem', 'one many')(set(), set())
        # 0.8 will have "inspect(cls).relationships" here
        mapper = class_mapper(cls)
        for item in mapper.iterate_properties:
            if isinstance(item, RelationshipProperty):
                if item.direction.name == ('MANYTOONE' or 'ONETOONE'):
                    local_column = mapper.get_property_by_column(item.local_side[0]).key
                    props.one.add((local_column, item.key))
                else:
                    props.many.add(item.key)
        return props

Base= declarative_base(cls=Lens)

meetings_topics = Table("meetings_topics", Base.metadata,
    Column('topic_id', Integer, ForeignKey('topic.id')),
    Column('meetings_id', Integer, ForeignKey('meetings.id')),
)
class Meeting(Base):
    __tablename__ = 'meetings'
    def __init__(self, memo):
        self.memo = memo
    id = Column(Integer, primary_key=True)
    date = Column(Date)
    memo = Column('note', String(60), nullable=True)
    category_name = Column('category', String(60), ForeignKey('categories.name'))
    category = relationship("Category", backref=backref('meetings'))
    topics = relationship("Topic",
        secondary=meetings_topics,
        backref="meetings")

class Category(Base):
    __tablename__ = 'categories'
    name = Column(String(50), primary_key=True)

class Topic(Base):
    __tablename__ = 'topic'
    id = Column(Integer, primary_key=True)

print Meeting.pk_columns
print Meeting.relations.one

# assignment is OK, since prop is memoized
Meeting.relations.one.add("FOO")

print Meeting.relations.one