Question

I have a pretty simple N:M relationship in SqlAlchemy 0.6.6. I have a class “attractLoop” that can contain a bunch of Media (Images or Videos). I need to have a list in which the same Media (let’s say image) can be appended twice. The relationship is as follows:

The media is a base class with most of the attributes Images and Videos will share.

class BaseMedia(BaseClass.BaseClass, declarativeBase):
    __tablename__ = "base_media"
    _polymorphicIdentity = Column("polymorphic_identity", String(20), key="polymorphicIdentity")
    __mapper_args__ = {
        'polymorphic_on': _polymorphicIdentity,
        'polymorphic_identity': None
    }

    _name = Column("name", String(50))
    _type = Column("type", String(50))
    _size = Column("size", Integer)
    _lastModified = Column("last_modified", DateTime, key="lastModified")
    _url = Column("url", String(512))
    _thumbnailFile = Column("thumbnail_file", String(512), key="thumbnailFile")
    _md5Hash = Column("md5_hash", LargeBinary(32), key="md5Hash")

Then the class who is going to use these “media” things:

class TestSqlAlchemyList(BaseClass.BaseClass, declarativeBase):
    __tablename__ = "tests"
    _mediaItems = relationship("BaseMedia",
                               secondary=intermediate_test_to_media,
                               primaryjoin="tests.c.id == intermediate_test_to_media.c.testId",
                               secondaryjoin="base_media.c.id == intermediate_test_to_media.c.baseMediaId",
                               collection_class=list,
                               uselist=True
                               )

    def __init__(self):
        super(TestSqlAlchemyList, self).__init__()
        self.mediaItems = list()

    def getMediaItems(self):
        return self._mediaItems

    def setMediaItems(self, mediaItems):
        if mediaItems:
            self._mediaItems = mediaItems
        else:
            self._mediaItems = list()

    def addMediaItem(self, mediaItem):
        self.mediaItems.append(mediaItem)
        #log.debug("::addMediaItem > Added media item %s to %s. Now length is %d (contains: %s)" % (mediaItem.id, self.id, len(self.mediaItems), list(item.id for item in self.mediaItems)))

    def addMediaItemById(self, mediaItemId):
        mediaItem = backlib.media.BaseMediaManager.BaseMediaManager.getById(int(mediaItemId))
        if mediaItem:
            if mediaItem.validityCheck():
                self.addMediaItem(mediaItem)
            else:
                raise TypeError("Media item with id %s didn't pass the validity check" % mediaItemId)
        else:
            raise KeyError("Media Item with id %s not found" % mediaItem)

    mediaItems = synonym('_mediaItems', descriptor=property(getMediaItems, setMediaItems))

And the intermediate class to link both of the tables:

intermediate_test_to_media = Table(
                                   "intermediate_test_to_media",
                                   Database.Base.metadata,
                                   Column("id", Integer, primary_key=True),
                                   Column("test_id", Integer, ForeignKey("tests.id"), key="testId"),
                                   Column("base_media_id", Integer, ForeignKey("base_media.id"), key="baseMediaId")
                                   )

When I append the same Media object (instance) twice to one instances of that TestSqlAlchemyList, it appends two correctly, but when I retrieve the TestSqlAlchemyList instance from the database, I only get one. It seems to be behaving more like a set.

The intermediate table has properly all the information, so the insertion seems to be working fine. Is when I try to load the list from the database when I don’t get all the items I had inserted.

mysql> SELECT * FROM intermediate_test_to_media;
+----+---------+---------------+
| id | test_id | base_media_id |
+----+---------+---------------+
|  1 |       1 |             1 |
|  2 |       1 |             1 |
|  3 |       1 |             2 |
|  4 |       1 |             2 |
|  5 |       1 |             1 |
|  6 |       1 |             1 |
|  7 |       2 |             1 |
|  8 |       2 |             1 |
|  9 |       2 |             1 |
| 10 |       2 |             2 |
| 11 |       2 |             1 |
| 12 |       2 |             1 |

As you can see, the “test” instance with id=1 should have the media [1, 1, 2, 2, 1, 1]. Well, it doesn’t. When I load it from the DB, it only has the media [1, 2]

I have tried to set any parameter in the relationship that could possibly smell to list… uselist, collection_class = list… Nothing…

You will see that the classes inherit from a BaseClass. That’s just a class that isn’t actually mapped to any table but contains a numeric field (“id”) that will be the primary key for every class and a bunch of other methods useful for the rest of the classes in my system (toJSON, toXML…). Just in case, I’m attaching an excerpt of it:

class BaseClass(object):
    _id = Column("id", Integer, primary_key=True, key="id")
    def __hash__(self):
        return int(self.id)

    def setId(self, id):
            try:
                    self._id = int(id)
            except TypeError:
                    self._id = None

    def getId(self):
        return self._id

    @declared_attr
    def id(cls):
        return synonym('_id', descriptor=property(cls.getId, cls.setId))

If anyone can give me a push, I’ll appreciate it a lot. Thank you. And sorry for the huge post… I don’t really know how to explain better.

Answer 1

I think what you want is a described in the documentation as “Extra Fields in Many-to-Many Relationships“. Rather than storing a unique row in the database foreach “link”, between attractLoop and Media, you would store a single association and specify (as a part of the link object model) how many times it is referenced and/or in which location(s) in the final list the media should appear. This is a different paradigm from where you started, so it’ll certainly require some re-coding, but I think it addresses your issue. You would likely need to use a property to redefine how to add or remove Media from the attactLoop.