Question

I’d like to link MyModel to AnotherModel using a many to many relationship without actually using an SQL relation: instead, I want to store a list of AnotherModel pks in a column of MyModel, and have a custom Field handle the conversion to a QuerySet (or a list of instances) and the reverse relation from AnotherModel to MyModel.

Do you know of anyone that has already done that, or do you have any tip for doing it in a simple way ? I’ve started to implement it myself, but I’m starting to realize how complicated it will be to fully implement the behavior of a ManyToManyField: I’m still fairly new to Django and doing it properly requires some familiarity with the inner workings of the framework.

So far I’ve got this:

class InlineManyToManyField(models.CharField):
    __metaclass__ = models.SubfieldBase

    def __init__(self, other_model, *args, **kwargs):
        try:
            assert not other_model._meta.abstract, "{0} cannot define a relation with abstract class {0}".format(
                self.__class__.__name__, to._meta.object_name)
        except AttributeError:
            assert isinstance(other_model, basestring), "{0}({1}) is invalid. First parameter to InlineManyToManyField must be either a model, a model name, or the string self".format(
                self.__class__.__name__,unicode(other_model))
        kwargs['max_length'] = kwargs.get('max_length', 255)
        kwargs['blank'] = kwargs.get('blank', True)
        self.other_model = other_model
        self.token = kwargs.pop('token', ' ')
        super(InlineManyToManyField, self).__init__(*args, **kwargs)

    def to_python(self, value):
        if not value: return
        if isinstance(value, basestring):
            pk_list = value.split(self.token)
            pk_list.pop(0)
            pk_list.pop()
            value = self.other_model._default_manager.filter(pk__in=pk_list)
        return value

    def get_db_prep_value(self, value, connection, prepared=False):
        if not value: return
        pk_list = [item.pk for item in value]
        pk_list.sort()
        return self.token + self.token.join(unicode(pk) for pk in pk_list) + self.token

    def contribute_to_class(self, cls, name):
        super(InlineManyToManyField, self).contribute_to_class(cls, name)
        if isinstance(self.other_model, basestring):
            def resolve_through_model(field, model, cls):
                field.other_model = model
            add_lazy_relation(cls, self, self.other_model, resolve_through_model)

    def value_to_string(self, obj):
        value = self._get_val_from_obj(obj)
        return self.get_db_prep_value(value)

    def get_db_prep_lookup(self, lookup_type, value, connection, prepared=False):
        if lookup_type in ('contains', 'icontains'):
            if isinstance(value, self.other_model):
                value = value.pk
            return ["%{0}{1}{0}%".format(self.token, connection.ops.prep_for_like_query(value))]
        return super(InlineManyToManyField, self).get_db_prep_lookup(
            lookup_type, value, connection=connection, prepared=prepared)

And here is how I use it:

class MyModel(models.Model):
    anotherlist = InlineManyToManyField(AnotherModel, token=':')

If the mymodel table contains a line with pk=1 and anotherlist=”:1:2:3:”, I can do this:

>>> m = MyModel.objects.get(pk=1)
>>> m.anotherlist
[<AnotherModel: 1>, <AnotherModel: 2>, <AnotherModel: 3>]
>>> MyModel.objects.filter(anotherlist__contains=2)
[<MyModel: 2>]

What I’d like to add next is the reverse relation: I’d love to have a mymodel_set on AnotherModel instances, using the “contains” code above for example, but I have a hard time understanding
how everything works in django/db/models/fields/related.py 🙂

So, before I spend days working on it, did you stumble on anything similar anywhere, or have you already written something similar yourself ?

Answer 1

Umm … why? and .. don’t!
All popular sql databases scales pretty darn well. You shouldn’t be afraid of inner-joins!.. They are ok!

What you trying to do is a bit of a pain … Leaving that aside, if you remove an AnotherModel instance, what will you do? Scan all the MyModel table for a string-id? What happens when Django 5.6.6.1 comes out and something is not compatible? Rewrite? Also think about the Admin modules and implications/changes over there!

Don’t overcommit to the tiniest bit of performance from the very beginning! Fix if is broken, test if is not!

Invest more time in the app’s architecture, and you will be fine!

To answer you question: Nope, I also think about something like that, search it, but haven’t found anything useful!