Question

Recently there’s been some data structure changes in our app, and we decided to use namespaces to separate different versions of of the data, and a mapreduce task that converts old entities to the new format.

Now that’s all fine, but we don’t want to always isolate the entire data set we have. The biggest part of our data is stored in a kind that’s pretty simple and doesn’t need to change often. So we decided to use per-kind namespaces.

Something like:

class Author(ndb.model.Model):
    ns = '2'

class Book(ndb.model.Model):
    ns = '1'

So, when migrating to version 2, we don’t need to convert all our data (and copy all ‘Book’ kinds to the other namespace), only entities of the ‘Author’ kind. Then, instead of defining the appengine_config.namespace_manager_default_namespace_for_request, we just the ‘namespace’ keyword arguments to our queries:

Author.query(namespace=Author.ns).get()

Question: how to store (i.e. put()) the different kinds using these different namespaces? Something like:

# Not an API
Author().put(namespace=Author.ns)

Of course, the above doesn’t work. (Yes, I could ask the datastore for an avaliable key in that namespace, and then use that key to store the instance with, but it’s an extra API call that I’d like to avoid.)

Answer 1

A Model instance will use the namespace you set with the namespace_manager[1] as you can see here: python/google/appengine/ext/db/init.py

What you could do is create a child class of Model which expects a class-level ‘ns’ attribute to be defined. This sub class then overrides put() and sets the namespace before calling original put and resets the namespace afterwards. Something like this:

'''
class MyModel(db.Model):

    ns = None

    def put(*args, **kwargs):
        if self.ns == None:
            raise ValueError('"ns" is not defined for this class.')
        original_namespace = namespace_manager.get_namespace()
        try:
            super(MyModelClass, self).put(*args, **kwargs)
        finally:
            namespace_manager.set_namespace(original_namespace)
'''

[1] http://code.google.com/appengine/docs/python/multitenancy/multitenancy.html