Question

I have an AppEngine application that I am considering upgrading to use the NDB database.

In my application, I have millions of objects that have old-style db references. I would like to know what the best migration path would be to get these ReferenceProperty values converted to KeyProperty values, or any other solution that would allow me to upgrade to NDB.

(I am hoping for something that doesn’t involve massive batch processing of all of the elements in the database and computing the KeyProperty based on the ReferenceProperty — something elegant would be nice)

Examples of models that I would like to upgrade from db.Model to ndb.Model are the following:

class UserModel(db.Model):
    ....

class MailMessageModel(db.Model):
    m_text = db.TextProperty()   
    m_from = db.ReferenceProperty(reference_class = UserModel)
    m_to = db.ReferenceProperty(reference_class = UserModel)

Answer 1

Good news, you don’t have to make any changes to your persisted data, as ext.db and ndb read and write the exact same data.

Here’s the quote from the NDB Cheat Sheet:

No Datastore Changes Needed!

In case you wondered, despite the different APIs, NDB and the old ext.db package write exactly the same data to the Datastore. That means you don’t have to do any conversion to your datastore, and you can happily mix and match NDB and ext.db code, as long as the schema you use is equivalent. You can even convert between ext.db and NDB keys using ndb.Key.from_old_key() and key.to_old_key().

The cheat sheet is a great guide to convert your model definitions. For example, changing your MailMessageModel should be as easy as:

before:

class MailMessage(db.Model):
    m_text = db.TextProperty()
    m_from = db.ReferenceProperty(reference_class=UserModel)
    m_to = db.ReferenceProperty(reference_class=UserModel)

after:

class MailMessage(ndb.Model):
    m_text = ndb.TextProperty()
    m_from = ndb.KeyProperty(kind=UserModel)
    m_to = ndb.KeyProperty(kind=UserModel)

I highly recommend using the cheat sheet to assist you with your migration.