Question

I have a problem where I insert a database item using a SQLAlchemy / Tastypie REST interface, but the item is missing when subsequently get the list of items. It shows up only after I get the list of items a second time.

I am using SQLAlchemy with Tastypie/Django running on Apache via mod_wsgi. I use a singleton Database Manager class to hold my engine and declarative_base, and with Tastypie, a separate class to get the session and make sure I roll-back if there is a problem with the commit. As in the update below, the problem occurs when I don’t close my session after inserting. Why is this necessary?

My original code was like this:

Session = scoped_session(sessionmaker(autoflush=True))

# Singleton Database Manager class for managing session
class DatabaseManager():
    engine = None
    base = None

    def ready(self):
       host='mysql+mysqldb://etc...'
       if self.engine and self.base:
            return True
        else:
            try:
                self.engine = create_engine(host, pool_recycle=3600)
                self.base = declarative_base(bind=self.engine)
                return True
            except:
                return False

    def getSession(self):
        if self.ready():
            session = Session()
            session.configure(bind=self.engine)
            return session
        else:
            return None

DM = DatabaseManager()

# A session class I use with Tastypie to ensure the session is destroyed at the
# end of the transaction, because Tastypie creates singleton Resources used for
# all threads
class MySession:
    def __init__(self):
        self.s = DM.getSession()

    def safeCommit(self):
        try:
            self.s.commit()
        except:
            self.s.rollback()
            raise

    def __del__(self):
        try:
            self.s.commit()
        except:
            self.s.rollback()
            raise


# ... Then ... when I get requests through Apache/mod_wsgi/Django/Tastypie
# First Request
obj_create():
    db = MySession()
    print db.s.query(DBClass).count() # returns 4
    newItem = DBClass()
    db.s.add(newItem)
    db.s.safeCommit()
    print db.s.query(DBClass).count() # returns 5


# Second Request after First Request returns
obj_get_list():
    db = MySession()
    print db.s.query(DBClass).count() # returns 4 ... should be 5


# Third Request is okay
obj_get_list():
    db = MySession()
    print db.s.query(DBClass).count() # returns 5

UPDATE

After further digging, it appears that the problem is my session needed to be closed after creating. Perhaps because Tastypie’s object_create() adds the SQLAlchemy object to it’s bundle, and I don’t know what happens after it leaves the function’s scope:

obj_create():
    db = MySession()
    newItem = DBClass()
    db.s.add(newItem)
    db.s.safeCommit()
    copiedObj = copyObj(newItem) # copy SQLAlchemy record into non-sa object (see below)
    db.s.close()
    return copiedObj

If someone cares to explain this in an answer, I can close the question. Also, for those who are curious, I copy my object out of SQLAlchemy like this:

class Struct:
    def __init__(self, **entries):
        self.__dict__.update(entries)

class MyTastypieResource(Resource):
    ...
    def copyObject(self, object):
        base = {}
        # self._meta is part of my tastypie resource
        for p in class_mapper(self._meta.object_class).iterate_properties:
            if p.key not in base and p.key not in self._meta.excludes:
                base[p.key] = getattr(object,p.key)
        return Struct(**base)

Answer 1

The problem was resolved by closing my session. The update in the answer didn’t solve the problem fully – I ended up adding a middleware class to close the session at the end of a transaction. This ensured everything was written to the database. The middleware looks a bit like this:

class SQLAlchemySessionMiddleWare(object):
    def process_response(self, request, response):
        try:
            session = MyDatabaseManger.getSession()
            session.commit()
            session.close()
        except Exception, err:
            pass
        return response

    def process_exception(self, request, exception):
        try:
            session = MyDatabaseManger.getSession()
            session.rollback()
            session.close()
        except Exception, err:
            pass