Home/ Questions/Q 9242679
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-18T08:37:49+00:00

I have had just discovered a really big problem with my Pyramid application. The

  • 0

I have had just discovered a really big problem with my Pyramid application. The mongo scaffold implied that event handlers should be used for handling database connections as well. The idea is, when new request comes in, connect to the database, when page is rendered disconnect.
So this is what I did, in my __init.py__

def connectDatabase(event):
    mongo = MongoDB()
    con = mongo.connectDatabase()
    db = con[Cfg_MongoDB_Database]
    event.request.con = con
    event.request.db = db
    redis = Redis()
    con = redis.connectDatabase()
    event.request.redis = con
    log.debug('newrequest')

def closeConnection(event):
    mongo = MongoDB()
    mongo.closeConnection(event.request.con) 
    log.debug('newresponse')   

def main(global_config, **settings):
    config = Configurator(settings=settings, root_factory=Dashboard)
    authentication = AuthTktAuthenticationPolicy(Cfg_Auth_Key, hashalg='sha512',\
        include_ip=False, timeout=3600*24*7, max_age=3600*24*7, reissue_time=3600)
    authorization = ACLAuthorizationPolicy()
    config.set_authentication_policy(authentication)
    config.set_authorization_policy(authorization)
    config.add_static_view('includes', 'includes', cache_max_age=3600)
    config.add_renderer(".html", "pyramid.mako_templating.renderer_factory")  
    config.add_route('dash', '/')
    config.add_subscriber(connectDatabase, NewRequest)
    config.add_subscriber(closeConnection, NewResponse)
    log.debug('main')
    config.scan()
    return config.make_wsgi_app()

However I’ve put some logger into method that is connecting to the database, and discovered the method is being called 10+ times, and if I disable caching in add_static_view over 100+ times. This implies that for each css, image, js, the new connection is made. Which is huge overhead!!! The site takes 300ms to connect to database 100 times!

/Users/jan/Documents/Test2/test2/data.py changed; reloading...
-------------------- Restarting --------------------
2013-02-01 18:32:48,351 DEBUG [test2.tools][MainThread] main
Starting server in PID 12374.
serving on http://127.0.0.1:6543
2013-02-01 18:34:14,451 DEBUG [test2.tools][Dummy-2] newrequest
2013-02-01 18:34:14,582 DEBUG [test2.tools][Dummy-2] newresponse
2013-02-01 18:34:14,952 DEBUG [test2.tools][Dummy-3] newrequest
2013-02-01 18:34:14,953 DEBUG [test2.tools][Dummy-3] newresponse
2013-02-01 18:34:17,459 DEBUG [test2.tools][Dummy-4] newrequest
2013-02-01 18:34:17,474 DEBUG [test2.tools][Dummy-4] newresponse
2013-02-01 18:34:17,482 DEBUG [test2.tools][Dummy-5] newrequest
2013-02-01 18:34:17,497 DEBUG [test2.tools][Dummy-5] newresponse
2013-02-01 18:34:19,158 DEBUG [test2.tools][Dummy-2] newrequest
2013-02-01 18:34:19,159 DEBUG [test2.tools][Dummy-2] newresponse

So the solution so far is to move the connection to Model instead. But how would I know than when can I close connection? Anyone have any better, more elegant solution for my problem?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    pymongo has built in connection pooling http://api.mongodb.org/python/current/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient

    Using connection pooling shouldn’t create a new connection on every request

    Also, I don’t think your pyramid app should be serving static media in production for exactly this reason, apache,nginx are optimized to serve files, and your requests won’t have to go through your app

    • 0