Question

I’m looking to store time-based counters using Redis, in a Python project. I want to store cumulative rolling counters for various events occurring in recent time periods (1, 5, 15, 30, 60 minutes or longer etc.) for a potentially large universe of data without resorting to SQL-based solutions.

I found a Ruby-based library that seems to offer exactly what I want. But I’m working with Python, so I’m hoping someone knows of a similar solution for us Pythonistas. I’d rather not reinvent the wheel, after all. I can port the Ruby library to Python, or roll my own if there aren’t any existing solutions.

So, ever heard of such a thing?

Edit: Why Redis? I’m running a WSGI web app with multiple instances, so I need to share the data across process boundaries. I know how to do what I want using SQL tables and queries, but I’m trying to keep recent event statistics in (something like) Redis rather than SQL (as an optimization.)

Answer 1

I have not seen the exact features that are offered by ruby library mentioned by you.
But there are nut and bolts available for you to build on for storing time series data in redis.

Checkout:

• http://lsbardel.github.com/python-stdnet/contrib/redis_timeseries.html
• http://pypi.python.org/pypi/kairos

If redis is optional then you should check out pytables which can be used for working on time series datasets.

• http://pytables.github.com/usersguide/