Question

I have an application for which I would like to record some statistics for later analysis but which are not at all needed within the application itself. i.e. The data will never need to be read by the application. Whatever system is used for storing these statistics, I just need to be able to take periodic dumps of the data.

I could use the logging built into AppEngine, but then it will display in the application logs. I would like to keep the application logs for debugging the application without having to see other custom information every time I check them.

How would you do this?

Edit: More details on the stats. Each time a user completed a particular task (1-3 times a day) we need to store details about that task. Such as time and order of events. These would be stored in a single string per session – 1-2kB.

Answer 1

I had a similar logging in my app, and what I did is to set up a simple servlet in a self-hosted server that receives the log string and level and store them in our local database. Each time I need this kind of log, I use asynchronous URLFetch to send the data from our app to our logging server.

I could store the log data in the datastore, but then I would lose the option to do full text search of my log strings, which in my experience is invaluable when you’re sifting through log files. I could also store the log in an external file and grep only the lines that I need.