Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
Do you have any experience logging to NoSQL databases for scalable apps? I have done some research on NoSQL databases for logging and found that MongoDB seems to be a good choice. Also, I found log4mongo-net which seems to be a very straightforward option.
Would you recommend this kind of approach? Are there any other suggestions?
I’ve decided to revise this accepted answer as the state of the art has moved significantly in the last 18 months, and much better alternatives exist.
New Answer
MongoDB is a sub-par choice for a scalable logging solution. There are the usual reasons for this (write performance under load for example). I’d like to put forward one more, which is that it only solves a single use case in a logging solution.
A strong logging solution needs to cover at least the following stages:
MongoDB as a choice only solves the Storage use case (albeit somewhat poorly). Once the complete chain is analysed, there are more appropriate solutions.
@KazukiOhta mentions a few options. My preferred end to end solution these days involves:
The underlying use of ElasticSearch for log data storage uses the current best of breed NoSQL solution for the logging and searching use case. The fact that Logstash-Forwarder / Logstash / ElasticSearch / Kibana3 are under the umbrella of ElasticSearch makes for an even more compelling argument.
Since Logstash can also act as a Graphite proxy, a very similar chain can be built for the associated problem of collecting and analysing metrics (not just logs).
Old Answer
MongoDB Capped Collections are extremely popular and suitable for logging, with the added bonus of being ‘schema less’, which is usually a semantic fit for logging. Often we only know what we want to log well into a project, or after certain issues have been found in production. Relational databases or strict schemas tend to be difficult to change in these cases, and attempts to make them ‘flexible’ tends just to make them ‘slow’ and difficult to use or understand.
But if you’d like to manage your logs in the dark and have lasers going and make it look like you’re from space there’s always Graylog2 which uses MongoDB as part of its overall infrastructure but provides a whole lot more on top such as a common, extensible format, a dedicated log collection server, distributed architecture and a funky UI.