I’m building a RESTful service in Symfony2, using the FOSRestBundle. I can track page usage in web clients using Google Analytics. However, this is is obviously not going to work for requests by non-HTML clients.
Before I embark on installing Redis, writing services, event dispatchers, etc, has this problem already been solved? Is there a solution that doesn’t have a serious impact on performance?
Based on stats of the project I’m replacing, I expect around 1,000 hits per hour with 90% of traffic coming from browsers. I won’t be in control of the non-HTML clients, so adding tracking there is not an option.
I need the data for the same reason that anybody needs analytics data – to make pretty graphs, and give quantitative evidence about where to focus development resources.
There are a few options, and it depends on exactly how much information you need. If you just want to know basic activity (how many calls to each web service, when, source IP, user agent), then the apache logs already have all that information.
Use CustomLog to add any additional fields you need. E.g. you mentioned the Accept header, which can be added like this:
If it is GET, POST, PUT, etc. is contained in the %r part.
If you want to know what is actually in the POST or PUT data, then it is more difficult. The extreme solution is to use the mod_dumpio module. That logs all of the client input (all headers, all cookies, all POST data). If people use your REST API to upload images then, for good or bad, you get the full image in your log. That could get very big.
The solution I favour is to log from PHP: a custom log either at the top of your PHP script, or when you process the requests. You then have full control over what to log, the easiest format to analyze, and you can also put it in context (e.g. log text data, but not logging image bytes). In development and for small sites I do this in parallel with apache logging. If Apache is taking too much CPU, then disable apache logging, or bypass Apache completely. (I’m currently evaluating the built-in webserver in php 5.4 – it has support for routing, so could be very suitable for web services.)
BTW, analyzing server logs is good to do in parallel with Google Analytics: it helps you appreciate the accuracy of each.