I have a python process (Pylons webapp) that is constantly using 10-30% of CPU. I’ll improve/tune logging to get some insight of what’s going on, but until then, are there any tools/techniques that allow to see what python process is doing, how many and how busy threads it has etc?
Update:
- configured access log which shows that there are no requests going on, webapp is just idling
- no point to plug in paste.profile in middleware chain since there are no requests, activity must be happening either in webapp’s worker threads or paster web server
- running paster like this: ‘python -m cProfile -o outfile /usr/bin/paster serve dev.ini’ and inspecting results shows that most time is spent in ‘posix.waitpid’. Paster runs webapp in subprocess, subprocess activity is not picked up by profiler
- looking into ;hacking PasteScript ‘serve’ command so that subprocesses would get profiled
Another update:
After much tinkering, sticking profiler in various places and getting familiar with PasteScript insides, I discovered that the constant CPU load goes away if application is started without ‘–reload’ parameter (this flag tells paster to restart itself if code changes, handy in development), which is fine in production environment.
Profiling might help you learn a bit of what it’s doing. If your sort the output by ‘time’ you will see which functions are chowing up cpu time, which should give you some good hints.