Question

I need to limit the amount of time and cpu taken by external command line apps I spawn from a python process using subprocess.call , mainly because sometimes the spawned process gets stuck and pins the cpu at 99%.

nice and ulimit seem like reasonable ways to do this, but I’m not sure how they’d interact with subprocess.

• The limits look something like:
  • Kill the process if it’s taking more than 60 seconds
  • Limit it to 20% of cpu
• I want to apply the resource limiting to the subprocess, not to the python process that’s spawning the subprocesses.

Is there a way to apply nice and ulimit to the subprocess.call spawned process? Are there better python-native alternatives?

This is on a linux (ubuntu) system.

Answer 1

You can set limits for subprocesses with the ulimit and nice shell commands like this:

import subprocess
subprocess.Popen('ulimit -t 60; nice -n 15 cpuhog', shell=True)

This runs cpuhog with a limit of 60 seconds of CPU time and a niceness adjustment of 15. Note that there is no simple way to set a 20% CPU throttle as such. The process will use 100% CPU unless another (less nice) process also needs the CPU.