Using Module 'Subprocess' With Timeout

Using module 'subprocess' with timeout

In Python 3.3+:

from subprocess import STDOUT, check_output

output = check_output(cmd, stderr=STDOUT, timeout=seconds)

output is a byte string that contains command's merged stdout, stderr data.

check_output raises CalledProcessError on non-zero exit status as specified in the question's text unlike proc.communicate() method.

I've removed shell=True because it is often used unnecessarily. You can always add it back if cmd indeed requires it. If you add shell=True i.e., if the child process spawns its own descendants; check_output() can return much later than the timeout indicates, see Subprocess timeout failure.

The timeout feature is available on Python 2.x via the subprocess32 backport of the 3.2+ subprocess module.

Python use timeout for subprocess with Popen

You can use the timeout or waitmax commands to set a time limit on the process you are running with Popen. For instance, to run a tail -f command for a maximum of 10 seconds -

import subprocess
process=subprocess.Popen(['timeout' ,'10', 'tail', '-f', '/var/log/syslog'], stdout=subprocess.PIPE)
out,err = process.communicate()

print out
Apr 26 21:40:01 linubuvma CRON[49447]: (smmsp) CMD (test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp)
Apr 26 21:45:01 linubuvma CRON[50065]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Apr 26 21:55:01 linubuvma CRON[51271]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Apr 26 22:00:01 linubuvma CRON[51871]: (smmsp) CMD (test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp)
Apr 26 22:05:01 linubuvma CRON[52491]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Apr 26 22:09:01 linubuvma CRON[52975]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime))
Apr 26 22:15:01 linubuvma CRON[53707]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Apr 26 22:17:01 linubuvma CRON[53951]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Apr 26 22:20:01 linubuvma CRON[54311]: (smmsp) CMD (test -x /etc/init.d/sendmail && /usr/share/sendmail/sendmail cron-msp)
Apr 26 22:25:01 linubuvma CRON[54937]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)

The tail commands terminated exactly after 10 seconds.

Adding timeout in subprocess.check_output

Python 2.7 does not support timeout parameter. You can instead use EasyProcess. This is a layer on top of subprocess module and pretty easy to use.

Python subprocess kill with timeout

The documentation explicitly states that the process should be killed:

from the docs for subprocess.run:

"The timeout argument is passed to Popen.communicate(). If the timeout expires, the child process will be killed and waited for. The TimeoutExpired exception will be re-raised after the child process has terminated."

But in your case you're using shell=True, and I've seen issues like that before, because the blocking process is a child of the shell process.

I don't think you need shell=True if you decompose your arguments properly and your scripts have the proper shebang. You could try this:

result=run(
  [os.path.join('utilities/shell_scripts',self.language_conf[key][1]), self.proc_dir, config.main_file],  # don't compose argument line yourself
            shell=False,  # no shell wrapper
            check=True,
            stdout=PIPE,
            stderr=PIPE, 
            universal_newlines=True, 
            timeout=30,
            bufsize=100)

note that I can reproduce this issue very easily on Windows (using Popen, but it's the same thing):

import subprocess,time

p=subprocess.Popen("notepad",shell=True)
time.sleep(1)
p.kill()

=> notepad stays open, probably because it manages to detach from the parent shell process.

import subprocess,time

p=subprocess.Popen("notepad",shell=False)
time.sleep(1)
p.kill()

=> notepad closes after 1 second

Funnily enough, if you remove time.sleep(), kill() works even with shell=True probably because it successfully kills the shell which is launching notepad.

I'm not saying you have exactly the same issue, I'm just demonstrating that shell=True is evil for many reasons, and not being able to kill/timeout the process is one more reason.

However, if you need shell=True for a reason, you can use psutil to kill all the children in the end. In that case, it's better to use Popen so you get the process id directly:

import subprocess,time,psutil

parent=subprocess.Popen("notepad",shell=True)
for _ in range(30): # 30 seconds
    if parent.poll() is not None:  # process just ended
      break
    time.sleep(1)
else:
   # the for loop ended without break: timeout
   parent = psutil.Process(parent.pid)
   for child in parent.children(recursive=True):  # or parent.children() for recursive=False
       child.kill()
   parent.kill()

(source: how to kill process and child processes from python?)

that example kills the notepad instance as well.

Capture output from subprocess with a timeout set

You can still use the capture_output attribute from your original code, and when the TimeoutExpired exception is thrown, you can collect stdout and stderr from there. For example:

try:
    proc = subprocess.run([commandAndFlags], capture_output=True, text=True, timeout=2000)
    outs = proc.stdout
    errs = proc.stderr
except subprocess.TimeoutExpired as timeErr:
    outs = timeErr.stdout
    errs = timeErr.stderr

Python subprocess timeout?

I would advise taking a look at the Timer class in the threading module. I used it to implement a timeout for a Popen.

First, create a callback:

def timeout( p ):
    if p.poll() is None:
        print 'Error: process taking too long to complete--terminating'
        p.kill()

Then open the process:

proc = Popen( ... )

Then create a timer that will call the callback, passing the process to it.

t = threading.Timer( 10.0, timeout, [proc] )
t.start()
t.join()

Somewhere later in the program, you may want to add the line:

t.cancel()

Otherwise, the python program will keep running until the timer has finished running.

EDIT: I was advised that there is a race condition that the subprocess p may terminate between the p.poll() and p.kill() calls. I believe the following code can fix that:

import errno

def timeout( p ):
    if p.poll() is None:
        try:
            p.kill()
            print 'Error: process taking too long to complete--terminating'
        except OSError as e:
            if e.errno != errno.ESRCH:
                raise

Though you may want to clean the exception handling to specifically handle just the particular exception that occurs when the subprocess has already terminated normally.

Executing popen with timeout

The Popen.wait takes an optional timeout parameter. You an use this to wait for completion only for a specific time. If the timeout triggers, you can terminate the process.

process = subprocess.call(cmd)
try:
    # if this returns, the process completed
    process.wait(timeout=30)
except subprocess.TimeoutExpired:
    process.terminate()

Since Python 3.5, you can also use the subprocess.run convenience function.

subprocess.run(cmd, timeout=30)

Note that this will still raise TimeoutExpired but automatically terminate the subprocess.

---

Related Topics