How to Suppress or Capture the Output of Subprocess.Run()

How to suppress or capture the output of subprocess.run()?

Here is how to suppress output, in order of decreasing levels of cleanliness. They assume you are on Python 3.

1. You can redirect to the special subprocess.DEVNULL target.

import subprocess

# To redirect stdout (only):
subprocess.run(['ls', '-l'], stdout=subprocess.DEVNULL)

# to redirect stderr to /dev/null as well:
subprocess.run(['ls', '-l'], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)

# Alternatively, you can merge stderr and stdout streams and redirect
# the one stream to /dev/null
subprocess.run(['ls', '-l'], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)

2. If you want a fully manual method, can redirect to /dev/null by opening the file handle yourself. Everything else would be identical to method #1.

import os
import subprocess

with open(os.devnull, 'w') as devnull:
    subprocess.run(['ls', '-l'], stdout=devnull)

---

Here is how to capture output (to use later or parse), in order of decreasing levels of cleanliness. They assume you are on Python 3.

NOTE: The below examples use text=True.

• This causes the STDOUT and STDERR to be captured as str instead of bytes.
  
  • Omit text=True to get bytes data
• text=True is Python >= 3.7 only, use universal_newlines=True on Python <= 3.6
  
  • universal_newlines=True is identical to text=True but more verbose to type but should exist on all Python versions

1. If you simply want to capture both STDOUT and STDERR independently, AND you are on Python >= 3.7, use capture_output=True.

import subprocess

result = subprocess.run(['ls', '-l'], capture_output=True, text=True)
print(result.stdout)
print(result.stderr)

2. You can use subprocess.PIPE to capture STDOUT and STDERR independently. This works on any version of Python that supports subprocess.run.

import subprocess

result = subprocess.run(['ls', '-l'], stdout=subprocess.PIPE, text=True)
print(result.stdout)

# To also capture stderr...
result = subprocess.run(['ls', '-l'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print(result.stdout)
print(result.stderr)

# To mix stdout and stderr into a single string
result = subprocess.run(['ls', '-l'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
print(result.stdout)

How to hide output of subprocess

For python >= 3.3, Redirect the output to DEVNULL:

import os
import subprocess

retcode = subprocess.call(['echo', 'foo'], 
    stdout=subprocess.DEVNULL,
    stderr=subprocess.STDOUT)

For python <3.3, including 2.7 use:

FNULL = open(os.devnull, 'w')
retcode = subprocess.call(['echo', 'foo'], 
    stdout=FNULL, 
    stderr=subprocess.STDOUT)

It is effectively the same as running this shell command:

retcode = os.system("echo 'foo' &> /dev/null")

subprocess.run not suppressing all console output

Based on Eryk Sun comment, I had to use

subprocess.run(
    'subinacl.exe /service "foo" display',
    stdout=subprocess.DEVNULL,
    stderr=subprocess.STDOUT, 
    creationflags=subprocess.CREATE_NO_WINDOW
)

How to capture output without preventing it from output in python

How about this?

from subprocess import Popen, PIPE

proc = Popen(["/usr/bin/nc", "-l", "9999"], stdout=PIPE)

buffer = []
line = proc.stdout.readline()
while line:
    buffer.append(line)
    print "LINE", line.strip()
    line = proc.stdout.readline()

print "buffer", ''.join(buffer)

Using another terminal send some text

nc localhost 9999
# type something.  the text should appear from the python code

Break the nc, and you get the output in buffer as well

Suppressing output in python subprocess call

You can use the stdout= and stderr= parameters to subprocess.call() to direct stdout or stderr to a file descriptor of your choice. So maybe something like this:

import os

devnull = open(os.devnull, 'w')
subprocess.call(shlex.split(
    '/usr/local/itms/bin/iTMSTransporter -m lookupMetadata '
    '-apple_id %s -destination %s' % (self,apple_id, self.destination)),
  stdout=devnull, stderr=devnull)

Using subprocess.PIPE, if you're not reading from the pipe, could cause your program to block if it generates a lot of output.

Update

As @yanlend mentions in a comment, newer (3.x) versions of Python include subprocess.DEVNULL to solve this problem in a more convenient and portable fashion. In that case, the code would look like:

subprocess.call(shlex.split(
    '/usr/local/itms/bin/iTMSTransporter -m lookupMetadata '
    '-apple_id %s -destination %s' % (self,apple_id, self.destination)),
  stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)

Suppress output of subprocess

Ok, the output is now clear. I do not exactly know why, but the command ssh -t -t puts the local terminal in raw mode. It makes sense anyway, because it is intended to allow you to directly use curses programs (such as vi) on the remote, and in that case, no conversion should be done, not even the simple \n -> \r\n that allows a simple new line to leave the cursor on first column. But I could not find a reference on this in ssh documentation.

It (-t -t) allows you to kill the remote process because the raw mode let the Ctrl + C to be sent to the remote instead of being processed by the local tty driver.

IMHO, this is design smell, because you only use a side effect of the pty allocation to pass a Ctrl + C to the remote and you suffer for another side effect which is the raw mode on local system. You should rather process the standard input (stdinput = subprocess.PIPE) and explicitely send a chr(3) when you input a special character on local keyboard, or install a signal handler for SIG-INT that does it.

Alternatively, as a workaround, you can simply use something like os.system("stty opost -igncr") (or better its subprocess equivalent) after starting the remote command to reset the local terminal in an acceptable mode.

Constantly print Subprocess output while process is running

You can use iter to process lines as soon as the command outputs them: lines = iter(fd.readline, ""). Here's a full example showing a typical use case (thanks to @jfs for helping out):

from __future__ import print_function # Only Python 2.x
import subprocess

def execute(cmd):
    popen = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
    for stdout_line in iter(popen.stdout.readline, ""):
        yield stdout_line 
    popen.stdout.close()
    return_code = popen.wait()
    if return_code:
        raise subprocess.CalledProcessError(return_code, cmd)

# Example
for path in execute(["locate", "a"]):
    print(path, end="")

---

Related Topics