Python: How to get stdout after running os.system?
If all you need is the stdout
output, then take a look at subprocess.check_output()
:
import subprocess
batcmd="dir"
result = subprocess.check_output(batcmd, shell=True)
Because you were using os.system()
, you'd have to set shell=True
to get the same behaviour. You do want to heed the security concerns about passing untrusted arguments to your shell.
If you need to capture stderr
as well, simply add stderr=subprocess.STDOUT
to the call:
result = subprocess.check_output([batcmd], stderr=subprocess.STDOUT)
to redirect the error output to the default output stream.
If you know that the output is text, add text=True
to decode the returned bytes value with the platform default encoding; use encoding="..."
instead if that codec is not correct for the data you receive.
Assign output of os.system to a variable and prevent it from being displayed on the screen
From "https://stackoverflow.com/questions/1410976/equivalent-of-backticks-in-python", which I asked a long time ago, what you may want to use is popen
:
os.popen('cat /etc/services').read()
From the docs for Python 3.6,
This is implemented using subprocess.Popen; see that class’s
documentation for more powerful ways to manage and communicate with
subprocesses.
Here's the corresponding code for subprocess
:
import subprocess
proc = subprocess.Popen(["cat", "/etc/services"], stdout=subprocess.PIPE, shell=True)
(out, err) = proc.communicate()
print("program output:", out)
How to get the output from os.system()?
Use subprocess
:
import subprocess
print(subprocess.check_output(['nslookup', 'google.com']))
If the return code is not zero it will raise a CalledProcessError
exception:
try:
print(subprocess.check_output(['nslookup', 'google.com']))
except subprocess.CalledProcessError as err:
print(err)
os.system only returns the exit code of the command. Here 0
means success. Any other number stands for an operating-system-dependent error. The output goes to stdout of this process. subprocess intends to replace os.system
.
subprocess.check_output is a convenience wrapper around subprocess.Popen that simplifies your use case.
How to store output of os.system() in a variable
The os.system
return the exit code of the command.
To capture the output of the command, you can use subprocess.check_output
output = subprocess.check_output('users', shell=True)
Running shell command and capturing the output
In all officially maintained versions of Python, the simplest approach is to use the subprocess.check_output
function:
>>> subprocess.check_output(['ls', '-l'])
b'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
check_output
runs a single program that takes only arguments as input.1 It returns the result exactly as printed to stdout
. If you need to write input to stdin
, skip ahead to the run
or Popen
sections. If you want to execute complex shell commands, see the note on shell=True
at the end of this answer.
The check_output
function works in all officially maintained versions of Python. But for more recent versions, a more flexible approach is available.
Modern versions of Python (3.5 or higher): run
If you're using Python 3.5+, and do not need backwards compatibility, the new run
function is recommended by the official documentation for most tasks. It provides a very general, high-level API for the subprocess
module. To capture the output of a program, pass the subprocess.PIPE
flag to the stdout
keyword argument. Then access the stdout
attribute of the returned CompletedProcess
object:
>>> import subprocess
>>> result = subprocess.run(['ls', '-l'], stdout=subprocess.PIPE)
>>> result.stdout
b'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
The return value is a bytes
object, so if you want a proper string, you'll need to decode
it. Assuming the called process returns a UTF-8-encoded string:
>>> result.stdout.decode('utf-8')
'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
This can all be compressed to a one-liner if desired:
>>> subprocess.run(['ls', '-l'], stdout=subprocess.PIPE).stdout.decode('utf-8')
'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
If you want to pass input to the process's stdin
, you can pass a bytes
object to the input
keyword argument:
>>> cmd = ['awk', 'length($0) > 5']
>>> ip = 'foo\nfoofoo\n'.encode('utf-8')
>>> result = subprocess.run(cmd, stdout=subprocess.PIPE, input=ip)
>>> result.stdout.decode('utf-8')
'foofoo\n'
You can capture errors by passing stderr=subprocess.PIPE
(capture to result.stderr
) or stderr=subprocess.STDOUT
(capture to result.stdout
along with regular output). If you want run
to throw an exception when the process returns a nonzero exit code, you can pass check=True
. (Or you can check the returncode
attribute of result
above.) When security is not a concern, you can also run more complex shell commands by passing shell=True
as described at the end of this answer.
Later versions of Python streamline the above further. In Python 3.7+, the above one-liner can be spelled like this:
>>> subprocess.run(['ls', '-l'], capture_output=True, text=True).stdout
'total 0\n-rw-r--r-- 1 memyself staff 0 Mar 14 11:04 files\n'
Using run
this way adds just a bit of complexity, compared to the old way of doing things. But now you can do almost anything you need to do with the run
function alone.
Older versions of Python (3-3.4): more about check_output
If you are using an older version of Python, or need modest backwards compatibility, you can use the check_output
function as briefly described above. It has been available since Python 2.7.
subprocess.check_output(*popenargs, **kwargs)
It takes takes the same arguments as Popen
(see below), and returns a string containing the program's output. The beginning of this answer has a more detailed usage example. In Python 3.5+, check_output
is equivalent to executing run
with check=True
and stdout=PIPE
, and returning just the stdout
attribute.
You can pass stderr=subprocess.STDOUT
to ensure that error messages are included in the returned output. When security is not a concern, you can also run more complex shell commands by passing shell=True
as described at the end of this answer.
If you need to pipe from stderr
or pass input to the process, check_output
won't be up to the task. See the Popen
examples below in that case.
Complex applications and legacy versions of Python (2.6 and below): Popen
If you need deep backwards compatibility, or if you need more sophisticated functionality than check_output
or run
provide, you'll have to work directly with Popen
objects, which encapsulate the low-level API for subprocesses.
The Popen
constructor accepts either a single command without arguments, or a list containing a command as its first item, followed by any number of arguments, each as a separate item in the list. shlex.split
can help parse strings into appropriately formatted lists. Popen
objects also accept a host of different arguments for process IO management and low-level configuration.
To send input and capture output, communicate
is almost always the preferred method. As in:
output = subprocess.Popen(["mycmd", "myarg"],
stdout=subprocess.PIPE).communicate()[0]
Or
>>> import subprocess
>>> p = subprocess.Popen(['ls', '-a'], stdout=subprocess.PIPE,
... stderr=subprocess.PIPE)
>>> out, err = p.communicate()
>>> print out
.
..
foo
If you set stdin=PIPE
, communicate
also allows you to pass data to the process via stdin
:
>>> cmd = ['awk', 'length($0) > 5']
>>> p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
... stderr=subprocess.PIPE,
... stdin=subprocess.PIPE)
>>> out, err = p.communicate('foo\nfoofoo\n')
>>> print out
foofoo
Note Aaron Hall's answer, which indicates that on some systems, you may need to set stdout
, stderr
, and stdin
all to PIPE
(or DEVNULL
) to get communicate
to work at all.
In some rare cases, you may need complex, real-time output capturing. Vartec's answer suggests a way forward, but methods other than communicate
are prone to deadlocks if not used carefully.
As with all the above functions, when security is not a concern, you can run more complex shell commands by passing shell=True
.
Notes
1. Running shell commands: the shell=True
argument
Normally, each call to run
, check_output
, or the Popen
constructor executes a single program. That means no fancy bash-style pipes. If you want to run complex shell commands, you can pass shell=True
, which all three functions support. For example:
>>> subprocess.check_output('cat books/* | wc', shell=True, text=True)
' 1299377 17005208 101299376\n'
However, doing this raises security concerns. If you're doing anything more than light scripting, you might be better off calling each process separately, and passing the output from each as an input to the next, via
run(cmd, [stdout=etc...], input=other_output)
Or
Popen(cmd, [stdout=etc...]).communicate(other_output)
The temptation to directly connect pipes is strong; resist it. Otherwise, you'll likely see deadlocks or have to do hacky things like this.
Getting output from os.system (no subprocess)
=> As Martijn Pieters says, there is no way to retrieve the stdout of a os.system() call (because it spawns a subshell = a black box of which we know nothing but its return code). The solution is then to make Python act as a proxy between the final user and CAST3M (pass-through configuration : Python is the "middleman" listening to the user request and transmitting it "as-is" to CAST3M, then capturing CAST3M answer and printing it back to the user). For this, you have to use subprocess, threading and queue modules. If you do not mind the complexity it brings (and the alteration of the original HCI experience), here is a quick summary of links which may reveal usefull :
- Wrap subprocess' stdout/stderr
- Can you make a python subprocess output stdout and stderr as usual, but also capture the output as a string?
- Running an interactive command from within python
- Non-blocking read on a subprocess.PIPE in python
- http://log.ooz.ie/2013/02/interactive-subprocess-communication-in.html
=> Robᵩ provides a workaround for Linux only, by appointing the logging task to the "script" Linux tool. This allows to keep the "user<>CAST3M" interactivity untouched (no proxy here).
Getting the output of os.system in python and processing it after
You can use subprocess.check_output:
f = subprocess.check_output("./script.sh ls -l test1/test2/test.log",shell=True)
print(f)
You can split into a list of individual args without using shell=True:
f = subprocess.check_output(['./script.sh', 'ls', '-l', 'test1/test2/test.log']))
Redirecting stdio from a command in os.system() in Python
You could consider running the program via subprocess.Popen
, with subprocess.PIPE
communication, and then shove that output where ever you would like, but as is, os.system
just runs the command, and nothing else.
from subprocess import Popen, PIPE
p = Popen(['command', 'and', 'args'], stdout=PIPE, stderr=PIPE, stdin=PIPE)
output = p.stdout.read()
p.stdin.write(input)
Much more flexible in my opinion. You might want to look at the full documentation: Python Subprocess module
Related Topics
How to Compare String and Integer in Python
Print a String as Hexadecimal Bytes
Search for String in All Pandas Dataframe Columns and Filter
Python: Can Executable Zip Files Include Data Files
Repeating Each Element of a Numpy Array 5 Times
Limit Number of Threads in Numpy
When Are Parentheses Required Around a Tuple
Python JSON Parser Allow Duplicate Keys
Capture Keyboardinterrupt in Python Without Try-Except
Example Use of "Continue" Statement in Python
Matching Nested Structures with Regular Expressions in Python