Python Script Prints Output of Os.System Before Print

Python script prints output of os.system before print

When you're outputting to a pipe, Python buffers your output that's written to sys.stdout and outputs it after a flush, or after it's overflown, or upon closing (when the program exits). While it'll buffer the print calls, system calls output directly into stdout and their output won't be buffered. That's why you're seeing such precedence. To avoid that, use python -u:

python -u test.py > test.out; cat test.out

See more info here.

EDIT: explanation on when the buffer will be flushed.

Assign output of os.system to a variable and prevent it from being displayed on the screen

From "Equivalent of Bash Backticks in Python", which I asked a long time ago, what you may want to use is popen:

os.popen('cat /etc/services').read()

From the docs for Python 3.6,

This is implemented using subprocess.Popen; see that class’s
documentation for more powerful ways to manage and communicate with
subprocesses.

---

Here's the corresponding code for subprocess:

import subprocess

proc = subprocess.Popen(["cat", "/etc/services"], stdout=subprocess.PIPE, shell=True)
(out, err) = proc.communicate()
print "program output:", out

Is there a way to print text with os.system in python?

It would seem that os.system and print use different buffers that are flushed in different times! (for explanation see here and crosslink)

The following works for me (force flushing the output):

import os
import sys
from time import sleep

i=0
while i<30:
    print "----CPU----"
    sys.stdout.flush()
    op=os.system("ps -e -o pid,%cpu,%mem,vsize,time,command | grep java | grep -v grep ")
    sys.stdout.flush()
    print i
    sys.stdout.flush()
    i+=1
    sleep(1.0/3.0)

Alternatively:

• use the -u flag to disable buffered output (python -u /tmp/test.py 1> /tmp/test.log)
• use subprocess.check_output() or
• pipe the idividual processes (instead of spawning a shell) which is a bit more long-winded (see here)

Hope it helps

How to avoid os.system() printing out return value in python

Actually, result is only the return status as an integer. The thing you're calling writes to stdout, which it inherits from your program, so you're seeing it printed out immediately. It's never available to your program.

Check out the subprocess module docs for more info:

http://docs.python.org/library/subprocess.html

Including capturing output, and invoking shells in different ways.

Python: How to get stdout after running os.system?

If all you need is the stdout output, then take a look at subprocess.check_output():

import subprocess

batcmd="dir"
result = subprocess.check_output(batcmd, shell=True)

Because you were using os.system(), you'd have to set shell=True to get the same behaviour. You do want to heed the security concerns about passing untrusted arguments to your shell.

If you need to capture stderr as well, simply add stderr=subprocess.STDOUT to the call:

result = subprocess.check_output([batcmd], stderr=subprocess.STDOUT)

to redirect the error output to the default output stream.

If you know that the output is text, add text=True to decode the returned bytes value with the platform default encoding; use encoding="..." instead if that codec is not correct for the data you receive.

Python os.system: Order of commands

Output is buffered. You have to flush this buffer:

import os
import sys
print('Hi')
sys.stdout.flush()
os.system('cat a.py')

How to get the output from os.system()?

Use subprocess:

import subprocess
print(subprocess.check_output(['nslookup', 'google.com']))

If the return code is not zero it will raise a CalledProcessError exception:

try:
    print(subprocess.check_output(['nslookup', 'google.com']))
except subprocess.CalledProcessError as err:
    print(err)

os.system only returns the exit code of the command. Here 0 means success. Any other number stands for an operating-system-dependent error. The output goes to stdout of this process. subprocess intends to replace os.system.

subprocess.check_output is a convenience wrapper around subprocess.Popen that simplifies your use case.

Getting the output of os.system in python and processing it after

You can use subprocess.check_output:

f = subprocess.check_output("./script.sh ls -l test1/test2/test.log",shell=True)
print(f)

You can split into a list of individual args without using shell=True:

f = subprocess.check_output(['./script.sh', 'ls', '-l', 'test1/test2/test.log']))

Hiding console output produced by os.system

To answer the question based on its title in the most generic form:

To suppress all output from os.system(), append >/dev/null 2>&1 to the shell command, which silences both stdout and stderr; e.g.:

import os
os.system('echo 3 | sudo tee /proc/sys/vm/drop_caches >/dev/null 2>&1')

Note that os.system() by design passes output from the calling process' stdout and stderr streams through to the console (terminal) - your Python code never sees them.

Also, os.system() does not raise an exception if the shell command fails and instead returns an exit code; note that it takes additional work to extract the shell command's true exit code: you need to extract the high byte from the 16-bit value returned, by applying >> 8 (although you can rely on a return value other than 0 implying an error condition).

---

Given the above limitations of os.system(), it is generally worthwhile to use the functions in the subprocess module instead:

For instance, subprocess.check_output() could be used as follows:

import subprocess
subprocess.check_output('echo 3 | sudo tee /proc/sys/vm/drop_caches', shell=True) 

The above will:

• capture stdout output and return it (with the return value being ignored in the example above)
• pass stderr output through; passing stderr=subprocess.STDOUT as an additional argument would also capture stderr.
• raise an error, if the shell command fails.

Note: Python 3.5 introduced subprocess.run(), a more flexible successor to both os.system() and subprocess.check_output() - see https://docs.python.org/3.5/library/subprocess.html#using-the-subprocess-module

---

Note:

• The reason that the OP is employing tee in the first place - despite not being interested in stdout output - is that a naïve attempt to use > ... instead would be interpreted before sudo is invoked, and thus fail, because the required privileges to write to /proc/sys/... haven't been granted yet.
• Whether you're using os.system() or a subprocess function, stdin is not affected by default, so if you're invoking your script from a terminal, you'll get an interactive password prompt when the sudo command is encountered (unless the credentials have been cached).

---

Related Topics