How to Get All of the Output from My .Exe Using Subprocess and Popen

How do I get all of the output from my .exe using subprocess and Popen?

To get all stdout as a string:

from subprocess import check_output as qx

cmd = r'C:\Tools\Dvb_pid_3_0.exe'
output = qx(cmd)

To get both stdout and stderr as a single string:

from subprocess import STDOUT

output = qx(cmd, stderr=STDOUT)

To get all lines as a list:

lines = output.splitlines()

To get lines as they are being printed by the subprocess:

from subprocess import Popen, PIPE

p = Popen(cmd, stdout=PIPE, bufsize=1)
for line in iter(p.stdout.readline, ''):
    print line,
p.stdout.close()
if p.wait() != 0:
   raise RuntimeError("%r failed, exit status: %d" % (cmd, p.returncode))

Add stderr=STDOUT to the Popen() call to merge stdout/stderr.

Note: if cmd uses block-buffering in the non-interactive mode then lines won't appear until the buffer flushes. winpexpect module might be able to get the output sooner.

To save the output to a file:

import subprocess

with open('output.txt', 'wb') as f:
    subprocess.check_call(cmd, stdout=f)

# to read line by line
with open('output.txt') as f:
    for line in f:
        print line,

If cmd always requires input even an empty one; set stdin:

import os

with open(os.devnull, 'rb') as DEVNULL:
    output = qx(cmd, stdin=DEVNULL) # use subprocess.DEVNULL on Python 3.3+

You could combine these solutions e.g., to merge stdout/stderr, and to save the output to a file, and to provide an empty input:

import os
from subprocess import STDOUT, check_call as x

with open(os.devnull, 'rb') as DEVNULL, open('output.txt', 'wb') as f:
    x(cmd, stdin=DEVNULL, stdout=f, stderr=STDOUT)

To provide all input as a single string you could use .communicate() method:

#!/usr/bin/env python
from subprocess import Popen, PIPE

cmd = ["python", "test.py"]
p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE, universal_newlines=True)
stdout_text, stderr_text = p.communicate(input="1\n\n")

print("stdout: %r\nstderr: %r" % (stdout_text, stderr_text))
if p.returncode != 0:
    raise RuntimeError("%r failed, status code %d" % (cmd, p.returncode))

where test.py:

print raw_input('abc')[::-1]
raw_input('press enter to exit')

If your interaction with the program is more like a conversation than you might need winpexpect module. Here's an example from pexpect docs:

# This connects to the openbsd ftp site and
# downloads the recursive directory listing.
from winpexpect import winspawn as spawn

child = spawn ('ftp ftp.openbsd.org')
child.expect ('Name .*: ')
child.sendline ('anonymous')
child.expect ('Password:')
child.sendline ('noah@example.com')
child.expect ('ftp> ')
child.sendline ('cd pub')
child.expect('ftp> ')
child.sendline ('get ls-lR.gz')
child.expect('ftp> ')
child.sendline ('bye')

To send special keys such as F3, F10 on Windows you might need SendKeys module or its pure Python implementation SendKeys-ctypes. Something like:

from SendKeys import SendKeys

SendKeys(r"""
    {LWIN}
    {PAUSE .25}
    r
    C:\Tools\Dvb_pid_3_0.exe{ENTER}
    {PAUSE 1}
    1{ENTER}
    {PAUSE 1}
    2{ENTER}
    {PAUSE 1}
    {F3}
    {PAUSE 1}
    {F10}
""")

It doesn't capture output.

How can I tell if an exe opened with subprocess has finished launching?

Adding this as a result from the discussion in the comments.

In order to detect the state of your exe beeing finished with loading you need the subprocess to signal that state to the calling script as there is no generic definition or way to detect that state (eg. through exit codes etc.).

A bit crude but also the most simple solution could be to let your exe print a defined string to stdout and make the subprocess handler above listen to it.

When you have your exe print a string like EXE Loaded. on stdout once it reaches your desired state (GUI visible or whatever),
you then would poll (or better use some sort of async io stuff :) ) the subprocess output from the calling script like so:

process = subprocess.Popen(['myFile.exe'], stdout=PIPE)

while True:
    output = process.stdout.readline()
    if process.poll() is not None and output == 'EXE Loaded.':
        break

This will basically poll the stdout of the launched exe and continue to do so until it receives EXE Loaded. as string.
Please be aware that this shall just explain the most basic idea behind it.
In a real example you would probably use some sort of more modern asynchronous wait stuff . Also think about using the run function like mentioned in the other answer. It's advised to not use POpen directly unless you cannot make due with the higher level API of subprocess.

Related Topics