How to Redirect Stderr in Python

How to redirect stderr in Python?

You can't do anything in Python code that can capture errors during the compilation of that same code. How could it? If the compiler can't finish compiling the code, it won't run the code, so your redirection hasn't even taken effect yet.

That's where your (undesired) subprocess comes in. You can write Python code that redirects the stdout, then invokes the Python interpreter to compile some other piece of code.

Temporarily Redirect stdout/stderr

To solve the issue that some function might have cached sys.stdout stream as a local variable and therefore replacing the global sys.stdout won't work inside that function, you could redirect at a file descriptor level (sys.stdout.fileno()) e.g.:

from __future__ import print_function
import os
import sys

def some_function_with_cached_sys_stdout(stdout=sys.stdout):
    print('cached stdout', file=stdout)

with stdout_redirected(to=os.devnull), merged_stderr_stdout():
    print('stdout goes to devnull')
    some_function_with_cached_sys_stdout()
    print('stderr also goes to stdout that goes to devnull', file=sys.stderr)
print('stdout is back')
some_function_with_cached_sys_stdout()
print('stderr is back', file=sys.stderr)

stdout_redirected() redirects all output for sys.stdout.fileno() to a given filename, file object, or file descriptor (os.devnull in the example).

stdout_redirected() and merged_stderr_stdout() are defined here.

Redirecting stderr and stdout to a file does not write errors when using with statements in Python

This happens because the with statement closes the resource before it can write the error output, due to the occurrence of an exception.
Take a look at it in the docs.

The following code:

with EXPRESSION as TARGET:
    SUITE

is semantically equivalent to:

manager = (EXPRESSION)
enter = type(manager).__enter__
exit = type(manager).__exit__
value = enter(manager)
hit_except = False

try:
    TARGET = value
    SUITE
except:
    hit_except = True
    if not exit(manager, *sys.exc_info()):
        raise
finally:
    if not hit_except:
        exit(manager, None, None, None)

The reason the first case handles the exception, is because the resource is closed when python exits, not because of the close statement.

Redirect stdout to a file in Python?

If you want to do the redirection within the Python script, setting sys.stdout to a file object does the trick:

# for python3
import sys
with open('file', 'w') as sys.stdout:
    print('test')

A far more common method is to use shell redirection when executing (same on Windows and Linux):

$ python3 foo.py > file

---

Related Topics