How to Prevent a C Shared Library to Print on Stdout in Python

How do I prevent a C shared library to print on stdout in python?

Based on @Yinon Ehrlich's answer. This variant tries to avoid leaking file descriptors:

import os
import sys
from contextlib import contextmanager

@contextmanager
def stdout_redirected(to=os.devnull):
    '''
    import os

    with stdout_redirected(to=filename):
        print("from Python")
        os.system("echo non-Python applications are also supported")
    '''
    fd = sys.stdout.fileno()

    ##### assert that Python and C stdio write using the same file descriptor
    ####assert libc.fileno(ctypes.c_void_p.in_dll(libc, "stdout")) == fd == 1

    def _redirect_stdout(to):
        sys.stdout.close() # + implicit flush()
        os.dup2(to.fileno(), fd) # fd writes to 'to' file
        sys.stdout = os.fdopen(fd, 'w') # Python writes to fd

    with os.fdopen(os.dup(fd), 'w') as old_stdout:
        with open(to, 'w') as file:
            _redirect_stdout(to=file)
        try:
            yield # allow code to be run with the redirected stdout
        finally:
            _redirect_stdout(to=old_stdout) # restore stdout.
                                            # buffering and flags such as
                                            # CLOEXEC may be different

Suppressing output of module calling outside library

Open /dev/null for writing, use os.dup() to copy stdout, and use os.dup2() to copy your open /dev/null to stdout. Use os.dup2() to copy your copied stdout back to the real stdout after.

devnull = open('/dev/null', 'w')
oldstdout_fno = os.dup(sys.stdout.fileno())
os.dup2(devnull.fileno(), 1)
makesomenoise()
os.dup2(oldstdout_fno, 1)

In python, how to capture the stdout from a c++ shared library to a variable

Python's sys.stdout object is simply a Python wrapper on top of the usual stdout file descriptor—changing it only affects the Python process, not the underlying file descriptor. Any non-Python code, whether it be another executable which was exec'ed or a C shared library which was loaded, won't understand that and will continue using the ordinary file descriptors for I/O.

So, in order for the shared library to output to a different location, you need to change the underlying file descriptor by opening a new file descriptor and then replacing stdout using os.dup2(). You could use a temporary file for the output, but it's a better idea to use a pipe created with os.pipe(). However, this has the danger for deadlock, if nothing is reading the pipe, so in order to prevent that we can use another thread to drain the pipe.

Below is a full working example which does not use temporary files and which is not susceptible to deadlock (tested on Mac OS X).

C shared library code:

// test.c
#include <stdio.h>

void hello(void)
{
  printf("Hello, world!\n");
}

Compiled as:

$ clang test.c -shared -fPIC -o libtest.dylib

Python driver:

import ctypes
import os
import sys
import threading

print 'Start'

liba = ctypes.cdll.LoadLibrary('libtest.dylib')

# Create pipe and dup2() the write end of it on top of stdout, saving a copy
# of the old stdout
stdout_fileno = sys.stdout.fileno()
stdout_save = os.dup(stdout_fileno)
stdout_pipe = os.pipe()
os.dup2(stdout_pipe[1], stdout_fileno)
os.close(stdout_pipe[1])

captured_stdout = ''
def drain_pipe():
    global captured_stdout
    while True:
        data = os.read(stdout_pipe[0], 1024)
        if not data:
            break
        captured_stdout += data

t = threading.Thread(target=drain_pipe)
t.start()

liba.hello()  # Call into the shared library

# Close the write end of the pipe to unblock the reader thread and trigger it
# to exit
os.close(stdout_fileno)
t.join()

# Clean up the pipe and restore the original stdout
os.close(stdout_pipe[0])
os.dup2(stdout_save, stdout_fileno)
os.close(stdout_save)

print 'Captured stdout:\n%s' % captured_stdout

How to block calls to print?

Python lets you overwrite standard output (stdout) with any file object. This should work cross platform and write to the null device.

import sys, os

# Disable
def blockPrint():
    sys.stdout = open(os.devnull, 'w')

# Restore
def enablePrint():
    sys.stdout = sys.__stdout__

print 'This will print'

blockPrint()
print "This won't"

enablePrint()
print "This will too"

If you don't want that one function to print, call blockPrint() before it, and enablePrint() when you want it to continue. If you want to disable all printing, start blocking at the top of the file.

Capturing print output from shared library called from python with ctypes module

We can use os.dup2() and os.pipe() to replace the entire stdout file descriptor (fd 1) with a pipe we can read from ourselves. You can do the same thing with stderr (fd 2).

This example uses select.select() to see if the pipe (our fake stdout) has data waiting to be written, so we can print it safely without blocking execution of our script.

As we are completely replacing the stdout file descriptor for this process and any subprocesses, this example can even capture output from child processes.

import os, sys, select

# the pipe would fail for some reason if I didn't write to stdout at some point
# so I write a space, then backspace (will show as empty in a normal terminal)
sys.stdout.write(' \b')
pipe_out, pipe_in = os.pipe()
# save a copy of stdout
stdout = os.dup(1)
# replace stdout with our write pipe
os.dup2(pipe_in, 1)

# check if we have more to read from the pipe
def more_data():
        r, _, _ = select.select([pipe_out], [], [], 0)
        return bool(r)

# read the whole pipe
def read_pipe():
        out = ''
        while more_data():
                out += os.read(pipe_out, 1024)

        return out

# testing print methods
import ctypes
libc = ctypes.CDLL('libc.so.6')

print 'This text gets captured by myStdOut'
libc.printf('This text fails to be captured by myStdOut\n')

# put stdout back in place 
os.dup2(stdout, 1)
print 'Contents of our stdout pipe:'
print read_pipe()

Rerouting stdin and stdout from C

Why use freopen()? The C89 specification has the answer in one of the endnotes for the section on <stdio.h>:

116. The primary use of the freopen function is to change the file associated with a standard
text stream (stderr,
stdin, or stdout), as those identifiers need not be
modifiable lvalues to which the value
returned by the fopen function
may be assigned.

freopen is commonly misused, e.g. stdin = freopen("newin", "r", stdin);. This is no more portable than fclose(stdin); stdin = fopen("newin", "r");. Both expressions attempt to assign to stdin, which is not guaranteed to be assignable.

The right way to use freopen is to omit the assignment: freopen("newin", "r", stdin);

Silence the stdout of a function in Python without trashing sys.stdout and restoring each function call

Assigning the stdout variable as you're doing has no effect whatsoever, assuming foo contains print statements -- yet another example of why you should never import stuff from inside a module (as you're doing here), but always a module as a whole (then use qualified names). The copy is irrelevant, by the way. The correct equivalent of your snippet is:

import sys
save_stdout = sys.stdout
sys.stdout = open('trash', 'w')
foo()
sys.stdout = save_stdout

Now, when the code is correct, is the time to make it more elegant or fast. For example, you could use an in-memory file-like object instead of file 'trash':

import sys
import io
save_stdout = sys.stdout
sys.stdout = io.BytesIO()
foo()
sys.stdout = save_stdout

for elegance, a context is best, e.g:

import contextlib
import io
import sys

@contextlib.contextmanager
def nostdout():
    save_stdout = sys.stdout
    sys.stdout = io.BytesIO()
    yield
    sys.stdout = save_stdout

once you have defined this context, for any block in which you don't want a stdout,

with nostdout():
    foo()

More optimization: you just need to replace sys.stdout with an object that has a no-op write method. For example:

import contextlib
import sys

class DummyFile(object):
    def write(self, x): pass

@contextlib.contextmanager
def nostdout():
    save_stdout = sys.stdout
    sys.stdout = DummyFile()
    yield
    sys.stdout = save_stdout

to be used the same way as the previous implementation of nostdout. I don't think it gets any cleaner or faster than this;-).

Capture print output from a c-module in python

The subprocess solution found here https://stackoverflow.com/a/5136686/4270148, actually works!

import subprocess
proc = subprocess.Popen(["python", "-c",
    "cnf = [[1, -5, 4], [-1, 5, 3, 4], [-3, -4]];\
    import pycosat;\
    pycosat.solve(cnf,verbose=5);"],
    stdout=subprocess.PIPE)
out = proc.communicate()[0]

I don't like that the way the program is passed (as an eval-string), but at least it works.

---

Related Topics