Python 2.X - Write Binary Output to Stdout

Python 2.x - Write binary output to stdout?

Which platform are you on?

You could try this recipe if you're on Windows (the link suggests it's Windows specific anyway).

if sys.platform == "win32":
    import os, msvcrt
    msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)

There are some references on the web that there would/should be a function in Python 3.1 to reopen sys.stdout in binary mode but I don't really know if there's a better alternative then the above for Python 2.x.

How to write binary data to stdout in python 3?

A better way:

import sys
sys.stdout.buffer.write(b"some binary data")

Writing bytes to standard output in a way compatible with both, python2 and python3

No need to test, just use getattr():

# retrieve stdout as a binary file object
output = getattr(sys.stdout, 'buffer', sys.stdout)

This retrieves the .buffer attribute on sys.stdout, but if it doesn't exist (Python 2) it'll return the sys.stdout object itself instead.

Python 2:

>>> import sys
>>> getattr(sys.stdout, 'buffer', sys.stdout)
<open file '<stdout>', mode 'w' at 0x100254150>

Python 3:

>>> import sys
>>> getattr(sys.stdout, 'buffer', sys.stdout)
<_io.BufferedWriter name='<stdout>'>

Take into account that in Python 2, stdout is still opened in text mode, newlines are still translated to os.linesep when writing. The Python 3 BufferedWriter object won't do this for you.

how to write bytes to stdout in python3.3

You can use os.fdopen to reopen stdout with different mode. You can get the file number for stdout with sys.stdout.fileno().

Example:

>>> fp = os.fdopen(sys.stdout.fileno(), 'wb')
>>> fp.write(b'Hello, world')
12
>>> sys.stdout.mode
'w'
>>> fp.mode
'wb'
>>> fp.flush()
Hello, world>>> 

What is the simplest way to write to stdout in binary mode?

You can use setmode(fileno(stdout), O_BINARY)

Wrap it in an ifdef if you want to keep it compatible with Linux.

See also: https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setmode?view=vs-2017

Why is the output different with Python 2.7 than Python 3.10?

I can't run this under Python 3.10.1 (Windows, 64-bit):

**Traceback (most recent call last):
  File ... simple.py", line 39, in <module>
    sys.stdout.write(chr(byte_acc))
  File  ... \lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\x80' in position 0: character maps to <undefined>

If I change the 2 instances of chr() to str() (so that it prints out a string representation of the byte's decimal value instead), it produces the same output under 3.10.1 and 2.7.11.

So you're getting burned by whatever default Unicode encoding scheme is used by your Python for sys.stdout under Python 3.

If I set an envar like so (syntax may differ under your OS):

set PYTHONIOENCODING=latin-1

then both Pythons produce the same output using chr().

One way

Here's one way to "fix it":

import sys
from sys import stdout

if hasattr(stdout, "buffer"): # Python >= 3
    def putbyte(b):
        assert 0 <= b < 256
        stdout.buffer.write(bytes([b]))
else: # before Python 3
    def putbyte(b):
        assert 0 <= b < 256
        stdout.write(chr(b))

and then change your code to use putbyte(byte_acc) instead of the current sys.stdout.write(chr(byte_acc)).

That's not quite enough, though. Writing to the internal buffer yourself also makes you responsible for buffer management across uses. After the current

sys.stdout.write("P4\n%d %d\n" % (w, h))

you also need to add

sys.stdout.flush()

to get the output string into the buffer before you add additional output bytes.

How can I export binary data from Python subprocess command through STDOUT?

Turns out, this has nothing to do with Python. It's a Ghostscript error. As pointed out in this post: Prevent Ghostscript from writing errors to standard output, Ghostscript writes errors to stdout, which corrupts files that are piped out.

Thanks to @Jean-François Fabre who suggested I look in the binary files.

---

Related Topics