How to Get a Thread Safe Print in Python 2.6

How do I get a thread safe print in Python 2.6?

Interesting problem -- considering all the things that happen within a print statement, including the setting and checking of the softspace attribute, making it "threadsafe" (meaning, actually: a thread that's printing only yields "control of standard output" to another thread when it's printing a newline, so that each entire line that's output is guaranteed to come from a single thread) was a bit of a challenge (the usual easy approach to actual thread safety -- delegating a separate thread to exclusively "own" and handle sys.stdout, communicate to it via Queue.Queue -- isn't all that useful, since the problem is not thread safety [[even with a plain print there is no risk of crashing and the characters that end up on standard output are exactly those which get printed]] but the need for mutual exclusion among threads for an extended range of operations).

So, I think I made it...:

import random
import sys
import thread
import threading
import time

def wait():
  time.sleep(random.random())
  return 'W'

def targ():
  for n in range(8):
    wait()
    print 'Thr', wait(), thread.get_ident(), wait(), 'at', wait(), n

tls = threading.local()

class ThreadSafeFile(object):
  def __init__(self, f):
    self.f = f
    self.lock = threading.RLock()
    self.nesting = 0

  def _getlock(self):
    self.lock.acquire()
    self.nesting += 1

  def _droplock(self):
    nesting = self.nesting
    self.nesting = 0
    for i in range(nesting):
      self.lock.release()

  def __getattr__(self, name):
    if name == 'softspace':
      return tls.softspace
    else:
      raise AttributeError(name)

  def __setattr__(self, name, value):
    if name == 'softspace':
      tls.softspace = value
    else:
      return object.__setattr__(self, name, value)

  def write(self, data):
    self._getlock()
    self.f.write(data)
    if data == '\n':
      self._droplock()

# comment the following statement out to get guaranteed chaos;-)
sys.stdout = ThreadSafeFile(sys.stdout)

thrs = []
for i in range(8):
  thrs.append(threading.Thread(target=targ))
print 'Starting'
for t in thrs:
  t.start()
for t in thrs:
  t.join()
print 'Done'

The calls to wait are intended to guarantee chaotically mixed output in the absence of this mutual exclusion guarantee (whence the comment). With the wrapping, i.e., the above code exactly as it looks there, and (at least) Python 2.5 and up (I believe this may run in earlier versions, too, but I don't have any easily at hand to check) the output is:

Thr W -1340583936 W at W 0
Thr W -1340051456 W at W 0
Thr W -1338986496 W at W 0
Thr W -1341116416 W at W 0
Thr W -1337921536 W at W 0
Thr W -1341648896 W at W 0
Thr W -1338454016 W at W 0
Thr W -1339518976 W at W 0
Thr W -1340583936 W at W 1
Thr W -1340051456 W at W 1
Thr W -1338986496 W at W 1
  ...more of the same...

The "serialization" efect (whereby the threads appear to "nicely round-robin" as above) is a side effect of the fact that the thread that gets to be the currently-printing one is seriously slower than the others (all those waits!-). Commenting out the time.sleep in wait, the output is instead

Thr W -1341648896 W at W 0
Thr W -1341116416 W at W 0
Thr W -1341648896 W at W 1
Thr W -1340583936 W at W 0
Thr W -1340051456 W at W 0
Thr W -1341116416 W at W 1
Thr W -1341116416 W at W 2
Thr W -1338986496 W at W 0
  ...more of the same...

i.e. a more typical "multithreaded output"... except for the guarantee that each line in the output comes entirely from one single thread.

Of course, a thread that does, e.g., print 'ciao', will keep "ownership" of standard output until it finally does perform a print without a trailing comma, and other threads wanting to print may sleep for quite a while (how else can one guarantee that each line in the output comes from a single thread? well, one architecture would be to accumulate partial lines to thread local storage instead of actually writing them to standard output, and only do the writing on receipt of the \n... delicate to interleave properly with softspace settings, I fear, but probably feasible).

Python 2.7: Print thread safe

from __future__ import print_function
print = lambda x: sys.stdout.write("%s\n" % x)

Is a nice cheap and dirty hack.

How do I do threading in python?

I think they when a thread didn't print a new line it's because the control was passed to another thread before the first thread printed the new line.

Is the `print` builtin function in Python 2.X atomic?

Your question is based on the central premise

Calling a function therefore is atomic as it's done with a single instruction.

which is thoroughly wrong.

First, executing the CALL_FUNCTION opcode can involve executing additional bytecode. The most obvious case of this is when the executed function is written in Python, but even built-in functions can freely call other code that may be written in Python. For example, print calls __str__ and write methods.

Second, Python is free to release the GIL even in the middle of C code. It commonly does this for I/O and other operations that might take a while without needing to perform Python API calls. There are 23 uses of the FILE_BEGIN_ALLOW_THREADS and Py_BEGIN_ALLOW_THREADS macros in the Python 2.7 file object implementation alone, including one in the implementation of file.write, which print relies on.

---

Related Topics