What Is the Global Interpreter Lock (Gil) in Cpython

What is the global interpreter lock (GIL) in CPython?

Python's GIL is intended to serialize access to interpreter internals from different threads. On multi-core systems, it means that multiple threads can't effectively make use of multiple cores. (If the GIL didn't lead to this problem, most people wouldn't care about the GIL - it's only being raised as an issue because of the increasing prevalence of multi-core systems.) If you want to understand it in detail, you can view this video or look at this set of slides. It might be too much information, but then you did ask for details :-)

Note that Python's GIL is only really an issue for CPython, the reference implementation. Jython and IronPython don't have a GIL. As a Python developer, you don't generally come across the GIL unless you're writing a C extension. C extension writers need to release the GIL when their extensions do blocking I/O, so that other threads in the Python process get a chance to run.

Global Interpreter lock: Jython vs CPython

Yes, Jython uses Java-Threads (even if you're using the threading modul of Python) and so it has no GIL. But this isn't the answer (otherwise it has to be 42, because the question is unclear :^) ).
The better Question is, what criteria you have and if CPython or Jython would be better.

If you want real multithreadding, it's your thing.
If you want to use Java and Python, use it.
If you want fast execution times .... then are other languages maybe better (you can try to messure the time in a thread task in Python and the same code in Jython, but I guess even with GIL CPython would be faster).

Greets,
Zonk

Python multithreading and Global Interpreter Lock

The Global Interpreter Lock ensures that only one thread is executing byte code at once. That execution could be interrupted at any time.

Consider this simple function which might be intended to atomically store related values to attributes on an instance x

def f(x, a, b):
    x.a, x.b = a, b

Here is its disassembly into bytecode

          0 LOAD_FAST                1 (a)
          3 LOAD_FAST                2 (b)
          6 ROT_TWO
          7 LOAD_FAST                0 (x)
         10 STORE_ATTR               0 (a)
         13 LOAD_FAST                0 (x)
         16 STORE_ATTR               1 (b)
         19 LOAD_CONST               0 (None)
         22 RETURN_VALUE

Suppose x is not protected by a mutex. Then any thread executing f(x, 1, 2) can easily be interrupted between storing a (at 10) and storing b (at 16). That interrupting thread will now see x in an inconsistent state.

python Global Interpreter Lock GIL problem

If you are opening each script by invoking a new process; you will not run afoul of the GIL. Each process gets its own interpreter and therefore its own interpreter lock.

Python Global Interpreter Lock (GIL) workaround on multi-core systems using taskset on Linux?

I have never heard of anyone using taskset for a performance gain with Python. Doesn't mean it can't happen in your case, but definitely publish your results so others can critique your benchmarking methods and provide validation.

Personally though, I would decouple your I/O threads from the CPU bound threads using a message queue. That way your front end is now completely network I/O bound (some with HTTP interface, some with message queue interface) and ideal for your threading situation. Then the CPU intense processes can either use multiprocessing or just be individual processes waiting for work to arrive on the message queue.

In the longer term you might also want to consider replacing your threaded I/O front-end with Twisted or some thing like eventlets because, even if they won't help performance they should improve scalability. Your back-end is now already scalable because you can run your message queue over any number of machines+cpus as needed.

Is my Python multithreading code affected by the Global Interpreter Lock

If your do_stuff_function is CPU-bound, then running it in multiple thread will not help, because the GIL only allows 1 thread to be executed at a time.

The way around this in Python is to use multiple process, just replace

from multiprocessing.dummy import Pool

with

from multiprocessing import Pool

---

Related Topics