Are Lists Thread-Safe

Are lists thread-safe?

Lists themselves are thread-safe. In CPython the GIL protects against concurrent accesses to them, and other implementations take care to use a fine-grained lock or a synchronized datatype for their list implementations. However, while lists themselves can't go corrupt by attempts to concurrently access, the lists's data is not protected. For example:

L[0] += 1

is not guaranteed to actually increase L[0] by one if another thread does the same thing, because += is not an atomic operation. (Very, very few operations in Python are actually atomic, because most of them can cause arbitrary Python code to be called.) You should use Queues because if you just use an unprotected list, you may get or delete the wrong item because of race conditions.

Thread-safe List T property

If you are targetting .Net 4 there are a few options in System.Collections.Concurrent Namespace

You could use ConcurrentBag<T> in this case instead of List<T>

Can list.append(foo) be not thread-safe?

list appending in python is thread safe.

For more details: what kinds of global value mutation are thread safe

You last thread gets stuck maybe due to other reason, for example: Memory allocating..

My first step to fix the stuck is use strace to trace the syscall.

And you can use gdb to print all threads' call stack too. Here is a wiki page: https://wiki.python.org/moin/DebuggingWithGdb

Concurrent access to list from multiple threads in python when data is appended constantly

You don't need locking when using list.append() or list.extend() with multiple threads. These operations are thread-safe.

Here is a brief overview of operations that are thread-safe: https://docs.python.org/3/faq/library.html#what-kinds-of-global-value-mutation-are-thread-safe

It's also worth mentioning that from a performance standpoint it's much faster to prepare sub-lists in separate threads, and then extend the main list with these sub-lists.

Why are we here, specifically, saying that ArrayList is not thread safe?

ArrayList is unsynchronized in implementation. When an object is unsynchronized it means that is is not locked while being modified structurally. A structural modification is any operation that adds or deletes one or more elements, or explicitly resizes the backing array; merely setting the value of an element is not a structural modification.
What you are referring to is an array which the elements are being added to or being deleted from and can be modified this differs from it having its value being set.

Reference is in regards with the pointer of the start of the array but how many elements are there is in question and having an unsynchronized object being modified in the sense of elements while the elements are being iterated over by another thread the integrity of the elements in the list is hard to guarantee. I hope I was able to convey the message plainly.

Look for more details here in Oracle: Array List and ConcurrentModificationException

ArrayList:

Note that this implementation is not synchronized. If multiple threads access an ArrayList instance concurrently, and at least one of the threads modifies the list structurally, it must be synchronized externally. (A structural modification is any operation that adds or deletes one or more elements, or explicitly resizes the backing array; merely setting the value of an element is not a structural modification.) This is typically accomplished by synchronizing on some object that naturally encapsulates the list. If no such object exists, the list should be "wrapped" using the Collections.synchronizedList method.

ConcurrentModificationException:

Note that fail-fast behavior cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification.

When lists are thread safe in python, is it safe to delete them?

Your code is not threadsafe, because your self.semaphore handling is not thread-safe. Two threads can read self.semaphore as False, before either of them have set it to True, as thread switches can take place at any point between instructions.

What you want to use is a proper thread lock object (a semaphore is the wrong primitive to use here).

When creating your instance, set a self.messages_lock = threading.Lock() attribute, and whenever you need to alter your messages list, use:

with self.messages_lock:
# locked block of code, guaranteed to be thread-safe.

or

try:
while not self.messages_lock.acquire(timeout=0.01):
print("waiting...")
# locked block of code, guaranteed to be thread-safe.
finally:
self.messages_lock.release()

if you must have a thread that prints out that it is waiting for a lock.

Is Python list slicing thread safe?

tl;dr both indexing and slicing are thread safe, but slicing consist of building slice and applying slice. Between them thread can modify our list.

In stackoverflow page you posted you can find a list of atomic operations for iterables and both, indexing and slicing are there.

Lets look at bytecode for slicing:

import dis

l = [1, 2, 3]

def f():
l[1:3]

dis.dis(f)
  7           0 LOAD_GLOBAL              0 (l)
2 LOAD_CONST 1 (1)
4 LOAD_CONST 2 (3)
6 BUILD_SLICE 2
8 BINARY_SUBSCR
(...)

BINARY_SUBSCR is a moment when you access the slice and it's indeed single operation. Check that:

import time
import threading

l = [0] * 1000000000

def append():
start = time.perf_counter()
time.sleep(0.1)
l.append(1) # Doesn't execute until main thread access slice
print("append", time.perf_counter() - start) # 3.4184917740058154

t = threading.Thread(target=append)
t.start()
start = time.perf_counter()
x = l[:] # Reading via slicing
print("main", time.perf_counter() - start) # 3.418436762993224
print("last_elem", x[-1]) # 0
t.join()

So fear not of changes in list while accessing a slice (and same with accessing an index) as it's simple opcode and is protected by GIL.

What can happen then?

# l = [1, 2, 3]
x = l[0:3]

You cannot assume x will contain all list.

  1. We first (BUILD_SLICE) with start 0, end 3.
  2. Then another thread can modify list e.g. by appending element.
  3. Only then we access slice with thread safe way.

Can it make our program crash? I doubt, as using out of bounds slice is safe.



Related Topics



Leave a reply



Submit