I'Ve Heard I++ Isn't Thread Safe, Is ++I Thread-Safe

I've heard i++ isn't thread safe, is ++i thread-safe?

You've heard wrong. It may well be that "i++" is thread-safe for a specific compiler and specific processor architecture but it's not mandated in the standards at all. In fact, since multi-threading isn't part of the ISO C or C++ standards ^(a), you can't consider anything to be thread-safe based on what you think it will compile down to.

It's quite feasible that ++i could compile to an arbitrary sequence such as:

load r0,[i]  ; load memory into reg 0
incr r0      ; increment reg 0
stor [i],r0  ; store reg 0 back to memory

which would not be thread-safe on my (imaginary) CPU that has no memory-increment instructions. Or it may be smart and compile it into:

lock         ; disable task switching (interrupts)
load r0,[i]  ; load memory into reg 0
incr r0      ; increment reg 0
stor [i],r0  ; store reg 0 back to memory
unlock       ; enable task switching (interrupts)

where lock disables and unlock enables interrupts. But, even then, this may not be thread-safe in an architecture that has more than one of these CPUs sharing memory (the lock may only disable interrupts for one CPU).

The language itself (or libraries for it, if it's not built into the language) will provide thread-safe constructs and you should use those rather than depend on your understanding (or possibly misunderstanding) of what machine code will be generated.

Things like Java synchronized and pthread_mutex_lock() (available to C/C++ under some operating systems) are what you need to look into ^(a).

^(a) This question was asked before the C11 and C++11 standards were completed. Those iterations have now introduced threading support into the language specifications, including atomic data types (though they, and threads in general, are optional, at least in C).

i++ thread safe

In the case of a single writer and many-readers to this single value it is thread-safe, but in the general case such operations are not thread-safe (thus needing a lock or using atomic operations). Also, what thread-safe means is very limited here.

If you simply do i++ in one thread, the other threads will either see the old value or the new value. On the two platforms you mention the values are atomically stored/loaded, so they cannot get half a value. This is however not true in general, for example a 64-bit value on x86 will not be atomic, so the reader could get half of the old value and half of the new value. So thread-safety here is very platform specific.

You still have to be careful however. If this is a plain int the optimizer may simply discard the load operation (perhaps keep a copy in a register). In this case the reader will never get a new value. This is vital if you are doing this in a loop. Unfortunately the only standards correct way of doing this is with C++0x using an atomic<T> type (volatile kind of serves this purpose now for some compilers).

If you do add a second writer the increment operator is of course not thread-safe at all. You could however here use an atomic add function which would make it thread safe again.

Why is reading not thread-safe?

Reading is thread safe, no problems..... until something writes to the location you're reading from, and then... well, hopefully you'll read before the data was changed, or read after the data was changed (in these cases, no worries), but sometimes, just when you really don't want it, you'll read half-way through the write and then you'll get compete garbage data.

The way to mitigate this is to ensure that you only read either before or after any writes, which requires you to check that a write is occurring and thus use a synchronisation lock of some sort. This makes things slower though, as you're obviously checking the lock and then reading instead of reading. If you're working with primitive data types (an int for example) then you can use a CPU synchronisation to speed this up dramatically.

As fr Python, chances are python data is always synchronised for you by the language runtime, if it isn't then you will get the same thread read problems sooner or later. (quick google says yes, Python will suffer the same problems is you're not careful)

Is putStrLn thread-safe?

Yes, it's thread-safe in the sense that you're asking about. A Handle is protected by an MVar which won't allow the buffer to become corrupted. As you pointed out, though, interleaving is a different matter.

Is CryptGenRandom() thread-safe?

Creating a separate HCRYPTPROV per thread doesn't make much sense. This is pointer to memory block from heap in all current implementations, primarily saved pointers to CSP entry points which used to call actual provider implementation (CPGenRandom in our case). The references themselves do not contain state of the CSP, unlike for example HCRYPTKEY which containing actual key state. So even if you create a separate HCRYPTPROV for every thread - this changes nothing.

There may be some global variables / data used by CSP internally during this call; this is however unknown as these would be implementation details. Of course we can serialize calls to CryptGenRandom in the code. However we cannot control that some other dll in our process also call CryptGenRandom concurrently. So serializing all calls to CryptGenRandom also impossible.

As result I think the CPGenRandom must be design to be thread-safe. and it my tests with a well known Microsoft CSP this is true. Internal synchronization is used in function, when need access global data and if multiple threads call CPGenRandom concurrently; every thread receives unique random data.

So my conclusion - CryptGenRandom is thread-safe, at least for all Microsoft CSP

Is this java class thread safe?

volatile is needed otherwise the optimizer might optimize the loop to if(b)while(true){}

but you can do this with wait and notify

public void add(int i){

    synchronized(this){
        while(!b){try{wait();}catch(InterruptedException e){}}//swallowing is not recommended log or reset the flag
            sum += i;
        b = true;   
        notify();
    }
}

public void mul(int i){
    synchronized(this){
        while(b){try{wait();}catch(InterruptedException e){}}
        sum *= i;
        b = false;  
        notify();
    }
}

however in this case (b checked inside the sync block) volatile is not needed

Achieving Thread-Safety

You can't really test for thread-safeness. All you can do is show that your code isn't thread-safe, but if you know how to do that you already know what to do in your program to fix that particular bug. It's the bugs you don't know that are the problem, and how would you write tests for those? Apart from that threading problems are much harder to find than other problems, as the act of debugging can already alter the behaviour of the program. Things will differ from one program run to the next, from one machine to the other. Number of CPUs and CPU cores, number and kind of programs running in parallel, exact order and timing of stuff happening in the program - all of this and much more will have influence on the program behaviour. [I actually wanted to add the phase of the moon and stuff like that to this list, but you get my meaning.]

My advice is to stop seeing this as an implementation problem, and start to look at this as a program design problem. You need to learn and read all that you can find about multi-threading, whether it is written for Delphi or not. In the end you need to understand the underlying principles and apply them properly in your programming. Primitives like critical sections, mutexes, conditions and threads are something the OS provides, and most languages only wrap them in their libraries (this ignores things like green threads as provided by for example Erlang, but it's a good point of view to start out from).

I'd say start with the Wikipedia article on threads and work your way through the linked articles. I have started with the book "Win32 Multithreaded Programming" by Aaron Cohen and Mike Woodring - it is out of print, but maybe you can find something similar.

Edit: Let me briefly follow up on your edited question. All access to data that is not read-only needs to be properly synchronized to be thread-safe, and sorting a list is not a read-only operation. So obviously one would need to add synchronization around all accesses to the list.

But with more and more cores in a system constant locking will limit the amount of work that can be done, so it is a good idea to look for a different way to design your program. One idea is to introduce as much read-only data as possible into your program - locking is no longer necessary, as all access is read-only.

I have found interfaces to be a very valuable aid in designing multi-threaded programs. Interfaces can be implemented to have only methods for read-only access to the internal data, and if you stick to them you can be quite sure that a lot of the potential programming errors do not occur. You can freely share them between threads, and the thread-safe reference counting will make sure that the implementing objects are properly freed when the last reference to them goes out of scope or is assigned another value.

What you do is create objects that descend from TInterfacedObject. They implement one or more interfaces which all provide only read-only access to the internals of the object, but they can also provide public methods that mutate the object state. When you create the object you keep both a variable of the object type and a interface pointer variable. That way lifetime management is easy, because the object will be deleted automatically when an exception occurs. You use the variable pointing to the object to call all methods necessary to properly set up the object. This mutates the internal state, but since this happens only in the active thread there is no potential for conflict. Once the object is properly set up you return the interface pointer to the calling code, and since there is no way to access the object afterwards except by going through the interface pointer you can be sure that only read-only access can be performed. By using this technique you can completely remove the locking inside of the object.

What if you need to change the state of the object? You don't, you create a new one by copying the data from the interface, and mutate the internal state of the new objects afterwards. Finally you return the reference pointer to the new object.

By using this you will only need locking where you get or set such interfaces. It can even be done without locking, by using the atomic interchange functions. See this blog post by Primoz Gabrijelcic for a similar use case where an interface pointer is set.

Java: Swing Libraries & Thread Safety

Never do long running tasks in response to a button, event, etc as these are on the event thread. If you block the event thread, the ENTIRE GUI will be completely unresponsive resulting in REALLY pissed off users. This is why Swing seems slow and crusty.
Use Threads, Executors, and SwingWorker to run tasks NOT ON THE EDT ( event dispatch thread).
Do not update or create widgets outside of the EDT. Just about the only call you can do outside of the EDT is Component.repaint(). Use SwingUtilitis.invokeLater to ensure certain code executes on the EDT.
Use EDT Debug Techniques and a smart look and feel (like Substance, which checks for EDT violation)

If you follow these rules, Swing can make some very attractive and RESPONSIVE GUIs

An example of some REALLY awesome Swing UI work: Palantir Technologies. Note: I DO NOT work for them, just an example of awesome swing. Shame no public demo... Their blog is good too, sparse, but good

I'Ve Heard I++ Isn't Thread Safe, Is ++I Thread-Safe