Using Stdlib'S Rand() from Multiple Threads

Using stdlib's rand() from multiple threads

srand() seeds the random number generator. You should only have to call srand(time(NULL)) once during startup.

That said, the documentation states:

The function rand() is not reentrant
or thread-safe, since it uses hidden
state that is modified on each call.
This might just be the seed value to
be used by the next call, or it might
be something more elaborate. In order
to get reproducible behaviour in a
threaded application, this state must
be made explicit. The function
rand_r() is supplied with a pointer to
an unsigned int, to be used as state.
This is a very small amount of state,
so this function will be a weak
pseudo-random generator. Try
drand48_r(3) instead.

The emphasized part of the above is probably the reason why all your threads get the same number.

Multiple threads invoking the same rand() function, how do I make them call their own instance of rand() function?

I have a question about how multithreading and function calls at low-level work. If I call the rand() function from multiple threads, do they access the same function?

Yes. They all access the rand function.

Where is it stored in the memory?

If by "it", you mean the code for the rand function, then it's stored in memory with the rest of the program's code.

Do all threads access the same memory location for the function? If they all access the same memory location, could it be that two threads are accessing the same function at the same time so one would deadlock or bottleneck/slow down the overall process?

The code is never modified. Two threads accessing data in memory that is never modified do not cause any deadlock, bottleneck, or slow down.

I want to make my threads completely parallel, so I want them to call their own instance of the rand() function. How could I go about doing that?

The instance of the function isn't the issue. The issue is the shared data. Threads accessing shared data will cause performance issues, even if they access the shared data through different functions entirely.

Most likely, you should just not use rand at all and use some code that meets whatever requirements you have. It may be that rand_r does it, but you're probably better off looking for a random-number generator that meets whatever requirements you have.

Random function in multi-threaded c program

Updated: Inserted direct answers to the OP's enumerated questions.

What is actually happening here?

Although some versions of the rand() function may be "thread safe" in some sense, there is no reason to believe or expect that without any external memory synchronization, the set of values returned by multiple rand() calls executed by different threads will be the same as the set of values returned by the same number of calls all executed by one thread. In particular, rand() maintains internal state that is modified on each call, and without any memory synchronization, it is entirely plausible that one thread will not see updates to that internal state that are performed by other threads. In that case, two or more threads may generate partially or wholly the same sequence of numbers.

How come the placement of the srand() function make a difference only to the main thread (thread 0)?

The only thing that can be said for certain is that if the srand() is outside the parallel block then it is executed only by the main thread, whereas if it is inside then it is executed separately by each thread. Inasmuch as your code is not properly synchronized, the effects of each case are not predictable from the source code, so my next comments are mostly speculative.

Supposing that time(), with its (only) one-second precision, returns the same value in each thread, placing srand() inside the parallel region ensures that every thread sees the same initial random number seed. If they then do not see each other's updates then they will generate the same sequences of pseudo-random numbers. Note, however, that you can neither safely rely on the threads seeing each other's updates nor safely rely on them not seeing each others updates.

If you put the srand() outside the parallel region, however, so that it is executed only by the main thread, then there are additional possibilities. If OMP maintains a thread pool whose members are already started before you enter the parallel section, then it may be that threads 1 and 2 fail to see the effect of thread 0's srand() call at all, and therefore both proceed with the default seed. There are other possibilities.

Why is it that either ways the the other 2 new threads always output same random number for the respective call to rand()?

It's impossible to say with any certainty. I'm inclined to guess, however, that none of the threads involved see each other's updates to rand()'s internal state.

How is this srand() and rand() even linked, to cause this abnormality?

The two functions are intimately linked. The purpose of srand() is to modify rand()'s internal state (to "seed" it, hence the "s" in "srand"), so as to start the psuedo-random number sequence it generates at a different (but still deterministic) point.

---

This problem can be solved in the same way that any problem involving multi-threaded access to shared variables can be solved: by applying synchronization. In this case, the most straightforward form of synchronization would probably be to protect rand() calls with a mutex. Since this is OMP code, your best bet might be to use OMP locks to implement a mutex, as it seems dicey to mix explicit pthreads objects with OMP declarations.

rand() with multiple threads

To give you a short and straight answer:

rand() is not thread safe.

It should not be called from multiple threads without explicit critical section.

Why am I generating the same number with different threads?

srand() and rand() are not guaranteed to be thread-safe.

Use the stuff available in #include <random>

why I need to use rand_r() in threads and why I need different seed for each threads?

Why I need different seed in each?

rand_r() is a pseudo-random number generator. That is to say, it generates a pseduo-random sequence of numbers: Each call returns the next number in the sequence.

"Random" means "unpredictable." If you have a generator for a truly random sequence of numbers, you will be unable to predict the next number in the sequence, no matter how many of the preceding numbers you already know.

A "Pseudo random" is like a random sequence in some ways—can be used as if it was random in some applications—but it isn't random at all. In fact, it is 100% predictable. All you need to know to predict the next number in the sequence is to know the state of the generator and the algorithm that it uses.

The seed for a pseudo-random generator provides a way to put the generator into a known, repeatable state. If you provide the same seed to two different instances of the generator, then both generators will return exactly the same sequence of values.

---

Do you want each thread to get exactly the same sequence as every other thread? It's up to you. If that's what you want, then seed each one with the same value. If you want them to get different "random" numbers, then seed each generator with a different value.

Also, if you want different runs of the program to get different "random" values, then you have to seed with a different value each time the program is run.

How to generate random numbers in multiple processes at the same time?

You can use a different seed for each process, based on the process id for example :

srand(getpid());

And then just use rand().

Canonical way to generate random numbers in Cython

Big pre-answer caveat: this answer recommends using C++ because the question specifically asks for a solution that runs without the GIL. If you don't have this requirement (and you probably don't...) then Numpy is the simplest and easiest solution. Provided that you're generating large amounts of numbers at a time you will find Numpy perfectly quick. Don't be misled into a complicated exercise in wrapping C++ because someone asked for a no-gil solution.

---

Original answer:

I think the easiest way to do this is to use the C++11 standard library which provides nice encapsulated random number generators and ways to use them. This is of course not the only options, and you could wrap pretty much any suitable C/C++ library (one good option might be to use whatever library numpy uses, since that's most likely already installed).

My general advice is to only wrap the bits you need and not bother with the full hierarchy and all the optional template parameters. By way of example I've shown one of the default generators, fed into a uniform float distribution.

# distutils: language = c++
# distutils: extra_compile_args = -std=c++11

cdef extern from "<random>" namespace "std":
    cdef cppclass mt19937:
        mt19937() # we need to define this constructor to stack allocate classes in Cython
        mt19937(unsigned int seed) # not worrying about matching the exact int type for seed
    
    cdef cppclass uniform_real_distribution[T]:
        uniform_real_distribution()
        uniform_real_distribution(T a, T b)
        T operator()(mt19937 gen) # ignore the possibility of using other classes for "gen"
        
def test():
    cdef:
        mt19937 gen = mt19937(5)
        uniform_real_distribution[double] dist = uniform_real_distribution[double](0.0,1.0)
    return dist(gen)

(The -std=c++11 at the start is for GCC. For other compilers you may need to tweak this. Increasingly c++11 is a default anyway, so you can drop it)

With reference to your criteria:

1. Cross platform on anything that supports C++. I believe the sequence should be specified so it's repeatable.
2. Thread safe, since the state is stored entirely within the mt19937 object (each thread should have its own mt19937).
3. No GIL - it's C++, with no Python parts
4. Reasonably easy.

---

Edit: about using discrete_distribution.

This is a bit harder because the constructors for discrete_distribution are less obvious how to wrap (they involve iterators). I think the easiest thing to do is to go via a C++ vector since support for that is built into Cython and it is readily convertable to/from a Python list

# use Cython's built in wrapping of std::vector
from libcpp.vector cimport vector

cdef extern from "<random>" namespace "std":
    # mt19937 as before
    
    cdef cppclass discrete_distribution[T]:
        discrete_distribution()
        # The following constructor is really a more generic template class
        # but tell Cython it only accepts vector iterators
        discrete_distribution(vector.iterator first, vector.iterator last)
        T operator()(mt19937 gen)

# an example function
def test2():
    cdef:
        mt19937 gen = mt19937(5)
        vector[double] values = [1,3,3,1] # autoconvert vector from Python list
        discrete_distribution[int] dd = discrete_distribution[int](values.begin(),values.end())
    return dd(gen)

Obviously that's a bit more involved than the uniform distribution, but it's not impossibly complicated (and the nasty bits could be hidden inside a Cython function).

---

Related Topics