Random Numbers for Multiple Threads

Random numbers for multiple threads

I'd use one instance to seed the others. I'm pretty sure you can do this safely fairly easily.

• Even small changes in the state space cause fairly large changes downstream - if you can ensure they don't have exactly the same starting space (and no identical state prefix), I wouldn't worry about producing identical numbers. For instance, using just the values 1,2,3 to seed three threads would work fine - you don't even need to seed the entire space. Another advantage: by using clearly predictable seeds you can easily discredit the idea that you're cherry-picking any runs (assuming you're trying to demonstrate something).
• It's trivial to seed in a way that means the resultant "children" are highly un-correlated. Just iterate in a breadth-first fashion; i.e. if you want to seed N x 623 int values, don't seed 623 values sequentially, but pick the first N and distribute, then the next N etc. Even if there's some correlation between the seeder and the children, the correlation between the various children should be virtually non-existant - and that's all you care about.
• I'd prefer an algorithm that allows deterministic execution whenever possible, so depending on urandom is not attractive. This makes debugging easier.
• Finally, and obviously - test. These PRNG are fairly robust, but by all means eyeball the results and do a few correlation tests inspired by what you're simulating. Most problems should be obvious - either you've seeded badly and there are obvious repeating subsequences, you you've seeded well, and then the quality is dictated by the PRNG limitations.
• For final executions, after you're done testing, you can seed the first of 623 state values using urandom for peace of mind and/or the thread ID.

Creating random numbers in a multithreaded program in C

rand() & srand() are not the ones that can be used safely in that case.

They both are neither re-entrant nor threadsafe. Generally speaking, neither C or the C++ standards pose any requirements about thread safety on any of the standard library functions.

Some implementations may indeed provide thread safe versions but it is not mandated by the standard.

To be able to use a random number generator in multithreaded environment You will need a implementation that allows passing in the state. This way, you can keep one state value per thread, and generate good quality random numbers without requiring synchronization.

The C standard library does not provide any choices. That makes 100% portability rather impossible.The choice of usage would then depend on your environment which you should mention as a part of your question to get accurate answers.

Have a look at GNU Scientific Library which claims to provide MultiThreaded Random Number generator.

Random function in multi-threaded c program

Updated: Inserted direct answers to the OP's enumerated questions.

What is actually happening here?

Although some versions of the rand() function may be "thread safe" in some sense, there is no reason to believe or expect that without any external memory synchronization, the set of values returned by multiple rand() calls executed by different threads will be the same as the set of values returned by the same number of calls all executed by one thread. In particular, rand() maintains internal state that is modified on each call, and without any memory synchronization, it is entirely plausible that one thread will not see updates to that internal state that are performed by other threads. In that case, two or more threads may generate partially or wholly the same sequence of numbers.

How come the placement of the srand() function make a difference only to the main thread (thread 0)?

The only thing that can be said for certain is that if the srand() is outside the parallel block then it is executed only by the main thread, whereas if it is inside then it is executed separately by each thread. Inasmuch as your code is not properly synchronized, the effects of each case are not predictable from the source code, so my next comments are mostly speculative.

Supposing that time(), with its (only) one-second precision, returns the same value in each thread, placing srand() inside the parallel region ensures that every thread sees the same initial random number seed. If they then do not see each other's updates then they will generate the same sequences of pseudo-random numbers. Note, however, that you can neither safely rely on the threads seeing each other's updates nor safely rely on them not seeing each others updates.

If you put the srand() outside the parallel region, however, so that it is executed only by the main thread, then there are additional possibilities. If OMP maintains a thread pool whose members are already started before you enter the parallel section, then it may be that threads 1 and 2 fail to see the effect of thread 0's srand() call at all, and therefore both proceed with the default seed. There are other possibilities.

Why is it that either ways the the other 2 new threads always output same random number for the respective call to rand()?

It's impossible to say with any certainty. I'm inclined to guess, however, that none of the threads involved see each other's updates to rand()'s internal state.

How is this srand() and rand() even linked, to cause this abnormality?

The two functions are intimately linked. The purpose of srand() is to modify rand()'s internal state (to "seed" it, hence the "s" in "srand"), so as to start the psuedo-random number sequence it generates at a different (but still deterministic) point.

---

This problem can be solved in the same way that any problem involving multi-threaded access to shared variables can be solved: by applying synchronization. In this case, the most straightforward form of synchronization would probably be to protect rand() calls with a mutex. Since this is OMP code, your best bet might be to use OMP locks to implement a mutex, as it seems dicey to mix explicit pthreads objects with OMP declarations.

c++ generate thread safe random numbers

I would probably generate the seeds in main, and pass a seed to each thread function. I wouldn't use the output of std::random_device directly either--I'd put numbers into something like an std::set or std::unordered_set until I got as many seeds as I wanted, to assure that I didn't give two threads the same seed (which would obviously be a waste of time).

Something along this general line:

int do_work(unsigned long long seed) {

  //Get random number generators
  typedef std::mt19937 MyRNG;

  //seed generator
  MyRNG rng(seed);

  //make my uniform distributions for each parameter
  std::uniform_real_distribution<> param1(-1,1);
  std::uniform_real_distribution<> param2(-1,1);

  double x,y;
  //Do my scan
  for (int i = 0; i < N; i++) {
    x = param1(rng);
    y = param2(rng);

   //Do things with x and y*
  }
}

static const int num_threads = 4;

int main() {
    std::set<unsigned long long> seeds;

    while (seeds.size() < num_threads)
       seeds.insert(std::random_device()());

    std::vector<std::thread> threads;

   for (auto const seed: seeds)
       threads.emplace_back(std::thread(do_work, seed));

    for (auto &t : threads)
        t.join();
}

As an aside, using a single result from random_device to seed an std::mt19937 restricts the generator quite a bit--you're giving it only 32 (or possibly 64) bits of seed, but it actually has 19937 bits of seed material. std::seed_seq attempts to ameliorate this to at least some degree (among other things, you can use a number of outputs from std::random_device to create the seed.

Oh, and given that your two instances of uniform_real_distribution use the same parameters, there's probably not a whole lot of need for two separate distribution objects either.

Is random number generation in multiple threads using C++11 random library slow just like using rand() in multiple threads?

Yes and no. Most of the C++11 random number generators are objects that encapsulate their own state, so as long as you create a separate generator object for each thread, each should be able to operate independently of the others (so you don't need any locking).

The specific case of std::random_device is a little different though: this is intended (but not guaranteed) to obtain truly non-deterministic data from some sort of random number generation hardware. The driver for that device may well impose locking requirements of its own -- and it's often fairly low bandwidth as well, so if you want to obtain a lot of numbers quickly, it's usually a poor choice.

In a typical case, you want to use one generator (other than std::random_device) per thread, and use std::random_device only to provide the initial seeds for those other generators. This may impose locks during that initialization, but then allows each thread to generate its random numbers without any interlock with other threads.

Using stdlib's rand() from multiple threads

srand() seeds the random number generator. You should only have to call srand(time(NULL)) once during startup.

That said, the documentation states:

The function rand() is not reentrant
or thread-safe, since it uses hidden
state that is modified on each call.
This might just be the seed value to
be used by the next call, or it might
be something more elaborate. In order
to get reproducible behaviour in a
threaded application, this state must
be made explicit. The function
rand_r() is supplied with a pointer to
an unsigned int, to be used as state.
This is a very small amount of state,
so this function will be a weak
pseudo-random generator. Try
drand48_r(3) instead.

The emphasized part of the above is probably the reason why all your threads get the same number.

How do I generate thread-safe uniform random numbers?

Have you tried this?

int intRand(const int & min, const int & max) {
    static thread_local std::mt19937 generator;
    std::uniform_int_distribution<int> distribution(min,max);
    return distribution(generator);
}

Distributions are extremely cheap (they will be completely inlined by the optimiser so that the only remaining overhead is the actual random number rescaling). Don’t be afraid to regenerate them as often as you need – in fact, resetting them would conceptually be no cheaper (which is why that operation doesn’t exist).

The actual random number generator, on the other hand, is a heavy-weight object carrying a lot of state and requiring quite some time to be constructed, so that should only be initialised once per thread (or even across threads, but then you’d need to synchronise access which is more costly in the long run).

Concurrent random number generation in Java

You can make any random number generator thread safe by passing the results through a BlockingQueue.

class SafeRandom implements Runnable {

    Random r = new Random();
    BlockingQueue<Double> q = new ArrayBlockingQueue<>(10);

    double get() throws InterruptedException {
        return q.take();
    }

    @Override
    public void run() {
        try {
            while (true) {
                q.put(r.nextDouble());
            }
        } catch (InterruptedException ie) {
        }
    }

}

---

Related Topics