Multiple Parallel.Foreach Loops in .Net

Nested Parallel.ForEach Loops on the same list?

Couldn't you just have one Parallel and one normal loop? So either

Parallel.ForEach(list, delegate(A element1)
{
  foreach(A element2 in list)
    foo(element1, element2)
});

or

foreach(A element1 in list)
{
  Parallel.ForEach(list, delegate(A element2)
  {
    foo(element1, element2);
  });
}

Should speed it up as well. There was never going to be a thread per cycle anyway, so this would probably be just as fast or slightly slower than nested parallel loops.

Nested Parallel.ForEach loops

A Parallel.ForEach does not necessarily execute in parallel -- it is just a request to do so if possible. Therefore, if the execution environment does not have the CPU power to execute the loops in parallel, it will not do so.

If the actions on the loops are not related (i.e., if they are separate and do not influence each other), I see no problem using Parallel.ForEach both on inner and outer loops.

It really depends on the execution environment. You could do timing tests if your test environment is similar enough to the production environment, and then determine what to do. When in doubt, test ;-)

Good luck!

Running two parallel.foreach() together

You can use tasks to run the two loops on separate threads. Use normal foreach loops.

var t1 = Task.Factory.StartNew(() => 
{
    foreach(var a in list1)
    {
        //do some operations
    }
});

var t2 = Task.Factory.StartNew(() => 
{
    foreach(var a in list1)
    {
        //do some operations
    }
});

// This will block the thread until both tasks have completed
Task.WaitAll(t1, t2);

Nested Parallel.For() loops and file creation problems

From MSDN:

Use the Parallel Loop pattern when you need to perform the same independent operation for each element of a collection or for a fixed number of iterations. The steps of a loop are independent if they don't write to memory locations or files that are read by other steps.

Parallel.For can speed up the processing of your rows by doing multi threading but it comes with a caveat that if it is not used correctly it will end with unexpected behavior of the program like the one you are having above.

The reason for following error :

DirectoryNotFoundException: Could not find a part of the path 'D:\Events\PATIENTID\EVENTID.txt'.

can be that the one thread goes to write and the directory is not there mean while the other thread creates that. Normally when doing parallelism there can be race conditions as we are doing multi-threading and if we don't use proper mechanics like locks or monitors then we end up with these kind of issues.

As you are doing file writing so multiple threads when trying to write to the same file end up with the error you have latter i.e.

The process cannot access the file 'D:\Events\PATIENTID\EVENTID.txt' because it is being used by another process.

as one thread is already writing to file so at that time other threads would fail to access the file for writing to it.

I would suggest to use a normal loop instead of parallelism here.

---

Related Topics