How to Limit the Maximum Number of Parallel Tasks in C#

How to limit the Maximum number of parallel tasks in c#

SemaphoreSlim is a very good solution in this case and I higly recommend OP to try this, but @Manoj's answer has flaw as mentioned in comments.semaphore should be waited before spawning the task like this.

Updated Answer: As @Vasyl pointed out Semaphore may be disposed before completion of tasks and will raise exception when Release() method is called so before exiting the using block must wait for the completion of all created Tasks.

int maxConcurrency=10;
var messages = new List<string>();
using(SemaphoreSlim concurrencySemaphore = new SemaphoreSlim(maxConcurrency))
{
    List<Task> tasks = new List<Task>();
    foreach(var msg in messages)
    {
        concurrencySemaphore.Wait();

        var t = Task.Factory.StartNew(() =>
        {
            try
            {
                 Process(msg);
            }
            finally
            {
                concurrencySemaphore.Release();
            }
        });

        tasks.Add(t);
    }

    Task.WaitAll(tasks.ToArray());
}

Answer to Comments
for those who want to see how semaphore can be disposed without Task.WaitAll
Run below code in console app and this exception will be raised.

System.ObjectDisposedException: 'The semaphore has been disposed.'

static void Main(string[] args)
{
    int maxConcurrency = 5;
    List<string> messages =  Enumerable.Range(1, 15).Select(e => e.ToString()).ToList();

    using (SemaphoreSlim concurrencySemaphore = new SemaphoreSlim(maxConcurrency))
    {
        List<Task> tasks = new List<Task>();
        foreach (var msg in messages)
        {
            concurrencySemaphore.Wait();

            var t = Task.Factory.StartNew(() =>
            {
                try
                {
                    Process(msg);
                }
                finally
                {
                    concurrencySemaphore.Release();
                }
            });

            tasks.Add(t);
        }

       // Task.WaitAll(tasks.ToArray());
    }
    Console.WriteLine("Exited using block");
    Console.ReadKey();
}

private static void Process(string msg)
{            
    Thread.Sleep(2000);
    Console.WriteLine(msg);
}

How to limit number of parallel tasks running?

There's a couple of problems with the code:

1. The task returned from LimitEncryptionSemaphore.WaitAsync is ignored, instead of being awaited.
2. LimitEncryptionSemaphore.WaitAsync is called in a different method than LimitEncryptionSemaphore.Release.

Fix:

private void WatcherOnCreated(string filePath, WatcherChangeTypes changeType)
{
  string fileName = Path.GetFileName(filePath);
  Logger.Debug($"A created item {fileName} was detected in drop folder.");

  TaskFactory.FireAndForget(async () =>
  {
    await LimitEncryptionSemaphore.WaitAsync();
    try
    {
      ...
    }
    catch (Exception ex)
    {
      ...
    }
    finally
    {
      LimitEncryptionSemaphore.Release();
    }
  });
}

Is there a general rule for a maximum amount of parallel tasks?

The TPL will automatically change how tasks are scheduled and add or remove ThreadPool threads over time. This means that, given enough time and similar work, the default behavior should improve to be the best option.

By default, it will start by using more threads than cores, since many tasks are not "pure CPU". Given that you're seeing extra tasks causing a slowdown, you likely either have resource contention (via locking), or your tasks are CPU bound, and having more tasks than processor cores will cause slowdowns. If this is going to be problematic, you can make a custom TaskScheduler that limits the number of tasks allowed at once, such as the LimitedConcurrencyTaskScheduler. This allows you to limit the number of tasks to the number of processors in pure CPU scenarios.

If your tasks are bound by other factors, such as IO, then you may need to profile to determine the best balance between # of concurrently scheduled tasks and throughput, though this will be system specific.

Best way to limit the number of active Tasks running via the Parallel Task Library

I just gave an answer which is very applicable to this question.

Basically, the TPL Task class is made to schedule CPU-bound work. It is not made for blocking work.

You are working with a resource that is not CPU: waiting for service replies. This means the TPL will mismange your resource because it assumes CPU boundedness to a certain degree.

Manage the resources yourself: Start a fixed number of threads or LongRunning tasks (which is basically the same). Decide on the number of threads empirically.

You can't put unreliable systems into production. For that reason, I recommend #1 but throttled. Don't create as many threads as there are work items. Create as many threads which are needed to saturate the remote service. Write yourself a helper function which spawns N threads and uses them to process M work items. You get totally predictable and reliable results that way.

Is there a way to limit the number of parallel Tasks globally in an ASP.NET Web API application?

I'm going to assume the code is fixed, i.e., Task.Run is removed and the WaitAsync / Release are adjusted to throttle the HTTP calls instead of List<T>.Add.

I am looking to limit the total number of Tasks that are being executed in parallel globally throughout the application, so as to allow scaling up of this application.

This does not make sense to me. Limiting your tasks limits your scaling up.

For example, if 50 requests to my API came in at the same time, this would start at most 250 tasks running parallel.

Concurrently, sure, but not in parallel. It's important to note that these aren't 250 threads, and that they're not 250 CPU-bound operations waiting for free thread pool threads to run on, either. These are Promise Tasks, not Delegate Tasks, so they don't "run" on a thread at all. It's just 250 objects in memory.

If I wanted to limit the total number of Tasks that are being executed at any given time to say 100, is it possible to accomplish this?

Since (these kinds of) tasks are just in-memory objects, there should be no need to limit them, any more than you would need to limit the number of strings or List<T>s. Apply throttling where you do need it; e.g., number of HTTP calls done simultaneously per request. Or per host.

Would the framework automatically prevent too many tasks from being executed?

The framework has nothing like this built-in.

Perhaps via a Queue? Or am I approaching this problem in the wrong way, and would I instead need to Queue the incoming requests to my application?

There's already a queue of requests. It's handled by IIS (or whatever your host is). If your server gets too busy (or gets busy very suddenly), the requests will queue up without you having to do anything.

Limit number of Threads in Task Parallel Library

You should not be using threads for this at all. There's a Task-based API for this, which is naturally asynchronous: CloudBlockBlob.UploadFromFileAsync. Use it with async/await and SemaphoreSlim to throttle the number of parallel uploads.

Example (untested):

const MAX_PARALLEL_UPLOADS = 5;

async Task UploadFiles()
{
    var files = new List<string>();
    // ... add files to the list

    // init the blob block and
    // upload files asynchronously
    using (var blobBlock = new CloudBlockBlob(url, credentials))
    using (var semaphore = new SemaphoreSlim(MAX_PARALLEL_UPLOADS))
    {
        var tasks = files.Select(async(filename) => 
        {
            await semaphore.WaitAsync();
            try
            {
                await blobBlock.UploadFromFileAsync(filename, FileMode.Create);
            }
            finally
            {
                semaphore.Release();
            }
        }).ToArray();

        await Task.WhenAll(tasks);
    }
}

Can I limit the amount of System.Threading.Tasks.Task objects that run simultaneously?

change the MaxDegreeOfParallelism property.

example

Limit the number of parallel threads in C#

Assuming you're building this with the TPL, you can set the ParallelOptions.MaxDegreesOfParallelism to whatever you want it to be.

Parallel.For for a code example.

Creating parallel tasks, how to limit new task creation?

There are lots of ways to do this. You can protect a counter (of current tasks) with a lock, using Monitor.Wait() to have the task-producing thread wait until the counter is decremented and it's safe to start another task.

Another approach is to use the Semaphore synchronization object. This does basically the same thing, but encapsulated in an easy-to-use object. You create a new Semaphore, specifying how many concurrent threads you want to be able to hold the lock at once. By having each task take a lock on the semaphore, then you can have the task-producer also lock on the semaphore and when you read the max number of tasks, the producer won't be able to continue and create another task until one of the currently running tasks completes.

---

Related Topics