Have a Set of Tasks with Only X Running at a Time

Have a set of Tasks with only X running at a time

SemaphoreSlim maxThread = new SemaphoreSlim(10);

for (int i = 0; i < 115; i++)
{
    maxThread.Wait();
    Task.Factory.StartNew(() =>
        {
            //Your Works
        }
        , TaskCreationOptions.LongRunning)
    .ContinueWith( (task) => maxThread.Release() );
}

Run multiple asynchronous Tasks continuously

What I would like is that anytime a task is completed, a new one is started.

This is a perfect use case for SemaphoreSlim:

private readonly SemaphoreSlim _mutex = new SemaphoreSlim(10);

public async Task AddTask(Func<Task> work)
{
  await _mutex.WaitAsync();
  try { await work(); }
  finally { _mutex.Release(); }
}

A timeout would be great as well, to prevent a task from being run undefinitely in case of a failure.

The standard pattern for timeouts is to use a CancellationTokenSource as the timer and pass a CancellationToken into the work that needs to support cancellation:

private readonly SemaphoreSlim _mutex = new SemaphoreSlim(10);

public async Task AddTask(Func<CancellationToken, Task> work)
{
  await _mutex.WaitAsync();
  using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));
  try { await work(cts.Token); }
  finally { _mutex.Release(); }
}

Run Async every x number of times in a for loop

Running tasks in "batch" is not a good idea in terms of performance. A long running task would make whole batch block. A better approach would be starting a new task as soon as one is finished.

This can be implemented with a queue as @MertAkcakaya suggested. But I will post another alternative based on my other answer Have a set of Tasks with only X running at a time

int maxTread = 3;
System.Net.ServicePointManager.DefaultConnectionLimit = 50; //Set this once to a max value in your app

var urls = new Tuple<string, string>[] {
    Tuple.Create("http://cnn.com","temp/cnn1.htm"),
    Tuple.Create("http://cnn.com","temp/cnn2.htm"),
    Tuple.Create("http://bbc.com","temp/bbc1.htm"),
    Tuple.Create("http://bbc.com","temp/bbc2.htm"),
    Tuple.Create("http://stackoverflow.com","temp/stackoverflow.htm"),
    Tuple.Create("http://google.com","temp/google1.htm"),
    Tuple.Create("http://google.com","temp/google2.htm"),
};
DownloadParallel(urls, maxTread);

---

async Task DownloadParallel(IEnumerable<Tuple<string,string>> urls, int maxThreads)
{
    SemaphoreSlim maxThread = new SemaphoreSlim(maxThreads);
    var client = new HttpClient();

    foreach(var url in urls)
    {
        await maxThread.WaitAsync();
        DownloadFile(client, url.Item1, url.Item2)
                    .ContinueWith((task) => maxThread.Release() );
    }
}


async Task DownloadFile(HttpClient client, string url, string fileName)
{
    var stream = await client.GetStreamAsync(url);
    using (var fileStream = File.Create(fileName))
    {
        await stream.CopyToAsync(fileStream);
    }
}

PS: DownloadParallel will return as soon as it starts the last download. So don't await it. If you really want to await it you should add for (int i = 0; i < maxThreads; i++) await maxThread.WaitAsync(); at the end of the method.

PS2: Don't forget to add exception handling to DownloadFile

C# Task thread pool - Running 100 tasks across only 10 threads

I think you're missing the point by focusing on threads, especially for asynchronous operations that don't require threads to execute.

.NET has a great ThreadPool you can use. You don't know how many threads are in it and you don't care. It just works (until it doesn't and you need to configure it yourself, but that's very advance).

Running tasks on the ThreadPool is very simple. Either create a task for each operation and throttle them using a SemaphoreSlim or use the ready-made TPL Dataflow blocks. For example:

var block = new ActionBlock<SomeData>(
    _ => _repository.WriteDataAsync(_), // What to do on each item
    new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 10 }); // How many items at the same time

foreach (var item in items)
{
    block.Post(item); // Post all items to the block
}

block.Complete(); // Signal completion
await block.Completion; // Asynchronously wait for completion.

If, however, you do plan on creating "dedicated" threads you can use Task.Factory.StartNew with the LongRunning option that creates a dedicated thread outside the ThreadPool. But keep in mind that async operations don't maintain the same thread throughout the operation as async operations don't need a thread. So starting on a dedicated thread may be pointless (more on that on my blog: LongRunning Is Useless For Task.Run With async-await)

Awaiting multiple Tasks with different results

After you use WhenAll, you can pull the results out individually with await:

var catTask = FeedCat();
var houseTask = SellHouse();
var carTask = BuyCar();

await Task.WhenAll(catTask, houseTask, carTask);

var cat = await catTask;
var house = await houseTask;
var car = await carTask;

You can also use Task.Result (since you know by this point they have all completed successfully). However, I recommend using await because it's clearly correct, while Result can cause problems in other scenarios.

Can I run multiple slow processes in the background so more than one task can run in parallel?

Convert the list to a list of "Upload" tasks, and await them all with Task.WhenAll():

public async Task Run()
{
    // This gets populated after calling the web-API and parsing out the result
    List<Stream> files = new List<MemoryStream>{.....};
    var tasks = files.Select(Upload);

    await Task.WhenAll(tasks);
}

See this post for some more information about tasks/await.

Run a task every x-minutes with Windows Task Scheduler

The task must be configured in two steps.

First you create a simple task that start at 0:00, every day. Then, you go in Advanced... (or similar depending on the operating system you are on) and select the Repeat every X minutes option for 24 hours.

The key here is to find the advanced properties. If you are using the XP wizard, it will only offer you to launch the advanced dialog once you created the task.

On more recent versions of Windows (7+ I think?):

1. Double click the task and a property window will show up.
2. Click the Triggers tab.
3. Double click the trigger details and the Edit Trigger window will show up.
4. Under Advanced settings panel, tick Repeat task every xxx minutes, and set Indefinitely if you need.
5. Finally, click ok.

---

Related Topics