C# Task.Waitall() in .Net and Mono

C# Task.WaitAll() in .Net and Mono

I’ll answer my own question although credit should go to Evk who guided me on the right track (thanks mate!)

The subject on this question is bad to say the least. The problem has nothing to do with Task.WaitAll but rather the Mono implementation of Dns.GetHostEntry. As Evk said in a comment:

That means (most likely) that Dns.GetHostEntry on linux starts new
non-background thread. Program cannot complete until all
non-background threads are finished.

The GetHostEntry() method is located in the source file Dns.cs and when called with a string it calls GetHostByName which then calls GetHostByName_internal which is an external C function located in w32socket.c. Finally mono_get_address_info (in networking-posix.c) is called and we are down in the libc function getaddrinfo. Phew!

I cannot see any new non-background threads being started, but I found this:

MONO_ENTER_GC_SAFE;
ret = getaddrinfo (hostname, service_name, &hints, &info);
MONO_EXIT_GC_SAFE;

MONO_ENTER_GC_SAFE and MONO_EXIT_GC_SAFE are macros defined in mono-threads-api.h

#define MONO_ENTER_GC_SAFE  \
    do {    \
        gpointer __gc_safe_dummy;   \
        gpointer __gc_safe_cookie = mono_threads_enter_gc_safe_region (&__gc_safe_dummy)

#define MONO_EXIT_GC_SAFE   \
        mono_threads_exit_gc_safe_region (__gc_safe_cookie, &__gc_safe_dummy);  \
    } while (0)

I did not dig any further, but I believe Evk is right.

So, the answer to my question: Dns.GetHostEntry() cannot be terminated or cancelled in Mono. A program calling this method will not terminate until all queries has been processed or timed out. That’s the way it is. My guess is that is has to do with the garbage collector (GC) which probably runs in non-background thread and hence cannot be cancelled/terminated.

Edit: (A few days later) The reason for this was obvious once I got down to the man-page for getaddrinfo. This function returns a linked list to the result. This list is of course allocated on the heap, and has to be freed at some point to avoid memory leakage. The function for that is freeaddrinfo.

Anyway, thanks again to Evk!

So how can we fire multiple DNS queries in parallel with a timeout (using Mono)? Easy! This function can be called in a Task and it will be happy to obey WaitAll with timeout:

private static string host(string query)
{
    ProcessStartInfo psi = new ProcessStartInfo("host", query);
    psi.UseShellExecute = false;
    psi.RedirectStandardOutput = true;
    Process p = Process.Start(psi);
    return p.StandardOutput.ReadToEnd();
}

ArgumentException isn't thrown on Task.WaitAll(new Task[0])

Its seems to be a wrong documentation. According to source code (Can be found here or via Resharper)
ArgumentException raised only when:

Argument is null:

5043     if (tasks == null)
5044     {
5045        throw new ArgumentNullException("tasks");
5046     }

Or one of array's element is `null':

5070    // Collects incomplete tasks in "waitedOnTaskList"
5071    for (int i = tasks.Length - 1; i >= 0; i--)
5072    {
5073        Task task = tasks[i];
5074
5075        if (task == null)
5076        {
5077            throw new ArgumentException(Environment.GetResourceString("Task_WaitMulti_NullTask"), "tasks");
5078        }

I cant find any checks for empty array in code.

PS: I was looking at .NET Framework 4.6.2 source code

F# task parallelism under Mono doesn't appear to execute in parallel

It looks like the Sleeps are ignored completely - see how the Task 2 loop is printed even before launching the next task, that's just silly - if the thread waited for 10ms, there's no way for that to happen.

I'd assume that the cause might be the timer resolution in the OS. The Sleep is far from accurate - it might very well be that Mono (or Mac OS) decides that since they can't reliably make you run again in 10ms, the best choice is to simply let you run right now. This is not how it works on Windows - there you're guaranteed to lose control as long as you don't Sleep(0); you'll always sleep at least as long as you wanted. It seems that on Mono / Mac OS, the idea is the reverse - the OS tries to let you sleep at most the amount of time you specified. If you want to sleep for less time than is the timer precision, too bad - no sleep.

But even if they are not ignored, there's still not a lot of pressure on the thread pool to give you more threads. You're only blocking for less than 100ms, for four tasks in a line - that's not nearly enough for the pool to start creating new threads to handle the requests (on MS.NET, new threads are only spooled after not having any free threads for 200ms, IIRC). You're simply not doing enough work for it to be worth it to spool up new threads!

The point you might be missing is that Task.Factory.StartNew is not actually starting any new threads, ever. Instead, it's scheduling the associated task on the default task scheduler - which just puts it in the thread pool queue, as tasks to execute "at earliest convenience", basically. If there's one free thread in the pool, the first tasks starts running there almost immediately. The second will run when there's another thread free etc. Only if the thread usage is "bad" (i.e. the threads are "blocked" - they're not doing any CPU work, but they're not free either) is the threadpool going to spawn new threads.

Prevent running multiple instances of a mono app

I came up with this answer. Call this method passing it a unique ID

    public static void PreventMultipleInstance(string applicationId)
    {
        // Under Windows this is:
        //      C:\Users\SomeUser\AppData\Local\Temp\ 
        // Linux this is:
        //      /tmp/
        var temporaryDirectory = Path.GetTempPath();

        // Application ID (Make sure this guid is different accross your different applications!
        var applicationGuid = applicationId + ".process-lock";

        // file that will serve as our lock
        var fileFulePath = Path.Combine(temporaryDirectory, applicationGuid);

        try
        {
            // Prevents other processes from reading from or writing to this file
            var _InstanceLock = new FileStream(fileFulePath, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None);
            _InstanceLock.Lock(0, 0);
            MonoApp.Logger.LogToDisk(LogType.Notification, "04ZH-EQP0", "Aquired Lock", fileFulePath);

            // todo investigate why we need a reference to file stream. Without this GC releases the lock!
            System.Timers.Timer t = new System.Timers.Timer()
            {
                Interval = 500000,
                Enabled = true,
            };
            t.Elapsed += (a, b) =>
            {
                try
                {
                    _InstanceLock.Lock(0, 0);
                }
                catch
                {
                    MonoApp.Logger.Log(LogType.Error, "AOI7-QMCT", "Unable to lock file");
                }
            };
            t.Start();

        }
        catch
        {
            // Terminate application because another instance with this ID is running
            Environment.Exit(102534); 
        }
    }         

C# .NET CORE: cancell other tasks if one returned some kind of result

Edit:
As @ckuri stated in the comments, it would be easier to leverage the already existing properties of Task instead of writing a custom result class. Adjusted my answer accordingly.

One solution would be to check for the result in the CreateTask() method and pass a CancellationToken into your TaskAction() method:

private static Task<bool> CreateTask(CancellationTokenSource cts)
{
    return Task.Run(async () =>
    {
        var result = await TaskAction(cts.Token);

        // If result is false, cancel all tasks
        if (!result)
            cts.Cancel();

        return result;
    });
}

private static async Task<bool> TaskAction(CancellationToken token)
{
    // Check for cancellation
    token.ThrowIfCancellationRequested();

    var delay = Rnd.Next(2, 5);
    // Pass the cancellation token to Task.Delay()
    await Task.Delay(delay * 1000, token);

    var taskResult = Rnd.Next(10) < 4;

    // Check for cancellation
    token.ThrowIfCancellationRequested();

    return taskResult;
}

Now you can do something like this in your Main method to receive all tasks which did not get cancelled:

try
{
    // Wait for all tasks inside a try catch block because `WhenAll` throws a `AggregationException` 
    // containing a `TaskCanceledException`, if the token gets canceled
    await Task.WhenAll(tasks);
} 
catch { }

var tasksWithResult = tasks.Where(t => !t.IsCanceled).ToList();

What happens to work scheduled by Task.Run() after the program terminates?

This means that Task.Run is actually a no-go for fire-and-forget scenarios.

Well, you don't want to forget - you want to wait until it's completed. So use the Task that's returned to you.

To do that, you'll need to keep track of all uncompleted tasks that you launch this way, and then use something like Task.WaitAll(tasks) in a non-background thread. You potentially don't need to remember the tasks themselves - you just need to have a counter which is decremented when each task completes, and then you just need to wait for that to get to zero.

It's hard to give more concrete advice than that without knowing more about your scenario, to be honest... but something like that would certainly work.

You can easily encapsulate this in your own convenience methods, of course.

Using tasks getting Object reference not set to an instance of an object

Task.WaitAll doesn't throw this exception. It will rethrow exceptions raised by one of its tasks. Without the full exception and call stack (as returned by Exception.ToString()) it's impossible to be sure, but the standard guidance applies here as well - somewhere, somehow, an unininitialized variable or parameter is used.

For example:

List<int> personIds;
List<int> meetingRoomIds;

SplitResourceIds(resourceIds, out personIds, out meetingRoomIds);

may well set both personIds and meetingRoomIds to null. This means that

var task = Task.Factory.StartNew(() => GetResourceFreeBusyOnline(requesterId,
                                                                 personIds.ToArray(), 
                                                                 start, end));

will throw an exception inside the task, that will be re-raised when Task.WaitAll is called.

To fix this just follow the advice in the duplicate question. Check parameters for null, debug your code, check the full call stack of the exception. Task.WaitAll will throw an AggregateException which contains all underlying exceptions in its InnerExceptions property.

At the very least you should catch the AggregateException to handle and log exceptions raised in tasks separately.

Finally, use async/await, Task.Run and await Task.WhenAll instead of StartNew and Task.WaitAll. await unwraps the AggregateException and raises the underlying exception, which makes debugging a lot easier.

UPDATE

From the call stack it appears that Tieto.MyMeetings.Tasks.FreeBusy.FreeBusyTasks.GetMeetingRoomsFreeBusyCached calls SharpArch.NHibernate.LinqRepositoryWithTypedId'2.FindAll and either passes a null parameter or a list of items that contain null. These values are probably used to create a session key, since SharpArch.NHibernate.Web.Mvc.WebSessionStorage.GetSessionForKey is called before the exception is thrown.

Finding the exact problem requires debugging the code and stepping into GetMeetingRoomsFreeBusyCached

---

Related Topics