Task.Yield - Real Usages

Task.Yield - real usages?

When you see:

await Task.Yield();

you can think about it this way:

await Task.Factory.StartNew( 
    () => {}, 
    CancellationToken.None, 
    TaskCreationOptions.None, 
    SynchronizationContext.Current != null?
        TaskScheduler.FromCurrentSynchronizationContext(): 
        TaskScheduler.Current);

All this does is makes sure the continuation will happen asynchronously in the future. By asynchronously I mean that the execution control will return to the caller of the async method, and the continuation callback will not happen on the same stack frame.

When exactly and on what thread it will happen completely depends on the caller thread's synchronization context.

For a UI thread, the continuation will happen upon some future iteration of the message loop, run by Application.Run (WinForms) or Dispatcher.Run (WPF). Internally, it comes down to the Win32 PostMessage API, which post a custom message to the UI thread's message queue. The await continuation callback will be called when this message gets pumped and processed. You're completely out of control about when exactly this is going to happen.

Besides, Windows has its own priorities for pumping messages: INFO: Window Message Priorities. The most relevant part:

Under this scheme, prioritization can be considered tri-level. All
posted messages are higher priority than user input messages because
they reside in different queues. And all user input messages are
higher priority than WM_PAINT and WM_TIMER messages.

So, if you use await Task.Yield() to yield to the message loop in attempt to keep the UI responsive, you are actually at risk of obstructing the UI thread's message loop. Some pending user input messages, as well as WM_PAINT and WM_TIMER, have a lower priority than the posted continuation message. Thus, if you do await Task.Yield() on a tight loop, you still may block the UI.

This is how it is different from the JavaScript's setTimer analogy you mentioned in the question. A setTimer callback will be called after all user input message have been processed by the browser's message pump.

So, await Task.Yield() is not good for doing background work on the UI thread. In fact, you very rarely need to run a background process on the UI thread, but sometimes you do, e.g. editor syntax highlighting, spell checking etc. In this case, use the framework's idle infrastructure.

E.g., with WPF you could do await Dispatcher.Yield(DispatcherPriority.ApplicationIdle):

async Task DoUIThreadWorkAsync(CancellationToken token)
{
    var i = 0;

    while (true)
    {
        token.ThrowIfCancellationRequested();

        await Dispatcher.Yield(DispatcherPriority.ApplicationIdle);

        // do the UI-related work item
        this.TextBlock.Text = "iteration " + i++;
    }
}

For WinForms, you could use Application.Idle event:

// await IdleYield();

public static Task IdleYield()
{
    var idleTcs = new TaskCompletionSource<bool>();
    // subscribe to Application.Idle
    EventHandler handler = null;
    handler = (s, e) =>
    {
        Application.Idle -= handler;
        idleTcs.SetResult(true);
    };
    Application.Idle += handler;
    return idleTcs.Task;
}

It is recommended that you do not exceed 50ms for each iteration of such background operation running on the UI thread.

For a non-UI thread with no synchronization context, await Task.Yield() just switches the continuation to a random pool thread. There is no guarantee it is going to be a different thread from the current thread, it's only guaranteed to be an asynchronous continuation. If ThreadPool is starving, it may schedule the continuation onto the same thread.

In ASP.NET, doing await Task.Yield() doesn't make sense at all, except for the workaround mentioned in @StephenCleary's answer. Otherwise, it will only hurt the web app performance with a redundant thread switch.

So, is await Task.Yield() useful? IMO, not much. It can be used as a shortcut to run the continuation via SynchronizationContext.Post or ThreadPool.QueueUserWorkItem, if you really need to impose asynchrony upon a part of your method.

Regarding the books you quoted, in my opinion those approaches to using Task.Yield are wrong. I explained why they're wrong for a UI thread, above. For a non-UI pool thread, there's simply no "other tasks in the thread to execute", unless you running a custom task pump like Stephen Toub's AsyncPump.

Updated to answer the comment:

... how can it be asynchronouse operation and stay in the same thread
?..

As a simple example: WinForms app:

async void Form_Load(object s, object e) 
{ 
    await Task.Yield(); 
    MessageBox.Show("Async message!");
}

Form_Load will return to the caller (the WinFroms framework code which has fired Load event), and then the message box will be shown asynchronously, upon some future iteration of the message loop run by Application.Run(). The continuation callback is queued with WinFormsSynchronizationContext.Post, which internally posts a private Windows message to the UI thread's message loop. The callback will be executed when this message gets pumped, still on the same thread.

In a console app, you can run a similar serializing loop with AsyncPump mentioned above.

When would I use Task.Yield()?

When you use async/await, there is no guarantee that the method you call when you do await FooAsync() will actually run asynchronously. The internal implementation is free to return using a completely synchronous path.

If you're making an API where it's critical that you don't block and you run some code asynchronously, and there's a chance that the called method will run synchronously (effectively blocking), using await Task.Yield() will force your method to be asynchronous, and return control at that point. The rest of the code will execute at a later time (at which point, it still may run synchronously) on the current context.

This can also be useful if you make an asynchronous method that requires some "long running" initialization, ie:

 private async void button_Click(object sender, EventArgs e)
 {
      await Task.Yield(); // Make us async right away

      var data = ExecuteFooOnUIThread(); // This will run on the UI thread at some point later

      await UseDataAsync(data);
 }

Without the Task.Yield() call, the method will execute synchronously all the way up to the first call to await.

await Task.Yield() and its alternatives

One situation where Task.Yield() is actually useful is when you are await recursively-called synchronously-completed Tasks. Because csharp’s async/await “releases Zalgo” by running continuations synchronously when it can, the stack in a fully synchronous recursion scenario can get big enough that your process dies. I think this is also partly due to tail-calls not being able to be supported because of the Task indirection. await Task.Yield() schedules the continuation to be run by the scheduler rather than inline, allowing growth in the stack to be avoided and this issue to be worked around.

Also, Task.Yield() can be used to cut short the synchronous portion of a method. If the caller needs to receive your method’s Task before your method performs some action, you can use Task.Yield() to force returning the Task earlier than would otherwise naturally happen. For example, in the following local method scenario, the async method is able to get a reference to its own Task safely (assuming you are running this on a single-concurrency SynchronizationContext such as in winforms or via nito’s AsyncContext.Run()):

using Nito.AsyncEx;
using System;
using System.Threading.Tasks;

class Program
{
    // Use a single-threaded SynchronizationContext similar to winforms/WPF
    static void Main(string[] args) => AsyncContext.Run(() => RunAsync());

    static async Task RunAsync()
    {
        Task<Task> task = null;
        task = getOwnTaskAsync();
        var foundTask = await task;
        Console.WriteLine($"{task?.Id} == {foundTask?.Id}: {task == foundTask}");

        async Task<Task> getOwnTaskAsync()
        {
            // Cause this method to return and let the 「task」 local be assigned.
            await Task.Yield();
            return task;
        }
    }
}

output:

3 == 3: True

I am sorry that I cannot think up any real-life scenarios where being able to forcibly cut short the synchronous portion of an async method is the best way to do something. Knowing that you can do a trick like I just showed can be useful sometimes, but it tends to be more dangerous too. Often you can pass around data in a better, more readable, and more threadsafe way. For example, you can pass the local method a reference to its own Task using a TaskCompletionSource instead:

using System;
using System.Threading.Tasks;

class Program
{
    // Fully free-threaded! Works in more environments!
    static void Main(string[] args) => RunAsync().Wait();

    static async Task RunAsync()
    {
        var ownTaskSource = new TaskCompletionSource<Task>();
        var task = getOwnTaskAsync(ownTaskSource.Task);
        ownTaskSource.SetResult(task);
        var foundTask = await task;
        Console.WriteLine($"{task?.Id} == {foundTask?.Id}: {task == foundTask}");

        async Task<Task> getOwnTaskAsync(
            Task<Task> ownTaskTask)
        {
            // This might be clearer.
            return await ownTaskTask;
        }
    }
}

output:

2 == 2: True

Task.Yield() in library needs ConfigureWait(false)

The exact equivalent of Task.Yield().ConfigureAwait(false) (which doesn't exist since ConfigureAwait is a method on Task and Task.Yield returns a custom awaitable) is simply using Task.Factory.StartNew with CancellationToken.None, TaskCreationOptions.PreferFairness and TaskScheduler.Current. In most cases however, Task.Run (which uses the default TaskScheduler) is close enough.

You can verify that by looking at the source for YieldAwaiter and see that it uses ThreadPool.QueueUserWorkItem/ThreadPool.UnsafeQueueUserWorkItem when TaskScheduler.Current is the default one (i.e. thread pool) and Task.Factory.StartNew when it isn't.

You can however create your own awaitable (as I did) that mimics YieldAwaitable but disregards the SynchronizationContext:

async Task Run(int input)
{
    await new NoContextYieldAwaitable();
    // executed on a ThreadPool thread
}

public struct NoContextYieldAwaitable
{
    public NoContextYieldAwaiter GetAwaiter() { return new NoContextYieldAwaiter(); }
    public struct NoContextYieldAwaiter : INotifyCompletion
    {
        public bool IsCompleted { get { return false; } }
        public void OnCompleted(Action continuation)
        {
            var scheduler = TaskScheduler.Current;
            if (scheduler == TaskScheduler.Default)
            {
                ThreadPool.QueueUserWorkItem(RunAction, continuation);
            }
            else
            {
                Task.Factory.StartNew(continuation, CancellationToken.None, TaskCreationOptions.PreferFairness, scheduler);
            }
        }

        public void GetResult() { }
        private static void RunAction(object state) { ((Action)state)(); }
    }
}

Note: I don't recommend actually using NoContextYieldAwaitable, it's just an answer to your question. You should be using Task.Run (or Task.Factory.StartNew with a specific TaskScheduler)

Why is an await Task.Yield() required for Thread.CurrentPrincipal to flow correctly?

How interesting! It appears that Thread.CurrentPrincipal is based on the logical call context, not the per-thread call context. IMO this is quite unintuitive and I'd be curious to hear why it was implemented this way.

In .NET 4.5., async methods interact with the logical call context so that it will more properly flow with async methods. I have a blog post on the topic; AFAIK that's the only place where it's documented. In .NET 4.5, at the beginning of every async method, it activates a "copy-on-write" behavior for its logical call context. When (if) the logical call context is modified, it will create a local copy of itself first.

You can see the "localness" of the logical call context (i.e., whether it has been copied) by observing System.Threading.Thread.CurrentThread.ExecutionContextBelongsToCurrentScope in a watch window.

If you don't Yield, then when you set Thread.CurrentPrincipal, you're creating a copy of the logical call context, which is treated as "local" to that async method. When the async method returns, that local context is discarded and the original context takes its place (you can see ExecutionContextBelongsToCurrentScope returning to false).

On the other hand, if you do Yield, then the SynchronizationContext behavior takes over. What actually happens is that the HttpContext is captured and used to resume both methods. In this case, you're not seeing Thread.CurrentPrincipal preserved from AuthenticateAsync to GetAsync; what is actually happening is HttpContext is preserved, and then HttpContext.User is overwriting Thread.CurrentPrincipal before the methods resume.

If you move the Yield into GetAsync, you see similar behavior: Thread.CurrentPrincipal is treated as a local modification scoped to AuthenticateAsync; it reverts its value when that method returns. However, HttpContext.User is still set correctly, and that value will be captured by Yield and when the method resumes, it will overwrite Thread.CurrentPrincipal.

Using Task.Yield to overcome ThreadPool starvation while implementing producer/consumer pattern

There are some good points left in the comments to your question. Being the user you quoted, I'd just like to sum it up: use the right tool for the job.

Using ThreadPool doesn't feel like the right tool for executing multiple continuous CPU-bound tasks, even if you try to organize some cooperative execution by turning them into state machines which yield CPU time to each other with await Task.Yield(). Thread switching is rather expensive; by doing await Task.Yield() on a tight loop you add a significant overhead. Besides, you should never take over the whole ThreadPool, as the .NET framework (and the underlying OS process) may need it for other things. On a related note, TPL even has the TaskCreationOptions.LongRunning option that requests to not run the task on a ThreadPool thread (rather, it creates a normal thread with new Thread() behind the scene).

That said, using a custom TaskScheduler with limited parallelism on some dedicated, out-of-pool threads with thread affinity for individual long-running tasks might be a different thing. At least, await continuations would be posted on the same thread, which should help reducing the switching overhead. This reminds me of a different problem I was trying to solve a while ago with ThreadAffinityTaskScheduler.

Still, depending on a particular scenario, it's usually better to use an existing well-established and tested tool. To name a few: Parallel Class, TPL Dataflow, System.Threading.Channels, Reactive Extensions.

There is also a whole range of existing industrial-strength solutions to deal with Publish-Subscribe pattern (RabbitMQ, PubNub, Redis, Azure Service Bus, Firebase Cloud Messaging (FCM), Amazon Simple Queue Service (SQS) etc).

How to create a Task which always yields?

First of all, the consumer of an async method shouldn't assume it will "yield" as that's nothing to do with it being async. If the consumer needs to make sure there's an offload to another thread they should use Task.Run to enforce that.

Second of all, I don't see how using Task.Run, or Task.Yield is problematic as it's used inside an async method which returns a Task and not a YieldAwaitable.

If you want to create a Task that behaves like YieldAwaitable you can just use Task.Yield inside an async method:

async Task Yield()
{
    await Task.Yield();
}

Edit:

As was mentioned in the comments, this has a race condition where it may not always yield. This race condition is inherent with how Task and TaskAwaiter are implemented. To avoid that you can create your own Task and TaskAwaiter:

public class YieldTask : Task
{
    public YieldTask() : base(() => {})
    {
        Start(TaskScheduler.Default);
    }

    public new TaskAwaiterWrapper GetAwaiter() => new TaskAwaiterWrapper(base.GetAwaiter());
}

public struct TaskAwaiterWrapper : INotifyCompletion
{
    private TaskAwaiter _taskAwaiter;

    public TaskAwaiterWrapper(TaskAwaiter taskAwaiter)
    {
        _taskAwaiter = taskAwaiter;
    }

    public bool IsCompleted => false;
    public void OnCompleted(Action continuation) => _taskAwaiter.OnCompleted(continuation);
    public void GetResult() => _taskAwaiter.GetResult();
}

This will create a task that always yields because IsCompleted always returns false. It can be used like this:

public static readonly YieldTask YieldTask = new YieldTask();

private static async Task MainAsync()
{
    await YieldTask;
    // something
}

Note: I highly discourage anyone from actually doing this kind of thing.

Is calling Task.Yield in Asp.Net Core ensuring continuation to run on the ThreadPool?

Assuming an Asp.Net Core application which does not have a SynchronizationContext, is there any functional difference from awaiting Task.Yield and passing that continuation to Task.Run?

There's practically no difference. await Task.Yield will queue the continuation to the thread pool and return to the caller. await Task.Run will queue its delegate to the thread pool and return to the caller.

Personally, I prefer Task.Run because it has a very explicit intent and because it's unambiguous (i.e., in other scenarios such as GUI threads, await Task.Yield will not queue to the thread pool, while await Task.Run always does the same thing regardless of context).

As others have noted in the comments, there's no reason to actually do either one on ASP.NET Core, which is already running on thread pool threads and does not have a request context. It would add a little overhead to the request while providing no benefit at all.

What's the difference between Task.Yield, Task.Run, and ConfigureAwait(false)?

Task.Yield is not a replacement for Task.Run and it has nothing to do with Task.ConfigureAwait.

Task.Yield - Generates an awaitable that completes just after it is checked for completion.
ConfigureAwait(false) - Generates an awaitable from a task that disregards the captured SynchronizationContext.
Task.Run - Executes a delegate on a ThreadPool thread.

Task.Yield is different than ConfigureAwait in that it is an awaitable all by itself, and not a configurable wrapper of another awaitable (i.e. the Task). Another difference is that Task.Yield does continue on the captured context.

Task.Run is different then both as it just takes a delegate and runs it on the ThreadPool, you can use it with ConfigureAwait(false) or without.

Task.Yield should be used to force an asynchronous point, not as a replacement to Task.Run. When await is reached in an async method it checks whether the task (or other awaitable) already completed and if so it continues on synchronously. Task.Yield prevents that from happening so it's useful for testing.

Another usage is in UI methods, where you don't want to be hogging the single UI thread, you insert an asynchronous point and the rest is scheduled to be executed in a later time.

Task.Yield - Real Usages