How to Write Asynchronous Linq Query

How to write Asynchronous LINQ query?

While LINQ doesn't really have this per se, the framework itself does... You can easily roll your own asynchronous query executor in 30 lines or so... In fact, I just threw this together for you :)

EDIT: Through writing this, I've discovered why they didn't implement it. It cannot handle anonymous types since they are scoped local. Thus, you have no way of defining your callback function. This is a pretty major thing since a lot of linq to sql stuff creates them in the select clause. Any of the below suggestions suffer the same fate, so I still think this one is the easiest to use!

EDIT: The only solution is to not use anonymous types. You can declare the callback as just taking IEnumerable (no type args), and use reflection to access the fields (ICK!!). Another way would be to declare the callback as "dynamic"... oh... wait... That's not out yet. :) This is another decent example of how dynamic could be used. Some may call it abuse.

Throw this in your utilities library:

public static class AsynchronousQueryExecutor
{
    public static void Call<T>(IEnumerable<T> query, Action<IEnumerable<T>> callback, Action<Exception> errorCallback)
    {
        Func<IEnumerable<T>, IEnumerable<T>> func =
            new Func<IEnumerable<T>, IEnumerable<T>>(InnerEnumerate<T>);
        IEnumerable<T> result = null;
        IAsyncResult ar = func.BeginInvoke(
                            query,
                            new AsyncCallback(delegate(IAsyncResult arr)
                            {
                                try
                                {
                                    result = ((Func<IEnumerable<T>, IEnumerable<T>>)((AsyncResult)arr).AsyncDelegate).EndInvoke(arr);
                                }
                                catch (Exception ex)
                                {
                                    if (errorCallback != null)
                                    {
                                        errorCallback(ex);
                                    }
                                    return;
                                }
                                //errors from inside here are the callbacks problem
                                //I think it would be confusing to report them
                                callback(result);
                            }),
                            null);
    }
    private static IEnumerable<T> InnerEnumerate<T>(IEnumerable<T> query)
    {
        foreach (var item in query) //the method hangs here while the query executes
        {
            yield return item;
        }
    }
}

And you could use it like this:

class Program
{

    public static void Main(string[] args)
    {
        //this could be your linq query
        var qry = TestSlowLoadingEnumerable();

        //We begin the call and give it our callback delegate
        //and a delegate to an error handler
        AsynchronousQueryExecutor.Call(qry, HandleResults, HandleError);

        Console.WriteLine("Call began on seperate thread, execution continued");
        Console.ReadLine();
    }

    public static void HandleResults(IEnumerable<int> results)
    {
        //the results are available in here
        foreach (var item in results)
        {
            Console.WriteLine(item);
        }
    }

    public static void HandleError(Exception ex)
    {
        Console.WriteLine("error");
    }

    //just a sample lazy loading enumerable
    public static IEnumerable<int> TestSlowLoadingEnumerable()
    {
        Thread.Sleep(5000);
        foreach (var i in new int[] { 1, 2, 3, 4, 5, 6 })
        {
            yield return i;
        }
    }

}

Going to go put this up on my blog now, pretty handy.

Asynchronous Linq Query

I am assuming you use EntityFamework here. In this case (as other answers point out) you can opt for .ToListAsync() like so:

public async IActionResult ProductsThatCostMoreThan(decimal price)
{ 
    var model = await db.Products
                               .Include(p => p.Category)
                               .Include(p => p.Supplier)
                               .Where(p => p.UnitPrice > price).ToListAsync();
     return View(model); 
}

The reason you have to have it at the end of your chain is because it returns Task, not IEnumerable, so you have to await for it. Technically you can enumerate your data source without filter, await for it and then apply .Where. But the way EF works it complies your .Where clause into SQL code and so it must be part of original method chain

Another point to make here it this extension method is part of EF assembly and therefore you might have had to look for alternatives had you not used EF. As other answers point out, actual query execution happens when you make a call to enumerate the IEnumerable, so you could technically write your own extension method with similar effect (easiest will be to examine the EF's extension source)

Creating an asynchronous method that contains a linq query

try this

  [HttpGet("GameDownloadLinks/{libraryId}")]
    public async Task<ActionResult<IEnumerable<GameDownloadLinks>>> GetGameLinksForlibraryAsync(Guid libraryID)
    {
        var libraryGameLinks = await (from gk in _context.GameLinks
                                   join gl in _context.GameList on gk.GameId equals gl.GameId
                                   where gl.libraryId == libraryId
                                   select new GameDownloadLinks
                                   {
                                       LibraryId = gl.libraryId,
                                       LinkText = gk.LinkText,
                                       Price = gk.Price,
                                       GameId = gl.GameId
                                   }).ToListAsync();

       // var asyncGameDownloadLinks = await Task.WhenAll(libraryGameLinks).toListAsync();
        return libraryGameLinks ;
    }

How to make async with linq

Let's look at your first action first:

[AllowAnonymous]
public ActionResult Register()
{
  return View();
}

Well, there's nothing here that does anything that would delay the current thread, so there's no benefit in doing anything asynchronously, so the best thing to do is to just leave this one alone.

Now, let's look at your second action. It contains:

dc.SubmitChanges();

That does indeed block the current thread while the changes are sent to the database. It probably won't be a big hit, but it is somewhere where we might benefit from using asynchronous code, especially because it's just so darn easy to do these days.

First change the signature:

public async Task<ActionResult> Register(RegisterViewModel model)

If you compile now it will work, but you'll get a warning:

 This async method lacks 'await' operators and will run synchronously. Consider using the 'await' operator to await non-blocking API calls, or 'await Task.Run(...)' to do CPU-bound work on a background thread.

But we've already done two things:

Changed from returning ActionResult to returning Task<ActionResult>.
Used async to mean that we can just do return View(model); and it gets turned into code that actually returns a Task<ActionResult> that when run will then run code like that in your original method.

But we've not actually gained anything. However instead of dc.SubmitChanges() we'll use dc.SubmitChangesAsync().

This is assuming it exists, which will depend on your data provider. More on that in a bit.

If it does exist then instead of returning void or some value T it would return Task or Task<T>.

We could write code to handle that task ourselves, but the easiest thing to do is:

await dc.SubmitChangesAsync();

Now the method we have is one that returns a Task<Action> that when initially Run (MVC will handle that bit for us) will execute the code as far as that await. Then it will release the thread running the code to do other things. After the SubmitChangesAsync has done its job, the execution of the method resumes at that point.

Similarly we could turn a method like:

var someList = someQuery.ToList();

Into:

var someList = await someQuery.ToListAsync();

Now, as I said, this all depends on the methods which result in database access actually having an async version. If there is no SubmitChangesAsync() then either you have to give up on this approach (boo!) or you have to write your own implementation (not trivial).

The current version of EntityFramework provides the following:

AllAsync
AnyAsync
AverageAsync
ContainsAsync
CountAsync
FirstAsync
FirstOrDefaultAsync
ForEachAsync
LoadAsync
LongCountAsync
MaxAsync
MinAsync
SingleAsync
SingleOrDefaultAsync
SumAsync
ToArrayAsync
ToDictionaryAsync
ToListAsync
SaveChangesAsync

That is, pretty much all of the methods that result in an actual database access have an Async version.

If you are not using a provider which offers this, then moving to asynchronous code will not be easy; it'll involve a more than can be put into a quick answer to do well.

One final caveat. Say you have an async method that always does a single await and that await is a tail-call:

public async Task<List<int>> GetList(int typeID)
{
  if(typeID < 1)
    throw new ArgumentOutOfRangeException();
  return await (from stuff in place where stuff.TypeID == typeID select stuff).ToListAsync();
}

Here you should not use async and await at all, because you are creating a task which will always result in a single task being awaited on, and so the extra await just complicates things behind the scenes for no good reason. Here you should just return the task:

public Task<List<int>> GetList(int typeID)
{
  if(typeID < 1)
    throw new ArgumentOutOfRangeException();
  return (from stuff in place where stuff.TypeID == typeID select stuff).ToListAsync();
}

But note that something that looks like a tail call inside a using block is not really a tail call, because it also implicitly does the Dispose() at the end of the using, so you can't simplify such calls in that way.

You can though simplify cases where there are different possible tail calls, as long as every path either does such a call, or throws an exception:

public async Task<List<int>> GetList(int? typeID)
{
  if(!typeID.HasValue)
    throw new ArgumentNullException();
  if(typeID.Value < 0)
    return await (from stuff in place).ToListAsync();
  return await (from stuff in place where stuff.TypeID == typeID select stuff).ToListAsync();
}

Can become:

public Task<List<int>> GetList(int? typeID)
{
  if(!typeID.HasValue)
    throw new ArgumentNullException();
  if(typeID.Value < 0)
    return (from stuff in place).ToListAsync();
  return (from stuff in place where stuff.TypeID == typeID select stuff).ToListAsync();
}

How to chain async calls in a complex linq query

The problem is that the lambda given to the deepest Select (.Select(async video => ...) is going to return a Task<T> (I assume Task<int> but not sure from the context).

Select doesn't understand how to use a Task and will just pass it through as is. You can convert these in bulk by using WhenAll (1) but you would have to make extra provisions on the database connection, as this will execute multiple queries in parallel. (2)

The most simple way in this instance is probably to scrap the LINQ and use foreach, like this:

var subtopics;

var topicModel = new TopicModel
{
    Id = 0,
    SubtopicModels = new List<SubtopicModel>()
};

foreach(var subtopic in subtopics)
{
    var subtopicModel = new SubtopicModel
    {
        Id = 0,
        VideoModels = new List<VideoModel>()
    };

    foreach(var video in subtopic.Videos.OrderBy(video => video.OrderInSubtopic))
    {
        subtopicModel.VideoModels.Add(new VideoModel
        {
            Id = 0,
            VideoLeftAtSeconds = (await _db.VideoActivities
                        .FirstOrDefaultAsync(activity => activity.UserId == userId && activity.VideoId == videoId))
                        .LeftAtSeconds
        });
    }

    topicModel.SubtopicModel.Add(subtopicModel);
}

Async await in linq select

var inputs = events.Select(async ev => await ProcessEventAsync(ev))
                   .Select(t => t.Result)
                   .Where(i => i != null)
                   .ToList();

But this seems very weird to me, first of all the use of async and await in the select. According to this answer by Stephen Cleary I should be able to drop those.

The call to Select is valid. These two lines are essentially identical:

events.Select(async ev => await ProcessEventAsync(ev))
events.Select(ev => ProcessEventAsync(ev))

(There's a minor difference regarding how a synchronous exception would be thrown from ProcessEventAsync, but in the context of this code it doesn't matter at all.)

Then the second Select which selects the result. Doesn't this mean the task isn't async at all and is performed synchronously (so much effort for nothing), or will the task be performed asynchronously and when it's done the rest of the query is executed?

It means that the query is blocking. So it is not really asynchronous.

Breaking it down:

var inputs = events.Select(async ev => await ProcessEventAsync(ev))

will first start an asynchronous operation for each event. Then this line:

                   .Select(t => t.Result)

will wait for those operations to complete one at a time (first it waits for the first event's operation, then the next, then the next, etc).

This is the part I don't care for, because it blocks and also would wrap any exceptions in AggregateException.

and is it completely the same like this?

var tasks = await Task.WhenAll(events.Select(ev => ProcessEventAsync(ev)));
var inputs = tasks.Where(result => result != null).ToList();

var inputs = (await Task.WhenAll(events.Select(ev => ProcessEventAsync(ev))))
                                       .Where(result => result != null).ToList();

Yes, those two examples are equivalent. They both start all asynchronous operations (events.Select(...)), then asynchronously wait for all the operations to complete in any order (await Task.WhenAll(...)), then proceed with the rest of the work (Where...).

Both of these examples are different from the original code. The original code is blocking and will wrap exceptions in AggregateException.

Async task method issue on a linq query return in c#

Linq uses the keyword yield return and this denotes that we won't actually do the operation until it's called upon. So your avoid code is trying to await the operation of a lazily invoked method.

per this MSDN

Although it's less code, take care when mixing LINQ with asynchronous code. Because LINQ uses deferred (lazy) execution, async calls won't happen immediately as they do in a foreach() loop unless you force the generated sequence to iterate with a call to .ToList() or .ToArray().

So, lose the wait. You're not actually executing the statement until the results are used

How to Write Asynchronous Linq Query