Is Async Httpclient from .Net 4.5 a Bad Choice for Intensive Load Applications

Is async HttpClient from .Net 4.5 a bad choice for intensive load applications?

Besides the tests mentioned in the question, I recently created some new ones involving much fewer HTTP calls (5000 compared to 1 million previously) but on requests that took much longer to execute (500 milliseconds compared to around 1 millisecond previously). Both tester applications, the synchronously multithreaded one (based on HttpWebRequest) and asynchronous I/O one (based on HTTP client) produced similar results: about 10 seconds to execute using around 3% of the CPU and 30 MB of memory. The only difference between the two testers was that the multithreaded one used 310 threads to execute, while the asynchronous one just 22. So in an application that would have combined both I/O bound and CPU bound operations the asynchronous version would have produced better results because there would have been more CPU time available for the threads performing CPU operations, which are the ones that actually need it (threads waiting for I/O operations to complete are just wasting).

As a conclusion to my tests, asynchronous HTTP calls are not the best option when dealing with very fast requests. The reason behind that is that when running a task that contains an asynchronous I/O call, the thread on which the task is started is quit as soon the as the asynchronous call is made and the rest of the task is registered as a callback. Then, when the I/O operation completes, the callback is queued for execution on the first available thread. All this creates an overhead, which makes fast I/O operations to be more efficient when executed on the thread that started them.

Asynchronous HTTP calls are a good option when dealing with long or potentially long I/O operations because it doesn't keep any threads busy on waiting for the I/O operations to complete. This decreases the overall number of threads used by an application allowing more CPU time to be spent by CPU bound operations. Furthermore, on applications that only allocate a limited number of threads (like it is the case with web applications), asynchronous I/O prevents thread pool thread depletion, which can happen if performing I/O calls synchronously.

So, async HttpClient is not a bottleneck for intensive load applications. It is just that by its nature it is not very well suited for very fast HTTP requests, instead it is ideal for long or potentially long ones, especially inside applications that only have a limited number of threads available. Also, it is a good practice to limit concurrency via ServicePointManager.DefaultConnectionLimit with a value that high enough to ensure a good level of parallelism, but low enough to prevent ephemeral port depletion. You can find more details on the tests and conclusions presented for this question here.

Is HttpClient async safe?

Will I get correct results back all the time as both calls come from the same thread now?

Yes. That's the indented usage of HttpClient.

"An HttpClient instance is a collection of settings applied to all requests executed by that instance. In addition, every HttpClient instance uses its own connection pool"

HttpClient Class

Is HttpClient safe to use concurrently?

According to Microsoft Docs, since .NET 4.5 The following instance methods are thread safe (thanks @ischell):

CancelPendingRequests
DeleteAsync
GetAsync
GetByteArrayAsync
GetStreamAsync
GetStringAsync
PostAsync
PutAsync
SendAsync
PatchAsync

HttpClient provide not truly async operations?

Does this mean that HttpClient provides not truly async operations?

Sort of. HttpClient is in an unusual position, since it's primary implementation uses HttpWebRequest, which is only partially asynchronous.

In particular, the DNS lookup is synchronous, and I think maybe the proxy resolution, too. After that, it's all asynchronous. So, for most scenarios, the DNS is fast (usually cached) and there isn't a proxy, so it acts asynchronously. Unfortunately, there are enough scenarios (particularly from within corporate networks) where the synchronous operations can cause significant lag.

So, when the team was writing HttpClient, they had three options:

1. Fix HttpWebRequest (and friends) allowing for fully-asynchronous operations. Unfortunately, this would have broken a fair amount of code. Due to the way inheritance is used as extension points in these objects, adding asynchronous methods would be backwards-incompatible.
2. Write their own HttpWebRequest equivalent. Unfortunately, this would take a lot of work and they'd lose all the interoperability with existing WebRequest-related code.
3. Queue requests to the thread pool to avoid the worst-case scenario (blocking synchronous code on the UI thread). Unfortunately, this has the side effects of degrading scalability on ASP.NET, being dependent on a free thread pool thread, and incurring the worst-case scenario cost even for best-case scenarios.

In an ideal world (i.e., when we have infinite developer and tester time), I would prefer (2), but I understand why they chose (3).

On a side note, the code you posted shows a dangerous use of StartNew, which has actually caused problems due to its use of TaskScheduler.Current. This has been fixed in .NET Core - not sure when the fix will roll back into .NET Framework proper.

How to effectively use HTTPClient (.Net 4.5) for sending request to different servers?

1. An HttpClient instance per host is totally appropriate here. Socket exhaustion typically happens at a number in the thousands; you should be fine with a "handful" of hosts. And a socket is specific to 1 host anyway, so you gain nothing in terms of socket conservation by using a single HttpClient instance with multiple hosts.

2. Yes, I would definitely consider boosting ServicePointManager.DefaultConnectionLimit in this scenario. How high? Depends on your scenario obviously, but with most hosting environments 100 or even more should be perfectly safe.

Unresponsiveness with async event handlers in WPF in .NET 4.5

WebClient uses HttpWebRequest, which unfortunately is not very asynchronous, even if you use the "asynchronous" methods. It does a blocking DNS lookup, at least. It may also block during proxy negotiation and/or the initial HTTP connection.

An older release of HttpClient was just using a wrapper around HttpWebRequest. I requested a truly-asynchronous HttpClient, but never heard a response. The last time I checked HttpClient, it was still part of MVC; ASP.NET Web API wasn't around at that time, so they may have fixed HttpClient since then. Or the difference in behavior between WebClient and HttpClient on your machine may just have to do with DNS caches or some such.

---

Related Topics