Linux and I/O Completion Ports

Linux and I/O completion ports?

So, on to my question... does linux support completion ports or even asynchronous I/O for sockets?

With regard to sockets, in 5.3 and later kernels, Linux has something analogous to completion ports in the shape of io_uring (for files/block devices io_uring support appeared in the 5.1 kernel).

IO Completion port Linux equivalent

I don't know of anything that does that directly, but you can combine a select() loop with a thread pool of your own to get similar behavior. When select() returns and you examine your fd_sets to see which file descriptors are ready, push each of those descriptors to a thread pool for processing. You'll still need a main thread to run the select() loop, independent of the thread pool for processing the I/O events.

Most of the complexity in this approach will be in how you keep track of which descriptors need to be included in the select() call at each iteration, and in the thread pool itself (since POSIX provides threads but not a standard thread pool API).

You might be interested in using a library like GLib, which provides a modular main loop that helps with polling on a varying set of file descriptors, as well as a thread pool implementation.

Multiple I/O Completion Ports

Yes. You can create as many IOCPs as you like*.

I expect you have a bug in your code or a standard 'deadlock caused by lock inversions'.

Can you break into the app in the debugger when it has deadlocked and see what the threads are doing?

(* subject to the usual resource limitations, memory, etc).

IOCPT/IOCP - does it exist only in Windows?

I/O Completion Ports are a Windows-only API, but the other common platforms do have roughly equivalent functionality when it comes to async I/O.

Linux has epoll and io_uring.
BSD/MacOS has kqueue. Windows also has Registered I/O.

The largest difference will be that IOCP has a special thread pooling feature designed to assist a tiny bit with otherwise very inefficient blocking I/O and contention scenarios, while the others leave that for you.

How do Completion Port Threads of the Thread Pool behave during async I/O in .NET / .NET Core?

Damien and Hans pointed me into the right direction in the comments, which I want to sum up in this answer.

Damien pointed to Stephen Cleary's awesome blog post which answers the first three points:

• The async I/O operation is dispatched on the calling thread. No IOCP thread is involved.
• Consequently, no IOCP threads block during async I/O.
• When the result is returned to the .NET application, an IOCP thread is borrowed to mark the task complete. The continuation is queued to the target SynchronizationContext or the thread pool.

Hans pointed out that there are similar mechanisms to IOCP in Linux (epoll) and
MacOS (kqueue).

UPDATE 2022-06-23: some people asked why IOCP threads do not block during I/O operations. It's important to understand how the Thread Pool manages its threads internally. The Thread Pool keeps a number of threads available, i.e. they reside in memory but are actually in a sleep state. This way when work comes in, you do not pay the cost of creating a new thread (my measurements on the topic show that creating a new thread instead of using an existing one is about 80 times slower). When work is available, it is queued to one of the sleeping threads, their state is changed from sleeping to ready to execute, and thus the operating system can pick this thread up in the next context switch (which usually occurs every 15ms) and assign it to one of your CPU cores. After your work is done, the thread is either put to sleep again or the next task can be executed on it. This is true for both worker threads and IOCP threads.

To conclude, IOCP threads do not block during the I/O operation, because only once the I/O completion port signals that the operation is complete, work is queued on an IOCP thread to mark the corresponding Task or Task<T> as completed and enqueue a possible continuation either on a worker thread or on the original caller thread if it has a synchronization context assigned to it and ConfigureAwait(false) was not called. During the I/O operation, the IOCP thread that will later execute the aforementioned work is either sleeping or handling completions from other I/O completion ports.

Is I/O Completion ports(Windows) or Asynchronous I/O (AIO) will improve performance of multithreaded servers handling large volume of requests?

Every request will be handled by
seperate detached thread.

That's not how I/O Completion Ports work on Windows... They allow you to use a very small number of threads to process vast amounts of I/O.

For example, here: http://www.lenholgate.com/blog/2005/10/the-64000-connection-question.html I talk about handling 64,000 concurrent active TCP connections with around 4 threads...

For Linux I expect the route to take is to use libevent.

For cross platform someone is sure to suggest ASIO.

IO COmpletion Ports for Mac OS X

Since you asked for a Windows specific feature for OS X, instead of using kqueue directly you may try libevent. It's a thin wrapper to different AIO mechanisms and it work on both platforms.

How to reregister for IO Completion ports for a handle

Referring to the documentation for CreateIoCompletionPort:

A handle can be associated with only one I/O completion port, and after the association is made, the handle remains associated with that I/O completion port until it [the handle] is closed.

[...] The I/O completion port handle and every file handle associated with that particular I/O completion port are known as references to the I/O completion port. The I/O completion port is released when there are no more references to it.

In other words, closing the I/O completion port handle doesn't achieve anything. The I/O completion port still exists and is permanently associated with the pipe handle. What you're attempting simply isn't possible; you will need to rearchitecture.

Note also:

It is best not to share a file handle associated with an I/O completion port by using either handle inheritance or a call to the DuplicateHandle function. Operations performed with such duplicate handles generate completion notifications. Careful consideration is advised.

---

Related Topics