Asynchronous Io Io_Submit Latency in Ubuntu Linux

How io_uring internally works?

Apparently, Linux already had an Asyn[c]-IO (AIO) API. I believe it is not fully asynchronous. So what was the issue with AIO?

If you're extra careful and match all its constraints, the "old" Linux AIO interface will behave asynchronously. However, if you "break" ANY of the (hidden) rules submission can suddenly (and silently) behave in a synchronous fashion (i.e. submission blocks non-deterministically and for inconveniently long periods of time). Some of the many "rules" are given in answers to asynchronous IO io_submit latency in Ubuntu Linux (the overall issues are also listed in section 1.0 of the io_uring document you linked).

how io_uring overcomes it?

  • It is a radically different interface (see this answer on the "Is there really no asynchronous block I/O on Linux?") which is harder to get wrong.
  • It has a workqueue/thread pool mechanism which it will punt requests to when it is aware that blocking will take place before the result of submission can be returned (and thus it is able to return control back to the caller). This allows it to retain asynchrony in more (hopefully all!) submission cases.
  • It has an optional privileged mode (IORING_SETUP_SQPOLL) where you don't even have to make syscalls to submit/retrieve results. If you're "just" manipulating memory contents it's going to be hard to be blocked on a call you never made!

How io_uring internally works?

There are two ring buffers where the first ring is used to queue submissions and when the work has been completed the "results" are announced via the second ring buffer (which contains completions). While it's hard to give you something more than a very high level view if you're uncomfortable with things like C structs/C function interfaces you may enjoy this video by Jens presenting io_uring nonetheless and find the explanations in https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-will-revolutionize-programming-in-linux/ and https://mattermost.com/blog/iouring-and-go/ more accessible.

io_uring's advantages over Linux AIO don't stop at better asynchrony though! See the aforementioned link for "Is there really no asynchronous block I/O on Linux?" for a list of other benefits...

Is there really no asynchronous block I/O on Linux?

The real answer, which was indirectly pointed to by Peter Teoh, is based on io_setup() and io_submit().
Specifically, the "aio_" functions indicated by Peter are part of the glibc user-level emulation based on threads, which is not an efficient implementation.
The real answer is in:

io_submit(2)
io_setup(2)
io_cancel(2)
io_destroy(2)
io_getevents(2)

Note that the man page, dated 2012-08, says that this implementation has not yet matured to the point where it can replace the glibc user-space emulation:

http://man7.org/linux/man-pages/man7/aio.7.html

this implementation hasn't yet matured to the point where the POSIX
AIO implementation can be completely reimplemented using the kernel
system calls.

So, according to the latest kernel documentation I can find, Linux does not yet have a mature, kernel-based asynchronous I/O model. And, if I assume that the documented model is actually mature, it still doesn't support partial I/O in the sense of recv() vs read().



Related Topics



Leave a reply



Submit