One-Shot *Level*-Triggered Epoll(): Does Epolloneshot Imply Epollet

epoll: difference between level triggered and edge triggered when EPOLLONESHOT specified

I think the bottom line answer is "there is not difference".

Looking at the code, it seems that the fd remembers the last set bits before being disabled by the one-shot. It remembers it was one shot, and it remembers whether it was ET or not.

Which is futile, because the fd is disabled until modified, and the next call to EPOLL_CTL_MOD will erase all of that, and replace with whatever the new MOD says.

Having said that, I do not understand why anyone would want both EPOLLET and EPOLLONESHOT. To me, the whole point of EPOLLET is that, unders certain programming models (namely, microthreads), it follows the semantics perfcetly. This means that I can add the fd to the epoll at the very start, and then never have to perform another epoll related system call.

EPOLLONESHOT, on the other hand, is used by people who want to keep a very strict control over when the fd is watched and when it isn't. That, by definition, is the opposite of what EPOLLET is used for. I just don't think the two are conceptually compatible.

Epoll with edge triggered and oneshot only reports once

However, I only receive the EPOLLIN event one time. Also, if I kill the client anytime after the first event is received, I do not get hangup events either.

man page for epoll_ctl(2) says that:

EPOLLONESHOT (since Linux 2.6.2)
Sets the one-shot behavior for the associated file descriptor.
This means that after an event is pulled out with
epoll_wait(2) the associated file descriptor is internally
disabled and no other events will be reported by the epoll
interface. The user must call epoll_ctl() with EPOLL_CTL_MOD
to rearm the file descriptor with a new event mask.

In your case, when you get the first event, epoll disables your sockfd.
When you re-enable your sockfd using EPOLL_CTL_MOD, it will notify all the events that are received by kernel after the re-registration. So any event between first notification and re-registration will be lost. This can be reason for not getting any hangup events or data.

Removing EPOLLONESHOT from events will correct your code, eventually you don't need to re-enable sockfd also.

And since you are using EPOLLET, there won't be any performance issue also.

Are epoll events being watched when not epoll_waiting

I'm experimenting with epoll's edge-mode which apparently only signals
files which have become ready for read/write (as opposed to level-mode
which signals all ready files, regardless of whether there were
already ready, or just became ready)

First let's get a clear view of the system, you need an accurate mental model of how the system works. Your view of epoll(7) is not really accurate.

The difference between edge-triggered and level-triggered is the definition of what exactly makes an event. The former generates one event for each action that has been subscribed on the file descriptor; once you consume the event, it is gone - even if you didn't consume all the data that generated such an event. OTOH, the latter keeps generating the same event over and over until you consume all the data that generated the event.

Here's an example that puts these concepts in action, blatantly stolen from man 7 epoll:

1. The file descriptor that represents the read side of a pipe (rfd) is registered on the epoll instance.

2. A pipe writer writes 2 kB of data on the write side of the pipe.

3. A call to epoll_wait(2) is done that will return rfd as a ready file descriptor.

4. The pipe reader reads 1 kB of data from rfd.

5. A call to epoll_wait(2) is done.

If the rfd file descriptor has been added to the epoll interface using
the EPOLLET (edge-triggered) flag, the call to epoll_wait(2) done in
step 5 will probably hang despite the available data still present in
the file input buffer; meanwhile the remote peer might be expecting a
response based on the data it already sent. The reason for this is
that edge-triggered mode delivers events only when changes occur on
the monitored file descriptor. So, in step 5 the caller might end up
waiting for some data that is already present inside the input buffer.
In the above example, an event on rfd will be generated because of the
write done in 2 and the event is consumed in 3. Since the read
operation done in 4 does not consume the whole buffer data, the call
to epoll_wait(2) done in step 5 might block indefinitely.

In short, the fundamental difference is in the definition of "event": edge-triggered treats events as a single unit that you consume once; level-triggered defines the consumption of an event as being equivalent to consuming all of the data belonging to that event.

Now, with that out of the way, let's address your specific questions.

in edge-mode, am I informed of readiness events that happen while I'm
not epoll_waiting

Yes, you are. Internally, the kernel queues up the interesting events that happened on each file descriptor. They are returned on the next call to epoll_wait(2), so you can rest assured that you won't lose events. Well, maybe not exactly on the next call if there are other events pending and the events buffer passed to epoll_wait(2) can't accommodate them all, but the point is, eventually these events will be reported.

What about events on one-shot files that haven't been rearmed yet?

Again, you never lose events. If the file descriptor hasn't been rearmed yet, should any interesting event arise, it is simply queued in memory until the file descriptor is rearmed. Once it is rearmed, any pending events - including those that happened before the descriptor was rearmed - will be reported in the next call to epoll_wait(2) (again, maybe not exactly the next one, but they will be reported). In other words, EPOLLONESHOT does not disable event monitoring, it simply disables event notification temporarily.

Ok, so will the last epoll_wait always be notified of the readiness of
socket S? Event if S is #1 (i.e. it's not rearmed)?

Given what I said above, by now it should be pretty clear: yes, it will. You won't lose any event. epoll offers strong guarantees, it's awesome. It's also thread-safe and you can wait on the same epoll fd in different threads and update event subscription concurrently. epoll is very powerful, and it is well worth taking the time to learn it!

What is the purpose of epoll's edge triggered option?

When an FD becomes read or write ready, you might not necessarily want to read (or write) all the data immediately.

Level-triggered epoll will keep nagging you as long as the FD remains ready, whereas edge-triggered won't bother you again until the next time you get an EAGAIN (so it's more complicated to code around, but can be more efficient depending on what you need to do).

Say you're writing from a resource to an FD. If you register your interest for that FD becoming write ready as level-triggered, you'll get constant notification that the FD is still ready for writing. If the resource isn't yet available, that's a waste of a wake-up, because you can't write any more anyway.

If you were to add it as edge-triggered instead, you'd get notification that the FD was write ready once, then when the other resource becomes ready you write as much as you can. Then if write(2) returns EAGAIN, you stop writing and wait for the next notification.

The same applies for reading, because you might not want to pull all the data into user-space before you're ready to do whatever you want to do with it (thus having to buffer it, etc etc). With edge-triggered epoll you get told when it's ready to read, and then can remember that and do the actual reading "as and when".

---

Related Topics