Do I Get a Notification from Epoll When a Fd Is Closed

What happen, when epoll file descriptor is closed?

Predictably, Linux does the same thing as it does for select(2). From the manual page:

For a discussion of what may happen if a file descriptor in an epoll
instance being monitored by epoll_wait() is closed in another thread,
see select(2).

And from the select(2) page:

If a file descriptor being monitored by select() is closed in another
thread, the result is unspecified. [...] On Linux (and some other
systems), closing the file descriptor in another thread has no effect
on select()

The tl;dr; is "don't do it":

In summary, any application that relies on a particular behavior in
this scenario must be considered buggy.

Epoll_wait returning events on closed file descriptor

Closing a file descriptor does not seem to remove it from the epoll. I tried it with very simple example on a 3.12.2. I'm inclined to call the man page wrong or inaccurate.

What I did in a test was:

• created a tcp socket
• bound it to localhost:5555
• set it to listen
• created an epoll
• added the socket there with hup, err and in
• slept a bit so I could optionally connect to with with nc
• closed the socket
• epoll_wait
• epoll_ctl del
• cleaned up

The wait works even though the socket had been closed whether I had connected to it or not.

Edit: The epoll_ctl_del did fail if the socket has been closed. And after reading the current man pages, it seems they're actually ok. The epoll page points to select(2) about closing a socket being monitored and that page says that the behaviour is unspecified.

Epoll: does it silently remove fds?

This is rather vague text there, but I guess it is just that if the descriptor is closed elsewhere, it is silently removed from the set. From Linux manpages, epoll(7):

Q6 Will closing a file descriptor cause it to be
removed from all epoll sets automatically?

A6 Yes, but be aware of the following point. A file
descriptor is a reference to an open file
description (see open(2)). Whenever a descriptor
is duplicated via dup(2), dup2(2), fcntl(2)
F_DUPFD, or fork(2), a new file descriptor refer‐
ring to the same open file description is cre‐
ated. An open file description continues to
exist until all file descriptors referring to it
have been closed. A file descriptor is removed
from an epoll set only after all the file
descriptors referring to the underlying open file
description have been closed (or before if the
descriptor is explicitly removed using
epoll_ctl(2) EPOLL_CTL_DEL). This means that
even after a file descriptor that is part of an
epoll set has been closed, events may be reported
for that file descriptor if other file descrip‐
tors referring to the same underlying file
description remain open.

So you have a socket with fd 42. It gets closed, and subsequently removed from the epoll object. But the kernel doesn't notify the libev about this through epoll_wait. Now the epoll_modify is called again with fd = 42. epoll_modify doesn't know whether this file descriptor 42 the same that already was in the epoll object or some other file description with the file descriptor number 42 reused.

One could also argue that the comments are just ranting and the design of the libev API is at fault here.

Is it necessary to deregister a socket from epoll before closing it?

From the man page:

Q6 Will closing a file descriptor cause it to be removed from all epoll sets
automatically?

A6 Yes, but be aware of the following point. A file descriptor is a
reference to an open file description (see open(2)). Whenever a
descriptor is duplicated via dup(2), dup2(2), fcntl(2) F_DUPFD, or
fork(2), a new file descriptor referring to the same open file description
is created. An open file description continues to exist until all file
descriptors referring to it have been closed. A file descriptor is
removed from an epoll set only after all the file descriptors referring to
the underlying open file description have been closed (or before if the
descriptor is explicitly removed using epoll_ctl(2) EPOLL_CTL_DEL). This
means that even after a file descriptor that is part of an epoll set has
been closed, events may be reported for that file descriptor if other file
descriptors referring to the same underlying file description remain open.

How does the Linux kernel remove a closed fd from the epoll interest list?

When fd1 is removed and same is dup to fd2 then kernel does not remove open file description but does remove entry from file descriptor table i.e internal data struct in kernel still remain intact but not associated with fd1 index anymore

Internally epoll fd keeps interest list in kernel data structure where it understand what all open file description needs to be cleaned up when close for one of interest list is invoked. Following link does have more information.
https://unix.stackexchange.com/questions/195057/what-is-an-open-file-description

Why does epoll_ctl need the filedescriptor twice?

This is a duplicate of about epoll_ctl()

The reason it needs it twice is that data inside event is a union. epoll_ctl does not know whether you actually provided a file descriptor or something else.

typedef union epoll_data {
    void        *ptr;
    int          fd;
    uint32_t     u32;
    uint64_t     u64;
} epoll_data_t;

epoll does not signal an event when socket is close

There are a few problems with your attempt.

1. You should not use EPOLLONESHOT unless you know what you are doing and you really need it. It disables the report of any other events to the epoll instance until you enable it again with EPOLL_CTL_MOD.

2. You should not use EPOLLHUP to determine if a connection was closed. The EPOLLHUP event may be raised before all data is read from the socket input stream, even if the client disconnects gracefully. I recommend you to only use EPOLLIN. If there is no input left (because of forceful or graceful disconnect), the read() would return 0 (EOF) and you can close the socket.

3. Your read() call will block the reading thread and consume the whole stream until EOF (connection closed). The whole point in using epoll() is to not have to use a while ( read(...) > 0 ) loop.

4. You should not use EPOLLET because "the code runs multithreaded" but because you need it. You can write multithreaded code without the edge-triggered mode. The use of EPOLLET requires a thorough knowledge of the differences between blocking and non-blocking I/O. You can easily run into pitfalls (as mentioned in the manual), like edge-triggered starvation.

---

Related Topics