Inotify Fd - Why Is The Limit Per User Id and Not Per Process

inotify FD - why is the limit per user id and not per process?

The reason is to prevent non-root users DoSing the system by watching lots of files using inotify. inotify structures require non-negligible amount of memory to maintain (and it can't be swapped out to disk), so there needs to be some limit on how much non-privileged can commit.

epoll used to have similar restrictions (max_user_instances and max_user_watches), although in the end max_user_instances was removed and max_user_watches was just set to be 4% of memory.

A similar patch should probably be submitted for inotify, but hasn't been so far.

File descriptors are limited on a per-process basis for a completely different reason: when a process starts a file descriptor table is allocated and its size is proportional to the maximum allowed number of file descriptors. Keeping this as small as possible reduces the per-process memory overhead.

Too many open files - Failed to initialize inotify: the user limit on the total number of inotify instances has been reached

This is usually a linux config issue. Increase the number of files open in the /etc/security/limits.conf:

It looks like you're using Mac OS in which case you should use sysctl. Add the following to the /etc/sysctl.conf:

kern.maxfiles=your new value kern.maxfilesperproc=your new value

how to make sure not to read a file before finishing the write to it

Based on your question, it sounds like you're currently monitoring the directory with the IN_CREATE (and maybe IN_OPEN) flag. Why not also use the IN_CLOSE flag so that you get notified when the file is closed? From there, it should be easy to keep track of whether something has the file open and you'll know that you don't want to try reading it yet.

Nodemon Error: System limit for number of file watchers reached

If you are using Linux, your project is hitting your system's file watchers limit

To fix this, on your terminal, try:

echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

---

Related Topics