Java Ioexception "Too Many Open Files"

Java IOException Too many open files

On Linux and other UNIX / UNIX-like platforms, the OS places a limit on the number of open file descriptors that a process may have at any given time. In the old days, this limit used to be hardwired¹, and relatively small. These days it is much larger (hundreds / thousands), and subject to a "soft" per-process configurable resource limit. (Look up the ulimit shell builtin ...)

Your Java application must be exceeding the per-process file descriptor limit.

You say that you have 19 files open, and that after a few hundred times you get an IOException saying "too many files open". Now this particular exception can ONLY happen when a new file descriptor is requested; i.e. when you are opening a file (or a pipe or a socket). You can verify this by printing the stacktrace for the IOException.

Unless your application is being run with a small resource limit (which seems unlikely), it follows that it must be repeatedly opening files / sockets / pipes, and failing to close them. Find out why that is happening and you should be able to figure out what to do about it.

FYI, the following pattern is a safe way to write to files that is guaranteed not to leak file descriptors.

Writer w = new FileWriter(...);
try {
    // write stuff to the file
} finally {
    try {
        w.close();
    } catch (IOException ex) {
        // Log error writing file and bail out.
    }
}

^{1 - Hardwired, as in compiled into the kernel. Changing the number of available fd slots required a recompilation ... and could result in less memory being available for other things. In the days when Unix commonly ran on 16-bit machines, these things really mattered.}

UPDATE

The Java 7 way is more concise:

try (Writer w = new FileWriter(...)) {
    // write stuff to the file
} // the `w` resource is automatically closed

UPDATE 2

Apparently you can also encounter a "too many files open" while attempting to run an external program. The basic cause is as described above. However, the reason that you encounter this in exec(...) is that the JVM is attempting to create "pipe" file descriptors that will be connected to the external application's standard input / output / error.

Too many open files throwing an IOException

I solved my problem following that tutorial

https://easyengine.io/tutorials/linux/increase-open-files-limit/

About causing too many open files with java?

I presume that the garbage collector is running, finding lots of unreachable BufferedReader objects and collecting them. That causes the underlying stream objects to be finalized ... which closes them.

To make this code break, add the BufferedReader objects to a list, so that they remain reachable.

And here's why I think that changing 7168 to MAXINT is working.

When a JVM starts, it will use a relatively small heap. One of the things that happens during GC is that the JVM decides if it needs to resize the heap. So here is what could be happening:

The JVM starts with a heap that is too small to hold 7168 open files + BufferedReader objects. (Remember that each of the latter probably has a preallocated buffer!)
You start opening files.
At about N = 7168 - 2538, the heap fills up with all of the BufferedReader objects + FileInputStream objects + various detritus from JVM startup / warmup.
The GC runs, and causes a (probably) all of the BufferedReader objects to be collected / finalized / closed.
Then the GC decides that it needs to expand the heap. You now have enough heap space for more open BufferedReader objects than your ulimit allows.
You resume opening files ... and then hit the open file limit.

That's one possible pattern.

If you really want to investigate this, I advise you turn on GC logging, and see if you can correlate the number of FDs reported by lsof with GC runs.

(You could try adding sleep calls between each open to make it easier to get lsof measurements. But that could change the JVM behavior in other ways ...)

Java too many open files exception

Try using this utility specifically designed for the purpose.

This Java agent is a utility that keeps track of where/when/who opened files in your JVM. You can have the agent trace these operations to find out about the access pattern or handle leaks, and dump the list of currently open files and where/when/who opened them.

When the exception occurs, this agent will dump the list, allowing you to find out where a large number of file descriptors are in use.

Tomcat too many open files error (Ubuntu 18.04)

The problem was with the Tomcat version. Version 9.0.13 would show "Too many open files" error, even when the total amount of opened files was around 5000 out of 100000 (limit). I updated the version to 9.0.14 and everything seems fine now.

Java Ioexception "Too Many Open Files"