Fatal Error - Too Many Open Files

Fatal Error - Too many open files

This can be a limitation on the server where the code is running. Every operating system only allows for a certain number of open files/handles/sockets. This limit is usually further reduced when the server is virtualized. On a Linux server you can check the current limit with ulimit -n, if you have root access you can increase it with the same command. I assume there is a method for Windows server as well. Otherwise there is not much you can do about it (except ask your hoster or administrator to increase it).

More configurable limits:

Change in /etc/security/limits.conf

soft nofile 1024
hard nofile 65535

Increase ulimit by ulimit -n 65535
or echo 65535 > /proc/sys/fs/file-max or in /etc/sysctl.conf:

fs.file-max=65535

How to fix `failed to open stream: Too many open files error` in PHP

Increase the ulimit -n or disable multipart uploading inside the aws-s3 driver for flysystem. it is flysystem that is keeping too much files open.

Ok a quick fix for you would be:

foreach ($images as $image) {

   $resourcesOpened = count(get_resources('stream'));

     while ( $resourcesOpened > 900 ) {
       sleep(10);
       $resourcesOpened = count(get_resources('stream'));
     }

     // download image
     $stream = fopen($image['path'], 'wb');
     write($stream, $image['content']);
     fclose($stream);

     // copy to S3
     $manager->copy('local://' . $image['p'], 's3://' . $image['p']);
   }

}

Too many open files' error xcode

I think you just have reached the maximum number of open file descriptors.

You can check the limit by

$ ulimit

And change it by:

$ ulimit -n 6666

But do not forget ulimit changes the limit only for current session. You have to put ulimit -n 6666 to .bash_profile or something like this

How to work around Too many open files error when writing arrow dataset with pyarrow?

There is no way to control this in the current code. This feature (max_open_files) was recently added to the C++ library and ARROW-13703 tracks adding it to the python library. I'm not certain if it will make the cutoff for 6.0 or not (6.0 should be releasing quite soon).

In the meantime, your limit for open files ((-n) 1024) is the default and is a bit conservative. You should be able to pretty safely increase the limit by a couple thousand. See this question for more discussion.

---

Related Topics