Cannot change the maximum open files per process with sysctl
For Ubuntu 17.04. See this solution.
Prior to Ubuntu 17.04:
I don't know why the above settings don't work but it seems you can get the same result by using the /etc/security/limits.conf
file.
Set the limit in /etc/security/limits.conf
sudo bash -c "echo '* - nofile 10240' >> /etc/security/limits.conf"
*
means all users. You can replace it by a specific username.-
means bothsoft
andhard
for the type of limit to be enforced. Hard can only be modified by the superuser. Soft can be modified by a non-root user and cannot be superior to hard.nofile
is the Maximum number of open files parameter.10240
is the new limit.
Reload
Logout and log back in. sudo sysctl -p
doesn't seem to be enough to reload.
You can check the new limit with:
ulimit -n
Tested on Ubuntu 16.04 and CentOS 6. Inspired by this answer.
Maximum number of threads per process in Linux?
Linux doesn't have a separate threads per process limit, just a limit on the total number of processes on the system (threads are essentially just processes with a shared address space on Linux) which you can view like this:
cat /proc/sys/kernel/threads-max
The default is the number of memory pages/4. You can increase this like:
echo 100000 > /proc/sys/kernel/threads-max
There is also a limit on the number of processes (and hence threads) that a single user may create, see ulimit/getrlimit
for details regarding these limits.
Maximum number of children processes on Linux
The number of child processes can be limited with setrlimit(2) using RLIMIT_NPROC
. Notice that fork(2) can fail for several reasons. You could use bash
builtin ulimit
to set that limit.
You can use getrlimit
(or parse /proc/self/limits
, see proc(5)) to get that information.
System-wide, you might use /proc/sys/kernel/threads-max
since:
This file specifies the system-wide limit on the number of threads
(tasks) that can be created on the system.
There is also /proc/sys/kernel/pid_max
This file specifies the value at which PIDs wrap around (i.e., the
value in this file is one greater than the maximum PID). PIDs
greater than this value are not allocated; thus, the value in this
file also acts as a system-wide limit on the total number of
processes and threads. The default value for this file, 32768,
results in the same range of PIDs as on earlier kernels. On 32-bit
platforms, 32768 is the maximum value for pid_max. On 64-bit
systems, pid_max can be set to any value up to 2^22 (PID_MAX_LIMIT,
approximately 4 million).
However, there could be other limitations (notably swap space).
A task for the kernel is either a single-threaded process or some thread inside some process - e.g. created by low-level syscall clone(2) (or some kernel thread like kworker
, ksoftirqd
etc...).
BTW, the practical number of processes is much more limited by available resources. A typical Linux desktop has only a few hundreds of them (right now, my Debian/x86-64 desktop with 32Gb RAM & i5-4690S has 227 processes). So a process is a quite expensive resource (it needs RAM, it needs CPU...). If you have too many of them you'll experience thrashing. And in practice, you don't want to have too many runnable processes or schedulable tasks (probably only a few dozens of them at most, perhaps no more than a few per core).
Max open files for working process
As a system administrator: The /etc/security/limits.conf
file controls this on most Linux installations; it allows you to set per-user limits. You'll want a line like myuser - nofile 1000
.
Within a process: The getrlimit and setrlimit calls control most per-process resource allocation limits. RLIMIT_NOFILE
controls the maximum number of file descriptors. You will need appropriate permissions to call it.
How to increase number of child proceses
In Linux you have the pid_max
limit:
$ cat /proc/sys/kernel/pid_max
32768
However, if your Linux is running on systemd you might hit user-slice
limits:
for root
$ cat /sys/fs/cgroup/pids/user.slice/user-0.slice/pids.max
for currently logged in user:
$ cat /sys/fs/cgroup/pids/user.slice/user-$(id -u).slice/pids.max
10813
The systemd equivalent would be:
$ systemd-analyze dump | sed -n "/-> Unit user-$(id -u).slice:/,/-> Unit /p"| grep -e "TasksMax="
TasksMax=10813
According to man logind.conf
UserTasksMax=
Sets the maximum number of OS tasks each user may run concurrently. > This controls the TasksMax= setting of the per-user slice unit, see systemd.resource-control(5) for details. If assigned the special value
"infinity", no tasks limit is applied. Defaults to 33%, which equals 10813 with the kernel's defaults on the host, but might be smaller in OS containers.
This limit is defined in /etc/systemd/logind.conf
, even though it might be commented out, 33%
out of 32768
.
#UserTasksMax=33%
When you modify the UserTasksMax
limit or increase sysctl kernel.pid_max
you'll have to restart systemd-logind
service:
service systemd-logind restart
On 64-bit system you should be able to increase the value up to 2^22, i.e.: 4194304
sysctl kernel.pid_max=4194304
Related Topics
What Is The Most Efficient Way to Exchange High Volume Data Between 2 Process
How to Give to Some User Permissions Only to Subfolder
Thread Quantum: How to Compute It
How to Launch a Job in a Shell Which Will Persist Even If The Shell Which Launches It Terminates
Linux - Create Animated Gif with Pan and Zoom
Linking a Static Library into a Shared Library
What Is The Correct Way to Define a Netfilter Hook Function
How to Clear Space on My Main System Drive on a Linux Centos System
Raising Hard Limit on Rlimit_Nofile System-Wide on Linux
When Is Posix Thread Cancellation Not Immediate
Trouble Ssh Tunneling to Remote Server
How to Use Ptrace(2) to Change Behaviour of Syscalls
Undefined Reference to Symbol 'Timer_Settime@@Glibc_2.3.3
Linux X86 Nasm - Subroutine: Print a Dword from Eax
How to Add a Carriage Return as a Character to a File
Interprocess Communication via Pipes