Maximum number of children processes on Linux
The number of child processes can be limited with setrlimit(2) using RLIMIT_NPROC
. Notice that fork(2) can fail for several reasons. You could use bash
builtin ulimit
to set that limit.
You can use getrlimit
(or parse /proc/self/limits
, see proc(5)) to get that information.
System-wide, you might use /proc/sys/kernel/threads-max
since:
This file specifies the system-wide limit on the number of threads
(tasks) that can be created on the system.
There is also /proc/sys/kernel/pid_max
This file specifies the value at which PIDs wrap around (i.e., the
value in this file is one greater than the maximum PID). PIDs
greater than this value are not allocated; thus, the value in this
file also acts as a system-wide limit on the total number of
processes and threads. The default value for this file, 32768,
results in the same range of PIDs as on earlier kernels. On 32-bit
platforms, 32768 is the maximum value for pid_max. On 64-bit
systems, pid_max can be set to any value up to 2^22 (PID_MAX_LIMIT,
approximately 4 million).
However, there could be other limitations (notably swap space).
A task for the kernel is either a single-threaded process or some thread inside some process - e.g. created by low-level syscall clone(2) (or some kernel thread like kworker
, ksoftirqd
etc...).
BTW, the practical number of processes is much more limited by available resources. A typical Linux desktop has only a few hundreds of them (right now, my Debian/x86-64 desktop with 32Gb RAM & i5-4690S has 227 processes). So a process is a quite expensive resource (it needs RAM, it needs CPU...). If you have too many of them you'll experience thrashing. And in practice, you don't want to have too many runnable processes or schedulable tasks (probably only a few dozens of them at most, perhaps no more than a few per core).
Create a program to calculate the maximum number of processes in the system
There are much easier ways to do this rather than via experimentation.
Most Unix systems provide ulimit
to show you the various soft (ie. user defined) and hard (ie. admin defined) limits on your account. For example, here's my soft and hard limits on OS X.
$ ulimit -a -S
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 256
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 709
virtual memory (kbytes, -v) unlimited
$ ulimit -a -H
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) unlimited
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 65532
cpu time (seconds, -t) unlimited
max user processes (-u) 1064
virtual memory (kbytes, -v) unlimited
While the system may support more processes, your program will be limited by these limits. I am limited to 709 (what an odd number) processes and can raise it to 1064.
The maximum number of processes at a single time is limited by the size of pid_t
and often by a restriction defined in the kernel. See this answer and also this answer for more detail.
What is the maximum number of child processes a supervisor can supervise?
gen_server
itself doesn't impose any particular restriction on the number of processes it can manage (apart from system limits) but if you add large enough amounts of children, you will (of course) eventually run into performance issues. gen_server
tries to be efficient regarding large number of children by e.g. not storing them in a linear list.
(If you really need to know the exact answer, you're probably designing your system in a suboptimal way, I would say.)
Maximum number of threads per process in Linux?
Linux doesn't have a separate threads per process limit, just a limit on the total number of processes on the system (threads are essentially just processes with a shared address space on Linux) which you can view like this:
cat /proc/sys/kernel/threads-max
The default is the number of memory pages/4. You can increase this like:
echo 100000 > /proc/sys/kernel/threads-max
There is also a limit on the number of processes (and hence threads) that a single user may create, see ulimit/getrlimit
for details regarding these limits.
Max number of open files per process in Linux
There is no issue here.
A pipe
has two ends, each gets its own file descriptor.
So, each end of a pipe
counts as a file against the limit.
The slight difference between 1024/2 = 512 and 510 is because your process has already opened the files stdin, stdout, and stderr, which counts against the limit.
Related Topics
Looping Through the Content of a File in Bash
Assembling 32-Bit Binaries on a 64-Bit System (Gnu Toolchain)
Why Should Eval Be Avoided in Bash, and What Should I Use Instead
Is There a Way For Non-Root Processes to Bind to "Privileged" Ports on Linux
What Registers Are Preserved Through a Linux X86-64 Function Call
Can You Run Gui Applications in a Linux Docker Container
How to Use Sudo to Redirect Output to a Location I Don't Have Permission to Write To
Argument List Too Long Error For Rm, Cp, Mv Commands
How to Fix 'Sudo: No Tty Present and No Askpass Program Specified' Error
How to Measure the Actual Memory Usage of an Application or Process
Iterate Over a List of Files With Spaces
Defining a Variable With or Without Export
How to Symlink a File in Linux
Use Expect in a Bash Script to Provide a Password to an Ssh Command
How Do the Likely/Unlikely Macros in the Linux Kernel Work and What Is Their Benefit