Max Thread Per Process in Linux

Maximum number of threads per process in Linux?

Linux doesn't have a separate threads per process limit, just a limit on the total number of processes on the system (threads are essentially just processes with a shared address space on Linux) which you can view like this:

cat /proc/sys/kernel/threads-max

The default is the number of memory pages/4. You can increase this like:

echo 100000 > /proc/sys/kernel/threads-max

There is also a limit on the number of processes (and hence threads) that a single user may create, see ulimit/getrlimit for details regarding these limits.

Why number of created thread is less than thread-max?

As mentioned by @1201ProgramAlarm and @Francois Andrieux in the comments. The thread-max value is a system wide limit and for creating much more threads we need to some modifications in kernel settings:

• Understanding the difference between pid_max, ulimit -u and thread_max
• Maximum number of threads per process in Linux?
  

UNIX: Maximum number of threads in a program

set ulimit -s 4000 from terminal. Now you can run more thread than before, but you will meet segmentation fault in some stage.

number of threads = total virtual memory / (stack size*1024*1024)

The number of threads per process can be increased by increasing total
virtual memory or by decreasing stack size. But, decreasing stack size
too much can lead to code failure due to stack overflow while max
virtual memory is equals to the swap memory.

further information see this post clearly explained.

Windows and linux max threads

For Windows, TechNet has an article by Mark Russinovich Pushing the Limits of Windows: Processes and Threads. The article also links to the testlimit tool, which lets you experiment on your system.

The TL;DR version is that it depends on Windows version, whether you're running 32- or 64-bit Windows, and whether your application is 32- or 64-bit, and for a 32-bit program whether you're linking your program as "large-address aware" (and the /3GB NTLDR option). It may also depend on ASLR. And it depends on whether you use the default thread stack reserve size, or adjust it.

A 32-bit non-large-adress-aware process running on a 32bit system without the /3GB switch, and the default 1meg stack reserve, will be limited to at most 2048 threads, but you'll very likely hit memory limits before that.

For a 64bit process with an adjusted stack size, you can probably expect hitting 40-50k threads with 2GB ram.

At any rate, if you're even close to hitting the 32-bit limits, you're Doing Things Wrong(TM). You don't want to be using blocking I/O with a thread per request - look into async I/O and thread pools (and all the fancy abstractions built on top of those).

Maximum size of stack of multi threaded process

each thread of a process gets a stack, while there's typically only
one heap for the process.

That's correct.

Is this limit applicable at process level or each thread can have
1MB/8MB stack?

Each thread gets its own stack; the stack-size-limit is per-thread (i.e. it is not a shared limit for all threads in the process)

And what happens to the memory allotted to stack after thread exit?

The memory pages are released and become available for use by other code in the future.

How to increase maximum number of JVM threads (Linux 64bit)

You can use a sample program to find out the current threads limit.

If you encounter Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread, check these:

1. In small memory machines

   Every Java thread consume its own stack memory. Default stack size is 1024k (= 1M). You can reduce the stack size like java -Xss512k .... JVM cannot be started if the stack size is too low.

   And beware heap memory configurations: (initial) -Xms and (maximum) -Xmx. The more memory is allocated to heap, the less available memory for stack.

2. System limits

   Some values in ulimit -a can affect a thread limit.

   • max memory size - unlimited on most 64bit machines
   • max user processes - linux treats threads like processes
   • virtual memory - unlimited on most 64bit machines. virtual memory usage is increased by -Xss configuration (default 1024k)

   You can change these values by (temporal) running ulimit command or (permanent) editing /etc/security/limits.conf.

3. sys.kernel.threads-max

   This value is the system-global (including non-JVM processes) maximum number of threads. Check cat /proc/sys/kernel/threads-max, and increase if necessary.

   echo 999999 > /proc/sys/kernel/threads-max
   
   or
   
   sys.kernel.threads-max = 999999 in /etc/sysctl.conf to change permanently.

4. sys.kernel.pid_max

   If cat /proc/sys/kernel/pid_max is similar to current limit, increase this. Linux treats threads like processes.

   echo 999999 > /proc/sys/kernel/pid_max
   
   or
   
   sys.kernel.pid_max = 999999 in /etc/sysctl.conf to change permanently.

   And you may need to increase sys.vm.max_map_count, too.

5. sys.vm.max_map_count

   cat /proc/sys/vm/max_map_count should be at least (2 x thread-count).

   Attempt to protect stack guard pages failed. and OpenJDK 64-Bit Server VM warning: Attempt to deallocate stack guard pages failed. error messages are emitted by JavaThread::create_stack_guard_pages(), and it calls os::guard_memory(). In Linux, this function is mprotect().

   echo 1999999 > /proc/sys/vm/max_map_count
   
   or
   
   sys.vm.max_map_count = 1999999 in /etc/sysctl.conf to change permanently.

---

Related Topics