Executors.Newcachedthreadpool() Versus Executors.Newfixedthreadpool()

Executors.newCachedThreadPool() versus Executors.newFixedThreadPool()

I think the docs explain the difference and usage of these two functions pretty well:

newFixedThreadPool

Creates a thread pool that reuses a
fixed number of threads operating off
a shared unbounded queue. At any
point, at most nThreads threads will
be active processing tasks. If
additional tasks are submitted when
all threads are active, they will wait
in the queue until a thread is
available. If any thread terminates
due to a failure during execution
prior to shutdown, a new one will take
its place if needed to execute
subsequent tasks. The threads in the
pool will exist until it is explicitly
shutdown.

newCachedThreadPool

Creates a thread pool that creates new
threads as needed, but will reuse
previously constructed threads when
they are available. These pools will
typically improve the performance of
programs that execute many short-lived
asynchronous tasks. Calls to execute
will reuse previously constructed
threads if available. If no existing
thread is available, a new thread will
be created and added to the pool.
Threads that have not been used for
sixty seconds are terminated and
removed from the cache. Thus, a pool
that remains idle for long enough will
not consume any resources. Note that
pools with similar properties but
different details (for example,
timeout parameters) may be created
using ThreadPoolExecutor constructors.

In terms of resources, the newFixedThreadPool will keep all the threads running until they are explicitly terminated. In the newCachedThreadPool Threads that have not been used for sixty seconds are terminated and removed from the cache.

Given this, the resource consumption will depend very much in the situation. For instance, If you have a huge number of long running tasks I would suggest the FixedThreadPool. As for the CachedThreadPool, the docs say that "These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks".

newFixedThreadPool() vs newCachedThreadPool()

newFixedThreadPool also creates threads lazily. Try this test:

    ThreadPoolExecutor p = (ThreadPoolExecutor) Executors.newFixedThreadPool(2);
    System.out.println(p.getPoolSize());
    p.execute(new Runnable() {public void run() {}});
    System.out.println(p.getPoolSize());

The differences are:

newFixedThreadPool's threads never expire, while newCachedThreadPool's threads expire 60 secs after last used
newCacheThreadPool's maximum number of active threads is unlimited

How to decide whether to use newCachedThreadPool or newFixedThreadPool?

So now I am wondering is - Should I use newFixedThreadPool or newCachedThreadPool in my code?

To quote from the Javadocs, the newFixedThreadPool():

Creates a thread pool that reuses a fixed number of threads...

This means that if you ask for 2 threads, it will start 2 threads and never start 3. On the other hand, the newCachedThreadPool():

Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available.

In your case, if you only have 2 thread to run, either will work fine since you will only be submitting 2 jobs to your pool. However, if you wanted to submit all 20 jobs at once but only have 2 jobs running at one time, you should use a newFixedThreadPool(2). If you used a cached pool then each of the 20 jobs will start a thread which will run at the same time which may not be optimal depending on how many CPUs you have.

Typically I use the newCachedThreadPool() when I need the thread to be spawned immediately, even if all of the threads currently running are busy. I recently used it when I was spawning timer tasks. The number of concurrent jobs are immaterial because I never spawn very many but I want them to run when they are requested and I want them to re-use dormant threads.

I used newFixedThreadPool() when I want to limit the number of concurrent tasks running at any one point to maximize performance and not swamp my server. For example if I am processing 100k lines from a file, one line at a time, I don't want each line to start a new thread but I want some level of concurrency so I allocate (for example) 10 fixed threads to run the tasks until the pool is exhausted.

Is it safe to have multiple Executors.newCachedThreadPool() in a Java program?

I don't think there is a clear yes / no answer on this.

On the one hand, there is not a finite number of threads that ThreadPoolExecutor instances consume. The JVM architecture itself doesn't limit the number of threads.

On the the second hand, the OS / environment may place some limits:

The OS may have hard limits on the total number of native threads it will support.
The OS may restrict the number of native threads that a given process (in this case the JVM) can create. This could be done using ulimit or cgroup limits, and potentially other ways.
A Java thread stack has a size of 1MB (by default) on a typical 64 bit JVM. If you attempt to start() too many threads, you may run out of memory and get an OOME.
If there are a large enough number of threads and/or too much thread context switching, the thread scheduler (in the OS) may struggle.
(Context switching typically happens when a thread does a blocking syscall or has to wait on a lock or a notification. Each time you switch context there are hardware related overheads: saving and restoring registers, switching virtual memory contexts, flushing memory caches, etc.)

On the third hand, there are other things than just the number and size of thread pools that could cause problems. For example, if the thread tasks interact with each other, you could experience problems due to:

deadlocking when locking shared objects,
too much contention on shared locks leading to resource starvation,
too much work leading to timeouts, or
priority inversion problems ... if you try to use priorities to "manage" the workload.

So ...

Is it safe to have several of these pools in a single program?
Or would I potentially run into a situation where one pool stalls on many threads and freezes up other pools.

It is unlikely you would get a "stall" ... unless the tasks are interacting in some way.

But if you have too many runnable threads competing for CPU, each one will get (on average) a smaller share of the finite number of cores available. And lock contention or too much context switching can slow things down further.

FixedThreadPool vs CachedThreadPool: the lesser of two evils

A CachedThreadPool seems appropriate for your situation as there are no negative consequence to using one for long running threads directly. The comment in the java doc about CachedThreadPools being suitable for short tasks merely suggest that they are particularly appropriate for such cases, not that they cannot be used for long running tasks.

The main concern with a CachedThreadPool is that it will create up to Integer.MAX_VALUE number of threads as it will always spawn a new thread if an unused one does not exist in the cache. So if you have long running tasks it is then more likely that you could grow the number of concurrent threads more than you desire since this type of thread pool will not limit how many execute concurrently itself. This does not seem to be a problem for your use case as described, but it is something to be aware of.

To elaborate further on the difference between a CachedThreadPool and a FixedThreadPool, Executors.newCachedThreadPool and Executors.newFixedThreadPool are both backed by the same thread pool implementation (at least in the open JDK) via an instance of ThreadPoolExecutor, just with different parameters. The differences just being their thread minimum, maximum, thread kill time, and queue type.

public static ExecutorService newFixedThreadPool(int nThreads) {
     return new ThreadPoolExecutor(nThreads, nThreads,
                                   0L, TimeUnit.MILLISECONDS,
                                   new LinkedBlockingQueue<Runnable>());
 }

public static ExecutorService newCachedThreadPool() {
    return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                                 60L, TimeUnit.SECONDS,
                                 new SynchronousQueue<Runnable>());
}

A FixedThreadPool does have its advantages when you do in fact want to work with a fixed number of threads, since then you can submit any number of tasks to the executor service while knowing that the number of threads will be maintained at the level you specified. If you explicitly want to grow the number of threads, then this is not the appropriate choice.

This does however mean that the one issue that you may have with the CachedThreadPool is in regards to limiting the number of threads that are running concurrently. The CachedThreadPool will not limit them for you, so you may need to write your own code to ensure that you do not run too many threads, which you can do relatively easily by instantiating your own ThreadPoolExecutor with your desired behaviour characteristics. This really depends on the design of your application and how tasks are submitted to the executor service.

Executors.newFixedThreadPool() - how expensive is this operation

How much expensive the above operation be..??. Creating fixedThreadPool at every given minute.

Creating a thread pool is a relatively expensive operation which can take milli-seconds. You don't want to be doing this many times per second.

A second is an eternity for a computer, if you have a 36 core machine it can execute as much as 100 billion instructions in that amount of time. A minute is a very, very long time, and if you only do something once a minute you could even restart your JVM every minute and still get reasonable throughput.

What can i do to optimize my solution, should i use a fixed thread pool, say (10), and maybe 3 or 5 or 6 or any number of threads getting utilized at any given minute.

Possibly, it depends on what you are doing. Without most analysis you could say for sure. Note: If you are using parallelStream(), if not you should see if you can, you can use the built in ForkJoinPool.commonPool() and not need to create another pool. But again, this depends on what you are doing.

when does executor really create new thread

Short answer is that in all cases you mentioned, threads are created only when you start executing the tasks, i.e. no threads will be created after this ExecutorService exec = Executors.newFixedThreadPool(3); line is executed, and first thread will be created only after this exec.execute(new LiftOff()); line is executed.

To better and fully understand this, as @Oliver has advised you need to walk through Java source code and also read about ThreadPoolExecutor, especially what is core pool size, max pool size etc. If you want to quickly understand this in layman's word then read this good answer.

Now, below are few good points to understand before I take you to some sample code:

Java's utility class Executors will create and return objects of ExecutorService (please note that ExecutorService is an interface).
Now in case of newCachedThreadPool(), newFixedThreadPool(int nThreads) you will get an object of ThreadPoolExecutor (please note that ThreadPoolExecutor implements ExecutorService interface).
As you can see in the source code, when you do Executors.newFixedThreadPool(3);, you only get an object of ThreadPoolExecutor with all instance variables like core pool size, max pool size etc. set, and in your case it will be set to 3.
At this point no threads are created, and actual thread creation only happens when you start executing tasks using execute, submit or invokeAll.
As you can see in source code, when you do first execute it will create only 1 new thread if current pool size is less than core pool size, so this is where your thread start getting created.

Please see below sample code which will help you understand and prove that threads are created only when you start executing the tasks. To explain you, main trick I did is I used the object reference variable of ThreadPoolExecutor so that I can call method like getPoolSize() (please note that getPoolSize() tells about how many threads are currently present in the pool), because when you use object reference variable of ExecutorService then you cannot call these methods (I am sure I need not to tell "why").

When you will run this example you will note that:

There were no threads created after ThreadPoolExecutor poolExecutor = (ThreadPoolExecutor) Executors.newFixedThreadPool(3);
After first executeRunnable(poolExecutor); you got one thread in the pool.
Since you have used core pool size as 3, so after creating 3 threads no more threads were created. (do read about core, max pool size and "queue" size)

package com.learn.stackoverflow.multithreading;

import java.util.Date;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;

/**
 * 
 * @author himanshu.agrawal
 *
 */
public class ExecutorServiceNumberOfThreads {
    public static void main(String[] args) {
        System.out.println("### Testing Executors.newFixedThreadPool()");
        testNewFixedThreadPool();
    }

    private static void testNewFixedThreadPool() {
        ThreadPoolExecutor poolExecutor = (ThreadPoolExecutor) Executors.newFixedThreadPool(3);
        System.out.println("Pool when no execute() : " + poolExecutor.getPoolSize());
        executeRunnable(poolExecutor);
        System.out.println("Pool after 1st execute() : " + poolExecutor.getPoolSize());
        executeRunnable(poolExecutor);
        System.out.println("Pool after 2nd execute() : " + poolExecutor.getPoolSize());
        executeRunnable(poolExecutor);
        System.out.println("Pool after 3rd execute() : " + poolExecutor.getPoolSize());
        executeRunnable(poolExecutor);
        System.out.println("Pool after 4th execute() : " + poolExecutor.getPoolSize());
        executeRunnable(poolExecutor);
        System.out.println("Pool after 5th execute() : " + poolExecutor.getPoolSize());
    }

    private static void executeRunnable(ThreadPoolExecutor poolExecutor) {
        poolExecutor.execute(new Runnable() {

            @Override
            public void run() {
                System.out.println("Running: " + Thread.currentThread().getId() + " | " + new Date());
            }
        });
    }

}

Output:

### Testing Executors.newFixedThreadPool()
Pool when no execute() : 0
Pool after 1st execute() : 1
Pool after 2nd execute() : 2
Pool after 3rd execute() : 3
Pool after 4th execute() : 3
Pool after 5th execute() : 3
Running: 10 | Sun Apr 23 19:50:32 IST 2017
Running: 8 | Sun Apr 23 19:50:32 IST 2017
Running: 10 | Sun Apr 23 19:50:32 IST 2017
Running: 9 | Sun Apr 23 19:50:32 IST 2017
Running: 8 | Sun Apr 23 19:50:32 IST 2017

Multiple newSingleThreadExecutor vs. newFixedThreadPool of ExecutorService

Given each task is a infinite loop, what I would used is a

newCachedThreadPool();

This would create a thread for every task which needed it (and no more)

The benefit of using a single threaded pool each is you could shutdown the pool individually, or give each thread a name, but if you don't need this, it's just overhead.

Note: you can change the name of a thread with setName("My task") which might be useful for debugging/profiling purposes.

One of the tricks of using an ExecutorService is that it captures any uncaught exception/errors and places it in the Future object returned. Often this Future is discarded which means that if your task dies unexpectedly it might also do it silently.

I suggest you do a try/catch(Throwable) outside the loop and log it so you can see if the thread ever dies unexpectedly. e.g OutOfMemoryError

Executors.Newcachedthreadpool() Versus Executors.Newfixedthreadpool()