Fixedthreadpool VS Cachedthreadpool: the Lesser of Two Evils

FixedThreadPool vs CachedThreadPool: the lesser of two evils

A CachedThreadPool seems appropriate for your situation as there are no negative consequence to using one for long running threads directly. The comment in the java doc about CachedThreadPools being suitable for short tasks merely suggest that they are particularly appropriate for such cases, not that they cannot be used for long running tasks.

The main concern with a CachedThreadPool is that it will create up to Integer.MAX_VALUE number of threads as it will always spawn a new thread if an unused one does not exist in the cache. So if you have long running tasks it is then more likely that you could grow the number of concurrent threads more than you desire since this type of thread pool will not limit how many execute concurrently itself. This does not seem to be a problem for your use case as described, but it is something to be aware of.

To elaborate further on the difference between a CachedThreadPool and a FixedThreadPool, Executors.newCachedThreadPool and Executors.newFixedThreadPool are both backed by the same thread pool implementation (at least in the open JDK) via an instance of ThreadPoolExecutor, just with different parameters. The differences just being their thread minimum, maximum, thread kill time, and queue type.

public static ExecutorService newFixedThreadPool(int nThreads) {
     return new ThreadPoolExecutor(nThreads, nThreads,
                                   0L, TimeUnit.MILLISECONDS,
                                   new LinkedBlockingQueue<Runnable>());
 }

public static ExecutorService newCachedThreadPool() {
    return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                                 60L, TimeUnit.SECONDS,
                                 new SynchronousQueue<Runnable>());
}

A FixedThreadPool does have its advantages when you do in fact want to work with a fixed number of threads, since then you can submit any number of tasks to the executor service while knowing that the number of threads will be maintained at the level you specified. If you explicitly want to grow the number of threads, then this is not the appropriate choice.

This does however mean that the one issue that you may have with the CachedThreadPool is in regards to limiting the number of threads that are running concurrently. The CachedThreadPool will not limit them for you, so you may need to write your own code to ensure that you do not run too many threads, which you can do relatively easily by instantiating your own ThreadPoolExecutor with your desired behaviour characteristics. This really depends on the design of your application and how tasks are submitted to the executor service.

CachedThreadPool vs FixedThreadPool

Go with newCachedThreadPool it is better fit for this situation, because your task are small and I/O (network) bound. Which means you should create threads (usually 1.5x ~ 2x times) greater than number of processor cores to get optimum output, but here I guess newCachedThreadPool will manage itself. So, newCachedThreadPool will have less overhead as compared to newFixedThreadPool and will help in your situation.

If you had CPU intensive tasks then newFixedThreadPool could have been a better choice.

---

Update

A list of addresses will be obtained about 10 addresses.

If you need only 10 address always, then it doesn't matter, go with newCachedThreadPool. But if you think that number of address can increase then use newFixedThreadPool with number of threads <= 1.5x to 2x times number of cores available.

---

From Java docs:

newFixedThreadPool

Creates a thread pool that reuses a
fixed number of threads operating off
a shared unbounded queue. At any
point, at most nThreads threads will
be active processing tasks. If
additional tasks are submitted when
all threads are active, they will wait
in the queue until a thread is
available. If any thread terminates
due to a failure during execution
prior to shutdown, a new one will take
its place if needed to execute
subsequent tasks. The threads in the
pool will exist until it is explicitly
shutdown.

newCachedThreadPool

Creates a thread pool that creates new
threads as needed, but will reuse
previously constructed threads when
they are available. These pools will
typically improve the performance of
programs that execute many short-lived
asynchronous tasks. Calls to execute
will reuse previously constructed
threads if available. If no existing
thread is available, a new thread will
be created and added to the pool.
Threads that have not been used for
sixty seconds are terminated and
removed from the cache. Thus, a pool
that remains idle for long enough will
not consume any resources. Note that
pools with similar properties but
different details (for example,
timeout parameters) may be created
using ThreadPoolExecutor constructors.

Java CachedThreadPool vs FixedThreadPool

The most important difference between a cached thread pool and a fixed thread pool in Java is that the cached thread pool has no upper limit on the number of threads it will spawn and use. Which one is preferred depends on what you want the scaling behavior to be like.

The main advantage of the cached thread pool is that threads will begin execution immediately even if you have an unanticipated large number of tasks to execute. For example, your business needs may increase over months or years, and your application may be moved to more powerful machines, and use of a cached thread pool will permit servicing the increased needs by using the increased available processing power without having to change the code. This can be an advantage since once the application has been in service for months or years, people may not remember the code well enough easily to identify the thread limit as a parameter that can be changed to improve performance.

The main advantage of the fixed thread pool is that the number of threads is more tightly controlled. This helps prevent other parts of the software installation - either within the application or in other applications - from being starved of processing power should this application receive a large number of tasks in a short period of time. In addition, the risk of running into the operating system threading limit is reduced, reducing the risks of software crashes that can result when a process needs to spawn a thread and is not able to do so.

Java CachedThreadPool vs FixedThreadPool for multithreaded server

I would recommend to use the Executors#cachedThreadPool if the size of clients isn't defined.

Documentation Executors#cachedThreadPool:

* Creates a thread pool that creates new threads as needed, but
* will reuse previously constructed threads when they are
* available.  These pools will typically improve the performance
* of programs that execute many short-lived asynchronous tasks.
* Calls to {@code execute} will reuse previously constructed
* threads if available. If no existing thread is available, a new
* thread will be created and added to the pool. Threads that have
* not been used for sixty seconds are terminated and removed from
* the cache. Thus, a pool that remains idle for long enough will
* not consume any resources.

Documentation Executors#fixedThreadPool:

* Creates a thread pool that reuses a fixed number of threads
* operating off a shared unbounded queue.  At any point, at most
* {@code nThreads} threads will be active processing tasks.
* If additional tasks are submitted when all threads are active,
* they will wait in the queue until a thread is available.

So if you aren't sure that you exactly for example need 5 threads you shouldn't use the Executors#fixedThreadPool.

Executors.newCachedThreadPool() versus Executors.newFixedThreadPool()

I think the docs explain the difference and usage of these two functions pretty well:

newFixedThreadPool

Creates a thread pool that reuses a
fixed number of threads operating off
a shared unbounded queue. At any
point, at most nThreads threads will
be active processing tasks. If
additional tasks are submitted when
all threads are active, they will wait
in the queue until a thread is
available. If any thread terminates
due to a failure during execution
prior to shutdown, a new one will take
its place if needed to execute
subsequent tasks. The threads in the
pool will exist until it is explicitly
shutdown.

newCachedThreadPool

Creates a thread pool that creates new
threads as needed, but will reuse
previously constructed threads when
they are available. These pools will
typically improve the performance of
programs that execute many short-lived
asynchronous tasks. Calls to execute
will reuse previously constructed
threads if available. If no existing
thread is available, a new thread will
be created and added to the pool.
Threads that have not been used for
sixty seconds are terminated and
removed from the cache. Thus, a pool
that remains idle for long enough will
not consume any resources. Note that
pools with similar properties but
different details (for example,
timeout parameters) may be created
using ThreadPoolExecutor constructors.

In terms of resources, the newFixedThreadPool will keep all the threads running until they are explicitly terminated. In the newCachedThreadPool Threads that have not been used for sixty seconds are terminated and removed from the cache.

Given this, the resource consumption will depend very much in the situation. For instance, If you have a huge number of long running tasks I would suggest the FixedThreadPool. As for the CachedThreadPool, the docs say that "These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks".

Why jvm recreate thread pool in case fixedThreadPool and don't do it in case of cachedThreadPool?

To answer your question; just look here:

private static long test() throws InterruptedException {
  ExecutorService executorService = Executors.newFixedThreadPool(100);

The JVM creates a new ThreadPool during each run of test(), because you tell it to do so.

In other words: if you intend to re-use the same threadpool, then avoid creating/shutting down your instances all the time.

In that sense, the simple fix is: move the creation of that ExecutorService into your main() method; and pass the service as argument to your test() method.

Edit: regarding your last comment on cached vs. fixed threadpool; you probably want to look into this question.

FixedThreadPool executor limited by some termination condition

Either use some kind of coordination mechanism (phaser, which was introduced in Java 7 is useful when you need to add more jobs on the fly) or just keep an external flag you set when done:

ExecutorService service = Executors.newFixedThreadPool(N);
volatile boolean done = false;

// add your jobs:
service.submit(() -> {
    while (!done) {
        // do something
    }
});

// set your flag
done = true;

It's enough for the flag to be volatile because only one thread is mutating the value; the threads inside the executor just need visibility into when it changes.

Java- FixedThreadPool with known pool size but unknown workers

Your solution is completely fine (the only point is that parallelism is perhaps not necessary if the workload of your WorkerThreads is very small).

With a thread pool, the number of submitted tasks is not relevant. There may be less or more than the number of threads in the pool, the thread pool takes care of that.

However, and this is important: You rely on some kind of order of the results of your WorkerThreads, but when using parallelism, this order is not guaranteed! It doesn't matter whether you use a thread pool, or how much worker threads you have, etc., it will always be possible that your results will be finished in an arbitrary order!

To keep the order right, give each WorkerThread the number of the current item in its constructor, and let them put their results in the right order after they are finished:

int noOfWorkItem = 0;
while((bytesRead = inBytes.read(buff)) != -1) {
   System.arraycopy(buff, BLOCK_SIZE-DICT_SIZE, dict, 0, DICT_SIZE);
   worker = new WorkerThread(buff, dict, noOfWorkItem++)   
   tpes.execute(worker);
}

ThreadPool executor and huge amount of clients connected simultaneously

If you are going to have really huge amount of clients, you should consider NIO for it, because creating thread for each client will be too expensive.

NIO uses selectors and channels and doesn't require to create new thread for each connection. See image.

Did you hear about netty ? I don't know what you are going to implement but seems like it will be useful.

---

Related Topics