Java Threads and Number of Cores

Java threads and number of cores

Processes vs Threads

In days of old, each process had precisely one thread of execution, so processes were scheduled onto cores directly (and in these old days, there was almost only one core to schedule onto). However, in operating systems that support threading (which is almost all moderns OS's), it is threads, not processes that are scheduled. So for the rest of this discussion we will talk exclusively about threads, and you should understand that each running process has one or more threads of execution.

Parallelism vs Concurrency

When two threads are running in parallel, they are both running at the same time. For example, if we have two threads, A and B, then their parallel execution would look like this:

CPU 1: A ------------------------->

CPU 2: B ------------------------->

When two threads are running concurrently, their execution overlaps. Overlapping can happen in one of two ways: either the threads are executing at the same time (i.e. in parallel, as above), or their executions are being interleaved on the processor, like so:

CPU 1: A -----------> B ----------> A -----------> B ---------->

So, for our purposes, parallelism can be thought of as a special case of concurrency*

Scheduling

But we are able to produce a thread pool(lets say 30) with a larger number than the number of cores that we posses(lets say 4) and have them run concurrently. How is this possible if we are only have 4 cores?

In this case, they can run concurrently because the CPU scheduler is giving each one of those 30 threads some share of CPU time. Some threads will be running in parallel (if you have 4 cores, then 4 threads will be running in parallel at any one time), but all 30 threads will be running concurrently. The reason you can then go play games or browse the web is that these new threads are added to the thread pool/queue and also given a share of CPU time.

Logical vs Physical Cores

According to my current understanding, a core can only perform 1 process at a time

This is not quite true. Due to very clever hardware design and pipelining that would be much too long to go into here (plus I don't understand it), it is possible for one physical core to actually be executing two completely different threads of execution at the same time. Chew over that sentence a bit if you need to -- it still blows my mind.

This amazing feat is called simultaneous multi-threading (or popularly Hyper-Threading, although that is a proprietary name for a specific instance of such technology). Thus, we have physical cores, which are the actual hardware CPU cores, and logical cores, which is the number of cores the operating system tells software is available for use. Logical cores are essentially an abstraction. In typical modern Intel CPUs, each physical core acts as two logical cores.

can someone explain how this works and also recommend some good reading on this?

I would recommend Operating System Concepts if you really want to understand how processes, threads, and scheduling all work together.

• The precise meanings of the terms parallel and concurrent are hotly debated, even here in our very own stack overflow. What one means by these terms depends a lot on the application domain.

How to scale threads according to CPU cores?

You can determine the number of processes available to the Java Virtual Machine by using the static Runtime method, availableProcessors. Once you have determined the number of processors available, create that number of threads and split up your work accordingly.

Update: To further clarify, a Thread is just an Object in Java, so you can create it just like you would create any other object. So, let's say that you call the above method and find that it returns 2 processors. Awesome. Now, you can create a loop that generates a new Thread, and splits the work off for that thread, and fires off the thread. Here's some pseudocode to demonstrate what I mean:

int processors = Runtime.getRuntime().availableProcessors();
for(int i=0; i < processors; i++) {
  Thread yourThread = new AThreadYouCreated();
  // You may need to pass in parameters depending on what work you are doing and how you setup your thread.
  yourThread.start();
}

For more information on creating your own thread, head to this tutorial. Also, you may want to look at Thread Pooling for the creation of the threads.

Tomcat thread pool size and number of cores

the number of cpu cores rarely exceeds 32.
As others have mentioned, threads provide concurrency - another thread can be executing while a different thread is in a blocked state. If all threads were in an active state, then it's true that some threads will context switch to another thread assigned to its core. But anyway, this also seems like a configurable parameter - https://www.baeldung.com/java-web-thread-pool-config

Number of threads vs processor core?

In an ideal world, where a thread has a continuous stream of steady work to do, having one thread per core might be a good idea.

In the real world, there are plenty of times where threads have to wait. They wait for a resource to be available, or for the user to do something productive. When they are waiting, the core sits idle and has nothing to do. Why not fill that time by switching to another thread and doing it's work?

Java multi-threading execution on CPU core

Each thread has an execution stack that tracks its state. It does not matter whether the two threads run on the same CPU or different CPU. In fact what usually happens is that a thread runs a little bit at a time, and the same CPU can be switch back and forth, running one thread for a while, and running the other for a while.

When the threads are allocated to different CPUs, then the possibility of conflicting updates to shared objects is greater, because the threads could be running at the same time causing fine-grained interweaving of updates to memory. If you design your threads to protect against concurrency problems, then it really does not matter much whether they run on the same CPU or different CPU. It does not matter if the thread is run sometimes on one CPU, and later by a different CPU. The thread holds all its own state and does not matter which CPU runs it.

It seems that Java delegates to the operating system the decision on where to run a particular thread.

---

Related Topics