Setting CPU Affinity of a Process from the Start on Linux

setting cpu affinity of a process from the start on linux

taskset can be used both to set the affinity of a running process or to launch a process with a certain affinity, see

• How to launch your application in a specific CPU in Linux (CPU affinity)?.
• man page for taskset

Synopsis

taskset [options] mask command [arg]...
taskset [options] -p [mask] pid

The below command will launch Google Chrome browser in CPU 1 & 2 (or 0 and 1). The mask is 0×00000003 and command is “google-chrome”.

taskset 0×00000003 google-chrome

How to set CPU affinity for a process from C or C++ in Linux?

You need to use sched_setaffinity(2).

For example, to run on CPUs 0 and 2 only:

#define _GNU_SOURCE
#include <sched.h>

cpu_set_t  mask;
CPU_ZERO(&mask);
CPU_SET(0, &mask);
CPU_SET(2, &mask);
int result = sched_setaffinity(0, sizeof(mask), &mask);

(0 for the first parameter means the current process, supply a PID if it's some other process you want to control).

See also sched_getcpu(3).

Setting CPU affinity to a process - C - Linux

OK.
This is the stupidest thing that one can ever do!

    CPU_SET(pid, &mask);

    CPU_SET(coreid, &mask);

changing the pid to the coreid will do it.

The other mistake was here:

    result = sched_setaffinity(pid, sizeof(mask), &mask);

Change default CPU affinity

From C program:

#define _GNU_SOURCE
#include <sched.h>

int sched_setaffinity(pid_t pid, size_t cpusetsize, cpu_set_t *mask);

see man sched_setaffinity for further information.

From the shell:

taskset <mask> <command> <args>

or

taskset -p <pid> <mask>

where <mask> is, for example, 0x00000001 for the first CPU.

cpu affinity, allowing only process to run on a specific cpu

I've discussed this in a series of comments to the original question, but I think it's "the answer to the underlying problem" rather than a specific answer to your specific question, so here we go:

I've heard this question a couple of times, and it was always being
asked out of a misunderstanding of how simultaneous multiprocessing
works. First of all: Why do you need your process on core #0? Do you
have a reason?

typically, the linux kernel is quite efficient at scheduling tasks to
processors in a manner that minimizes the negative effects that either
process migration or a single-core-bottleneck would bring. In fact,
it's very rare to see someone actually gain performance by manually
setting affinities, and that usually only happens when the userland
process is closely communicating with a kernel module which for some
hardware or implementation reason is inherently single-threaded and
can't easily be migrated.

Your question itself shows a minor misunderstanding: Affinity means
that the kernel knows that it should schedule that process on the
given core. Hence, other processes will automatically be migrated away
from that core and only be running there if your desired task leaves a
lot of that core unused. To change the "priority" your process has in
CPU allocation, just change its nice value.

The reason is performance measurement by isolating the running process. Regarding the second comment, that mean we have to rely on OS
scheduler because it will not run a background process on a core which
is currently utilized 100% while there are idle cores.

what you're measuring is not the performance of the process
isolatedly, but of the process, bound to a single CPU! For a
single-threaded process that's ok, but imagine that your process might
have multiple threads -- these all would normally run on different
cores, and overall performance would be much higher. Generally, try
just minimizing non-process workloads on your machine (ie. run without
a window manager/session manager running, stop all non-essential
services) and use a very small nice value -- that measurement might be
relatively precise.

Also, the time command allows you to know how much time a process
spend totally (including waiting), occupying CPU as userland process,
and occupying CPU in system calls -- I think that might fit your needs
well enough :)

Assigning a cpu core to a process - Linux

There are two ways of assigning cpu core/cores to a running process.

First method:

taskset -cp 0,4 9030

Pretty clear ! assigning cpu cores 0 and 4 to the pid 9030.

Second Method:

taskset -p 0x11 9030

This is a bit more complex. The hexadecimal number that follows -p is a bitmask. An explanation can be found here, an excerpt of which is given below :

The CPU affinity is represented as a bitmask, with the lowest order
bit corresponding to the first logical CPU and the highest order bit
corresponding to the last logical CPU. Not all CPUs may exist on a
given system but a mask may specify more CPUs than are present. A
retrieved mask will reflect only the bits that correspond to CPUs
physically on the system. If an invalid mask is given (i.e., one that
corresponds to no valid CPUs on the current system) an error is
returned. The masks are typically given in hexadecimal.

Still confused? Look at the image below :

I have added the binaries corresponding to the hexadecimal number and the processors are counted from left starting from zero. In the first example there is a one in the bitmask corresponding to the zeroth processor, so that processor will be enabled for a process. All the processors which have zero to their corresponding position in the bitmask will be disabled. In fact this is the reason why it is called a mask.

Having said all these, using taskset to change the processor affinity requires that :

A user must possess CAP_SYS_NICE to change the CPU affinity of a
process. Any user can retrieve the affinity mask.

Please check the Capabalities Man Page.

You might be interested to look at this SO Question that deals with CAP_SYS_NICE.

My Resources

1. Tutorials
   Point

2. XModulo

Force Linux to schedule processes on CPU cores that share CPU cache

Newer Linux may do this for you: Cluster-Aware Scheduling Lands In Linux 5.16 - there's support for scheduling decisions to be influenced by the fact that some cores share resources.

If you manually pick a CCX, you could give them each the same affinity mask that allows them to schedule on any of the cores in that CCX.

An affinity mask can have multiple bits set.

---

I don't know of a way to let the kernel decide which CCX, but then schedule both tasks to cores within it. If the parent checks which core it's currently running on, it could set a mask to include all cores in the CCX containing it, assuming you have a way to detect how core #s are grouped, and a function to apply that.

You'd want to be careful that you don't end up leaving some CCXs totally unused if you start multiple processes that each do this, though. Maybe every second, do whatever top or htop do to check per-core utilization, and if so rebalance? (i.e. change the affinity mask of both processes to the cores of a different CCX). Or maybe put this functionality outside the processes being scheduled, so there's one "master control program" that looks at (and possibly modifies) affinity masks for a set of tasks that it should control. (Not all tasks on the system; that would be a waste of work.)

Or if it's looking at everything, it doesn't need to do so much checking of current load average, just count what's scheduled where. (And assume that tasks it doesn't know about can pick any free cores on any CCX, like daemons or the occasional compile job. Or at least compete fairly if all cores are busy with jobs it's managing.)

---

Obviously this is not helpful for most parent/child processes, only ones that do a lot of communication via shared memory (or maybe pipes, since kernel pipe buffers are effectively shared memory).

It is true that Zen CPUs have varying inter-core latency within / across CCXs, as well as just cache hit effects from sharing L3. https://www.anandtech.com/show/16529/amd-epyc-milan-review/4 did some microbenchmarking on Zen 3 vs. 2-socket Xeon Platinum vs. 2-socket ARM Ampere.

---

Related Topics