How to Calculate CPU Utilization of a Process & All Its Child Processes in Linux

How to calculate CPU utilization of a process & all its child processes in Linux?

You can find this information in /proc/PID/stat where PID is your parent process's process ID. Assuming that the parent process waits for its children then the total CPU usage can be calculated from utime, stime, cutime and cstime:

utime %lu

Amount of time that this process has been scheduled in user mode,
measured in clock ticks (divide by sysconf(_SC_CLK_TCK). This includes
guest time, guest_time (time spent running a virtual CPU, see below),
so that applications that are not aware of the guest time field do not
lose that time from their calculations.

stime %lu

Amount of time that this process has been scheduled in kernel mode,
measured in clock ticks (divide by sysconf(_SC_CLK_TCK).

cutime %ld

Amount of time that this process's waited-for children have been
scheduled in user mode, measured in clock ticks (divide by
sysconf(_SC_CLK_TCK). (See also times(2).) This includes guest time,
cguest_time (time spent running a virtual CPU, see below).

cstime %ld

Amount of time that this process's waited-for children have been
scheduled in kernel mode, measured in clock ticks (divide by
sysconf(_SC_CLK_TCK).

See proc(5) manpage for details.

How to calculate the CPU usage of a process by PID in Linux from C?

You need to parse out the data from /proc/<PID>/stat. These are the first few fields (from Documentation/filesystems/proc.txt in your kernel source):

Table 1-3: Contents of the stat files (as of 2.6.22-rc3)
..............................................................................
 Field          Content
  pid           process id
  tcomm         filename of the executable
  state         state (R is running, S is sleeping, D is sleeping in an
                uninterruptible wait, Z is zombie, T is traced or stopped)
  ppid          process id of the parent process
  pgrp          pgrp of the process
  sid           session id
  tty_nr        tty the process uses
  tty_pgrp      pgrp of the tty
  flags         task flags
  min_flt       number of minor faults
  cmin_flt      number of minor faults with child's
  maj_flt       number of major faults
  cmaj_flt      number of major faults with child's
  utime         user mode jiffies
  stime         kernel mode jiffies
  cutime        user mode jiffies with child's
  cstime        kernel mode jiffies with child's

You're probably after utime and/or stime. You'll also need to read the cpu line from /proc/stat, which looks like:

cpu  192369 7119 480152 122044337 14142 9937 26747 0 0

This tells you the cumulative CPU time that's been used in various categories, in units of jiffies. You need to take the sum of the values on this line to get a time_total measure.

Read both utime and stime for the process you're interested in, and read time_total from /proc/stat. Then sleep for a second or so, and read them all again. You can now calculate the CPU usage of the process over the sampling time, with:

user_util = 100 * (utime_after - utime_before) / (time_total_after - time_total_before);
sys_util = 100 * (stime_after - stime_before) / (time_total_after - time_total_before);

Make sense?

Efficient way to calculate cpu usage of multiple processes in linux

You should do it using /proc as you have already discovered. Or perhaps using some higher-level library if your language of choice offers one. Parsing the output of ps and top is not winning you anything. The data you need are all in /proc, you just need to be careful to check the identity of each process over time, in case PIDs get recycled while you are collecting data (on most systems PID recycling won't happen in the span of mere seconds, so it's workable).

Find cpu time of a process and it children while the process is still running

You can calculate it using the:

utime %lu

Amount of time that this process has been scheduled in user mode,
measured in clock ticks (divide by sysconf(_SC_CLK_TCK). This includes
guest time, guest_time (time spent running a virtual CPU, see below),
so that applications that are not aware of the guest time field do not
lose that time from their calculations.

stime %lu

Amount of time that this process has been scheduled in kernel mode,
measured in clock ticks (divide by sysconf(_SC_CLK_TCK).

cutime %ld

Amount of time that this process's waited-for children have been
scheduled in user mode, measured in clock ticks (divide by
sysconf(_SC_CLK_TCK). (See also times(2).) This includes guest time,
cguest_time (time spent running a virtual CPU, see below).

cstime %ld

Amount of time that this process's waited-for children have been
scheduled in kernel mode, measured in clock ticks (divide by
sysconf(_SC_CLK_TCK).

C Program to get CPU usage for a PID and all its children

Yes, the parent needs to wait for the CPU time of the children to be added in (see manual entry for getrusage link). Also see this answer for more details.

How do I get the total CPU usage of an application from /proc/pid/stat?

Preparation

To calculate CPU usage for a specific process you'll need the following:

/proc/uptime
- #1 uptime of the system (seconds)
/proc/[PID]/stat
- #14 utime - CPU time spent in user code, measured in clock ticks
- #15 stime - CPU time spent in kernel code, measured in clock ticks
- #16 cutime - Waited-for children's CPU time spent in user code (in clock ticks)
- #17 cstime - Waited-for children's CPU time spent in kernel code (in clock ticks)
- #22 starttime - Time when the process started, measured in clock ticks
Hertz (number of clock ticks per second) of your system.
- In most cases, getconf CLK_TCK can be used to return the number of clock ticks.
- The sysconf(_SC_CLK_TCK) C function call may also be used to return the hertz value.

Calculation

First we determine the total time spent for the process:

total_time = utime + stime

We also have to decide whether we want to include the time from children processes. If we do, then we add those values to total_time:

total_time = total_time + cutime + cstime

Next we get the total elapsed time in seconds since the process started:

seconds = uptime - (starttime / Hertz)

Finally we calculate the CPU usage percentage:

cpu_usage = 100 * ((total_time / Hertz) / seconds)

How to get CPU usage from related processes?

What you're looking for is a program or script that will gather the CPU usage of all child processes spawned by scrapy.

If you wanted to script this yourself, you could look at the output of ps -p {scrapy pid} -L to get all the threads spawned by the instantiation of scrapy.

Or, you could chain together a couple Linux commands to have a one-liner:

ps -C scrapy -o pcpu= | awk '{cpu_usage+=$1} END {print cpu_usage}'

ps:

-C specifies the command name to output

-o pcou= tells ps to only display cpu usage

awk:

{cpu_usage+=$1} END loops over the response from ps

{print cpu_usage} will send the sum to STDOUT.

Get cpu of child process

check it in parent process through /proc interface
check it in child processes, and send it back to parent process through IPC, such as shared memory or socket, etc..