Understanding Load Average VS. CPU Usage

Understanding load average vs. cpu usage

top shows CPU utilization for running processes while load average shows (since 1993) number of running processes plus number of processes in the uninterruptible state. Processes waiting for work do not consume CPU. As a result top CPU utilization is less that 7/8 * 100%.

Source: http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html

CPU idle % vs load average

I think you might need to revise your conception of what a 'high' load average is. Since you have something running on cpu/19, you have at least 19 CPU threads in your system -- probably 24 or 32. So to get the idle down close to 0, you'll need a load average of MORE than that. With only ~7, you'll only be able to keep at most around 1/4 of the CPU threads busy, so the other 75% will be idle.

Linux(Ubuntu) load average higher than total-true-utilization?

There does not appear to be anything wrong with your system. What is wrong seems to be your understanding of CPU accounting. In particular, load average has nearly nothing at all to do with CPU usage. Load average is based on the number of processes that are ready to run (not waiting on I/O, network, keyboard input, etc...), if there is an available CPU for them to be scheduled on. While it's true that, given an 8 core system, if all 8 cores are 100% busy with a single CPU-bound thread each, your load average should be around 8.00, it is entirely possible to have a load average of 200.0 with near-0% CPU utilization. All that would indicate is you have 200 processes that are ready to run, but as soon as they get scheduled, they do almost nothing before they go back to waiting for input of some sort.

Your top output shows that vlc seems to be using roughly the equivalent of 5 of your cores, but it doesn't indicate whether you have 5 cores at 100% each, or if all 8 cores are at 62.5% each. All of the other processes listed by top also contribute to your load average, as well as CPU usage. In particular, top running with a short delay like your example of 0.1 seconds, will probably increase your load average by almost 1 itself, even though, overall, it's not using a lot of CPU time.

Does load average affect performance?

No, but that doesn't mean your benchmark will run the same.

The answer to your question is no. The load average is a reported value. It is meant to give you an idea of the state of the system, averaged over several periods of time. Since it is averaged, it takes time for it to go back to 0 after a heavy load was placed on the system.

This is just a report, however. Your system isn't really loaded, and the CPU isn't currently taken. A new benchmark you'll run is unaffected by the system's state 5 minutes ago.

With that said, what is true for CPU may not be true for memory. If your loader uses a lot of memory, the kernel might push less used memory into the swap. It will also reduce the amount of memory it has for file cache. Depending on your benchmark, that might affect the benchmark's performance.

What Would Happen if CPU Load Average is High

It's just that: if you run absurdly high load averages, the overall efficiency will suffer: the CPU processing power will go to waste.

This is caused by several factors; the most immediate being more CPU time needed for scheduling the competing tasks. One not at all insignificant factor is that several competing processes will also overutlize the CPU cache; each task switch effectively throwing out the cache contents and replacing them with new ones. Further choke points come in forms of bottlenecks in memory and storage bandwidths.

What is the CPU load unit of measure in cgroup (difference between CPU Usage and CPU Load)?

It is the number of tasks. A very nice explanation can be found here: https://serverfault.com/questions/667078/high-cpu-utilization-but-low-load-average

what is CPU LOAD AVERAGE?

I can see there is no CPU load on your server, If you are facing server load issues then I will suggest you please monitor your server through top and ps -aufx command. With this command you can find out exact root cause of your server load issues.

CPU Load average rule for 5 minutes

The following expression should work:

expr: node_load5 / count by (instance, job) (node_cpu_seconds_total{mode="idle"}) >= 0.95

---

Related Topics