Linux: How to Put a Load on System Memory

Linux: How to put a load on system memory?

I didn't understand very well if you want to generate arbitrary CPU load or CPU utilization. Yes, they are different things indeed. I'll try to cover both problems.

First of all: load is the average number of processes in the running, runnable or waiting for CPU scheduler queues in a given amount of time, "the one that wants your CPU" so to speak.

So, if you want to generate arbitrary load (say 0.3) you have to run a process for 30% of the time and then remove it from the run queue for 70% of the time, moving it to the sleeping queue or killing it, for example.

You can try this script to do that:

export LOAD=0.3
while true
     do yes > /dev/null &
     sleep $LOAD
     killall yes
     sleep `echo "1 - $LOAD" | bc`
done

Note that you have to wait some time (1, 10 and 15 minutes) to get the respective numbers to come up, and it will be influenced by other processes in your system. The more busy your system is the more this numbers will float. The last number (15 minutes interval) tends to be the most accurate.

CPU usage is, instead, the amount of time for which CPU was used for processing instructions of a computer program.

So, if you want to generate arbitrary CPU usage (say 30%) you have to run a process that is CPU bound 30% of the time and sleeps 70% of it.

I wrote an example to show you that:

#include <stdlib.h>
#include <unistd.h>
#include <err.h>
#include <math.h>
#include <sys/time.h>
#include <stdarg.h>
#include <sys/wait.h>

#define CPUUSAGE 0.3      /* set it to a 0 < float < 1 */
#define PROCESSES 1       /* number of child worker processes */
#define CYCLETIME 50000   /* total cycle interval in microseconds */

#define WORKTIME (CYCLETIME * CPUUSAGE)
#define SLEEPTIME (CYCLETIME - WORKTIME)

/* returns t1-t2 in microseconds */
static inline long timediff(const struct timeval *t1, const struct timeval *t2)
{
  return (t1->tv_sec - t2->tv_sec) * 1000000 + (t1->tv_usec - t2->tv_usec);
}

static inline void gettime (struct timeval *t)
{
  if (gettimeofday(t, NULL) < 0)
  {
    err(1, "failed to acquire time");
  }
}

int hogcpu (void)
{
  struct timeval tWorkStart, tWorkCur, tSleepStart, tSleepStop;
  long usSleep, usWork, usWorkDelay = 0, usSleepDelay = 0;

  do
  {
    usWork = WORKTIME - usWorkDelay;
    gettime (&tWorkStart);
    do
    {
      sqrt (rand ());
      gettime (&tWorkCur);
    }
    while ((usWorkDelay = (timediff (&tWorkCur, &tWorkStart) - usWork)) < 0);

    if (usSleepDelay <= SLEEPTIME)
      usSleep = SLEEPTIME - usSleepDelay;
    else
      usSleep = SLEEPTIME;

    gettime (&tSleepStart);
    usleep (usSleep);
    gettime (&tSleepStop);
    usSleepDelay = timediff (&tSleepStop, &tSleepStart) - usSleep;
  }
  while (1);
  return 0;
}

int main (int argc, char const *argv[])
{
  pid_t pid;
  int i;
  for (i = 0; i < PROCESSES; i++)
  {
    switch (pid = fork ())
    {
    case 0:
      _exit (hogcpu ());
    case -1:
      err (1, "fork failed");
      break;
    default:
      warnx ("worker [%d] forked", pid);
    }
  }
  wait(NULL);
  return 0;
}

If you want to eat up a fixed amount of RAM you can use the program in the cgkanchi's answer.

Retrieve CPU usage and memory usage of a single process on Linux?

ps -p <pid> -o %cpu,%mem,cmd

(You can leave off "cmd" but that might be helpful in debugging).

Note that this gives average CPU usage of the process over the time it has been running.

How to get the memory and cpu usage of a remote server?

After searching online and combining a few answers from other questions on stackflow. I get the following solution.

Solution

On your local computer, you might want to have the following bash script, named, say, usage_ssh

START=1
END=3
date
for i in $(seq $START $END)
do
        printf '=%.0s' {1..50};
        printf '\n'
        echo myservery$i
        ssh myserver$i -o LogLevel=QUIET -t "~/bin/usage"
done
printf '=%.0s' {1..50};
printf '\n'
printf 'CPU Load: \n'
printf 'First Field\tprocesses per processor\n'
printf 'Second Filed\tidling percentage in last 5 minutes\n'
printf '\n'
printf '\n'

On your remote server, you should have the following bash script named usage. This script should be located in ~/bin.

free -m | awk 'NR==2{printf "Memory Usage\t%s/%sMB\t\t%.2f%\n", $3, $2, $3/$2*100}';
top -n 1 | grep load | awk '{printf "CPU Load\t%.2f\t\t\t%.2f\n", $(NF-2), $(NF-1)}';

Explanation

The idea is that You will call the use ssh -t <your command> to run executable on your remote file and get the output on the screen of your local computer.

Output

Sat Mar 28 10:32:34 CDT 2020
==================================================
myserver1
Memory Usage    47418/48254MB       98.27%
CPU Load        0.01                0.02
==================================================
myserver2
Memory Usage    47421/48254MB       98.27%
CPU Load        0.01                0.02
==================================================
myserver3
Memory Usage    4300/84541MB        5.09%
CPU Load        0.02                0.02
==================================================
CPU Load: 
First Field processes per processor
Second Filed    idling percentage in last 5 minutes

How to log the memory consumption on Linux?

A small script like

rm memory.log
while true; do free >> memory.log; sleep 1; done

How do we Load Linux Image to appropiate location in Memory

from file position 0x1000 means what it says. You have it in the dump:

[Nr]      Name           Type            Addr     Off    Size   ES Flg Lk Inf Al
...
[ 1]   .text            PROGBITS        c5000000 001000 5ac13e 00  AX  0   0 4096

It's where the .text section starts in the file, at offset 0x1000.

But one thing we can see here .text section is greater than DRAM end address

Nope, it's not greater (not in the sense of bigger, at least), it's compiled in the expectation that it'll be loaded at address 0xc5000000 in the memory.

so image should not be loded properly though we are not getting any error after loading first section it keeps on loading other sections

The image can be loaded anywhere, it's just data for the purpose of loading.

OTOH, if loading section to address 0xc5000000 means what it says, the file gets loaded into nowhere since your RAM ends at 0x7fffffff.

but after this message it hangs.

And that's expected. Machine code is rarely position-independent and so if you load it at a location different from where it's supposed to be loaded, it'll not work. Or if it doesn't even get loaded, then what are you going to execute? Garbage.

Is there any way to set these section addresses before compilation??

Depending on the system you may have one of the two below options or both:

• set up page translation in such a way that virtual addresses from 0xc5000000 and up map to physical addresses from 0x5000000 and up for the entire program
• find the linker script that your compiler is using and change the initial section address from 0xc5000000 to 0x5000000, google this up, see compiler/linker documentation

Also, it's a bit odd that the entry point is at 0x5000000. Not that this is necessarily wrong, it's just that it's rarely the case. I'd make sure that the start label (or _start or whatever it is) indeed receives the same address as the beginning of the .text section. If, for some reason, it's not the case, there's something wrong either with the linker script or the compiler/linker command-line options or with the loader.

Linux CPU and memory percentages

Put the following snippet somewhere in a script:

#!/bin/bash

CPU=$(lscpu | grep '\(CPU\|max\) MHz:' | xargs echo | awk '{printf "%3.0f\n", $3*100/$7}')
MEM=$(free | grep Mem | awk '{printf "%3.0f\n", $3*100/$2}')
echo CPU $CPU% \| MEM $MEM%

And call it from genmon as bash /path/to/this/script.sh.

Behavior of the Kernel under high memory load

Sorry to answer my own question, but I like to have a record of solutions. The article linked by sawdust contains what I needed.

• The kernal activates the out-of-memory manager (mm/oom_kill.c in the Linux kernel) when it runs out of allocatable memory.
• The OOM manager uses some heuristics to determine which process should be killed. Total runtime of a process reduces the chance of death, while memory allocated increases it. There are other factors, but they don't matter for me.
• After picking a process, the OOM sends it SIGTERM.

There are two reasons for why, in my case, all processes except the one that's hogging the memory get killed.

1. My process ignores SIGTERM during the region where it is allocating memory. This might be because the process is actively receiving lots of other signals during this time, and/or because the process is blocking for I/O during much of the remaining time. In anycase, it ignores SIGTERM.

2. The process that is hogging all the memory typically has been running for a long time, accumulating RAM for a few hours. Even though it has about 4 times as much as any other process, it's long runtime (several hundred times longer than others) might cause the OOM manager to pick other processes to terminate first.

Solutions:

1. Running:

   ulimit -v memamount

for a particular user changes the maximum amount of memory the user can allocate with a single process to memamount. This can prevent the OOM manager from activating. Instead, malloc calls will fail, which I can detect.

2: Writing a handler for SIGTERM that gracefully cleans up might help, but only if the OOM is actually sending SIGTERM to the process, and the process is ignoring or failing to receive SIGTERM.

3: Set the memory limit from within your code (C):

//resource limit structure with both hard and soft max set to 2GB.
struct rlimit memmax; memmax.rlim_max=0x7FFFFFFF; memmax.rlim_cur = 0x7FFFFFFF;
setrlimit(RLIMIT_MEMLOCK,&memmax); //set maximum virtual memory space to 2GB. 

How to set CPU load on a Red Hat Linux box?

This is exactly what you need (internet archive link):
https://web.archive.org/web/20120512025754/http://weather.ou.edu/~apw/projects/stress/stress-1.0.4.tar.gz

From the homepage:
"stress is a simple workload generator for POSIX systems. It imposes a configurable amount of CPU, memory, I/O, and disk stress on the system. It is written in C, and is free software licensed under the GPL."

Memory usage of current process in C

You can always just open the 'files' in the /proc system as you would a regular file (using the 'self' symlink so you don't have to look up your own pid):

FILE* status = fopen( "/proc/self/status", "r" );

Of course, you now have to parse the file to pick out the information you need.

system loads in GNU screen's hardstatus line

To keep it short:

The three numbers stand for average cpu load during the last minute, 5 minutes and 15 minutes where 0 means 0% load and 1.0 means 100% load.

Have a look at this excellent explanation of the cpu load.

For the second question I'm not sure if you can express the system load with only one number in the hardstatus line of screen (I don't think so), but I hope my answer helped a little bit ;-) *Jost

---

Related Topics