Limit the Memory and CPU Available for a User in Linux

Limit the memory and cpu available for a user in Linux

For process related limits, you can have a look in /etc/security/limits.conf (read the comments in the file, use google or use man limits.conf for more information). And as jpalecek points out, you may use ulimit -a to see (and possibly modify) all such limits currently in effect.

You can use the command quota to see if a disk quota is in effect.

Limit memory usage for a single Linux process

There's some problems with ulimit. Here's a useful read on the topic: Limiting time and memory consumption of a program in Linux, which lead to the timeout tool, which lets you cage a process (and its forks) by time or memory consumption.

The timeout tool requires Perl 5+ and the /proc filesystem mounted. After that you copy the tool to e.g. /usr/local/bin like so:

curl https://raw.githubusercontent.com/pshved/timeout/master/timeout | \
  sudo tee /usr/local/bin/timeout && sudo chmod 755 /usr/local/bin/timeout

After that, you can 'cage' your process by memory consumption as in your question like so:

timeout -m 500 pdftoppm Sample.pdf

Alternatively you could use -t <seconds> and -x <hertz> to respectively limit the process by time or CPU constraints.

The way this tool works is by checking multiple times per second if the spawned process has not oversubscribed its set boundaries. This means there actually is a small window where a process could potentially be oversubscribing before timeout notices and kills the process.

A more correct approach would hence likely involve cgroups, but that is much more involved to set up, even if you'd use Docker or runC, which among things, offer a more user-friendly abstraction around cgroups.

How to limit program runtime, memory usage and as a specific user programmatically in Linux

You'll want to use the setrlimit syscall to limit memory (Process::RLIMIT_AS). To limit the runtime of the program, you can only control the total number of seconds a process gets CPU time (so that doesn't account for time spent sleeping or waiting on I/O). That's done with Process::CPU.

Drop privileges with Process::Sys.setgid followed by Process::Sys.setuid after setting these rlimits, but before calling your target process with Process::exec.

Example target program:

#include <stdio.h>
#include <stdlib.h>

#include <unistd.h>

#define ALLOC_SIZE_1  1024
#define ALLOC_SIZE_2  (1024 * 1024 * 5)

int
main(int argc, char *argv[])
{
  char *buf;

  fprintf(stderr, "[+] uid: %d, gid: %d\n", getuid(), getgid());

  fprintf(stderr, "[+] trying to allocate %d bytes (should succeed)...\n", ALLOC_SIZE_1);
  if (NULL == (buf = malloc(ALLOC_SIZE_1))) {
    fprintf(stderr, "[!] failed!\n");
    return -1;
  }

  fprintf(stderr, "[+] success.\n");
  free(buf);

  fprintf(stderr, "[+] trying to allocate %d bytes (should fail)...\n", ALLOC_SIZE_2);
  if (NULL != (buf = malloc(ALLOC_SIZE_2))) {
    fprintf(stderr, "[!] succeeded!  (should have failed.)\n");
    return -1;
  }

  fprintf(stderr, "[+] ok.  now doing infinite loop (should get killed pretty soon)...\n");
  for (;;);

  return 0;
}

And accompanying Ruby script to invoke it (run this script as root with, e.g. sudo /tmp/foo.rb):

#!/usr/bin/env ruby

TARGET_GID = 99
TARGET_UID = 99

Process::setrlimit(Process::RLIMIT_AS, 1024 * 1024 * 5)
Process::setrlimit(Process::RLIMIT_CPU, 3)

Process::Sys.setgid(TARGET_GID)
Process::Sys.setuid(TARGET_UID)

Process::exec('/tmp/test')

And finally, output of running on my machine:

$ sudo ./run.rb
[+] uid: 99, gid: 99
[+] trying to allocate 1024 bytes (should succeed)...
[+] success.
[+] trying to allocate 5242880 bytes (should fail)...
[+] ok.  now doing infinite loop (should get killed pretty soon)...
$

Limit available memory to a program

cgroups is the feature or modern linux kernels which allow you to limit resources like memory for group of processes (or for single process with threads). More about cgroups: https://en.wikipedia.org/wiki/Cgroups https://man7.org/linux/man-pages/man7/cgroups.7.html

The cgroups feature should be already enabled in your ubuntu 18.04 kernels. There are some descriptions how to use cgroups to limit memory:

• for systemd managed service: https://www.paranoids.at/cgroup-ubuntu-18-04-howto/,
• for docker containers docker run --memory=1G ….
• for custom process tree with cgcreate/cgset/cgexec commands: https://dev.to/vga/how-to-see-and-limit-memory-consumption-of-an-application-5bfl something like

# Create a group for memory named “limited_group_1”
cgcreate -g "memory:limited_group_1" -t USERNAME:GROUPNAME

# Specify memory limit to 1G for this group
cgset -r memory.limit_in_bytes=1G "limited_group_1"

# Launch the application in this group
cgexec -g "memory:limited_group_1" ./YOUR_APPLICATION

# If needed, we can remove the group
cgdelete "memory:limited_group_1"

https://unix.stackexchange.com/questions/44985/limit-memory-usage-for-a-single-linux-process/279175#279175 was also mentioned in https://dev.to/vga/how-to-see-and-limit-memory-consumption-of-an-application-5bfl

PS: Default memory allocators in older glibc versions (malloc, new) has awful behavior for freed regions: they are not returned back without periodic malloc_trim() library calls. You should try to link your application with libjemalloc or libtcmalloc which will replace malloc implementation of glibc with some code better in memory returning.

Limiting process memory/CPU usage on linux

You might want to investigate cgroups as well as the older (obsolete) ulimit.

Setting limit to total physical memory available in Linux

I found the answer I was looking for. Basically, the parameter that sets the total available physical memory is "mem=MEMORY_LIMIT". And this is a kernel boot parameter. You need to add, say "mem=1G" for maximum of 1GB available physical memory to the kernel boot parameter. For more info on how to add kernel boot parameters look at https://wiki.ubuntu.com/Kernel/KernelBootParameters

Memory Limit and CPU Limit in Docker Container

UPDATE

This is the answer that works: https://stackoverflow.com/a/41827812/3209177

---

I have just found this answer:

Limit memory on a docker container doesn't work

I am going to try that and come back with my observation.

---

Related Topics