How to Determine The Available Physical Memory in Linux

How can I find out the total physical memory (RAM) of my linux box suitable to be parsed by a shell script?

If you're interested in the physical RAM, use the command dmidecode. It gives you a lot more information than just that, but depending on your use case, you might also want to know if the 8G in the system come from 2x4GB sticks or 4x2GB sticks.

How can the physical RAM size be determined in Linux programatically?

#include <unistd.h>

long long physical_mem_bytes = (long long) sysconf (_SC_PHYS_PAGES) * sysconf (_SC_PAGESIZE);

Other than the command line ulimit, I don't know of a way of finding maximum memory for an individual process.

Find exact physical memory usage in Ubuntu/Linux

TL;DR - Virtual memory is complicated.

The best measure of a Linux processes current usage of physical memory is RES.

The RES value represents the sum of all of the processes pages that are currently resident in physical memory. It includes resident code pages and resident data pages. It also includes shared pages (SHR) that are currently RAM resident, though these pages cannot be exclusively ascribed to >>this<< process.

The VIRT value is actually the sum of all notionally allocated pages for the process, and it includes pages that are currently RAM resident, pages that are currently swapped to disk.

See https://stackoverflow.com/a/56351211/1184752 for another explanation.

---

Note that RES is giving you (roughly) instantaneous RAM usage. That is what you asked about ...

The "actual" memory usage over time is more complicated because the OS's virtual memory subsystem is typically be swapping pages in and out according to demand. So, for example, some of your application's pages may not have been accesses recently, and the OS may then swap them out (to swap space) to free up RAM for other pages required by your application ... or something else.

The VIRT value while actually representing virtual address space, is a good approximation of total (virtual) memory usage. However, it may be an over-estimate:

• Some pages in a processes address space are shared between multiple processes. This includes read-only code segments, pages shared between parent and child processes between vfork and exec, and shared memory segments created using mmap.

• Some pages may be set to have illegal access (e.g. for stack red-zones) and may not be backed by either RAM or swap device pages.

• Some pages of the address space in certain states may not have been committed to either RAM or disk yet ... depending on how the virtual memory system is implemented. (Consider the case where a process requests a huge memory segment and neither reads from it or writes to it. It is possible that the virtual memory implementation will not allocate RAM pages until the first read or write in the page. And if you use lazy swap reservation, swap pages not be committed either. But beware that you can get into trouble with lazy swap reservation.)

VIRT can also be under-estimate because the OS usually reserves swap space for all pages ... whether they are currently swapped in or swapped out. So if you count the RAM and swap versions of a given page as separate units of storage, VIRT usually underestimates the total storage used.

---

Finally, if your real goal is to limit your application to using at most
700 MB (of virtual address space) then you can use ulimit -v ... to do this. If the application tries to request memory beyond its limit, the request fails.

how to get physical memory including reserved memory from linux kernel?

physpages = get_num_physpages();

pr_info("Memory: %luK/%luK available (%luK kernel code, %luK rwdata, %luK rodata, %luK init, %luK bss, %luK reserved, %luK cma-reserved"
#ifdef  CONFIG_HIGHMEM
        ", %luK highmem"
#endif
        "%s%s)\n",
        nr_free_pages() << (PAGE_SHIFT - 10),
        physpages << (PAGE_SHIFT - 10),
        codesize >> 10, datasize >> 10, rosize >> 10,
        (init_data_size + init_code_size) >> 10, bss_size >> 10,
        (physpages - totalram_pages() - totalcma_pages) << (PAGE_SHIFT - 10),
        totalcma_pages << (PAGE_SHIFT - 10),
#ifdef  CONFIG_HIGHMEM
        totalhigh_pages() << (PAGE_SHIFT - 10),
#endif
        str ? ", " : "", str ? str : "");

So get_num_physpages << (PAGE_SHIFT - 10) will get the whole physical memory.

How to access physical addresses from user space in Linux?

You can map a device file to a user process memory using mmap(2) system call. Usually, device files are mappings of physical memory to the file system.
Otherwise, you have to write a kernel module which creates such a file or provides a way to map the needed memory to a user process.

Another way is remapping parts of /dev/mem to a user memory.

Edit:
Example of mmaping /dev/mem (this program must have access to /dev/mem, e.g. have root rights):

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>

int main(int argc, char *argv[]) {
    if (argc < 3) {
        printf("Usage: %s <phys_addr> <offset>\n", argv[0]);
        return 0;
    }

    off_t offset = strtoul(argv[1], NULL, 0);
    size_t len = strtoul(argv[2], NULL, 0);

    // Truncate offset to a multiple of the page size, or mmap will fail.
    size_t pagesize = sysconf(_SC_PAGE_SIZE);
    off_t page_base = (offset / pagesize) * pagesize;
    off_t page_offset = offset - page_base;

    int fd = open("/dev/mem", O_SYNC);
    unsigned char *mem = mmap(NULL, page_offset + len, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, page_base);
    if (mem == MAP_FAILED) {
        perror("Can't map memory");
        return -1;
    }

    size_t i;
    for (i = 0; i < len; ++i)
        printf("%02x ", (int)mem[page_offset + i]);

    return 0;
}

Get total physical memory in Python

your best bet for a cross-platform solution is to use the psutil package (available on PyPI).

import psutil
    
psutil.virtual_memory().total  # total physical memory in Bytes

Documentation for virtual_memory is here.

How can I measure the actual memory usage of an application or process?

With ps or similar tools you will only get the amount of memory pages allocated by that process. This number is correct, but:

• does not reflect the actual amount of memory used by the application, only the amount of memory reserved for it

• can be misleading if pages are shared, for example by several threads or by using dynamically linked libraries

If you really want to know what amount of memory your application actually uses, you need to run it within a profiler. For example, Valgrind can give you insights about the amount of memory used, and, more importantly, about possible memory leaks in your program. The heap profiler tool of Valgrind is called 'massif':

Massif is a heap profiler. It performs detailed heap profiling by taking regular snapshots of a program's heap. It produces a graph showing heap usage over time, including information about which parts of the program are responsible for the most memory allocations. The graph is supplemented by a text or HTML file that includes more information for determining where the most memory is being allocated. Massif runs programs about 20x slower than normal.

As explained in the Valgrind documentation, you need to run the program through Valgrind:

valgrind --tool=massif <executable> <arguments>

Massif writes a dump of memory usage snapshots (e.g. massif.out.12345). These provide, (1) a timeline of memory usage, (2) for each snapshot, a record of where in your program memory was allocated. A great graphical tool for analyzing these files is massif-visualizer. But I found ms_print, a simple text-based tool shipped with Valgrind, to be of great help already.

To find memory leaks, use the (default) memcheck tool of valgrind.

---

Related Topics