On Linux: We See Following: Physical, Real, Swap, Virtual Memory - Which Should We Consider for Sizing

On Linux: We see following: Physical, Real, Swap, Virtual Memory - Which should we consider for sizing?

Real and Physical

Physical memory is the amount of DRAM which is currently used. Real memory shows how much your applications are using system DRAM memory. It is roughly lower than physical memory. Linux system caches some of disk data. This caching is the difference between physical and real memory. Actually, when you have free memory Linux goes to use it for caching. Do not worry, as your applications demand memory they gonna get the cached space back.

Swap and Virtual

Swap is additional space to your actual DRAM. This space is borrowed from disk space and once you application fill-out entire DRAM, Linux transfers some unused memory to swap to let all application stay alive. Total of swap and physical memory is the virtual memory.

Do you need extra memory?

In answer to your question, you need to check real memory. If your real memory is full, you need to get some RAM. Use free command to check the amount of actual free memory. For example on my system free says:

$ free
                 total       used       free     shared    buffers     cached
    Mem:      16324640    9314120    7010520          0     433096    8066048
    -/+ buffers/cache:     814976   15509664
    Swap:      2047992          0    2047992

You need to check buffer/cache section. As shown above, there are real 15 GB free DRAM (second line) on my system. Check this on your system and find out whether you need more memory or not. The lines represent physical, real, and swap memory, respectively.

Linux `top` command: how much process memory is physically stored in swap space?

The behavior depends on a version of procps you are using. For instance, in version 3.0.5 SWAP value equals:

task->size - task->resident

and it is exactly what you are encountering. Man top.1 says:

VIRT = SWAP + RES

Procps-ng, however, reads /proc/pid/status and sets SWAP correctly

https://gitlab.com/procps-ng/procps/blob/master/proc/readproc.c#L383

So, you can update procps or look at /proc/pid/status directly

How can virtual memory exist when there is no swap space?

The virtual memory these programs (htop and the like) are counting is just the size of the address space the processes have requested. You have physical memory, actual RAM, and a virtual address space which maps addresses as user space programs see them to physical memory. They are separate. A 0x0ff84560 pointer probably doesn't actually reference that part of RAM. The OS gets to set up a mapping that decides where in RAM you're actually referencing. Even further, it can set up the mapping before it has RAM to back it up. It's an event driven process. The OS will set up the mapping on request with no real backing, no physical memory allocated, and only actually map it to real RAM when you try to use the virtual memory.

The size of virtual memory is the size of this mapping. But not all of it has to be backed by physical RAM, so it can be larger than RAM even when there's no swap. But this causes problems programs try to actually use more memory than there is RAM. It's no problem at all if they only request it, only if they use it.

Additionally, as Thilo mentioned, memory mapped files can add to this. You can map an entire 100TB file into your virtual address space no problem. The OS handles the logistics in the background: bringing in the parts you need (the parts you try to access) and reaping the parts it has to to clear physical memory.

Why memory usage is more than physical RAM in Linux?

VSZ is virtual memory size which is used by the process. It's normal that it's higher than the size of your physical memory because this is one of the main ideas of this. You should rather look at Resident size (RSS) which is the actual physical memory used by the process.

Look at this example:

I have an nginx process running:

 ps -o rss,vsz,cmd ax | grep -i nginx | head -n1
  956  31248 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf

rss - 956 kB
vsz - 31248 kB

So, it means this process is using 956kB of physical memory, and 31MB of virtual memory.

Disabling swap (swapoff -a), like you did, doesn't disable using virtual memory.

Read about virtual memory here:
Virtual memory

Calculating % memory used on Linux

Yes, this is essentially correct. The actual numbers might be (very) marginally lower, but for all intents and purposes, if you have x physical memory and y virtual memory (swap in linux), then you have x + y memory available to the operating system and any programs running underneath the OS.

What's the difference between virtual memory and swap space?

There's an excellent explantation of virtual memory over on superuser.

Simply put, virtual memory is a combination of RAM and disk space that running processes can use.

Swap space is the portion of virtual memory that is on the hard disk, used when RAM is full.

As for why 32bit CPU is limited to 4gb virtual memory, it's addressed well here:

By definition, a 32-bit processor uses
32 bits to refer to the location of
each byte of memory. 2^32 = 4.2
billion, which means a memory address
that's 32 bits long can only refer to
4.2 billion unique locations (i.e. 4 GB).

How can I measure the actual memory usage of an application or process?

With ps or similar tools you will only get the amount of memory pages allocated by that process. This number is correct, but:

• does not reflect the actual amount of memory used by the application, only the amount of memory reserved for it

• can be misleading if pages are shared, for example by several threads or by using dynamically linked libraries

If you really want to know what amount of memory your application actually uses, you need to run it within a profiler. For example, Valgrind can give you insights about the amount of memory used, and, more importantly, about possible memory leaks in your program. The heap profiler tool of Valgrind is called 'massif':

Massif is a heap profiler. It performs detailed heap profiling by taking regular snapshots of a program's heap. It produces a graph showing heap usage over time, including information about which parts of the program are responsible for the most memory allocations. The graph is supplemented by a text or HTML file that includes more information for determining where the most memory is being allocated. Massif runs programs about 20x slower than normal.

As explained in the Valgrind documentation, you need to run the program through Valgrind:

valgrind --tool=massif <executable> <arguments>

Massif writes a dump of memory usage snapshots (e.g. massif.out.12345). These provide, (1) a timeline of memory usage, (2) for each snapshot, a record of where in your program memory was allocated. A great graphical tool for analyzing these files is massif-visualizer. But I found ms_print, a simple text-based tool shipped with Valgrind, to be of great help already.

To find memory leaks, use the (default) memcheck tool of valgrind.

---

Related Topics