What Does "Private_Dirty" Memory Mean in Smaps

What does Private_Dirty memory mean in smaps?

Memory is either private, meaning it is exclusive to this process, or shared, meaning multiple processes may have it mapped and in use (think shared library code, etc.). Memory can also be clean - it hasn't been modified since it was loaded from disk or provided as zero-filled pages or whatever, and so if it needs to be freed to provide memory pages for other processes, it can just be discarded, and reloaded/re-filled if it is ever needed again - or dirty, which means if it needs to be freed up, it must be written out to a swap area so that the modified contents can be recovered when necessary.

It's not necessarily unusual to see large amounts of private dirty data in a process. The problem is when the sum of all private dirty data across all processes in the system becomes a significant portion (exact numbers depend greatly on your workload and acceptable performance) of your overall physical memory, and stuff has to start being swapped in/out...

Shared_dirty vs Private_dirty in shared memory

The distinction between Clean and Dirty refers to whether or not the pages have been written-out to the backing store since being written to in memory. For a mapping of /dev/zero, pages are obviously never written-out, so Clean pages have only been read whereas Dirty pages have been written to.

For a shared mapping, the distinction between Private and Shared is whether the pages have only been referenced by the process you're examining, or whether they've been referenced by multiple processes.

So in summary:

• Shared_Clean are the pages in the mapping that have been referenced by this process and at least one other process, but not written by any process;
• Shared_Dirty are the pages in the mapping that have been referenced by this process and at least one other process, and written by at least one of those processes;
• Private_Clean are the pages in the mapping that have been read and not written by this process but not referenced by any other process;
• Private_Dirty are the pages in the mapping that have been written by this process but not referenced by any other process.

Pages can move from Clean to Dirty when they're written to, and from Private to Shared when another process references them.

If you map a real disk file, then pages can also move from Dirty to Clean when they're written-out by the kernel.

What is dirty private memory?

Talking about the memory non-decrease even after freeing some chunks, you'd better use mmap in anonymous mode like this:

mmap(NULL, chunck_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

which maps no file descriptor and returns a pointer to a memory chunk which is immediately returned to the OS when you unmap it. However, mmap requires a system call, which is slower than malloc. So, you should use mmap for allocating large pages.

What does pss mean in /proc/pid/smaps

Quoting from lwn.net

The "proportional set size" (PSS) of a process is the count of pages
it has in memory, where each page is divided by the number of
processes sharing it. So if a process has 1000 pages all to itself,
and 1000 shared with one other process, its PSS will be 1500

From Linux Kernel Documentation,

The /proc/PID/smaps is an extension based on maps, showing the memory
consumption for each of the process's mappings. For each of mappings there
is a series of lines such as the following:

08048000-080bc000 r-xp 00000000 03:02 13130      /bin/bash
Size:               1084 kB
Rss:                 892 kB
Pss:                 374 kB
Shared_Clean:        892 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
Referenced:          892 kB
Anonymous:             0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:              374 kB

The first of these lines shows the same information as is displayed
for the mapping in /proc/PID/maps. The remaining lines show the size
of the mapping (size), the amount of the mapping that is currently
resident in RAM (RSS), the process' proportional share of this mapping
(PSS), the number of clean and dirty private pages in the mapping.
Note that even a page which is part of a MAP_SHARED mapping, but has
only a single pte mapped, i.e. is currently used by only one process,
is accounted as private and not as shared. "Referenced" indicates the
amount of memory currently marked as referenced or accessed.
"Anonymous" shows the amount of memory that does not belong to any
file. Even a mapping associated with a file may contain anonymous
pages: when MAP_PRIVATE and a page is modified, the file page is
replaced by a private anonymous copy. "Swap" shows how much
would-be-anonymous memory is also used, but out on swap.

This Question on Unix and Linux Stackexchange covers almost the topic. See Mat's excellent answer which will surely clear all your doubts.

What [vectors] mean in smaps?

[vectors] indicates a page used by the VDSO mechanism. VDSO is a way of accelerating common system calls by eliminating the overhead of context switching.
Basically the kernel just shares a bit of its memory with the result of common system calls (think gettimeofday() and the like) where your user space process can read it.

You should not count it as used memory, as the same memory will be shared by all processes.

Want to make /proc/*/smaps output anonymous region sum and /proc/meminfo AnonPages to match to trace actual memory usage precisely

I've found this diff which was done 2010 October, which just answer my question.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b40d4f84becd69275451baee7f0801c85eb58437

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/Documentation/filesystems/proc.txt?id=b40d4f84becd69275451baee7f0801c85eb58437&id2=d16e15f5b029fc7d03540ba0e5fb23b0abb0ebe0

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/fs/proc/task_mmu.c?id=b40d4f84becd69275451baee7f0801c85eb58437&id2=d16e15f5b029fc7d03540ba0e5fb23b0abb0ebe0

---

Related Topics