Maximum Resident Set Size Does Not Make Sense

Maximum resident set size does not make sense

I stumbled across this a while ago. It's a bug in GNU time, values are 4 times too large, as it assumes a size in pages and converts it into kB, even though it is kB already in the first place.
You might wanna check:

http://groups.google.com/group/gnu.utils.help/browse_thread/thread/bb530eb072f86e18/83599c4828de175b

http://forums.whirlpool.net.au/archive/1693957

What does maximum resident set size mean?

Resident set size (RSS) means, roughly, the total amount of physical memory assigned to a process at a given point in time. It does not count pages that have been swapped out, or that are mapped from a file but not currently loaded into physical memory.

"Maximum RSS" means the maximum of the RSS since the process's birth, i.e. the largest it has ever been. So this number tells you the largest amount of physical memory your process has ever been using at any one instant.

It can vary from one run to the next if, for instance, the OS decided to swap out different amounts of your program's memory at different times. This decision would depend in part on what the rest of the system is doing, and where else physical memory is needed.

Linux getrusage() maxrss maximum resident set size not increasing with allocation (C++)

Allocated memory isn't actually mapped until you access it.
If you initialize the array with values, Linux is forced to actually allocate and map new pages:

#include <iostream>
#include <sys/time.h>
#include <sys/resource.h>
using namespace std;
int main() {
  struct rusage r_usage;
  getrusage(RUSAGE_SELF, &r_usage);
  cout << r_usage.ru_maxrss << "kb\n";
  cout << "Allocating...\n";
  int a = 1000000;                 // Sufficiently large
  int* memleaktest = new int[a](); // Initialized to zero
  if(!memleaktest)
    cout << "Allocation failed";
  getrusage(RUSAGE_SELF, &r_usage);
  cout << "after allocation " << r_usage.ru_maxrss << "kb\n";
  return 0;
}

On my system, this results in:

4900kb
Allocating...
after allocation 6844kb

Note that compiler optimizations may decide that the array is unused or should be allocated up front, so prefer compiling without them or rewriting the test case in such a way that it can't be optimized.

Does 'maximum resident set size' equal 'required RAM size for the process at least'?

Of sorts, yes. However, the OS may over allocate memory to the process,
or under allocate (and therefore, use swap space). At any rate, it is a good approximation.

See Peak memory usage of a linux/unix process

Unit of maximum resident set size in the output of time

The unit of Maximum Resident Size is bytes.

NodeJS v14 memory issues: Resident Set grows a lot

Within the v8 memory chart, Dynatrace does not only show the heap memory, it also compares it with the resident set size of the whole process. Some may find this confusing, but this helps tremendously to separate JS memory leaks from native memory leaks. In your case there seems to be a potential native memory leak that is caused by native modules.

You mentioned the --max-memory-restart parameter, so I guess you are using PM2. It's official documentation is not clear about whether the tool monitors just the memory used on the heap or overall memory usage. According to this article it seems that PM2 is not able to cover also native memory. Even more interesting are this bug report (closed but according to latest comments still not fixed) and the accepted answer to this question on Stack Overflow. It seems that even your version of PM2 may contain native memory leaks.

Here you can find more information on how to analyze a native memory leak in NodeJS. The accepted answer of the mentioned question from above lists also some links to alterntives for PM2.

By the way, Dynatrace will tell you if the increasing memory usage should have any measurable negtive effect on your services / applications or on end user experience.

node process memory usage, resident set size constantly increasing

In js garbage collector does not run immediately after execution of your code. Thus the memory is not freed immediately after execution. You can run GC independently, after working with large objects, if you care about memory consumption. More information you can find here.

setTimeout(function() {
    global.gc();
    console.log("After sending = ", process.memoryUsage());
}, 5000);

To look at your memory allocation you can run your server with v8-profiler and get a Heap snapshot. More info here.

How to get Resident Set Size (RSS)

You can parse /proc/self/statm file. It contains only integers so it is easy to parse. Second field is RSS measured in pages.

func getRss() (int64, error) {
    buf, err := ioutil.ReadFile("/proc/self/statm")
    if err != nil {
        return 0, err
    }

    fields := strings.Split(string(buf), " ")
    if len(fields) < 2 {
        return 0, errors.New("Cannot parse statm")
    }

    rss, err := strconv.ParseInt(fields[1], 10, 64)
    if err != nil {
        return 0, err
    }

    return rss * int64(os.Getpagesize()), err
}

---

Related Topics