Ruby Memory Management

Memory Management in Ruby

You are correct, it's not GC that changed the physical memory requirements, it's the OS kernel.

You need to look at the VIRT column, not the RES column. As you can see VIRT stays exactly the same.

RES is physical (resident) memory, VIRT is virtual (allocated, but currently unused) memory.

When the process sleeps it's not using its memory or doing anything, so the OS memory manager decides to swap out part of the physical memory and move it into virtual space.

Why keep an idle process hogging physical memory for no reason? So the OS is smart, and swaps out as much unused physical memory as possible, that's why you see a reduction in RES.

I suspect you would see the same effect even without array = nil, by just sleeping long enough. Once you stop sleeping and access something in the array, then RES will jump back up again.

You can read some more discussion through these:

What is RSS and VSZ in Linux memory management

http://www.darkcoding.net/software/resident-and-virtual-memory-on-linux-a-short-example/

What's the difference between "virtual memory" and "swap space"?

http://www.tldp.org/LDP/tlk/mm/memory.html

Ruby Memory Management

Don't do this:

def method(x)
  x.split( doesn't matter what the args are )
end

or this:

def method(x)
  x.gsub( doesn't matter what the args are )
end

Both will permanently leak memory in ruby 1.8.5 and 1.8.6. (not sure about 1.8.7 as I haven't tried it, but I really hope it's fixed.) The workaround is stupid and involves creating a local variable. You don't have to use the local, just create one...

Things like this are why I have lots of love for the ruby language, but no respect for MRI

Object memory allocation

The answer depends on the actual implementation. Here I assume you are asking about MRI.

Ruby objects are allocated on the heap. There is no concept of the stack when talking about object allocations.

The heap is split up into pages, each consisting of 16kb. Each page is carved up into fixed size slots which can hold Ruby objects. A page can hold ~408 objects, since each object (which is an RVALUE struct) occupies 40bytes.

All of this is managed by the VM (ie. YARV).

source: http://timetobleed.com/garbage-collection-slides-from-la-ruby-conference/

Regarding your example, variables just hold references to objects, so m actually points to an allocated MyClass object.

The C struct (RClass) that backs up MyClass internally, contains a pointer to a table with the user-defined methods like #mySecondMethod and a pointer to a table with the names of the instance variables that its objects have.

Each object (which is backed up by RObject since the Object class is the default root of all objects) internally contains a pointer to the values of its instance variables.

The newly defined #mySecondMethod is available because of the dynamic nature of the language and the fact that method lookup happens at runtime.

What are your strategies to keep the memory usage low?

1. Choose date structures that are efficient representations, scale well, and do what you need.
2. Use algorithms that work using efficient data structures rather than bloated, but easier ones.
3. Look else where. Ruby has a C bridge and its much easier to be memory conscious in C than in Ruby.

Ruby Memory Usage gone wild

I believe it's all about the http client you are using: Rest-Client. Unfortunately it has some bad reputation of being memory-hungry. You should definitely look for some awesome gem that is both memory/time efficient.

I would highly recommend HTTP.rb or its http/2 successor HTTPX

For a good benchmark, please have a look at this awesome article by the author of another awesome gem Shrine: https://twin.github.io/httprb-is-great/

Here is what I found after replacing Rest-Client with HTTP.rb on my local machine:

Versions: Ruby: 2.5.3p105, HTTP.rb: 4.0.0, OS: Ubuntu 16.04

Total download size: 96.92Mb through 118 unique requests.

Memory consumption:

Total allocated: 7107283 bytes (83437 objects)
Total retained:  44221 bytes (385 objects)

So it allocated only 7Mb while downloading 96.92Mb compared to roughly 1Gb using Rest-Client.

Here is the snippet: https://gist.github.com/mtrolle/96f55822122ecabd3cc46190a6dc18a5#gistcomment-2774405

How much memory is consumed if I create a ruby object?

You could use ruby-prof, a wonderful ruby profiler that will tell you everything your code is doing, including memory allocation. The usage is really simple:

require 'ruby-prof'

# Profile the code
result = RubyProf.profile do
  ...
  [code to profile]
  ...
end

# Print a flat profile to text
printer = RubyProf::FlatPrinter.new(result)
printer.print(STDOUT)

It can output results as text, text graph, html graph, call stack and more. In the readme there is also a section about profiling rails applications. The installation is immediate, so give it a try:

gem install ruby-prof

---

Related Topics