What Do the Fields of Ruby's Gc.Stat Mean

How to monitor garbage collection when running Cucumber?

The easiest way to watch Ruby's GC work is to use GC::Profiler. You can use it with cucumber by putting something like this in features/support/gc.rb:

Before do
  GC::Profiler.enable
  GC.start
end

After do
  puts GC::Profiler.report
end

If you want more detailed information, assuming you're using MRI (C Ruby), you can get garbage collection statistics from GC.stat. You can print information from GC.stat after every Cucumber scenario by putting something like this in features/support/gc.rb:

After do
  puts "GC: total #{GC.stat[:count]}, major #{GC.stat[:minor_gc_count]}, minor #{GC.stat[:minor_gc_count]}"
end

Different versions of Ruby have different garbage collection, so GC.stat returns different information. The GC.stat rdoc doesn't document what each hash key means. A blog post by Sam Saffron documents GC.stat's hash keys for Ruby 2.0 (and says that this SO answer is inaccurate). A blog post by Thorsten Ball documents GC.stat's hash keys for Ruby 2.1.

Could someone help me understand VkPhysicalDeviceMemoryProperties?

Vulkan recognizes two distinct concepts when it comes to memory. There are the actual physical pieces of RAM that the device can talk to. Then there are ways to allocate memory from one of those pools of RAM.

A heap represents a specific piece of RAM. VkMemoryHeap is the object that describes one of the available heaps of RAM that the device can talk to. There really aren't that many things that define a particular heap. Just two: the number of bytes of that RAMs storage and the storage's location relative to the Vulkan device (local vs. non-local).

A memory type is a particular means of allocating memory from a specific heap. VkMemoryType is the object that describes a particular way of allocating memory. And there are a lot more descriptive flags for how you can allocate memory from a heap.

For a more concrete example, consider a standard PC setup with a discrete GPU. The device has its own local RAM, but the discrete GPU can also access CPU memory. So a Vulkan device will have two heaps: one of them will be local, the other non-local.

However, there will usually be more than two memory types. You usually have one memory type that represents local memory, which does not have the VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT set. That means you can't map the memory; you can only access it via transfer operations from some other memory type (or from rendering operations or whatever).

But you will often have two memory types that both use the same non-local heap. They will both be VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, thus allowing mapping. However, one of them will likely have the VK_MEMORY_PROPERTY_HOST_CACHED_BIT flag set, while the other will be VK_MEMORY_PROPERTY_HOST_COHERENT_BIT. This allows you to choose whether you want cached CPU access (thus requiring an explicit flush of ranges of modified memory) or uncached CPU access.

But while they are two separate memory types, they both allocate from the same heap. Which is why VkMemoryType has an index that refers to the heap who's memory it is allocating from.

---

Only thing I'm not getting is how the two DEVICE_LOCAL flags interact.

Did you look at the specification? It's not exactly hiding how this works:

if propertyFlags has the VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT bit set, memory allocated with this type is the most efficient for device access. This property will only be set for memory types belonging to heaps with the VK_MEMORY_HEAP_DEVICE_LOCAL_BIT set.

Is it saying that if the memory is local then all types corresponding to that memory are local, or that they can be local?

You seem to be trying to impose the wrong meaning to these things. Just look at what the specification says and take it at face value.

PROPERTY_DEVICE_LOCAL denotes a memory type which will achieve the best device access performance. The only connection between this and MEMORY_DEVICE_LOCAL is that memory types with PROPERTY_DEVICE_LOCAL will only be associated with memory heaps that use MEMORY_DEVICE_LOCAL.

That's the only relevant meaning here.

If you want an example of when a memory heap would be device local, yet have memory types that aren't, consider a GPU that has no memory of its own. There's only one heap, which is therefore MEMORY_DEVICE_LOCAL.

However, allocating memory from that pool in a way that makes it host-visible may decrease the performance of device access to that memory. Therefore, for such hardware, the host-visible memory types for the same heap will not use PROPERTY_DEVICE_LOCAL.

Then again, other hardware doesn't lose performance from making memory host-visible. So they only have one memory type, which has all of the available properties. For Intel, their on-chip GPUs apparently have access to some level of the CPU's caches.

What Pastie alternatives are there? (installable)

There is for instance LodgeIt by the Pocoo guys.

Ruby: Destructors?

You can use ObjectSpace.define_finalizer when you create the image file, and it will get invoked when the garbage man comes to collect. Just be careful not to reference the object itself in your proc, otherwise it won't be collected by the garbage man. (Won't pick up something that's alive and kicking)

class MyObject
  def generate_image
    image = ImageMagick.do_some_magick
    ObjectSpace.define_finalizer(self, proc { image.self_destruct! })
  end
end

GOGCTRACE and collection time

The trace does not show an absolute time but one relative to the output.
When the line occurs the GC happened 0 + 1 + 0 ms ago.

See the corresponding output line in the code for reference.

If you prefix each line on stdout with a timestamp you then get the time of the GC run
(timestamp - time in milliseconds = time the GC ran).

Example:

GOGCTRACE=1 ./myprog 2>&1 | while read line; do echo $(date +%s) $line; done

If you just want to have a feeling when a GC run happened with accuracy of seconds
then this is totally sufficient. The output will most likely not lag behind much and
even if it does, you'd still have the timestamps to get the correct time.

Another solution would be to use ReadGCStats from runtime/debug which gives you
the GCStats struct, which in turn has a
field LastGC of type time.Time, holding the absolute time of the last GC run.
Alternatively you can use runtime.ReadMemStats, which gives you even more information.

Of course you would still need to know when a garbage collection happened. For this you could
use a finalizer on a object which you create solely for the purpose of being garbage
collected and apply runtime.SetFinalizer on it.
In the finalizer you would then read the LastGC time and print it.

Example (on play):

type Garbage struct{ a int }

func notify(f *Garbage) {
    stats := &runtime.MemStats{}
    runtime.ReadMemStats(stats)

    fmt.Println("Last GC was:", stats.LastGC)

    go ProduceFinalizedGarbage()
}

func ProduceFinalizedGarbage() {
    x := &Garbage{}
    runtime.SetFinalizer(x, notify)
}

func main() {
    go ProduceFinalizedGarbage()

    for {
        runtime.GC()
        time.Sleep(30 * time.Second) // Give GC time to run
    }
}

---

Related Topics