How to Determine the Memory Footprint (Size) of a Variable

How to determine the memory footprint (size) of a variable?

You Probably need a Memory Profiler. I have gathered information fro SO but I have copied the some important thing which may help you also.

As you probably know, Xdebug dropped the memory profiling support since the 2.* version. Please search for the "removed functions" string here: http://www.xdebug.org/updates.php

Removed functions

Removed support for Memory profiling as that didn't work properly.

Other Profiler Options

php-memory-profiler

https://github.com/arnaud-lb/php-memory-profiler. This is what I've done on my Ubuntu server to enable it:

sudo apt-get install libjudy-dev libjudydebian1
sudo pecl install memprof
echo "extension=memprof.so" > /etc/php5/mods-available/memprof.ini
sudo php5enmod memprof
service apache2 restart

And then in my code:

<?php
memprof_enable();
// do your stuff
memprof_dump_callgrind(fopen("/tmp/callgrind.out", "w"));

Finally open the callgrind.out file with KCachegrind

Using Google gperftools (recommended!)

First of all install the Google gperftools by downloading the latest package here: https://code.google.com/p/gperftools/

Then as always:

sudo apt-get update
sudo apt-get install libunwind-dev -y
./configure
make
make install

Now in your code:

memprof_enable();

// do your magic

memprof_dump_pprof(fopen("/tmp/profile.heap", "w"));

Then open your terminal and launch:

pprof --web /tmp/profile.heap

pprof will create a new window in your existing browser session with something like shown below:

Xhprof + Xhgui (the best in my opinion to profile both cpu and memory)

With Xhprof and Xhgui you can profile the cpu usage as well or just the memory usage if that's your issue at the moment.
It's a very complete solutions, it gives you full control and the logs can be written both on mongo or in the filesystem.

For more details see here.

Blackfire

Blackfire is a PHP profiler by SensioLabs, the Symfony2 guys https://blackfire.io/

If you use puphpet to set up your virtual machine you'll be happy to know it's supported ;-)

Xdebug and tracing memory usage

XDEBUG2 is a extension for PHP. Xdebug allows you to log all function calls, including parameters and return values to a file in different formats.There are three output formats. One is meant as a human readable trace, another one is more suited for computer programs as it is easier to parse, and the last one uses HTML for formatting the trace. You can switch between the two different formats with the setting. An example would be available here

forp

forp simple, non intrusive, production-oriented, PHP profiler. Some of features are:

• measurement of time and allocated memory for each function

• CPU usage

• file and line number of the function call

• output as Google's Trace Event format

• caption of functions

• grouping of functions

• aliases of functions (useful for anonymous functions)

DBG

DBG is a a full-featured php debugger, an interactive tool that helps you debugging php scripts. It works on a production and/or development WEB server and allows you debug your scripts locally or remotely, from an IDE or console and its features are:

• Remote and local debugging

• Explicit and implicit activation

• Call stack, including function calls, dynamic and static method calls, with their parameters

• Navigation through the call stack with ability to evaluate variables in corresponding (nested) places

• Step in/Step out/Step over/Run to cursor functionality

• Conditional breakpoints

• Global breakpoints

• Logging for errors and warnings

• Multiple simultaneous sessions for parallel debugging

• Support for GUI and CLI front-ends

• IPv6 and IPv4 networks supported

• All data transferred by debugger can be optionally protected with SSL

Variable's memory size in Python

Use sys.getsizeof to get the size of an object, in bytes.

>>> from sys import getsizeof
>>> a = 42
>>> getsizeof(a)
12
>>> a = 2**1000
>>> getsizeof(a)
146
>>>

Note that the size and layout of an object is purely implementation-specific. CPython, for example, may use totally different internal data structures than IronPython. So the size of an object may vary from implementation to implementation.

Determine memory consumption of each variable/table in kdb process

There are some great summarization routines found the ws namespace in GitHub.

These routines list the names of objects of particular classes (functions, variables, or tables), or provide a type- dependent synopsis of their properties. Names can be specified either explicitly or by referencing a parent namespace (in which case all objects in all namespaces below it are considered). Object size is approximated and does not include attribute overhead.

https://github.com/LeslieGoldsmith/ws

How to get memory size of variable?

You can use the unsafe.Sizeof function for this.
It returns the size in bytes, occupied by the value you pass into it.
Here's a working example:

package main

import "fmt"
import "unsafe"

func main() {
    a := int(123)
    b := int64(123)
    c := "foo"
    d := struct {
        FieldA float32
        FieldB string
    }{0, "bar"}

    fmt.Printf("a: %T, %d\n", a, unsafe.Sizeof(a))
    fmt.Printf("b: %T, %d\n", b, unsafe.Sizeof(b))
    fmt.Printf("c: %T, %d\n", c, unsafe.Sizeof(c))
    fmt.Printf("d: %T, %d\n", d, unsafe.Sizeof(d))
}

Take note that some platforms explicitly disallow the use of unsafe, because it is.. well, unsafe. This used to include AppEngine. Not sure if that is still the case today, but I imagine so.

As @Timur Fayzrakhmanov notes, reflect.TypeOf(variable).Size() will give you the same information. For the reflect package, the same restriction goes as for the unsafe package. I.e.: some platforms may not allow its use.

How can I determine the size of variable in memory WITHOUT creating it?

For matrices of standard numeric types all you need to know is the number of elements in your matrix and the number of bytes in the data type. For your example, your matrix will be of type double by default, which is 8 bytes, so your total matrix size will be:

matrixSize = [500 500 500];
byteSize = prod(matrixSize)*8;

You can figure out the byte size for a given data type from a scalar variable of that type using whos:

temp = uint8(0);           % Sample uint8 variable
varData = whos('temp');    % Get variable data from whos
varBytes = varData.bytes;  % Get number of bytes

varBytes =

     1                     % uint8 takes 1 byte

As mentioned by Sam, container classes like cell arrays and structures make it a little bit more complicated to compute the total byte usage since they require some memory overhead.

How to get memory size of variable in Go?

unsafe.SizeOf() and reflect.Type.Size() only return the size of the passed value without recursively traversing the data structure and adding sizes of pointed values.

The slice is relatively a simple struct: reflect.SliceHeader, and since we know it references a backing array, we can easily compute its size "manually", e.g.:

s := make([]int32, 1000)

fmt.Println("Size of []int32:", unsafe.Sizeof(s))
fmt.Println("Size of [1000]int32:", unsafe.Sizeof([1000]int32{}))
fmt.Println("Real size of s:", unsafe.Sizeof(s)+unsafe.Sizeof([1000]int32{}))

Output (try it on the Go Playground):

Size of []int32: 12
Size of [1000]int32: 4000
Real size of s: 4012

Maps are a lot more complex data structures, I won't go into details, but check out this question+answer: Golang: computing the memory footprint (or byte length) of a map

Calculating size of any variable or structure (recursively)

If you want "real" numbers, you may take advantage of the testing tool of Go, which can also perform memory benchmarking. Pass the -benchmem argument, and inside the benchmark function allocate only whose memory you want to measure:

func BenchmarkSlice100(b *testing.B) {
    for i := 0; i < b.N; i++ { getSlice(100) }
}
func BenchmarkSlice1000(b *testing.B) {
    for i := 0; i < b.N; i++ { getSlice(1000) }
}
func BenchmarkSlice10000(b *testing.B) {
    for i := 0; i < b.N; i++ { getSlice(10000) }
}
func BenchmarkMap100(b *testing.B) {
    for i := 0; i < b.N; i++ { getMap(100) }
}
func BenchmarkMap1000(b *testing.B) {
    for i := 0; i < b.N; i++ { getMap(1000) }
}
func BenchmarkMap10000(b *testing.B) {
    for i := 0; i < b.N; i++ { getMap(10000) }
}

(Remove the timing and printing calls from getSlice() and getMap() of course.)

Running with

go test -bench . -benchmem

Output is:

BenchmarkSlice100-4    3000000        471 ns/op        1792 B/op      1 allocs/op
BenchmarkSlice1000-4    300000       3944 ns/op       16384 B/op      1 allocs/op
BenchmarkSlice10000-4    50000      39293 ns/op      163840 B/op      1 allocs/op
BenchmarkMap100-4       200000      11651 ns/op        2843 B/op      9 allocs/op
BenchmarkMap1000-4       10000     111040 ns/op       41823 B/op     12 allocs/op
BenchmarkMap10000-4       1000    1152011 ns/op      315450 B/op    135 allocs/op

B/op values tell you how many bytes were allocated per op. allocs/op tells how many (distinct) memory allocations occurred per op.

On my 64-bit architecture (where the size of int is 8 bytes) it tells that the size of a slice having 2000 elements is roughly 16 KB (in line with 2000 * 8 bytes). A map with 1000 int-int pairs required approximately to allocate 42 KB.

What is the proper way to know how much memory is occupied by a variable

In your code,

 cout << sizeof(p) << endl;

gives you the size of the variable p, which is of type int *.

This is not the same as

cout << sizeof(*p) << endl;

or

cout << sizeof(int) << endl;

which will give you the size occupied by an int variable.

Having said that, just to clarify, to get to know the size occupied by a variable, you need to use the sizeof operator on that variable, not to a pointer-to-that variable. ( What you learnt in first year is correct ).

Note, size of a pointer is dependent on the architecture, so it can vary. In some architecture, size of a pointer can be 32 bits (sizeof will return 4), in some others it can be 64 bits (sizeof will return 8).

python- how to display size of all variables

You can iterate over both the key and value of a dictionary using .items()

from __future__ import print_function  # for Python2
import sys

local_vars = list(locals().items())
for var, obj in local_vars:
    print(var, sys.getsizeof(obj))

---

Related Topics