"Sys.Getsizeof(Int)" Returns an Unreasonably Large Value

sys.getsizeof(int) returns an unreasonably large value?

The short answer

You're getting the size of the class, not of an instance of the class. Call int to get the size of an instance:

>>> sys.getsizeof(int())
24

If that size still seems a little bit large, remember that a Python int is very different from an int in (for example) c. In Python, an int is a fully-fledged object. This means there's extra overhead.

Every Python object contains at least a refcount and a reference to the object's type in addition to other storage; on a 64-bit machine, that takes up 16 bytes! The int internals (as determined by the standard CPython implementation) have also changed over time, so that the amount of additional storage taken depends on your version.

Some details about int objects in Python 2 and 3

Here's the situation in Python 2. (Some of this is adapted from a blog post by Laurent Luce). Integer objects are represented as blocks of memory with the following structure:

typedef struct {
    PyObject_HEAD
    long ob_ival;
} PyIntObject;

PyObject_HEAD is a macro defining the storage for the refcount and the object type. It's described in some detail by the documentation, and the code can be seen in this answer.

The memory is allocated in large blocks so that there's not an allocation bottleneck for every new integer. The structure for the block looks like this:

struct _intblock {
    struct _intblock *next;
    PyIntObject objects[N_INTOBJECTS];
};
typedef struct _intblock PyIntBlock;

These are all empty at first. Then, each time a new integer is created, Python uses the memory pointed at by next and increments next to point to the next free integer object in the block.

I'm not entirely sure how this changes once you exceed the storage capacity of an ordinary integer, but once you do so, the size of an int gets larger. On my machine, in Python 2:

>>> sys.getsizeof(0)
24
>>> sys.getsizeof(1)
24
>>> sys.getsizeof(2 ** 62)
24
>>> sys.getsizeof(2 ** 63)
36

In Python 3, I think the general picture is the same, but the size of integers increases in a more piecemeal way:

>>> sys.getsizeof(0)
24
>>> sys.getsizeof(1)
28
>>> sys.getsizeof(2 ** 30 - 1)
28
>>> sys.getsizeof(2 ** 30)
32
>>> sys.getsizeof(2 ** 60 - 1)
32
>>> sys.getsizeof(2 ** 60)
36

These results are, of course, all hardware-dependent! YMMV.

The variability in integer size in Python 3 is a hint that they may behave more like variable-length types (like lists). And indeed, this turns out to be true. Here's the definition of the C struct for int objects in Python 3:

struct _longobject {
    PyObject_VAR_HEAD
    digit ob_digit[1];
};

The comments that accompany this definition summarize Python 3's representation of integers. Zero is represented not by a stored value, but by an object with size zero (which is why sys.getsizeof(0) is 24 bytes while sys.getsizeof(1) is 28). Negative numbers are represented by objects with a negative size attribute! So weird.

sys.getsizeof(int) return 400 bytes

In your code you're getting the size of the class not of an instance of the class.

Call int to get the size of an instance, like the following code

>>> sys.getsizeof(int())
24

sys.getsizeof(int) returns an unreasonably large value?

The short answer

You're getting the size of the class, not of an instance of the class. Call int to get the size of an instance:

>>> sys.getsizeof(int())
24

If that size still seems a little bit large, remember that a Python int is very different from an int in (for example) c. In Python, an int is a fully-fledged object. This means there's extra overhead.

Every Python object contains at least a refcount and a reference to the object's type in addition to other storage; on a 64-bit machine, that takes up 16 bytes! The int internals (as determined by the standard CPython implementation) have also changed over time, so that the amount of additional storage taken depends on your version.

Some details about int objects in Python 2 and 3

Here's the situation in Python 2. (Some of this is adapted from a blog post by Laurent Luce). Integer objects are represented as blocks of memory with the following structure:

typedef struct {
    PyObject_HEAD
    long ob_ival;
} PyIntObject;

PyObject_HEAD is a macro defining the storage for the refcount and the object type. It's described in some detail by the documentation, and the code can be seen in this answer.

The memory is allocated in large blocks so that there's not an allocation bottleneck for every new integer. The structure for the block looks like this:

struct _intblock {
    struct _intblock *next;
    PyIntObject objects[N_INTOBJECTS];
};
typedef struct _intblock PyIntBlock;

These are all empty at first. Then, each time a new integer is created, Python uses the memory pointed at by next and increments next to point to the next free integer object in the block.

I'm not entirely sure how this changes once you exceed the storage capacity of an ordinary integer, but once you do so, the size of an int gets larger. On my machine, in Python 2:

>>> sys.getsizeof(0)
24
>>> sys.getsizeof(1)
24
>>> sys.getsizeof(2 ** 62)
24
>>> sys.getsizeof(2 ** 63)
36

In Python 3, I think the general picture is the same, but the size of integers increases in a more piecemeal way:

>>> sys.getsizeof(0)
24
>>> sys.getsizeof(1)
28
>>> sys.getsizeof(2 ** 30 - 1)
28
>>> sys.getsizeof(2 ** 30)
32
>>> sys.getsizeof(2 ** 60 - 1)
32
>>> sys.getsizeof(2 ** 60)
36

These results are, of course, all hardware-dependent! YMMV.

The variability in integer size in Python 3 is a hint that they may behave more like variable-length types (like lists). And indeed, this turns out to be true. Here's the definition of the C struct for int objects in Python 3:

struct _longobject {
    PyObject_VAR_HEAD
    digit ob_digit[1];
};

The comments that accompany this definition summarize Python 3's representation of integers. Zero is represented not by a stored value, but by an object with size zero (which is why sys.getsizeof(0) is 24 bytes while sys.getsizeof(1) is 28). Negative numbers are represented by objects with a negative size attribute! So weird.

the instance of int has much less bytes than its inherited class

int is a class, that means it is a type object:

>>> type(int)
<class 'type'>
>>> from sys import getsizeof
>>> getsizeof(int)
400

sys.getsizeof returns the size of that object, not its instance. Use () to create an instance of int and see its size

>>> getsizeof(int())
24
>>> getsizeof(int(2))
28

>>> getsizeof(0)
24
>>> getsizeof(2)
28

This doesn't just apply to the built-in types, same behavior can be observed using user-defined classes:

>>> class a:
...     pass
... 
>>> getsizeof(a)
1056
>>> getsizeof(a())
56
>>> obj = a()
>>> getsizeof(obj)
56

How do I determine the size of my array in C?

Executive summary:

int a[17];
size_t n = sizeof(a)/sizeof(a[0]);

---

Full answer:

To determine the size of your array in bytes, you can use the sizeof
operator:

int a[17];
size_t n = sizeof(a);

On my computer, ints are 4 bytes long, so n is 68.

To determine the number of elements in the array, we can divide
the total size of the array by the size of the array element.
You could do this with the type, like this:

int a[17];
size_t n = sizeof(a) / sizeof(int);

and get the proper answer (68 / 4 = 17), but if the type of
a changed you would have a nasty bug if you forgot to change
the sizeof(int) as well.

So the preferred divisor is sizeof(a[0]) or the equivalent sizeof(*a), the size of the first element of the array.

int a[17];
size_t n = sizeof(a) / sizeof(a[0]);

Another advantage is that you can now easily parameterize
the array name in a macro and get:

#define NELEMS(x)  (sizeof(x) / sizeof((x)[0]))

int a[17];
size_t n = NELEMS(a);

How to find the size of an array (from a pointer pointing to the first element array)?

No, you can't. The compiler doesn't know what the pointer is pointing to. There are tricks, like ending the array with a known out-of-band value and then counting the size up until that value, but that's not using sizeof().

Another trick is the one mentioned by Zan, which is to stash the size somewhere. For example, if you're dynamically allocating the array, allocate a block one int bigger than the one you need, stash the size in the first int, and return ptr+1 as the pointer to the array. When you need the size, decrement the pointer and peek at the stashed value. Just remember to free the whole block starting from the beginning, and not just the array.

---

Related Topics