Force C++ Structure to Pack Tightly

Force C++ structure to pack tightly

If you're using GCC, you can do struct __attribute__ ((packed)) { short a; int b; }

On VC++ you can do #pragma pack(1). This option is also supported by GCC.

#pragma pack(push, 1)
struct { short a; int b; }
#pragma pack(pop)

Other compilers may have options to do a tight packing of the structure with no padding.

pack struct / avoid padding

#include <stdio.h>
#include <stdint.h>

struct SkipListNode{
    void        *data;      // 8 bytes
    uint8_t     size;       // 1 byte
    void *next[1];
    };

struct SkipListNode_pack{
    void        *data;      // 8 bytes
    uint8_t     size;       // 1 byte
    void *next[1];
    } __attribute__((packed));

int main(int argc, char** argv)
{
    printf("%d:%d\n", sizeof(struct SkipListNode), sizeof(struct SkipListNode_pack));
    return 0;
}

The output from the above code:

ishaypeled@arania sandbox]$ ./test
24:17

So yeah, you should definitely use __attribute__((packed)) in your case if you want to save memory. On the other hand, like @MarioTheSpoon states - it may come with a performance penalty.

I'd check the padding solution to see if you can live with that penalty on your specific machines. My bet is you won't even notice it unless you're really performance critical.

How to declare packed struct (without padding) for LLVM?

Did you actually try it? I just tested it on my machine, and __attribute__((packed)) compiled fine using clang.

Edit: I got the same warning ("Warning: packed attribute unused") for

typedef struct {
    int a;
    char c;
} mystruct __attribute__((packed));

and in this case sizeof(mystruct) was 8.

However,

typedef struct __attribute__((packed)) {
    int a;
    char c;
} mystruct;

worked just fine, and sizeof(mystruct) was 5.

Conclusion: it seems that the attribute needs to preceed the struct label in order to get this working.

How to find all the structs that could be made smaller by changing the order of their members

pahole is a utility written for this specific purpose. It'll analyse your compiled object files (compiled with debugging enabled), and show you the structure holes.

are bits in structs guaranteed to be contiguous?

Most compilers will give you bit fields that are packed together, although the behaviour is implementation defined.

One way to check would be to add a static_assert to compare the size of your struct definitions with and without the bitfields. This way you would get a nice compile time error once the implementation doesn't do what you expect.

#include <cstdint>

struct Pair_size_check{
    uint64_t    created;    // 8 bytes
    uint32_t    expires;    // 4 bytes
    uint16_t    keylen;     // 2 bytes, 4 bits are used for vallen. 2 bits are reserved for future versions. 10 bytes for keylen.
    uint16_t    vallen;     // 2  bytes
};

struct Pair{
    uint64_t    created;
    uint32_t    expires;
    uint8_t     vallen2 : 4;
    uint8_t     future  : 2;
    uint16_t    keylen  : 10;
    uint16_t    vallen; 
};

static_assert(sizeof(Pair) == sizeof(Pair_size_check));

Compile-time check to make sure that there is no padding anywhere in a struct

Since C++17 you might be able to use std::has_unique_object_representations.

#include <type_traits>

static_assert(std::has_unique_object_representations_v<This_Should_Succeed>); // succeeds
static_assert(std::has_unique_object_representations_v<This_Should_Fail>); // fails

Although, this might not do exactly what you want it to do. Check the linked cppreference page for details.

Are packed structs portable?

You should never use structs across compile domains, against memory (hardware registers, picking apart items read from a file or passing data between processors or the same processor different software (between an app and a kernel driver)). You are asking for trouble as the compiler has somewhat free will to choose alignment and then the user on top of that can make it worse by using modifiers.

No there is no reason to assume you can do this safely across platforms, even if you use the same gcc compiler version for example against different targets (different builds of the compiler as well as the target differences).

To reduce your odds of failure start with the largest items first (64 bit then 32 bit the 16 bit then lastly any 8 bit items) Ideally align on 32 minimum perhaps 64 which one would hope arm and x86 do, but that can always change as well as the default can be modified by whomever builds the compiler from sources.

Now if this is a job security thing, sure go ahead, you can do regular maintenance on this code, likely going to need a definition of each structure for each target (so one copy of the source code for the structure definition for ARM and another for x86, or will need this eventually if not immediately). And then every or every few product releases you get to be called in to do work on the code...Nice little maintenance time bombs that go off...

If you want to safely communicate between compile domains or processors the same or different architectures, use an array of some size, a stream of bytes a stream of halfwords or a stream of words. Significantly reduces your risk of failure and maintenance down the road. Do not use structures to pick apart those items that just restores the risk and failure.

The reason why folks seem to think this is okay because of using the same compiler or family against the same target or family (or compilers derived from other compilers choices), as you understand the rules of the language and where the implementation defined areas are you will eventually run across a difference, sometimes it takes decades in your career, sometimes it takes weeks...Its the "works on my machine" problem...

Force a C++ class to have aligned attributes

With the MSVC compiler (you have included the VS-2015 tag), you can use the #pragma pack directive to set padding options for classes and structures. This also allows you to save/restore 'default' settings, using the push and pop options.

The following short example demonstrates this:

#include <iostream>

class C3 {
public:
    uint16_t p1;
    uint16_t p2;
    uint8_t p3;
};

#pragma pack(push, 1) // Save current packing and set to single-byte
class C4 {
public:
    uint16_t p1;
    uint16_t p2;
    uint8_t p3;
};
#pragma pack(pop) // Restore saved packing

class C5 {
public:
    uint16_t p1;
    uint16_t p2;
    uint8_t p3;
};

int main()
{
    std::cout << sizeof(C3) << std::endl; // 6
    std::cout << sizeof(C4) << std::endl; // 5
    std::cout << sizeof(C5) << std::endl; // 6 (again)
    return 0;
}

Many other compilers support directives equivalent to the MSVC #pragma pack.

Is gcc's __attribute__((packed)) / #pragma pack unsafe?

Yes, __attribute__((packed)) is potentially unsafe on some systems. The symptom probably won't show up on an x86, which just makes the problem more insidious; testing on x86 systems won't reveal the problem. (On the x86, misaligned accesses are handled in hardware; if you dereference an int* pointer that points to an odd address, it will be a little slower than if it were properly aligned, but you'll get the correct result.)

On some other systems, such as SPARC, attempting to access a misaligned int object causes a bus error, crashing the program.

There have also been systems where a misaligned access quietly ignores the low-order bits of the address, causing it to access the wrong chunk of memory.

Consider the following program:

#include <stdio.h>
#include <stddef.h>
int main(void)
{
    struct foo {
        char c;
        int x;
    } __attribute__((packed));
    struct foo arr[2] = { { 'a', 10 }, {'b', 20 } };
    int *p0 = &arr[0].x;
    int *p1 = &arr[1].x;
    printf("sizeof(struct foo)      = %d\n", (int)sizeof(struct foo));
    printf("offsetof(struct foo, c) = %d\n", (int)offsetof(struct foo, c));
    printf("offsetof(struct foo, x) = %d\n", (int)offsetof(struct foo, x));
    printf("arr[0].x = %d\n", arr[0].x);
    printf("arr[1].x = %d\n", arr[1].x);
    printf("p0 = %p\n", (void*)p0);
    printf("p1 = %p\n", (void*)p1);
    printf("*p0 = %d\n", *p0);
    printf("*p1 = %d\n", *p1);
    return 0;
}

On x86 Ubuntu with gcc 4.5.2, it produces the following output:

sizeof(struct foo)      = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = 0xbffc104f
p1 = 0xbffc1054
*p0 = 10
*p1 = 20

On SPARC Solaris 9 with gcc 4.5.1, it produces the following:

sizeof(struct foo)      = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = ffbff317
p1 = ffbff31c
Bus error

In both cases, the program is compiled with no extra options, just gcc packed.c -o packed.

(A program that uses a single struct rather than array doesn't reliably exhibit the problem, since the compiler can allocate the struct on an odd address so the x member is properly aligned. With an array of two struct foo objects, at least one or the other will have a misaligned x member.)

(In this case, p0 points to a misaligned address, because it points to a packed int member following a char member. p1 happens to be correctly aligned, since it points to the same member in the second element of the array, so there are two char objects preceding it -- and on SPARC Solaris the array arr appears to be allocated at an address that is even, but not a multiple of 4.)

When referring to the member x of a struct foo by name, the compiler knows that x is potentially misaligned, and will generate additional code to access it correctly.

Once the address of arr[0].x or arr[1].x has been stored in a pointer object, neither the compiler nor the running program knows that it points to a misaligned int object. It just assumes that it's properly aligned, resulting (on some systems) in a bus error or similar other failure.

Fixing this in gcc would, I believe, be impractical. A general solution would require, for each attempt to dereference a pointer to any type with non-trivial alignment requirements either (a) proving at compile time that the pointer doesn't point to a misaligned member of a packed struct, or (b) generating bulkier and slower code that can handle either aligned or misaligned objects.

I've submitted a gcc bug report. As I said, I don't believe it's practical to fix it, but the documentation should mention it (it currently doesn't).

UPDATE: As of 2018-12-20, this bug is marked as FIXED. The patch will appear in gcc 9 with the addition of a new -Waddress-of-packed-member option, enabled by default.

When address of packed member of struct or union is taken, it may
result in an unaligned pointer value. This patch adds
-Waddress-of-packed-member to check alignment at pointer assignment and warn unaligned address as well as unaligned pointer

I've just built that version of gcc from source. For the above program, it produces these diagnostics:

c.c: In function ‘main’:
c.c:10:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
   10 |     int *p0 = &arr[0].x;
      |               ^~~~~~~~~
c.c:11:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
   11 |     int *p1 = &arr[1].x;
      |               ^~~~~~~~~

---

Related Topics