C/C++ With Gcc: Statically Add Resource Files to Executable/Library

C/C++ with GCC: Statically add resource files to executable/library

With imagemagick:

convert file.png data.h

Gives something like:

/*
  data.h (PNM).
*/
static unsigned char
  MagickImage[] =
  {
    0x50, 0x36, 0x0A, 0x23, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x64, 0x20, 
    0x77, 0x69, 0x74, 0x68, 0x20, 0x47, 0x49, 0x4D, 0x50, 0x0A, 0x32, 0x37, 
    0x37, 0x20, 0x31, 0x36, 0x32, 0x0A, 0x32, 0x35, 0x35, 0x0A, 0xFF, 0xFF, 
    0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 

....

For compatibility with other code you can then use either fmemopen to get a "regular" FILE * object, or alternatively std::stringstream to make an iostream. std::stringstream is not great for this though and you can of course just use a pointer anywhere you can use an iterator.

If you're using this with automake don't forget to set BUILT_SOURCES appropriately.

The nice thing about doing it this way is:

1. You get text out, so it can be in version control and patches sensibly
2. It is portable and well defined on every platform

Embedding resources in executable using GCC

There are a couple possibilities:

• use ld's capability to turn any file into an object (Embedding binary blobs using gcc mingw):

  ld -r -b binary -o binary.o foo.bar  # then link in binary.o
  

• use a bin2c/bin2h utility to turn any file into an array of bytes (Embed image in code, without using resource section or external images)

---

Update: Here's a more complete example of how to use data bound into the executable using ld -r -b binary:

#include <stdio.h>

// a file named foo.bar with some example text is 'imported' into 
// an object file using the following command:
//
//      ld -r -b binary -o foo.bar.o foo.bar
//
// That creates an bject file named "foo.bar.o" with the following 
// symbols:
//
//      _binary_foo_bar_start
//      _binary_foo_bar_end
//      _binary_foo_bar_size
//
// Note that the symbols are addresses (so for example, to get the 
// size value, you have to get the address of the _binary_foo_bar_size
// symbol).
//
// In my example, foo.bar is a simple text file, and this program will
// dump the contents of that file which has been linked in by specifying
// foo.bar.o as an object file input to the linker when the progrma is built

extern char _binary_foo_bar_start[];
extern char _binary_foo_bar_end[];

int main(void)
{
    printf( "address of start: %p\n", &_binary_foo_bar_start);
    printf( "address of end: %p\n", &_binary_foo_bar_end);

    for (char* p = _binary_foo_bar_start; p != _binary_foo_bar_end; ++p) {
        putchar( *p);
    }

    return 0;
}

---

Update 2 - Getting the resource size: I could not read the _binary_foo_bar_size correctly. At runtime, gdb shows me the right size of the text resource by using display (unsigned int)&_binary_foo_bar_size. But assigning this to a variable gave always a wrong value. I could solve this issue the following way:

unsigned int iSize =  (unsigned int)(&_binary_foo_bar_end - &_binary_foo_bar_start)

It is a workaround, but it works good and is not too ugly.

Embedding resources into executable using GCC on Arm?

I suppose I cannot reuse the same myv2.o?

You can't that; file is an ELF 64bit x86_64 ABI object file, you need it for your target architecture

Do I have to regenerate the file on Arm

no, there's no reason ld needs to run on an ARM machine,

using ld from an appropriate Arm toolchain?

exactly, just replace ld with your correctly targeting linker, e.g. arm-none-eabi-ld if this is for baremetal. Again, this is normal cross-development: no need to run on the target architecture, just build for the target architecture.

Embed resources (eg, shader code; images) into executable/library with CMake

One of the easiest ways to do this is to include a small, portable C program in your build that reads the resource and generates a C file that contains the length of the resource data and the actual resource data as an array of constant character literals. This will be entirely platform independent, but should only be used for resources that are reasonably small. For larger resources, you probably don't want to embed the files in your program.

For resource "foo", the generated C file "foo.c" would contain:

const char foo[] = { /* bytes of resource foo */ };
const size_t foo_len = sizeof(foo);

To access the resource from C++, you declare the following two symbols in either a header or the cpp file where they're used:

extern "C" const char foo[];
extern "C" const size_t foo_len;

To generate foo.c in the build, you need a target for the C program (call it embedfile.c), and you need to use the ADD_CUSTOM_COMMAND command to call this program:

add_executable(embedfile embedfile.c)

add_custom_command(
  OUTPUT foo.c
  COMMAND embedfile foo foo.rsrc
  DEPENDS foo.rsrc)

Then, include foo.c on the source list of a target that requires the "foo" resource. You now have access to the bytes of "foo".

The program embedfile.c is:

#include <stdlib.h>
#include <stdio.h>

FILE* open_or_exit(const char* fname, const char* mode)
{
  FILE* f = fopen(fname, mode);
  if (f == NULL) {
    perror(fname);
    exit(EXIT_FAILURE);
  }
  return f;
}

int main(int argc, char** argv)
{
  if (argc < 3) {
    fprintf(stderr, "USAGE: %s {sym} {rsrc}\n\n"
        "  Creates {sym}.c from the contents of {rsrc}\n",
        argv[0]);
    return EXIT_FAILURE;
  }

  const char* sym = argv[1];
  FILE* in = open_or_exit(argv[2], "r");

  char symfile[256];
  snprintf(symfile, sizeof(symfile), "%s.c", sym);

  FILE* out = open_or_exit(symfile,"w");
  fprintf(out, "#include <stdlib.h>\n");
  fprintf(out, "const char %s[] = {\n", sym);

  unsigned char buf[256];
  size_t nread = 0;
  size_t linecount = 0;
  do {
    nread = fread(buf, 1, sizeof(buf), in);
    size_t i;
    for (i=0; i < nread; i++) {
      fprintf(out, "0x%02x, ", buf[i]);
      if (++linecount == 10) { fprintf(out, "\n"); linecount = 0; }
    }
  } while (nread > 0);
  if (linecount > 0) fprintf(out, "\n");
  fprintf(out, "};\n");
  fprintf(out, "const size_t %s_len = sizeof(%s);\n\n",sym,sym);

  fclose(in);
  fclose(out);

  return EXIT_SUCCESS;
}

Performance of embedding resource files into executable

The disadvantage to this resource allocation method is that all resources will be present in virtual memory for the entire duration of your application.

For example, if you have 10 levels in your game, using a more common file-based storage solution you would only load the data for whichever level you are going to play. By loading all the game data at runtime you are allocating more RAM than necessary. A solution which uses an optimally minimal amount of RAM would only load the resources it needs for the duration it needs them.

However, if you were to use a file-based storage solution for your resources and load them all at the start of execution your RAM usage would be the same as the statically allocated resource method.

Also, if your target machine is a simple game console that does not do multitasking then you are usually free to use the available resources in whatever means suits you the best. There is no requirement to "play nice" as there's typically only one game running at a time.

If you're dealing with a few megabytes on a modern system the penalty is negligible, but the performance penalty when you get into multiple gigabytes of data is that you will be creating an unnecessary strain on the available memory resources of the system.

edit: be sure to see points raised in the comments

---

Related Topics