Linux Core Dumps Are Too Large!

reduce core dump size on linux

Consider using Breakpad: https://code.google.com/p/google-breakpad/
https://code.google.com/p/google-breakpad/wiki/GettingStartedWithBreakpad :

Breakpad is a library and tool suite that allows you to distribute an
application to users with compiler-provided debugging information
removed, record crashes in compact "minidump" files, send them back to
your server, and produce C and C++ stack traces from these minidumps.
Breakpad can also write minidumps on request for programs that have
not crashed.

How to limit the size of core dump file when generating it using GDB

GDB does not respect 'ulimit -c', only the kernel does.

It's not clear whether you run GDB on target board, or on a development host (and using gdbserver on target). You probably should use the latter, which will allow you to collect full core dump.

Truncated core dumps are a pain anyway, as often they will not contain exactly the info you need to debug the problem.

Why core file is more than virtual memory?

If you are running a 64-bit OS then you can have file-backed mappings that exceed many times the amount of available physical memory + swap space.

Since kernel version 2.6.23, Linux provides a mechanism to control what gets included in the core dump file, called core dump filter. The value of the filter is a bit-field manipulated via the /proc/<pid>/coredump_filter file (see core(5) man page):

• bit 0 (0x01) - anonymous private mappings (e.g. dynamically allocated memory)
• bit 1 (0x02) - anonymous shared mappings
• bit 2 (0x04) - file-backed private mappings
• bit 3 (0x08) - file-backed shared mappings (e.g. shared libraries)
• bit 4 (0x10) - ELF headers
• bit 5 (0x20) - private huge pages
• bit 6 (0x40) - shared huge pages

The default value is 0x33 which corresponds to dumping all anonymous mappings as well as the ELF headers (but only if kernel is compiled with CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS) and the private huge pages. Reading from this file returns the hexadecimal value of the filter. Writing a new hexadecimal value to coredump_filter changes the filter for the particular process, e.g. to enable dump of all possible mappings one would:

echo 0x7f > /proc/<pid>/coredump_filter

(where <pid> is the PID of the process)

The value of the core dump filter is iherited in child processes created by fork().

Some Linux distributions might change the filter value for the init process early in the OS boot stage, e.g. to enable dumping the file-backed mappings. This would then affect any process started later.

why does gdb complain that my core files are too small and then fail to produce a meaningful stack trace?

The Core Dump File Format

On a modern Linux system, core dump files are formatted using the ELF object file format, with a specific configuration.
ELF is a structured binary file format, with file offsets used as references between data chunks in the file.

• Core Dump Files
• The ELF object file format

For core dump files, the e_type field in the ELF file header will have the value ET_CORE.

Unlike most ELF files, core dump files make all their data available via program headers, and no section headers are present.
You may therefore choose to ignore section headers in calculating the size of the file, if you only need to deal with core files.

Calculating Core Dump File Size

To calculate the ELF file size:

1. Consider all the chunks in the file:
   
   
   • chunk description (offset + size)
   • the ELF file header (0 + e_ehsize) (52 for ELF32, 64 for ELF64)
   • program header table (e_phoff + e_phentsize * e_phnum)
   • program data chunks (aka "segments") (p_offset + p_filesz)
   • the section header table (e_shoff + e_shentsize * e_shnum) - not required for core files
   • the section data chunks - (sh_offset + sh_size) - not required for core files
2. Eliminate any section headers with a sh_type of SHT_NOBITS, as these are merely present to record the position of data that has been stripped and is no longer present in the file (not required for core files).
3. Eliminate any chunks of size 0, as they contain no addressable bytes and therefore their file offset is irrelevant.
4. The end of the file will be the end of the last chunk, which is the maximum of the offset + size for all remaining chunks listed above.

If you find the offsets to the program header or section header tables are past the end of the file, then you will not be able to calculate an expected file size, but you will know the file has been truncated.

Although an ELF file could potentially contain unaddressed regions and be longer than the calculated size, in my limited experience the files have been exactly the size calculated by the above method.

Truncated Core Files

gdb likely performs a calculation similar to the above to calculate the expected core file size.

In short, if gdb says your core file is truncated, it is very likely truncated.

One of the most likely causes for truncated core dump files is the system ulimit. This can be set on a system-wide basis in /etc/security/limits.conf, or on a per-user basis using the ulimit shell command [footnote: I don't know anything about systems other than my own].

Try the command "ulimit -c" to check your effective core file size limit:

$ ulimit -c
unlimited

Also, it's worth noting that gdb doesn't actually refuse to operate because of the truncated core file. gdb still attempts to produce a stack backtrace and in your case only fails when it tries to access data on the stack and finds that the specific memory locations addressed are off the end of the truncated core file.

---

Related Topics