Why Are Core Dump Files Generated

How to generate a core dump in Linux on a segmentation fault?

This depends on what shell you are using. If you are using bash, then the ulimit command controls several settings relating to program execution, such as whether you should dump core. If you type

ulimit -c unlimited

then that will tell bash that its programs can dump cores of any size. You can specify a size such as 52M instead of unlimited if you want, but in practice this shouldn't be necessary since the size of core files will probably never be an issue for you.

In tcsh, you'd type

limit coredumpsize unlimited

Why are core dump files generated?

A process dumps core when it is terminated by the operating system due to a fault in the program. The most typical reason this occurs is because the program accessed an invalid pointer value. Given that you have a sporadic dump, it's likely that you are using an uninitialized pointer.

Can you post the code that is causing the fault? Other than vague generalizations it's hard to guess what's wrong without actually seeing code.

As for what a core dump actually is, check out this Wikipedia article:

http://en.wikipedia.org/wiki/Core_dump

What is a core dump file?

The core dump files are not generated by Slurm, but by the operating system. They keep the memory record of a process that have crashed.

They can be used for tracing the execution or debugging.

You can adjust the core dump file size including this command in your .bashrc:

limit coredumpsize 2500

More info:

Limit the size of a core dump file

Core dump file

Who generate a core dump file? Kernel or glibc?

The kernel itself generates the coredump. See the core handling routines in the linux kernel source here:

http://lxr.linux.no/linux+v3.12.6/fs/coredump.c

If the process receives any of the following signals 1, the kernel responds by attempting a coredump.

#define SIG_KERNEL_COREDUMP_MASK (\
rt_sigmask(SIGQUIT)   |  rt_sigmask(SIGILL)    | \
rt_sigmask(SIGTRAP)   |  rt_sigmask(SIGABRT)   | \
rt_sigmask(SIGFPE)    |  rt_sigmask(SIGSEGV)   | \
rt_sigmask(SIGBUS)    |  rt_sigmask(SIGSYS)    | \
rt_sigmask(SIGXCPU)   |  rt_sigmask(SIGXFSZ)   | \
SIGEMT_MASK

This coredump is configurable, and can be disabled or controlled in several ways, including the file /proc/sys/kernel/core_pattern, and ulimit. One can also control the delivery of these signals through the signal handling mechanisms.

Core dump file is not generated

Make sure your current directory (at the time of crash -- server may change directories) is writable. If the server calls setuid, the directory has to be writable by that user.

Also check /proc/sys/kernel/core_pattern. That may redirect core dumps to another directory, and that directory must be writable. More info here.

How to generate core dump file in Linux?

The generation of core dump files it is not always enabled. Try with the ulimit command.

Why Coredump files is not generating here?

kill -9 will not generate a core file. The command kill -l gives a list of supported signals. kill -6 or kill -SIGABRT should produce a core file. As well as most other signals such as kill -BUS, kill -SEGV, etc.

What is a core dump file in Linux? What information does it provide?

It's basically the process address space in use (from the mm_struct structure which contains all the virtual memory areas), and any other supporting information^*a, at the time it crashed.

For example, let's say you try to dereference a NULL pointer and receive a SEGV signal, causing you to exit. As part of that process, the operating system tries to write your information to a file for later post-mortem analysis.

You can load the core file into a debugger along with the executable file (for symbols and other debugging information, for example) and poke around to try and discover what caused the problem.

^*a: in kernel version 2.6.38, fs/exec.c/do_coredump() is the one responsible for core dumps and you can see that it's passed the signal number, exit code and registers. It in turn passes the signal number and registers to a binary-format-specific (ELF, a.out, etc) dumper.

The ELF dumper is fs/binfmt_elf.c/elf_core_dump() and you can see that it outputs non-memory-based information, like thread details, in fs/binfmt_elf.c/fill_note_info(), then returns to output the process space.