Debugging in Linux Using Core Dumps

How do I analyze a program's core dump file with GDB when it has command-line parameters?

You can use the core with GDB in many ways, but passing parameters which is to be passed to the executable to GDB is not the way to use the core file. This could also be the reason you got that error. You can use the core file in the following ways:

gdb <executable> <core-file> or gdb <executable> -c <core-file> or

gdb <executable>
...
(gdb) core <core-file>

When using the core file you don't have to pass arguments. The crash scenario is shown in GDB (checked with GDB version 7.1 on Ubuntu).

For example:

$ ./crash -p param1 -o param2
Segmentation fault (core dumped)
$ gdb ./crash core
GNU gdb (GDB) 7.1-ubuntu
...
Core was generated by `./crash -p param1 -o param2'. <<<<< See this line shows crash scenario
Program terminated with signal 11, Segmentation fault.
#0  __strlen_ia32 () at ../sysdeps/i386/i686/multiarch/../../i586/strlen.S:99
99    ../sysdeps/i386/i686/multiarch/../../i586/strlen.S: No such file or directory.
    in ../sysdeps/i386/i686/multiarch/../../i586/strlen.S
(gdb)

If you want to pass parameters to the executable to be debugged in GDB, use --args.

For example:

$ gdb --args ./crash -p param1 -o param2
GNU gdb (GDB) 7.1-ubuntu
...
(gdb) r
Starting program: /home/@@@@/crash -p param1 -o param2

Program received signal SIGSEGV, Segmentation fault.
__strlen_ia32 () at ../sysdeps/i386/i686/multiarch/../../i586/strlen.S:99
99    ../sysdeps/i386/i686/multiarch/../../i586/strlen.S: No such file or directory.
    in ../sysdeps/i386/i686/multiarch/../../i586/strlen.S
(gdb)

Man pages will be helpful to see other GDB options.

Most useful commands are:

• bt (backtrace)
• info locals (show values of local variables)
• info registers (show values of CPU registers)
• frame X (change to stack frame X)
• up and down (navigate in the stack frame (call chain))

Debugging in Linux using core dumps

It sounds like there are other differences between your release and debug build then simply the absence/presence of the -g flag. Assuming that's the case, there is nothing you can do right now, but you can adjust your build to handle this better:

Here's what we do at my place of work.

1. Include the -g flag when building the release version.
2. Archive that version.
3. run strip --strip-unneeded on the binary before shipping it to customers.

Now, when we get a crash we can use the archived version with symbols to do debugging.

One thing to note is that if your release version includes optimization, debugging may be difficult even with symbols. For example, the optimizer can reorder your code so even though the debugger will say you crashed on line N, you can't assume that the code actually executed line N-1.

I have generated a linux core dump with debug symbols in target system. But I am unable to debug the dump using GDB in host system

How should I link the shared libraries in debug symbols in host system

Your problem has nothing to do with any shared libraries (you may have a problem with them as well, but you should solve the immediate problem first).

Your immediate problem is that your binary: coredump.dbg has been corrupted in some way. This is the key message from readelf: Error: Unable to read in 0x900 bytes of section headers.

This could happen in a few different ways:

1. You have a buggy linker (that is very unlikely).
2. You are using some combination of objcopy and/or strip, and are not doing it correctly.
3. You transfer the file between target and host in a way that corrupts it (using e.g. ASCII-mode FTP transfer).

gdb debugging of core.# file - getting the full command which caused the crash

You can load core dump with matching binary (the one for which core dump was generated) and print argv values in the frame where main function resides.

Something like this:

gdb /tools/graphmap/bin/Linux-x64/graphmap /4thExp/core.82912

Then go up in stack trace to initial frame where int main(int argc, char *argv[]) resides. Now you can print the number of arguments and their values from gdb prompt.

Update:

It appears that your binary is multithreaded and crash happened in some auxiliary thread. You should therefore find main thread and switch to it. Here is an example of how to do it for Firefox with many threads:

(gdb) t a a bt -1

Thread 59 (Thread 0x7f691deff700 (LWP 25924)):
#12 0x00007f69dce93f6f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105
..........
..........
many threads are listed here
..........
..........
Thread 1 (Thread 0x7f69de01a740 (LWP 4143)):
#17 0x000056374cb38817 in main ()
(gdb) t 1
[Switching to thread 1 (Thread 0x7f69de01a740 (LWP 4143))]
#0  0x00007f69dce8800d in poll () at ../sysdeps/unix/syscall-template.S:84
84  T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)

Now gdb is switched to main thread (Thread 1).

GDB With a coredump file in linux

If your core file is core just run

gdb ./my_prog core

to do some post-mortem analysis with the core.

If you want to use gdb without your core, consider

gdb --args ./my_prog arg

or else run just gdb ./my_prog then issue the set args command to gdb.

If your process is still running as pid 1234, you could with gdb ./my_prog 1234 attach the gdb to the running process.

You really should read the gdb documentation.

---

Related Topics