How to Prepend a Directory the Library Path When Loading a Core File in Gdb on Linux

How do I prepend a directory the library path when loading a core file in gdb on Linux

Start gdb without specifying the executable or core file, then type the following commands:

set solib-absolute-prefix ./usr
file path/to/executable
core-file path/to/corefile

You will need to make sure to mirror your library path exactly from the target system. The above is meant for debugging targets that don't match your host, that is why it's important to replicate your root filesystem structure containing your libraries.

If you are remote debugging a server that is the same architecture and Linux/glibc version as your host, then you can do as fd suggested:

set solib-search-path <path>

If you are trying to override some of the libraries, but not all then you can copy the target library directory structure into a temporary place and use the solib-absolute-prefix solution described above.

make gdb load a shared library from a specific path

The simplest solution is to temporarily restore /opt/mydir/lib/libmylib.so.0 to the copy that was used at crash time (i.e. the one now in /tmp), analyse the core, then restore back the new version.

If you don't want to do that, set solib-search-path and set sysroot are your friends.

Note that you must set both before loading the core. This sequence should work:

(gdb) set sysroot /no/such/file
(gdb) set solib-search-path /tmp:/usr/lib:/lib
(gdb) core /tmp/core

generate core file with gdb

I want to know if all the mmap data will be dump to core file?

Usually the kernel only dumps writable mmaps, but not read-only ones. However, this is configurable: see core(5) man page (the "Controlling which mappings are written to the core dump" part).

background: we had an issue on production, and the binary on production don't have symbol info in it.

The "standard" approach is to debug such binaries remotely with gdbserver and connect to it with gdb that does have access to full-debug binary.

How to get the Shared Library lists when check JVM core dump with Serviceability Agent?

Use info sharedlibrary or info proc mapping gdb command.

$ gdb -core core.3539 /usr/java/jdk1.8.0_102/bin/java

(gdb) info sharedlibrary
From                To                  Syms Read   Shared Object Library
0x00007f5f11569a70  0x00007f5f11576ab1  Yes         /lib/x86_64-linux-gnu/libpthread.so.0
0x00007f5f11350310  0x00007f5f1135d598  Yes (*)     /usr/java/jdk1.8.0_102/bin/../lib/amd64/jli/libjli.so
0x00007f5f1114ada0  0x00007f5f1114b98e  Yes         /lib/x86_64-linux-gnu/libdl.so.2
0x00007f5f10da08b0  0x00007f5f10ef3334  Yes         /lib/x86_64-linux-gnu/libc.so.6
0x00007f5f11781ac0  0x00007f5f1179f640  Yes         /lib64/ld-linux-x86-64.so.2
0x00007f5f0ffaf840  0x00007f5f10885f58  Yes (*)     /usr/java/jdk1.8.0_102/jre/lib/amd64/server/libjvm.so
0x00007f5f0fa8d600  0x00007f5f0fafed0a  Yes         /lib/x86_64-linux-gnu/libm.so.6
0x00007f5f0f882100  0x00007f5f0f8851df  Yes         /lib/x86_64-linux-gnu/librt.so.1
0x00007f5f0f6752a0  0x00007f5f0f67c2a8  Yes (*)     /usr/java/jdk1.8.0_102/jre/lib/amd64/libverify.so
0x00007f5f0f4525e0  0x00007f5f0f468a88  Yes (*)     /usr/java/jdk1.8.0_102/jre/lib/amd64/libjava.so
0x00007f5f0f23d2d0  0x00007f5f0f242bf1  Yes         /lib/x86_64-linux-gnu/libnss_compat.so.2
0x00007f5f0f026ff0  0x00007f5f0f0341e1  Yes         /lib/x86_64-linux-gnu/libnsl.so.1
0x00007f5f0ee190b0  0x00007f5f0ee1f8ce  Yes         /lib/x86_64-linux-gnu/libnss_nis.so.2
0x00007f5f0ec071b0  0x00007f5f0ec0d2a1  Yes         /lib/x86_64-linux-gnu/libnss_files.so.2
0x00007f5f0e9ec7f0  0x00007f5f0e9fd7a8  Yes (*)     /usr/java/jdk1.8.0_102/jre/lib/amd64/libzip.so
0x00007f5ef2822a10  0x00007f5ef2830c68  Yes (*)     /usr/java/jdk1.8.0_102/jre/lib/amd64/libnet.so
(*): Shared library is missing debugging information.

many core files( e.g core.1678 etc ),how to find where the exactly error is. By using gdb?

many core files( e.g core.1678 etc )...

This indicates that your same program or different programs in that particular directory is continuously crashing. When your machine is configured to generate dump file, it creates the file in the form of core.(PID). You may refer many useful article regarding the core dump file. You may refer my blog as well which explains about core dump analysis and its internal.

http://mantoshopensource.blogspot.sg/2011/02/core-dump-analysis-part-ii.html

The basic command to load and analyze the core dump file using GDB is as follows:

mantosh@ubuntu:~$ gdb
// This is how you would open the core dump file.
(gdb) core core.23515
(no debugging symbols found)
Core was generated by `./otest LinuxWorldRocks 10'.
Program terminated with signal 11, Segmentation fault.
[New process 23515]
==> Signal 11(SIGSEGV) was the reason for this core-dump file
==> pid of a program is 23515
#0 0x080485f8 in ?? ()

// Load the debug symbol of your program(build with -g option)
(gdb) symbol ./otest
Reading symbols from /home/mantosh/Desktop/otest...done.

// Now you can execute any normal command which you perform while debugging(except breakpoints).
(gdb) bt
#0 0x080485f8 in printf_info (info=0x8ec5008 "LinuxWorld") at test.c:58
#1 0x080485c2 in my_memcpy (dest=0x8ec5012 "", source=0xbfb9c6fe "Rocks",
length=10) at test.c:47
#2 0x0804855d in main (argc=3, argv=0xbfb9b3f4) at test.c:33

EDIT

cored-ump file is the snapshot of that particular program at the time of exception/segmentation fault. So once you load core-dump in GDB you would only be able to execute the command to read
the memory information. You can not use the debugging commands like breakpoints, continue, run ...etc..........

many core files( e.g core.1678 etc ),how to find where the exactly error is. By using gdb?

many core files( e.g core.1678 etc )...

This indicates that your same program or different programs in that particular directory is continuously crashing. When your machine is configured to generate dump file, it creates the file in the form of core.(PID). You may refer many useful article regarding the core dump file. You may refer my blog as well which explains about core dump analysis and its internal.

http://mantoshopensource.blogspot.sg/2011/02/core-dump-analysis-part-ii.html

The basic command to load and analyze the core dump file using GDB is as follows:

mantosh@ubuntu:~$ gdb
// This is how you would open the core dump file.
(gdb) core core.23515
(no debugging symbols found)
Core was generated by `./otest LinuxWorldRocks 10'.
Program terminated with signal 11, Segmentation fault.
[New process 23515]
==> Signal 11(SIGSEGV) was the reason for this core-dump file
==> pid of a program is 23515
#0 0x080485f8 in ?? ()

// Load the debug symbol of your program(build with -g option)
(gdb) symbol ./otest
Reading symbols from /home/mantosh/Desktop/otest...done.

// Now you can execute any normal command which you perform while debugging(except breakpoints).
(gdb) bt
#0 0x080485f8 in printf_info (info=0x8ec5008 "LinuxWorld") at test.c:58
#1 0x080485c2 in my_memcpy (dest=0x8ec5012 "", source=0xbfb9c6fe "Rocks",
length=10) at test.c:47
#2 0x0804855d in main (argc=3, argv=0xbfb9b3f4) at test.c:33

EDIT

cored-ump file is the snapshot of that particular program at the time of exception/segmentation fault. So once you load core-dump in GDB you would only be able to execute the command to read
the memory information. You can not use the debugging commands like breakpoints, continue, run ...etc..........

Why is gdb refusing to load my shared objects and what is the validation operation

The big thing here is to make sure you're using the exact same binary that is on the target (that the program runs over). This is often quite difficult with libc, especially because libc/ldqnx are sometimes "the same thing" and it confuses gdb.

The easiest way to do this is to log your mkifs output (on the linux host):

make 2>&1 | tee build-out.txt

and read through that, search for libc.so.4, and copy the binary that's being pulled onto the target into . (wherever you're running gdb) so you don't need to mess with SOLIB paths (the lazy solution).

Alternatively, scp/ftp a new libc (one that you want to use, and ideally one that you have associated symbols for) into /tmp and use LD_LIBRARY_PATH to pull that one (and DL_DEBUG=libs to confirm, if you need). Use that same libc to debug

source: I work at QNX and even we struggle with gdb + libc sometimes

---

Related Topics