Having Trouble Finding the Method _Kernel_Vsyscall Within the Linux Kernel

What is __kernel_vsyscall?

__kernel_vsyscal is the method used by linux-gate.so (a part of the Linux kernel) to make a system call using the fastest available method, preferably the sysenter instruction. The thing is properly explained by Johan Petersson.

gdb: always stop at 0xffffe410 in __kernel_vsyscall ()

How did I get this situation?

That situation is completely normal for when you attach to a process which is blocked in a system call (waiting for message, or for read to complete).

How to make it continue?

You type continue (at which point the application would again block, waiting for a message). If you want to debug some part of the application, set breakpoints before continuing.

_kernel_vsyscall () in core files

The error is in free(), which is likely a function you call in your program, and a common place to make mistakes. You likely freed an invalid pointer (possibly via double-free?). The stack trace shows all those other functions because those were called below free(). This is common when calling library functions incorrectly, so you generally just keep an eye out for things that you recognize. As a rule of thumb, you'll want to start looking at the furthest thing down a stack trace that you recognize (i.e. is in your program), though if there are other memory corruptions further up the stack or in your program, looking there won't help too much.

Edit for clarity: "Down the stack" means "toward the top of the list," since as sixlettervariables points out, you want to find the most recent place you were that you recognize. I realize that my initial wording could be confusing.

Linux Kernel systemcall call with an int 0x80

For 64-bit systems the Linux system call ABI is completely different from i*86 one unless there's a layer of compatibility.
This may help:
http://callumscode.com/blog/3

I also found the syscall source in the eglibc, it looks different indeed:
http://www.eglibc.org/cgi-bin/viewvc.cgi/trunk/libc/sysdeps/unix/sysv/linux/x86_64/syscall.S?view=markup

So it looks like int $0x80 does not work for x86_64 Linux kernels, you need to use syscall instead.

---

Related Topics