Capturing User-Space Assembly with Ftrace and Kprobes (By Using Virtual Address Translation)

register_kprobe returns EINVAL (-22) error for instructions involving rip

Turns out, register_kprobe does have limitations with instructions invoving rip relative addressing for x86_64.

Here is snippet of __copy_instruction function code causing error (register_kprobe -> prepare_kprobe -> arch_prepare_kprobe -> arch_copy_kprobe -> __copy_instruction )

#ifdef CONFIG_X86_64
    if (insn_rip_relative(&insn)) {
        s64 newdisp;
        u8 *disp;
        kernel_insn_init(&insn, dest);
        insn_get_displacement(&insn);
        /*
         * The copied instruction uses the %rip-relative addressing
         * mode.  Adjust the displacement for the difference between
         * the original location of this instruction and the location
         * of the copy that will actually be run.  The tricky bit here
         * is making sure that the sign extension happens correctly in
         * this calculation, since we need a signed 32-bit result to
         * be sign-extended to 64 bits when it's added to the %rip
         * value and yield the same 64-bit result that the sign-
         * extension of the original signed 32-bit displacement would
         * have given.
         */
        newdisp = (u8 *) src + (s64) insn.displacement.value - (u8 *) dest;
        if ((s64) (s32) newdisp != newdisp) {
            pr_err("Kprobes error: new displacement does not fit into s32 (%llx)\n", newdisp);
            pr_err("\tSrc: %p, Dest: %p, old disp: %x\n", src, dest, insn.displacement.value);
            return 0;
        }
        disp = (u8 *) dest + insn_offset_displacement(&insn);
        *(s32 *) disp = (s32) newdisp;
    }
#endif

http://elixir.free-electrons.com/linux/v3.10/ident/__copy_instruction

A new displacement value is calculated based new instruction address (where orig insn is copied). If that value doesn't fit in 32 bit, it returns 0 which results in EINVAL error. Hence the failure.

As a workaround, we can set kprobe handler post previous instruction or pre next instruction based on need (works for me).

Good explanation/documentation on ftrace

I don't have an explanation, but the documentation is here.

ftrace: Is there any way to view my trace with kernelshark without using trace-cmd?

1. Capturing an oops (from startup) to the serial console

The function calls leading up to a kernel-panic can be captured by passing the following command-line options to the Linux kernel:

ftrace=function ftrace_dump_on_oops

2. Capturing traces automatically upon boot

You can use the following kernel command line parameters to automatically generate a trace at boot. This can then be opened using parsers like Kernelshark and pytimechart.

trace_event=sched:*,timer:*,irq:* trace_buf_size=40M

---

For more details checkout this Ftrace wiki.

---

Related Topics