Difference Between Trap Flag (Tf) and Monitor Trap Flag

Need an overview of debugging process from the hardware layer

The x86 ISA includes a single-byte int3 encoding that's intended for software breakpoints. GDB uses this (via ptrace) by default for breakpoints.

(Why Single Stepping Instruction on X86?)

x86 also has a Trap Flag (TF) in EFLAGS for single-step mode. (https://en.wikipedia.org/wiki/Trap_flag). See also Difference between trap flag (TF) and monitor trap flag?

There are even "debug registers" for setting hardware breakpoints, without modifying the machine code to be run. And also hardware support for watch points, to break on write to a certain address. This makes GDB watch points efficient, not requiring it to single-step and manually decode the instruction to see where it writes.

https://wiki.osdev.org/CPU_Registers_x86#Debug_Registers

Implementing hardware breakpoints using x86 debug register osdev forum thread might be relevant.

---

Some other ISAs exist without nearly as much HW support for debugging. e.g. without a single-step flag, a debugger might have to always decode the current instruction (pointed to by program counter) to find the next one to be executed, and set a software breakpoint there.

ARM Linux used to do that to implement ptrace single-step, but that disassembler code was removed from the kernel and now just returns -EIO. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=425fc47adb5bb69f76285be77a09a3341a30799e is the commit that removed it.

GDB Setting Trap Flag After Popf (possible bug?)

I believe it is a kernel bug. The kernel has to set TF when single-stepping but user mode also may be modifying TF. To handle this, the kernel tries to maintain who set TF:

    /* Set TF on the kernel stack.. */
    regs->flags |= X86_EFLAGS_TF;

    /*
     * ..but if TF is changed by the instruction we will trace,
     * don't mark it as being "us" that set it, so that we
     * won't clear it by hand later.
     *
     * Note that if we don't actually execute the popf because
     * of a signal arriving right now or suchlike, we will lose
     * track of the fact that it really was "us" that set it.
     */
    if (is_setting_trap_flag(child, regs)) {
            clear_tsk_thread_flag(child, TIF_FORCED_TF);
            return 0;
    }

Note it even admits some corner cases might make it lose track.
What's worse is that is_setting_trap_flag only checks if the instruction will modify TF, it doesn't check if it is actually setting it:

    switch (opcode[i]) {
    /* popf and iret */
    case 0x9d: case 0xcf:
            return 1;

As such it will mark TF as user-set even if it has been cleared. In get_flags it will try to mask TF if it was set by kernel as follows:

    /*
     * If the debugger set TF, hide it from the readout.
     */
    if (test_tsk_thread_flag(task, TIF_FORCED_TF))
            retval &= ~X86_EFLAGS_TF;

Since TIF_FORCED_TF has been erroneously cleared, this condition will not be true, hence the TF that has in fact been set by the kernel for single-stepping will be returned to the debugger.

I think this can be fixed by modifying is_setting_trap_flag so that it checks the stack for the new value of the flags and only returns 1 if TF is actually being set.

Why Single Stepping Instruction on X86?

int 3 is a special 1-byte interrupt. Invoking it will break into the debugger if one is present, otherwise the application will typically crash.

When the debugger sets the trap flag, this causes the processor to automatically execute an int 1 interrupt after every instruction. This allows the debugger to single-step by instructions, without having to insert an int 3 instruction. You do not have to invoke this interrupt explicitly.

How to do a VMEXIT in kvm on every branch instruction?

BTF flag is not in the VMCS; it's in the IA32_DEBUGCTL MSR.
You need to set it explicitly by doing wrmsr before the VM entry.
For KVM you could add this in vmx_vcpu_run.

In order to cause a VM exit for all debug exceptions, set bit 1 of the exception bitmap in the VMCS. The information about the debug exception that caused the VM exit is saved in the exit qualification field of the VMCS. The hypervisor can then process the debug exception itself or inject it into the guest.

---

Related Topics