X86 Gnu Assembler Strange Change Seg Fault

X86 assembly: got segmentation fault after executing ret in the main function, why?

Check your preamble in _myfunc:

_myfunc:
    pushq %rbp
    movq %rsp, %rbp

And the postamble, which "undoes" the preamble:

    movq %rbp, %rsp
    pop %rbp
    movq $0, %rax
    ret

Now check the postamble to _main:

    movq %rbp, %rsp
    pop %rbp
    ret

Where's _main's preamble?

You're popping off the return address into rbp - assuming that rbp contained what you wanted in the previous instruction!

[Edit]

@RaymondChen makes a point about the fact that you are pushing the parameters onto the stack, but not explicitly cleaning them up. This is bad practice - but (luckily?) you're not affected by this because the stack is "fixed up" immediately after the call anyway.

When you push parameters onto the stack, something has to take them off again. There are only two ways this can be done: by the function itself as it returns, or by the caller after it is returned to.

For example, if you push four integers onto the stack, then call a function, the function could do a ret $4*8 to pop those four eight-byte registers. Your _myfunc function could do this.

The other alternative is used when the called function doesn't know how many parameters have been pushed, like printf() doesn't. Your example uses registers rather than the stack for the arguments to printf(), but there are times when the stack is used - especially if there are a lot of parameters! If four parameters for printf() are pushed onto the stack, printf can't do a ret $4*8 because it doesn't know how many there were. So instead, the caller (your _myfunc) should add $4 * 8 to rsp straight after the return.

Like I said, the next instruction is "move rbp to rsp" anyway, so the add is (strictly) unnecessary - but good practice nevertheless!

Assembly Hello world program segmentation fault

The problem

System V ABI requires you to align your stack at 16-byte before you call a function. In order to make it easy, the ABI guarantees that "On function entry, if you sub your stack pointer with 8 * n (n is an odd number), your stack will be 16-byte aligned".

If you don't follow this calling convention, other libraries may crash because they can't align their stack frame properly if they need to use instructions that need special alignment, like movdqa for example.

Solution

ammarfaizi2@integral:/tmp/test_asm$ cat test.S
.intel_syntax noprefix
    
.data

    message: .asciz "Hello World!\n"

.text

.global main

main:
    sub rsp, 8
    xor eax, eax
    lea rdi, [rip + message]
    call printf
    add rsp, 8
    ret
ammarfaizi2@integral:/tmp/test_asm$ gcc test.S -o test
ammarfaizi2@integral:/tmp/test_asm$ ./test
Hello World!
ammarfaizi2@integral:/tmp/test_asm$ 

Recommendation

If you call a function and the next thing you do is ret, you can simplify the code with tail call. It uses jmp to the target function to be called. Make sure you undo the current function stack frame before jump if you setup it before.

To support PIE and PIC, consider to use RIP relative addressing to access static storage. It also improves security. Compilers these days usually compile the target to PIE by default.

This part is the example of accessing static storage with RIP relative addressing:

lea rdi, [rip + message]

Execution

ammarfaizi2@integral:/tmp/test_asm$ cat test.S
.intel_syntax noprefix
    
.data

    message: .asciz "Hello World!\n"

.text

.global main

main:
    xor eax, eax
    lea rdi, [rip + message]
    jmp printf

ammarfaizi2@integral:/tmp/test_asm$ gcc test.S -o test
ammarfaizi2@integral:/tmp/test_asm$ ./test
Hello World!
ammarfaizi2@integral:/tmp/test_asm$ 

Edit

Added xor eax, eax for safety. See: glibc scanf Segmentation faults when called from a function that doesn't align RSP

Related Topics