How can I use ftrace to get the in-kernel call graph called by a system call?
First you need to get the function name right - e.g. the function name to use for tracing open
syscalls is sys_open
.
To do this the "proper" way, it's necessary to have function_graph
support in the kernel. On the x86 architecture this depends on CC_OPTIMIZE_FOR_SIZE being disabled, but on x86_64 it doesn't.
In my case I didn't bother to compile a custom kernel to disable CC_OPTIMIZE_FOR_SIZE, I just did
trace-cmd record -p function --func-stack
and included various functions that looked like they might be called by adding several -l
options. This was enough to figure out what I wanted to know.
How to capture all generated events with ftrace without any loss
I solved this problem by also capturing the event syscalls:sys_enter_execve
. Between the two of them I was able to get every instance of exec called.
Does ftrace allow capture of system call arguments to the Linux kernel, or only function names?
I have limited experience with ftrace
, although I have used it for for function stack traces and latency issues. (People with more experience can possibly suggest) Its pretty much the same experience using trace-cmd
and kernelshark
.
However, if you want to trace syscalls, function params, kernel APIs and return values etc. within the kernel space a better choice would be to go with systemtap
. It has an extensive list of Samples & Doc which is good for function call tracing, argument values passed etc. You may want to look at some samples and taylor them to your requirement. See general/para-callgraph-verbose.stp
and process/sleeptime.stp
"
general/para-callgraph-verbose.stp - Callgraph Tracing with Verbose Arguments
keywords: TRACE CALLGRAPH
Print a timed per-thread microsecond-timed callgraph, complete with pretty-printed function parameters and return values. The first parameter names the function probe points to trace. The optional second parameter names the probe points for trigger functions, which acts to enable tracing for only those functions that occur while the current thread is nested within the trigger.
stap para-callgraph-verbose.stp 'kernel.function("*@fs/proc*.c")' \
'kernel.function("vfs_read")' -c "cat /proc/sys/vm/* || true"
process/strace.stp - Trace system calls
keywords: _BEST PROCESS SYSCALL
The script loosely emulates strace, when applied to individual processes or hierarchies (via -c/-x), or the entire system (without -c/-x). A few output configuration parameters may be set with -G.
stap strace.stp -c "sleep 1"
"
Note you will need to install the correct version of the debug kernel
and kernel-devel
rpms/deb for stap
to work correctly. For this just use stap-prep
and install the dependencies shown depending on the flavour you are on.
Related Topics
Using Bash Script to Feed Input to Command Line
How to *Only* Get the Number of Bytes Available on a Disk in Bash
Iterating Over Each Line of Ls -L Output
Setting a Gdb Exit Breakpoint Not Working
Using 'Find -Perm' to Find When a Permission Is Not Set
Avrisp Mkii Doesn't Work with Avrdude on Linux
Linux Mint - Adding Environment Variables Permanently
How to Analyse a Crash Dump File Using Gdb
What Is The Equivalent of "Aptitude" in MAC Osx
How to Debug a Futex Contention Shown in Strace
How to Create a Statically Linked Position Independent Executable Elf in Linux
Counting Number of Directories in a Specific Directory
Walking Page Tables of a Process in Linux
How to Find Files Modified in Last X Minutes (Find -Mmin Does Not Work as Expected)
Accurately Calculating CPU Utilization in Linux Using /Proc/Stat
Where to Start Learning About Linux Dma/Device Drivers/Memory Allocation