How to Enable or Configure Ftrace Module

How to Enable or configure ftrace module

Add below kernel config options in your yocto project conf/local.conf file & then clean & rebuild linux kernel.

KERNEL_CONFIG_DYNAMIC_FTRACE="y"
KERNEL_CONFIG_DEBUG_FS="y"
KERNEL_CONFIG_FTRACE="y"
KERNEL_CONFIG_FUNCTION_TRACER="y"
KERNEL_CONFIG_FUNCTION_GRAPH_TRACER="y"
KERNEL_CONFIG_IRQSOFF_TRACER="y"
KERNEL_CONFIG_PREEMPT_TRACER="y"
KERNEL_CONFIG_SCHED_TRACER="y"
KERNEL_CONFIG_STACK_TRACER="y"
KERNEL_CONFIG_BLK_DEV_IO_TRACE="y"
KERNEL_CONFIG_FUNCTION_PROFILER="y"
KERNEL_CONFIG_FTRACE_MCOUNT_RECORD="y"

Booting with this kernel shall have ftrace enabled

How to make a linux kernel function available to ftrace function_graph tracer?

The problem is that those functions are annotated with __init and __devinit, which are black listed by ftrace function tracer.

Why? Because as module init functions (or kernel init functions) they are loaded during initialization and removed when the initialization is complete. Every function that ftrace traces is kept in a special compact table. Currently, there's no way to tell ftrace that those functions have been removed (freed) and that ftrace should remove them from its table. If we were to just ignore that, then when function tracing is enabled, ftrace will try to modify locations that no longer exist and can cause all sorts of issues (remember the e1000e bug?).

If you really want to trace them, then remove those annotations. Then they should appear in the list of functions to trace.

How to make a linux kernel function available to ftrace?

The functions that i wanted to measure were in a suspend resume flow.
And apparently suspend & resume are very problematic for tracing functions(dynamic tracing), and ftrace disables the tracing during this process.
So for now you cannot trace those functions.

How to use ftrace for tcp probe?

tldr;

Unfortunately, I could not find any way to get TCP tracepoints to work in Mininet, which is what ftrace would uses. The reason for this is that the mininet's /sys/kern/debug directory is empty, i.e., tracing cannot be enabled.

Options:

1. Using mininet-tracing (not recommended)

There probably is a way to get the kernel to include this, or you could use https://github.com/mininet/mininet-tracing which might get you what you need, but I have seen reports that it is slow, and has been updated 9 years ago...

2. Writing a new kernel module (I have tested this and it works)

What I found as a solution instead, was to force printing for the TCP I had in mind and then take a look at the results that way. In order to enable this, you would essentially need to extend some of TCP's behaviour and (quite possibly) reuse the TCP module you have in mind. And create a new kernel module.

Here I have provided an example that you can use. It logs socket information on each ACK. I also included a Makefile and a script to load/unload the kernel module. After you enable the module and let some traffic flow (assuming you are on a debian-based linux) you should be able to find the logs of your TCP in /var/log/kern.log.

Note:
This is a hacky way around the issue, but was good enough for my needs, and hopefully can help someone else too.

---

Related Topics