Origin of a Kworker-Thread

Origin of a kworker-thread

(Seems to me this is rather off topic here, but here's the answer I posted on unix.stackexchange.com.)

I found this thread on lkml that answers your question a little. (It seems even Linus himself was puzzled as to how to find out the origin of those threads.)

Basically, there are two ways of doing this:

$ echo workqueue:workqueue_queue_work > /sys/kernel/debug/tracing/set_event
$ cat /sys/kernel/debug/tracing/trace_pipe > out.txt
(wait a few secs)

For this you will need ftrace to be compiled in your kernel.

This will output what threads are all doing, and is useful for tracing multiple small jobs.

cat /proc/THE_OFFENDING_KWORKER/stack

This will output the stack of a single thread doing a lot of work. It may allow you to find out what caused this specific thread to hog the CPU (for example). THE_OFFENDING_KWORKER is the pid of the kworker in the process list.

how many ways we can create a process in linux using c

exec family of system call does not call fork, neither it creates a new process.
It only overwrites the existing process with the new binary.

In linux user programs, fork is the only function to create new process. Though fork internally calls clone and other system calls.

In other hands, system is only a wrapper to fork and exec. The actual task of creating a process is done by fork in system. So system is not a way to create new process.

What does kernel tainted mean?

To (over) simplify, 'tainted' means that the kernel is in a state other than what it would be in if it were built fresh from the open source origin and used in a way that it had been intended. It is a way of flagging a kernel to warn people (e.g., developers) that there may be unknown reasons for it to be unreliable, and that debugging it may be difficult or impossible.

In this case, 'GD' means that all modules are licensed as GPL or compatible (ie not proprietary), and that a crash or BUG() occurred.

The reasons are listed below:

See: oops-tracing.txt

---------------------------------------------------------------------------
Tainted kernels:

Some oops reports contain the string 'Tainted: ' after the program
counter. This indicates that the kernel has been tainted by some
mechanism.  The string is followed by a series of position-sensitive
characters, each representing a particular tainted value.

  1: 'G' if all modules loaded have a GPL or compatible license, 'P' if
     any proprietary module has been loaded.  Modules without a
     MODULE_LICENSE or with a MODULE_LICENSE that is not recognised by
     insmod as GPL compatible are assumed to be proprietary.

  2: 'F' if any module was force loaded by "insmod -f", ' ' if all
     modules were loaded normally.

  3: 'S' if the oops occurred on an SMP kernel running on hardware that
     hasn't been certified as safe to run multiprocessor.
     Currently this occurs only on various Athlons that are not
     SMP capable.

  4: 'R' if a module was force unloaded by "rmmod -f", ' ' if all
     modules were unloaded normally.

  5: 'M' if any processor has reported a Machine Check Exception,
     ' ' if no Machine Check Exceptions have occurred.

  6: 'B' if a page-release function has found a bad page reference or
     some unexpected page flags.

  7: 'U' if a user or user application specifically requested that the
     Tainted flag be set, ' ' otherwise.

  8: 'D' if the kernel has died recently, i.e. there was an OOPS or BUG.

  9: 'A' if the ACPI table has been overridden.

 10: 'W' if a warning has previously been issued by the kernel.
     (Though some warnings may set more specific taint flags.)

 11: 'C' if a staging driver has been loaded.

 12: 'I' if the kernel is working around a severe bug in the platform
     firmware (BIOS or similar).

 13: 'O' if an externally-built ("out-of-tree") module has been loaded.

 14: 'E' if an unsigned module has been loaded in a kernel supporting
     module signature.

 15: 'L' if a soft lockup has previously occurred on the system.

 16: 'K' if the kernel has been live patched.

The primary reason for the 'Tainted: ' string is to tell kernel
debuggers if this is a clean kernel or if anything unusual has
occurred.  Tainting is permanent: even if an offending module is
unloaded, the tainted value remains to indicate that the kernel is not
trustworthy.

Linux(Ubuntu) load average higher than total-true-utilization?

There does not appear to be anything wrong with your system. What is wrong seems to be your understanding of CPU accounting. In particular, load average has nearly nothing at all to do with CPU usage. Load average is based on the number of processes that are ready to run (not waiting on I/O, network, keyboard input, etc...), if there is an available CPU for them to be scheduled on. While it's true that, given an 8 core system, if all 8 cores are 100% busy with a single CPU-bound thread each, your load average should be around 8.00, it is entirely possible to have a load average of 200.0 with near-0% CPU utilization. All that would indicate is you have 200 processes that are ready to run, but as soon as they get scheduled, they do almost nothing before they go back to waiting for input of some sort.

Your top output shows that vlc seems to be using roughly the equivalent of 5 of your cores, but it doesn't indicate whether you have 5 cores at 100% each, or if all 8 cores are at 62.5% each. All of the other processes listed by top also contribute to your load average, as well as CPU usage. In particular, top running with a short delay like your example of 0.1 seconds, will probably increase your load average by almost 1 itself, even though, overall, it's not using a lot of CPU time.

How to get the start time of a long-running Linux process?

You can specify a formatter and use lstart, like this command:

ps -eo pid,lstart,cmd

The above command will output all processes, with formatters to get PID, command run, and date+time started.

Example (from Debian/Jessie command line)

$ ps -eo pid,lstart,cmd
  PID CMD                                          STARTED
    1 Tue Jun  7 01:29:38 2016 /sbin/init                  
    2 Tue Jun  7 01:29:38 2016 [kthreadd]                  
    3 Tue Jun  7 01:29:38 2016 [ksoftirqd/0]               
    5 Tue Jun  7 01:29:38 2016 [kworker/0:0H]              
    7 Tue Jun  7 01:29:38 2016 [rcu_sched]                 
    8 Tue Jun  7 01:29:38 2016 [rcu_bh]                    
    9 Tue Jun  7 01:29:38 2016 [migration/0]               
   10 Tue Jun  7 01:29:38 2016 [kdevtmpfs]                 
   11 Tue Jun  7 01:29:38 2016 [netns]                     
  277 Tue Jun  7 01:29:38 2016 [writeback]                 
  279 Tue Jun  7 01:29:38 2016 [crypto]                    
      ...

You can read ps's manpage or check Opengroup's page for the other formatters.

Could the affinity of thread created by kernel be set by cpuset?

cpuset(7) is a manual page that describes a Linux userspace API in general. As the page states, you can use the sched_setaffinity(2) syscall to restrict a task to a specific set of CPUs.

The fact that sched_setaffinity(2) is a syscall should already make you notice that the functionality is intended for userspace usage. If you are writing kernel code, kernel threads have different internal APIs for this purpose (see kthread.h):

• kthread_bind(), which can be used to bind the kthread to a single CPU specified by its numerical ID.
• kthread_bind_mask(), which can be used to bind the kthread to one or more CPUs defined by a struct cpumask. You can initialize the right struct cpumask through cpumask_set_cpu(). This API is similar to the sched_setaffinity(2) syscall, but for kthreads.

---

Related Topics