Kernel Level Thread Library

How user level thread talk with kernel level thread

Kernel threads are like a specialized task responsible for doing a specific operation (not meant to last long). They are not threads waiting for incoming request from user-land threads. Moreover, a system call does not systematically create a kernel thread (see this post for more information and this one for some context about system calls): a kernel thread is started when a background task is required like for dealing with IO requests for example (this post shows a good practical case though the description is a bit deep). Basic system calls just runs in the same user-thread but with higher privileges. Note the kernel functions use a dedicated kernel stack: each user-level thread has 2 stacks on Linux: one for user-land functions and one for kernel-land functions (for sake of security).

As a result, in practice, I think all answers are wrong in usual cases (ie. assuming the target operation do not require to create a kernel thread). If the target system calls done actually require to create kernel threads, then b) is the correct answer. Indeed, kernel threads are like a one-shot specialized task as previously stated. Creating/Destroying new kernel-threads is not much expensive since it is just basically a relatively lightweight task_struct data structure internally.

What exactly does it mean to schedule user-level thread to run on an available LWP?

Just like a process is a container for memory, the LWP (= kernel-level thread) is a container for fibers (= user-level threads, essentially).

The kernel's thread scheduler only sees kernel-level threads (LWPs), and it schedules LWPs on and off the CPUs - namely, an LWP has a time slice where it gets to run on a CPU. The user-level thread library (= fiber scheduler) owns the LWPs of that process, and it decides which fibers get to use the timeslices allocated by the kernel's scheduler to those LWPs.

When a fiber decides to yield the CPU, but the LWP's timeslice is not over yet, the fiber scheduler schedules another fiber to run within the LWP on that CPU. But while that other fiber is running, the LWP's timeslice might run out, and the kernel's scheduler will schedule that LWP off the CPU. The fiber scheduler will not have a say on the matter - the fiber scheduler won't even get to run, because it's in userspace and the kernel is not aware of it.

Would it makes the kernel level thread clearly preferable to user level thread if system calls is as fast as procedure calls?

Some web searching results told me that the only deficiency of kernel-level thread is the slow speed of its management(create, switch, terminate, etc.).

It's not that simple. To understand, think about what causes task switches. Here's a (partial) list:

• a device told a device driver that an operation completed (some data arrived, etc) causing a thread that was waiting for the operation to unblock and then preempt the currently running thread. For this case you're running kernel code when you find out that a task switch is needed, so kernel task switching is faster.

• enough time passed; either causing an "end of time slice" task switch, or causing a sleeping thread to unblock and preempt. For this case you're running kernel code when you find out that a task switch is needed, so kernel task switching is faster.

• the thread accessed virtual memory that isn't currently accessible, triggering the kernel's page fault handler which finds out that the current task has to wait while the kernel fetches data from from swap space or from a file (if the virtual memory is part of a memory mapped file), or has to wait for kernel to free up RAM by sending other pages to swap space (if virtual memory was involved in some kind of "copy on write"); causing a task switch because the currently running task can't continue. For this case you're running kernel code when you find out that a task switch is needed, so kernel task switching is faster.

• a new process is being created, and its initial thread preempts the currently running thread. For this case you're running kernel code when you find out that a task switch is needed, so kernel task switching is faster.

• the currently running thread asked kernel to do something with a file and kernel got "VFS cache miss" that prevents the request from being performed without any task switches. For this case you're running kernel code when you find out that a task switch is needed, so kernel task switching is faster.

• the currently running thread releases a mutex or sends some data (e.g. using a pipe or socket); causing a thread that belongs to a different process to unblock and preempt. For this case you're running kernel code when you find out that a task switch is needed, so kernel task switching is faster.

• the currently running thread releases a mutex or sends some data (e.g. using a pipe or socket); causing a thread that belongs to the same process to unblock and preempt. For this case you're running user-space code when you find out that a task switch is needed, so in theory user-space task switching is faster, but in practice it can just as easily be an indicator of poor design (using too many threads and/or far too much lock contention).

• a new thread is being created for the same process; and the new thread preempts the currently running thread. For this case you're running user-space code when you find out that a task switch is needed, so in user-space task switching is faster; but only if kernel isn't informed (e.g. so that utilities like "top" can properly display details for threads) - if kernel is informed anyway then it doesn't make much difference where the task switch happens.

For most software (which doesn't use very many threads); doing task switches in the kernel is faster. Of course it's also (hopefully) fairly irrelevant for performance (because time spent switching tasks should be tiny compared to time spend doing other work).

And I always have an instinct that such management should be done by the OS automatically because only OS knows which thread would be suitable to run at a specific time.

Yes; but possibly not for the reason you think.

Another problem with user-space threading (besides making most task switches slower) is that it can't support global thread priorities without becoming a severe security disaster. Specifically; a process can't know if its own thread is higher or lower priority than a thread belonging to a different process (unless it has information about all threads for the entire OS, which is information that normal processes shouldn't be trusted to have); so user-space threading leads to wasting CPU time doing unimportant work (for one process) when there's important work to do (for a different process).

Another problem with user-space threading is that (for some CPUs - e.g. most 80x86 CPUs) the CPUs are not independent, and there may be power management decisions involved with scheduling. For examples; most 80x86 CPUs have hyper-threading (where a core is shared by 2 logical processors), where a smart scheduler may say "one logical processor in the core is running a high priority/important thread, so the other logical processor in the same core should not run a low priority/unimportant thread because that would make the important work slower"; most 80x86 CPUs have "turbo boost" (with similar "don't let low priority threads ruin the turbo-boost/performance of high priority thread" possibilities); and most CPUs have thermal management (where scheduler might say "Hey, these threads are all low priority, so let's underclock the CPU so that it cools down and can go faster later (has more thermal headroom) when there's high priority/more important work to do!").

Would it makes the kernel level thread clearly preferable to user level thread if system calls is as fast as procedure calls?

If system calls were as fast as normal procedure calls, then the performance differences between user-space threading and kernel threading would disappear (but all the other problems with user-space threading would remain). However, the reason why system calls are slower than normal procedure calls is that they pass through a kind of "isolation barrier" (that isolates kernel's code and data from malicious user-space code); so to make system calls as fast as normal procedure calls you'd have to get rid of the isolation (effectively turning the kernel into a kind of "global shared library" that can be dynamically linked) but without that isolation you'll have an extreme security disaster. In other words; to have any hope of achieving acceptable security, system calls must be slower than normal procedure calls.

Is Pthread library actually a user thread solution?

Q: I understand Pthread is a thread library meeting POSIX standard

A: Yes. Actually, "Pthreads" stands for "Posix threads":
http://en.wikipedia.org/wiki/Pthreads

Q: It is available in Unix-like OS.

A: Actually, it's available for many different OSs ... including Windows, MacOS ... and, of course, Linux, BSD and Solaris.

Q: About thread, I read that there are three different models

Now you're getting fuzzy. "Threads" is a very generic term. There are many, many different models. And many, many different ways you can characterize and/or implement "threads". Including stuff like the Java threading model, or the Ada threading model.

Q: When I call pthread_create() to create a thread, did I create a
user level thread?

A: Yes: Just about everything you do in user space is "protected" in your own, private "user space".

Q: User level thread: the kernel does not know it.

A: No. The kernel knows everything :)

Q: Kernel level thread: kernel directly supports multiple threads of
control in a process.

A: Yes, there is such a thing as "kernel threads".

And, as it happens, Linux makes EXTENSIVE use of kernel threads. For example, every single process in a Linux system is a "kernel thread". And every user-created pthread is ALSO implemented as a new "kernel thread". As are "worker threads" (which are completely invisible to any user-level process).

But this is an advanced topic you do NOT need to understand in order to effectively use pthreads. Here's a great book that discussed this - and many other topics - in detail:

[Linux Kernel Development, Robert Love][1]

Remember: "Pthreads" is an interface. How it's implemented depends on the platform. Linux uses kernel threads; Windows uses Win32 threads, etc.

===========================================================================
ADDENDUM:

Since people still seem to be hitting this old thread, I thought it would be useful to reference this post:

https://stackoverflow.com/a/11255174/421195

Linux typically uses two implementations of pthreads:
LinuxThreads and Native
POSIX Thread Library(NPTL),
although the former is largely obsolete. Kernel from 2.6 provides
NPTL, which provides much closer conformance to SUSv3, and perform
better especially when there are many threads.

You can query the
specific implementation of pthreads under shell using command:

getconf GNU_LIBPTHREAD_VERSION

You can also get a more detailed implementation difference in The
Linux Programming Interface.

"Pthreads" is a library, based on the Posix standard. How a pthreads library is implemented will differ from platform to platform and library to library.
[1]: http://www.amazon.com/Linux-Kernel-Development-Robert-Love/dp/0672329468

---

Related Topics