Why Can't We Use C Standard Library Functions in Kernel Development

Why can't we use C standard library functions in kernel development?

Because the GNU C Library which you are familiar with is implemented for user mode, not kernel mode. The kernel cannot access a userspace API (which might invoke a syscall to the Linux kernel).

From the KernelNewbies FAQ

Can I use library functions in the kernel ?

System libraries (such as glibc, libreadline, libproplist, whatever) that are typically available to userspace programmers are unavailable to kernel programmers. When a process is being loaded the loader will automatically load any dependent libraries into the address space of the process. None of this mechanism is available to kernel programmers: forget about ISO C libraries, the only things available is what is already implemented (and exported) in the kernel and what you can implement yourself.

Note that it is possible to "convert" libraries to work in the kernel; however, they won't fit well, the process is tedious and error-prone, and there might be significant problems with stack handling (the kernel is limited to a small amount of stack space, while userspace programs don't have this limitation) causing random memory corruption.

Many of the commonly requested functions have already been implemented in the kernel, sometimes in "lightweight" versions that aren't as featureful as their userland counterparts. Be sure to grep the headers for any functions you might be able to use before writing your own version from scratch. Some of the most commonly used ones are in include/linux/string.h.

Whenever you feel you need a library function, you should consider your design, and ask yourself if you could move some or all the code into user-space instead.

If you need to use functions from standard library, you have to re-implement that functionality because of a simple reason - there is no standard C library.

C library is basically implemented on the top of the Linux kernel (or other operating system's kernel).

For instance, C library's mkdir(3) function is basically nothing more than a wrapper for Linux kernel's system call mkdir(2).

http://linux.die.net/man/3/mkdir
http://linux.die.net/man/2/mkdir

Is shared standard C library first initialized by kernel?

You are wrong.

1.) There is no need to put libc into kernel. It doesn't affect any low-level system or hardware dependent components.

2.) libc.so is ordinary dynamic library.

Now some more details:

When you launch your application, f.e. from bash console, bash forks and execs new process. What does it mean. Actually, this means that OS creates address space environment and loads .text .data .bss from ELF file, preserves virtual space for stack. You can see this mappings here:

sudo cat /proc/1118/maps 
00400000-00407000 r-xp 00000000 08:01 1845158                            /sbin/getty
00606000-00607000 r--p 00006000 08:01 1845158                            /sbin/getty
00607000-00608000 rw-p 00007000 08:01 1845158                            /sbin/getty
00608000-0060a000 rw-p 00000000 00:00 0 
00ff3000-01014000 rw-p 00000000 00:00 0                                  [heap]
...
7f728efd3000-7f728efd5000 rw-p 001bf000 08:01 466797                     /lib/x86_64-linux-gnu/libc-2.19.so
7f728efd5000-7f728efda000 rw-p 00000000 00:00 0 
7f728efda000-7f728effd000 r-xp 00000000 08:01 466799                     /lib/x86_64-linux-gnu/ld-2.19.so

7f728f1fe000-7f728f1ff000 rw-p 00000000 00:00 0 
7fffa122b000-7fffa124c000 rw-p 00000000 00:00 0                          [stack]
7fffa1293000-7fffa1295000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

But there are more. After loading thoose segments, Linux kernel will also load ld-linux.so into memory (you can see it in mappings). This stuff called dynamic linker, and actually ld-linux is responsible for all dynamic libraries loading. As you might know, at the moment the application have been compiled, you already know the list of shared libraries you will use. You can check it via ldd command

ldd /sbin/getty 
linux-vdso.so.1 =>  (0x00007fff4cfa6000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2af2832000)
/lib64/ld-linux-x86-64.so.2 (0x00007f2af2c24000)

This stuff must be held somewhere in the ELF (don't know where exactly). So after loading, ld-linux uses this list and finds all needed libraries at predefined (standart) paths like /usr/lib and so on. Now ld-linux can just mmap regions for located dynamic libraries. That is how libc will be loaded to process address space.

Linux kernel: What kind of C Linux kernel is using?

C isn't built ontop of linux. C in itself is a compiled programming language, that a compiler translates into machine code. Based on your OS, the compiler may do it differently (for some C code).

But the language C itself really is just a very long list of things functions should do and how things should behave, and compilers just obey these rules. Thats what is called the C "standard". There is a comittee that sets it, and there are multiple versions.

Linux Kernel was indeed written in C. So someone wrote it and then compiled it using a standard-compliant C compiler.

As for libraries, they're optional. The Linux kernel is developed without dependencies, that means it implements everything it needs itself, in plain C. These includes you see are just files from the kernel itself, defining its functions, types etc.

How can third-party libraries do things that the standard library can't do?

How can third-party libraries do things that the standard library can't do?

At the lowest level, they use APIs provided by the operating system.

In most cases, there is typically a stack of libraries with increasing levels of abstraction. On top of operating system API, there is often a system libraries (such as glibc, libDRM). On top of system libraries, there is often a portable cross-platform abstraction (such as libuv, Mesa3D). On top of those, there are more complex libraries and frameworks that combine multiple interfaces (such as a web server framework, SDL).

Suppose I am a developer who wants to create a library that can draw shapes on the screen, how do I even make such a library without using any other third-party library?

Since using operating system API's is operating system specific, let's use Linux as an example.

You have a choice between multiple graphics API's. Perhaps most commonly used are the framebuffer device, and direct rendering manager.

However, if you want your program to be windowed with other graphical programs, then there must be co-ordination between the programs. This is the job of a display server (typically Xorg). Most graphical Linux programs do not interact with the graphics APIs directly but instead through such server. If you wish to not use a client library provided by the server, then you must re-implement their client side protocol.

Question about overriding C standard library functions and how to link everything together

If you are working with glibc you can use alternative non-overriden function names:

[max@truth ~]$ nm --defined-only --dynamic /lib64/libc.so.6 | egrep "malloc\b"
0000003e56079540 T __libc_malloc
0000003e56079540 T malloc

Note the same function address in the above. In other words, malloc() function is given two names, so that the original malloc() version is available under __libc_malloc() name in case malloc() has been interposed.

A quick grep on glibc sources reveals the only caller of __libc_malloc() is mcheck. These function aliases are a glibc implementation detail and there is no header for them. malloc/mcheck.c declares the internal functions as below:

extern __typeof (malloc) __libc_malloc;
extern __typeof (free) __libc_free;
extern __typeof (realloc) __libc_realloc;

Other C libraries may have differently named aliases, so using dlsym() to get the original malloc() address is more portable.

How does the C++ standard library work behind the scenes?

Does it use the OS specific API (since the library is cross platform), or what?

At some point, the OS specific API is used. The fstream implementation does not necessarily call an OS function directly. It might use other classes, which call functions inherited from C, etc., but eventually the call chain will lead to an OS call. (Yes, the details are often too complicated for an intermediate programmer to follow. So, as a self-described beginner, your findings are not surprising.)

The library is cross-platform in the sense that on your end (the C++ programmer), the interface is the same regardless of platform. It is not, however, the same library on every platform. Each platform has its own library, exposing the same interface on the C++ side, but making use of different OS calls. (In fact, the same platform might have multiple standard libraries, as the library implementation is provided by your toolchain, not by the standards committee.)

And, if the standard library can do it, aren't I supposed to be able to mess with some files as well without calling the standard fstream file (which to my experience I can't do)?

Yes, you are allowed to. Apparently, you have not been able to yet, but with some practice and guidance you should be able to. Everything in the standard library can be recreated in your own code. The point of the standard library (and most libraries, for that matter) is to save you time, not to enable something that was otherwise unavailable. For example, you don't have to implement a file stream for every program you write; it's in the standard library so you can focus on more interesting aspects of your project.

GCC linker does not link standard library

You cannot use the C library for a kernel as it is build for an existing kernel and relies on the syscalls of its target OS. Instead, you have to write a driver for keyboards and everything else you need to get characters from anywhere. getc() is a very advanced function from that point of view and you should consider making the basic functions of the kernel stable before programming anything to interact with.

By the way, you should really build a cross compiler. It has many advantages over feeding the system compiler with awkward options. After all, the kernel is meant to run on different machines, so it should be compiled for bare x86, which is what a cross-compiler does.

Keep coding this thing!
leitimmel

CUDA: calling library function in kernel

Yes, the only way to use stdlib's functions from kernel is to reimplement them. But I strongly advice you to reconsider this idea, since it's highly unlikely you would need to run code that uses strcmp() on GPU. Please, add additional details about your problem, so a better solution could be proposed (I highly doubt that serial string comparison on GPU is what you really need).

It's barely possible to simply recompile all stdlib for GPU, since it depends a lot on some system calls (like memory allocation), which could not be used on GPU (well, in recent versions of CUDA toolkit you can allocate device memory from kernel, but it's not "cuda-way", is supported only by newest hardware and is very bad for performance).
Besides, CPU versions of most functions is far from being "good" for GPUs. So, in vast majority of cases compiling your ordinary CPU functions for GPU would lead to no good, so the compiler doesn't even try it.

---

Related Topics