How to Manipulate Page Cache in Linux

How to manipulate page cache in Linux?

Firstly, the kernel does not maintain a master list of all files in the page cache, because it has no need for such information. Instead, given an inode you can look up the associated page cache pages, and vice-versa.

For each page cache struct page, page_mapping() will return the struct address_space that it belongs to. The host member of struct address_space identifies the owning struct inode, and from there you can get the inode number and device.

Page Cache and Mode Switch

A simplified explanation:

The page mapping does not change when you change to kernel mode. However, the kernel's own memory space becomes accessible (due to the ring change). When in kernel mode, one can still access the userspace memory of the process. Therefore for standard I/O calls there is nothing to do - userspace can be accessed directly. However, in many circumstances this is not a good idea as the pointers passed may point to unmapped memory, or paged out memory, or disappear half-way through the call. Hence normally copy_to_user and copy_from_user are used.

There may be some system calls that do change the memory map. For instance, fork() creates CoW copies of the page map. exec and friends remap pages to an executable on disk. These are however the exception.

Also, a system call may context switch. For instance sleep() is almost guaranteed to. However, that is not for the purposes of accessing the calling program's memory.

Page cache vs L1 cache?

Yes, those are known as hardware/CPU caches. They store "cache lines" which are usually 64 bytes and are pieces of memory pulled from RAM and stored for closer/faster access for the CPU. This is usually behind the scenes, since its handled in hardware, you generally can't modify it nor does the OS have direct access to its contents. However, you can flush the contents using flush_cache_ family of functions.

The purpose of the page cache is similar: to speed up future accesses by storing memory that might be used soon. It deals with memory pages of size 4KB, normally, and holds memory loaded in from disk onto RAM. It is an OS-level cache. The Linux kernel has complete control over how the page cache works, which you can modify yourself in the source code. Take a look at this discussion if you want to know more about what is stored in the page cache.

How does Page Cache work in memory? Specifically in Linux

There are many operating systems out there, so if we're talking about any arbitrary real, future or theoretical operating system the answer could be "either".

However, for all real operating systems that I'm aware of, the page will only be in memory once. It would be very inefficient to create a copy of each page, and doing so would not create any benefit.

Linux memory management (caching)

In linux two caches were distinct: Files were in the page cache, disk blocks were in the buffer cache. Given that most files are represented by a filesystem on a disk, data was represented twice, once in each of the caches. Many Unix systems follow a similar pattern.

The buffer cache remains, however, as the kernel still needs to perform block I/O in terms of blocks, not pages. As most blocks represent file data, most of the buffer cache is represented by the page cache. But a small amount of block data isn't file backed—metadata and raw block I/O for example—and thus is solely represented by the buffer cache.

How to stop page cache for disk I/O in my linux system?

I'm not sure that changing something will affect overall performance.

Maybe you might use posix_fadvise(2) and sync_file_range(2) in your program (and more rarely fsync(2) or fdatasync(2) or sync(2) or syncfs(2), ...). Also look at madvise(2), mlock(2) and munlock(2), and of course mmap(2) and munmap(2). Perhaps ionice(1) could help.

In the reader process, you might perhaps use readhahead(2) (perhaps in a separate thread).

Upgrading your kernel (to a 3.6 or better) could certainly help: Linux has improved significantly on these points since 2.6.32 which is really old.

---

Related Topics