How to Stop Page Cache for Disk I/O in My Linux System

How to stop page cache for disk I/O in my linux system?

I'm not sure that changing something will affect overall performance.

Maybe you might use posix_fadvise(2) and sync_file_range(2) in your program (and more rarely fsync(2) or fdatasync(2) or sync(2) or syncfs(2), ...). Also look at madvise(2), mlock(2) and munlock(2), and of course mmap(2) and munmap(2). Perhaps ionice(1) could help.

In the reader process, you might perhaps use readhahead(2) (perhaps in a separate thread).

Upgrading your kernel (to a 3.6 or better) could certainly help: Linux has improved significantly on these points since 2.6.32 which is really old.

Disabling disk cache in linux

You need root access to do this. You can run hdparm -W 0 /dev/sda command to disable write caching, where you have to replace /dev/sda with device for your drive:

#include <stdlib.h>
...
system("hdparm -W 0 /dev/sda1");

You can also selectively disable write caching to individual partitions like this: hdparm -W 0 /dev/sda1.

To reenable caching again just use the -W 1 argument.

man hdparm, man system

How to purge disk I/O caches on Linux?

Sounds like you want the sync command, or the sync() function.

If you want disk cache flushing: echo 3 | sudo tee /proc/sys/vm/drop_caches

Prevent backup reads from getting into linux page cache

if you re using rsync there is the flag --drop-cache according to this question
the nocache utility which

minimize the effect an application has on the Linux file system cache
Use case: backup processes that should not interfere with the present state of the cache.

using dd there is direct I/O to bybass cache according to this question
the dd also has the option nocache option check the command info coreutils 'dd invocation'for details

Telling Linux not to keep a file in the cache when it is written to disk

Can I tell the OS not to keep parts of the large file in cache ?

Yes, you probably want to use some system call like posix_fadvise(2) or madvise(2). In weird cases, you might use readahead(2) or userfaultfd(2) or Linux-specific flags to mmap(2). Or very cleverly handle SIGSEGV (see signal(7), signal-safety(7) and eventfd(2) and signalfd(2)) You'll need to write your C program doing that.

But I am not sure that it is worth your development efforts. In many cases, the behavior of a recent Linux kernel is good enough.

Most of the time, it is simply not worth the effort to explicitly code something to manage your page cache.

I guess that mount(2) options in your /etc/fstab file (see fstab(5)...) are in practice more important. Or changing or tuning your file system (e.g. ext4(5), xfs(5)..). Or read(2)-ing in large pieces (1Mbytes).

Play with dd(1) to measure. See also time(7)

Most applications are not disk-bound, and for those who are disk bound, renting more disk space is cheaper that adding and debugging extra code.

don't forget to benchmark, e.g. using strace(1) and time(1)

PS. Don't forget your developer costs. They often are a lot above the price of a RAM module (or of some faster SSD disk).

Are file reads served from dirtied pages in the page cache?

The file read will fetch data from page cache without writing to disk. From Linux Kernel Development 3rd Edition by Robert Love:

Whenever the kernel begins a read operation—for example, when a
process issues the read() system call—it first checks if the requisite
data is in the page cache. If it is, the kernel can forgo accessing
the disk and read the data directly out of RAM.This is called a cache
hit. If the data is not in the cache, called a cache miss, the kernel
must schedule block I/O operations to read the data off the disk.

Writeback to disk happens periodically, separate from read:

The third strategy, employed by Linux, is called write-back. In a
write-back cache, processes perform write operations directly into the
page cache.The backing store is not immediately or directly updated.
Instead, the written-to pages in the page cache are marked as dirty
and are added to a dirty list. Periodically, pages in the dirty list
are written back to disk in a process called writeback, bringing the
on-disk copy in line with the inmemory cache.

How to Stop Page Cache for Disk I/O in My Linux System