Does the Linux Filesystem Cache Files Efficiently

Does the Linux filesystem cache files efficiently?

Yes, if you do not modify the file each time you open it.

Linux will hold the file's information in copy-on-write pages in memory, and "loading" the file into memory should be very fast (page table swap at worst).

Edit: Though, as cdhowie points out, there is no 'linux filesystem'. However, I believe the relevant code is in linux's memory management, and is therefore independent of the filesystem in question. If you're curious, you can read in the linux source about handling vm_area_struct objects in linux/mm/mmap.c, mainly.

Does Linux cache (small) files to optimize I/O?

Yes. If you start a program like htop and observe the yellow part of the memory usage, this is the amount of memory currently being used for disk cache. However, accessing the file will result in a disk-write to update the access time of that file, this can be disabled by adding the "noatime" option to the relevant partition line in /etc/fstab

Which is faster/better for caching, File System or Memcached?

Memcached is faster, but the memory is limited. HDD is huge, but I/O is slow compared to memory. You should put the hottest things to memcached, and all the others can go to cache files.

(Or man up and invest some money into more memory like these guys :)

For some benchmarks see: Cache Performance Comparison (File, Memcached, Query Cache, APC)

In theory:

Read 1 MB sequentially from memory       250,000 ns
Disk seek                             10,000,000 ns

http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf

Telling Linux not to keep a file in the cache when it is written to disk

Can I tell the OS not to keep parts of the large file in cache ?

Yes, you probably want to use some system call like posix_fadvise(2) or madvise(2). In weird cases, you might use readahead(2) or userfaultfd(2) or Linux-specific flags to mmap(2). Or very cleverly handle SIGSEGV (see signal(7), signal-safety(7) and eventfd(2) and signalfd(2)) You'll need to write your C program doing that.

But I am not sure that it is worth your development efforts. In many cases, the behavior of a recent Linux kernel is good enough.

Most of the time, it is simply not worth the effort to explicitly code something to manage your page cache.

I guess that mount(2) options in your /etc/fstab file (see fstab(5)...) are in practice more important. Or changing or tuning your file system (e.g. ext4(5), xfs(5)..). Or read(2)-ing in large pieces (1Mbytes).

Play with dd(1) to measure. See also time(7)

Most applications are not disk-bound, and for those who are disk bound, renting more disk space is cheaper that adding and debugging extra code.

don't forget to benchmark, e.g. using strace(1) and time(1)

PS. Don't forget your developer costs. They often are a lot above the price of a RAM module (or of some faster SSD disk).

Which is faster/better for caching, File System or Memcached?

For some benchmarks see: Cache Performance Comparison (File, Memcached, Query Cache, APC)

In theory:

Read 1 MB sequentially from memory       250,000 ns
Disk seek                             10,000,000 ns

http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf

Prevent backup reads from getting into linux page cache

if you re using rsync there is the flag --drop-cache according to this question
the nocache utility which

minimize the effect an application has on the Linux file system cache
Use case: backup processes that should not interfere with the present state of the cache.

using dd there is direct I/O to bybass cache according to this question
the dd also has the option nocache option check the command info coreutils 'dd invocation'for details

Write-through RAM disk, or massive caching of file system?

By default, Linux will use free RAM (almost all of it) to cache disk accesses, and will delay writes. The heuristics used by the kernel to decide the caching strategy are not perfect, but beating them in a specific situation is not easy. Also, on journalling filesystems (i.e. all the default filesystems nowadays), actual writes to the disk will be performed in a way which is resilient the crashes; this implies a bit of overhead. You may want to try to fiddle with filesystem options. E.g., for ext3, try mounting with data=writeback or even async (these options may improve filesystem performance, at the expense of reduced resilience towards crashes). Also, use noatime to reduce filesystem activity.

Programmatically, you might also want to perform disk accesses through memory mappings (with mmap). This is a bit hand-on, but it gives more control about data management and optimization.

How can I show size of the filesystem cache that has to be synced?

/proc/meminfo can tell you this information:

cat /proc/meminfo | grep -e Dirty -e Writeback

According to kernel documentation,

Dirty
              Memory which is waiting to get written back to the disk
Writeback
              Memory which is actively being written back to the disk

I don't think there is a way to determine how much Dirty or Writeback memory is specific to a device though.

Linux file system automatically backed by disk but hosted entirely in memory?

Why don't you try to create a RAID mirror between a ramdisk and a physical disk ?

Not sure if it's efficient though. If the mirror must always be synchronized, it will have to wait for the disk anyway when you write, but for reading you should gain something.
But yeah, to me it looks very much a complicated, wheel reinvented square IO caching :)

Would be a nice experiment, though.

Does the Linux Filesystem Cache Files Efficiently