Write-Through Ram Disk, or Massive Caching of File System

Write-through RAM disk, or massive caching of file system?

By default, Linux will use free RAM (almost all of it) to cache disk accesses, and will delay writes. The heuristics used by the kernel to decide the caching strategy are not perfect, but beating them in a specific situation is not easy. Also, on journalling filesystems (i.e. all the default filesystems nowadays), actual writes to the disk will be performed in a way which is resilient the crashes; this implies a bit of overhead. You may want to try to fiddle with filesystem options. E.g., for ext3, try mounting with data=writeback or even async (these options may improve filesystem performance, at the expense of reduced resilience towards crashes). Also, use noatime to reduce filesystem activity.

Programmatically, you might also want to perform disk accesses through memory mappings (with mmap). This is a bit hand-on, but it gives more control about data management and optimization.

Which is faster/better for caching, File System or Memcached?

Memcached is faster, but the memory is limited. HDD is huge, but I/O is slow compared to memory. You should put the hottest things to memcached, and all the others can go to cache files.

(Or man up and invest some money into more memory like these guys :)

For some benchmarks see: Cache Performance Comparison (File, Memcached, Query Cache, APC)

In theory:

Read 1 MB sequentially from memory       250,000 ns
Disk seek                             10,000,000 ns

http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf

Saving files programatically in ImDisk's RAM DISK is not faster

When you write to a file, it does not yet mean that the data is actually written to a disk. Operating Systems typically use buffer cache to defer writes to the device.

So, your benchmark basically measures the performance of image encoding plus writing to a buffer cache. If you want to measure actual device I/O, the performance test should at least involve FileChannel.force (which calls fsync under the hood).

Besides that, exclude image encoding from the benchmark, as it also may take significant amount of time.

Which is faster/better for caching, File System or Memcached?

Memcached is faster, but the memory is limited. HDD is huge, but I/O is slow compared to memory. You should put the hottest things to memcached, and all the others can go to cache files.

(Or man up and invest some money into more memory like these guys :)

For some benchmarks see: Cache Performance Comparison (File, Memcached, Query Cache, APC)

In theory:

Read 1 MB sequentially from memory       250,000 ns
Disk seek                             10,000,000 ns

http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf

Imread & Imwrite do not achieve expected gains on a Ramdisk

File caching. Probably, Windows' file cache is already speeding up your I/O activity here, so the RAM disk isn't giving you an additional speedup. When you write out the file, it's written to the file cache and then asynchronously flushed to the disk, so your Matlab code doesn't have to wait for the physical disk writes to complete. And when you immediately read the same file back in to memory, there's a high chance it's still present in the file cache, so it's served from memory instead of incurring a physical disk read.

If that's your actual code, you're re-writing the same file over and over again, which means all the activity may be happening inside the disk cache, so you're not hitting a bottleneck with the underlying storage mechanism.

Rewrite your test code so it looks more like your actual workload: writing to different files on each pass if that's what you'll be doing in practice, including the image processing code, and actually running multiple processes in parallel. Put it in the Matlab profiler, or add finer-grained tic/toc calls, to see how much time you're actually spending in I/O (e.g. imread and imwrite, and the parts of them that are doing file I/O). If you're doing nontrivial processing outside the I/O, you might not see significant, if any, speedup from the RAM disk because the file cache would have time to do the actual physical I/O during your other processing.

And since you say there's a maximum of 10 MB that gets written over and over again, that's small enough that it could easily fit inside the file cache in the first place, and your actual physical I/O throughput is pretty small: if you write a file, and then overwrite its contents with new data before the file cache flushes it to disk, the OS never has to flush that first set of data all the way to disk. Your I/O might already be mostly happening in memory due to the cache so switching to a RAM disk won't help because physical I/O isn't a bottleneck.

Modern operating systems do a lot of caching because they know scenarios like this happen. A RAM disk isn't necessarily going to be a big speedup. There's nothing specific to Matlab or imread/imwrite about this behavior; the other RAM disk questions like RAMdisk slower than disk? are still relevant.

How to view paging system (demand paging) as another layer of cache?

To start; your answers for line size, associativity and number of lines are right.

Tag size in bits? 14 (Number of lines in cache is 2^14 which gives us 14 bits for tag)

Tag size would be "location on disk / line size", plus some other bits (e.g. for managing the replacement policy). We don't know how big the disk is (other than "large").

However; it's possibly not unreasonable to work this out backwards - start from the assumption that the OS was designed to support a wide range of different hard disk sizes and that the tag is a nice "power of 2"; then assume that 4 bits are used for other purposes (e.g. "age" for LRU). If the tag is 32 bits (a very common "power of 2") then it would imply the OS could support a maximum disk size of 2 TiB ("1 << (32-4) * 8 KiB"), which is (keeping "future proofing" in mind) a little too small for an OS designed in the last 10 years or so. The next larger "power of 2" is 64 bits, which is very likely for modern hardware, but less likely for older hardware (e.g. 32-bit CPUs, smaller disks). Based on "128 MiB of RAM" I'd suspect that the hardware is very old (e.g. normal desktop/server systems started having more than 128 MiB in the late 1990s), so I'd go with "32 bit tag".

Replacement policy? I am not sure at all (maybe clock algorithm which approximates LRU)

Writeback or write-through? write back (It is not consistent with Disk at all times) Write-allocate? yes, because after page fault we bring the page to memory for both writing and reading

There isn't enough information to be sure.

A literal write-through policy would be a performance disaster (imagine having to write 8 KiB to a relatively slow disk every time anything pushed data on the stack). A write back policy would be very bad for fault tolerance (e.g. if there's a power failure you'd lose far too much data).

This alone is enough to imply that it's some kind of custom design (neither strictly write-back nor strictly write-through).

To complicate things more; an OS could take into account "eviction costs". E.g. if the data in memory is already on the disk then the page can be re-used immediately, but if the data in memory has been modified then that page would have to be saved to disk before the memory can be re-used; so if/when the OS needs to evict data from cache to make room (e.g. for more recently used data) it'd be reasonable to prefer evicting an unmodified page (which is cheaper to evict). In addition; for spinning disks, it's common for an OS to optimize disk access to minimize head movement (where the goal is to reduce seek times and improve disk IO performance).

The OS might combine all of these factors when deciding when modified data is written to disk.

Exclusivity/Inclusivity? I think non-inclusive and non exclusive (NINE), maybe because memory mapped files are partially in memory and partially in swap file or ELF file (program text). Forexample stack of process is only in memory except when we run out of memory and send it to a swap file. Am I right?

If RAM is treated as a cache of disk, then the system is an example of single-level store (see https://en.wikipedia.org/wiki/Single-level_store ) and isn't a normal OS (with normal virtual memory - e.g. swap space and file systems). Typically systems that use single-level store are built around the idea of having "persistent objects" and do not have files at all. With this in mind; I don't think it's reasonable to make assumptions that would make sense for a normal operating system (e.g. assume that there are executable files, or that memory mapped files are supported, or that some part of the disk is "swap space" and another part of the disk is not).

However; I would assume that you're right about "non-inclusive and non exclusive (NINE)" - inclusive would be bad for performance (for the same reason write-through would be bad for performance) and exclusive would be very bad for fault tolerance (for the same reason that write-back is bad for fault tolerance).

---

Related Topics