What Is the Fastest Method for High Performance Sequential File I/O in C++

What is the Fastest Method for High Performance Sequential File I/O in C++?

Are there generally accepted guidelines for achieving the fastest possible sequential file I/O in C++?

Rule 0: Measure. Use all available profiling tools and get to know them. It's almost a commandment in programming that if you didn't measure it you don't know how fast it is, and for I/O this is even more true. Make sure to test under actual work conditions if you possibly can. A process that has no competition for the I/O system can be over-optimized, fine-tuned for conditions that don't exist under real loads.

1. Use mapped memory instead of writing to files. This isn't always faster but it allows the opportunity to optimize the I/O in an operating system-specific but relatively portable way, by avoiding unnecessary copying, and taking advantage of the OS's knowledge of how the disk actually being used. ("Portable" if you use a wrapper, not an OS-specific API call).

2. Try and linearize your output as much as possible. Having to jump around memory to find the buffers to write can have noticeable effects under optimized conditions, because cache lines, paging and other memory subsystem issues will start to matter. If you have lots of buffers look into support for scatter-gather I/O which tries to do that linearizing for you.

Some possible considerations:

• Guidelines for choosing the optimal buffer size

Page size for starters, but be ready to tune from there.

• Will a portable library like boost::asio be too abstracted to expose the intricacies
  of a specific platform, or can they be assumed to be optimal?

Don't assume it's optimal. It depends on how thoroughly the library gets exercised on your platform, and how much effort the developers put into making it fast. Having said that a portable I/O library can be very fast, because fast abstractions exist on most systems, and it's usually possible to come up with a general API that covers a lot of the bases. Boost.Asio is, to the best of my limited knowledge, fairly fine tuned for the particular platform it is on: there's a whole family of OS and OS-variant specific APIs for fast async I/O (e.g. epoll, /dev/epoll, kqueue, Windows overlapped I/O), and Asio wraps them all.

• Is asynchronous I/O always preferable to synchronous? What if the application is not otherwise CPU-bound?

Asynchronous I/O isn't faster in a raw sense than synchronous I/O. What asynchronous I/O does is ensure that your code is not wasting time waiting for the I/O to complete. It is faster in a general way than the other method of not wasting that time, namely using threads, because it will call back into your code when I/O is ready and not before. There are no false starts or concerns with idle threads needing to be terminated.

Fastest way to read a text file of strings line by line

As I understand your question your objective is to read a file of words and insert each word into some data structure. You want this read+insertion to be as fast as possible. (I won't debate the rationale for or the wisdom of this, I'll just accept is as a requirement. :-) )
If my understanding is correct, then perhaps an alternative approach would be to write a utility program that will read the file of words, insert them into the data structure, and then serialize that data structure to a file (say BLOB.dat, for example). Then your main program will deserialize BLOB.dat into the data structure that you require. Essentially you pre-process the words file into some intermediate binary format that can be loaded into your data structure most efficiently. Or would this be cheating in your scenario??

Why mmap() is faster than sequential IO?

I heard (read it on the internet somewhere) that mmap() is faster than sequential IO. Is this correct? If yes then why it is faster?

It can be - there are pros and cons, listed below. When you really have reason to care, always benchmark both.

Quite apart from the actual IO efficiency, there are implications for the way the application code tracks when it needs to do the I/O, and does data processing/generation, that can sometimes impact performance quite dramatically.

1. mmap() is not reading sequentially.
   2) mmap() has to fetch from the disk itself same as read() does
   3) The mapped area is not sequential - so no DMA (?).

So mmap() should actually be slower than read() from a file? Which of my assumptions above are wrong?

1. is wrong... mmap() assigns a region of virtual address space corresponding to file content... whenever a page in that address space is accessed, physical RAM is found to back the virtual addresses and the corresponding disk content is faulted into that RAM. So, the order in which reads are done from the disk matches the order of access. It's a "lazy" I/O mechanism. If, for example, you needed to index into a huge hash table that was to be read from disk, then mmaping the file and starting to do access means the disk I/O is not done sequentially and may therefore result in longer elapsed time until the entire file is read into memory, but while that's happening lookups are succeeding and dependent work can be undertaken, and if parts of the file are never actually needed they're not read (allow for the granularity of disk and memory pages, and that even when using memory mapping many OSes allow you to specify some performance-enhancing / memory-efficiency tips about your planned access patterns so they can proactively read ahead or release memory more aggressively knowing you're unlikely to return to it).

2. absolutely true

3. "The mapped area is not sequential" is vague. Memory mapped regions are "contiguous" (sequential) in virtual address space. We've discussed disk I/O being sequential above. Or, are you thinking of something else? Anyway, while pages are being faulted in, they may indeed be transferred using DMA.

Further, there are other reasons why memory mapping may outperform usual I/O:

• there's less copying:
  
  • often OS & library level routines pass data through one or more buffers before it reaches an application-specified buffer, the application then dynamically allocates storage, then copies from the I/O buffer to that storage so the data's usable after the file reading completes
  • memory mapping allows (but doesn't force) in-place usage (you can just record a pointer and possibly length)
    
    • continuing to access data in-place risks increased cache misses and/or swapping later: the file/memory-map could be more verbose than data structures into which it could be parsed, so access patterns on data therein could have more delays to fault in more memory pages
• memory mapping can simplify the application's parsing job by letting the application treat the entire file content as accessible, rather than worrying about when to read another buffer full
• the application defers more to the OS's wisdom re number of pages that are in physical RAM at any single point in time, effectively sharing a direct-access disk cache with the application
• as well-wisher comments below, "using memory mapping you typically use less system calls"
• if multiple processes are accessing the same file, they should be able to share the physical backing pages

The are also reasons why mmap may be slower - do read Linus Torvald's post here which says of mmap:

...page table games along with the fault (and even just TLB miss)
overhead is easily more than the cost of copying a page in a nice
streaming manner...

And from another of his posts:

• quite noticeable setup and teardown costs. And I mean noticeable. It's things like following the page tables to unmap everything cleanly. It's the book-keeping for maintaining a list of all the mappings. It's The TLB flush needed after unmapping stuff.

• page faulting is expensive. That's how the mapping gets populated, and it's quite slow.

Linux does have "hugepages" (so one TLB entry per 2MB, instead of per 4kb) and even Transparent Huge Pages, where the OS attempts to use them even if the application code wasn't written to explicitly utilise them.

FWIW, the last time this arose for me at work, memory mapped input was 80% faster than fread et al for reading binary database records into a proprietary database, on 64 bit Linux with ~170GB files.

Read sequential file - Compressed file vs Uncompressed

Yes, it is possible to improve disk read by using compression.

This effect is most likely to happen if you use a multi-threaded reader : while one thread reads compressed data from disk, the other one decode the previous compressed block within memory.

Considering the speed of LZ4, the decoding operation is likely to finish before the other thread complete reading the next block. This way, you'll achieved a bandwidth improvement, proportional to the compression ratio of the tested file.

Obviously, there are other effects to consider when benchmarking. For example, seek times of HDD are several order of magnitude larger than SSD, and under bad circumstances, it can become the dominant part of the timing, reducing any bandwidth advantage to zero.

---

Related Topics