Is Std::Ifstream Significantly Slower Than File

Is std::ifstream significantly slower than FILE?

I don't think that'd make a difference. Especially if you're reading char by char, the overhead of I/O is likely to completely dominate anything else.
Why do you read single bytes at a time? You know how extremely inefficient it is?

On a 326kb file, the fastest solution will most likely be to just read it into memory at once.

The difference between std::ifstream and the C equivalents, is basically a virtual function call or two. It may make a difference if executed a few tens of million times per second, otherwise, not reall. file I/O is generally so slow that the API used to access it doesn't really matter. What matters far more is the read/write pattern. Lots of seeks are bad, sequential reads/writes good.

Why is my file reading via fstream::read slow on another computer with similar hardware?

Could be a lot of things: amount of RAM, how many programs and services are running, health of the drive, how full the drive is, etc.

The first thing I'd check for is antivirus interference, which if you've sent your colleague an exe is probably scanning everything you're writing.

fstream faster than fopen? and gets faster as buffers increase?

In WriteCPP, you have to set the buffer before opening the file, like so:

std::ofstream file;
char buffer[BUFF]; file.rdbuf()->pubsetbuf(buffer, BUFF);   // set buffer
file.open ("test.txt", std::ios::binary);                   // open file

Then you get the sort of results that you might you expect (times are for writing 20MB with the buffer sizes shown):

writeCPP, 32: 2.15278
writeCPP, 128: 1.21372
writeCPP, 512: 0.857389

I also benchmarked WriteC with your change from fprintf to fputc and got the following (again writing 20MB):

writeC, 32: 1.41433
writeC, 128: 0.524264
writeC, 512: 0.355097

Test program is here:

https://wandbox.org/permlink/F2H2jcrMVsc5VNFf

Should C++ file read be slower than Ruby or C#?

Based on the comments and the originally posted code (it has now been fixed [now deleted]) there was previously a coding error (i++ missing) that stopped the C++ program from outputting anything. This plus the while(true) loop in the complete code sample would present symptoms consistent with those stated in the question (i.e. user waits 500s sees no output and force terminates the program). This is because it would complete reading the file without outputting anything and enter into the deliberately added infinite loop.

The revised complete source code correctly completes (according to the comments) in ~1.6s for a 1.2 million file. My advice for improving performance would be as follows:

1. Make sure you are compiling in release mode (not debug mode). Given the user has specified they are using Visual Studio 2017, I would recommend viewing the official Microsoft documentation (https://msdn.microsoft.com/en-us/library/wx0123s5.aspx) for a thorough explanation.

2. To make it easier to diagnose problems do not add an infinite loop at the end of your program. Instead run the executable from powershell / (cmd) and confirm that it terminates correctly.

EDIT: I would also add:

3. For accurate timings you also need to take into account the OS disk cache. Run each benchmark multiple times to 'warm-up' the disk cache.

Will reading a file be faster with a FILE* or an std::ifstream?

ifstream supports random access with seekg. FILE* might be faster but you should measure it.

Why is my C++ disk write test much slower than a simply file copy using bash?

First, you're not really measuring the disk writing speed, but (partly) the speed of writing data to the OS disk cache. To really measure the disk writing speed, the data should be flushed to disk before calculating the time. Without flushing there could be a difference depending on the file size and the available memory.

There seems to be something wrong in the calculations too. You're not using the value of MB.

Also make sure the buffer size is a power of two, or at least a multiple of the disk page size (4096 bytes): char buffer[32 * 1024];. You might as well do that for payload too. (looks like you changed that from 1024 to 1000 in an edit where you added the calculations).

Do not use streams to write a (binary) buffer of data to disk, but instead write directly to the file, using FILE*, fopen(), fwrite(), fclose(). See this answer for an example and some timings.

---

To copy a file: open the source file in read-only and, if possible, forward-only mode, and using fread(), fwrite():

while fread() from source to buffer
  fwrite() buffer to destination file

This should give you a speed comparable to the speed of an OS file copy (you might want to test some different buffer sizes).

This might be slightly faster using memory mapping:

open src, create memory mapping over the file
open/create dest, set file size to size of src, create memory mapping over the file
memcpy() src to dest

For large files smaller mapped views should be used.

Why is ifstream::read much faster than using iterators?

read is a single iostream setup (part of every iostream operation) and a single call to the OS, reading directly into the buffer you provided.

The iterator works by repeatedly extracting a single char with operator>>. Because of the buffer size, this might mean more OS calls, but more importantly it also means repeated setting up and tearing down of the iostream sentry, which might mean a mutex lock, and usually means a bunch of other stuff. Furthermore, operator>> is a formatted operation, whereas read is unformatted, which is additional setup overhead on every operation.

Edit: Tired eyes saw istream_iterator instead of istreambuf_iterator. Of course istreambuf_iterator does not do formatted input. It calls sbumpc or something like that on the streambuf. Still a lot of calls, and using the buffer, which is probably smaller than the entire file.

---

Related Topics