How to Read a Binary File into a Vector of Unsigned Chars

How to read a binary file into a vector of unsigned chars

When testing for performance, I would include a test case for:

std::vector<BYTE> readFile(const char* filename)
{
    // open the file:
    std::ifstream file(filename, std::ios::binary);

    // Stop eating new lines in binary mode!!!
    file.unsetf(std::ios::skipws);

    // get its size:
    std::streampos fileSize;

    file.seekg(0, std::ios::end);
    fileSize = file.tellg();
    file.seekg(0, std::ios::beg);

    // reserve capacity
    std::vector<BYTE> vec;
    vec.reserve(fileSize);

    // read the data:
    vec.insert(vec.begin(),
               std::istream_iterator<BYTE>(file),
               std::istream_iterator<BYTE>());

    return vec;
}

My thinking is that the constructor of Method 1 touches the elements in the vector, and then the read touches each element again.

Method 2 and Method 3 look most promising, but could suffer one or more resize's. Hence the reason to reserve before reading or inserting.

I would also test with std::copy:

...
std::vector<byte> vec;
vec.reserve(fileSize);

std::copy(std::istream_iterator<BYTE>(file),
          std::istream_iterator<BYTE>(),
          std::back_inserter(vec));

In the end, I think the best solution will avoid operator >> from istream_iterator (and all the overhead and goodness from operator >> trying to interpret binary data). But I don't know what to use that allows you to directly copy the data into the vector.

Finally, my testing with binary data is showing ios::binary is not being honored. Hence the reason for noskipws from <iomanip>.

Read binary file into unsigned char vector buffer

const std::vector<unsigned char> buffer(bufferSize);

You declared a const object here. By definition, a const object cannot be modified. Your plans to modify this object, by reading something into it, are already doomed to a big, abysmal, failure at this point. But there's also a second problem.

file.read(buffer.data(), bufferSize);

If you actually read your compiler's error message, slowly, it tells you exactly what the problem is.

First of all, read()s first parameter is a char *, a pointer to a character.

But you are passing a const unsigned char *. That's because data(), given that buffer is const, obviously returns a const unsigned char *.

And that's why you get a compilation error. If you now reread the compilation error, skipping about half of its words, it makes perfect sense now:

Cannot initialize a parameter of type ... 'char *' with an rvalue of
type ... 'const unsigned char *'

To fix it, your buffer should not be a const object. Preferrably, it should be a std::vector<char>, so you end up really passing a char * to read().

But, if you insist, you can keep it a vector of unsigned chars and use reinterpret_cast to cast the result of data() from unsigned char * to char *.

Read a binary file into a std::vector uint16_t instead of std::vector char

With c++11, you can use the data() member of std::vector and read all of file (or big chunks, if the file is too big).

Something like

#include <vector>
#include <fstream>
#include <iostream>

using myType = uint16_t;

int main ()
 {
   std::ifstream is;
   std::vector<myType> rawfilebuffer;

   is.open("uint16_t_file.raw", std::ios::binary);
   is.seekg(0, std::ios::end);
   size_t filesize=is.tellg();
   is.seekg(0, std::ios::beg);

   rawfilebuffer.resize(filesize/sizeof(myType));

   is.read((char *)rawfilebuffer.data(), filesize);

   for ( auto const & ui : rawfilebuffer )
      std::cout << '[' << ui << ']';

   std::cout << '\n';

   return 0;
}

Attention to the file size. If it's an exact multiple of sizeof(myType), well.

Otherwise, you should modify you resize instruction in this way

rawfilebuffer.resize(filesize/sizeof(myType)+(filesize%sizeof(myType)?1U:0U));

std::vector unsigned char remains empty after reading binary file using std::ifstream

reserve() allocates memory, but it doesn't register the allocated region as valid elements.

You should use resize() to add elements and size() to count the elements.

    // reserve capacity
    in.seekg(0, std::ios::end);
    //fileContent.reserve(in.tellg());
    fileContent.resize(in.tellg());
    in.clear();
    in.seekg(0, std::ios::beg);

    // read into vector
    //in.read(reinterpret_cast<char *>(fileContent.data()), fileContent.capacity());
    in.read(reinterpret_cast<char *>(fileContent.data()), fileContent.size());

How do you read n bytes from a file and put them into a vector uint8_t using iterators?

You can use ifstream::read for that.

std::vector<BYTE> v(100);
while ( file.read(reinterpret_cast<char*>(v.data()), 100) )
{
   // Find out how many characters were actually read.
   auto count = file.gcount();

   // Use v up to count BTYEs.
}

Reading binary file into vector char reading less than full file

There are multiple stream iterators. The class template std::istream_iterator<T> is for formatted input, i.e., it is going go skip leading whitespace before trying to read an object of type T.

From the looks of it you want std::istreambuf_iterator<char> which is used to iterate over the characters in a file, not doing any skipping.

---

Related Topics