"Proper" Way to Store Binary Data with C++/Stl

Proper way to store binary data with C++/STL

vector of char is nice because the memory is contiguious. Therefore you can use it with a lot of C API's such as berkley sockets or file APIs. You can do the following, for example:

  std::vector<char> vect;
  ...
  send(sock, &vect[0], vect.size());

and it will work fine.

You can essentially treat it just like any other dynamically allocated char buffer. You can scan up and down looking for magic numbers or patters. You can parse it partially in place. For receiving from a socket you can very easily resize it to append more data.

The downside is resizing is not terribly efficient (resize or preallocate prudently) and deletion from the front of the array will also be very ineficient. If you need to, say, pop just one or two chars at a time off the front of the data structure very frequently, copying to a deque before this processing may be an option. This costs you a copy and deque memory isn't contiguous, so you can't just pass a pointer to a C API.

Bottom line, learn about the data structures and their tradeoffs before diving in, however vector of char is typically what I see used in general practice.

C++ STL's String eqivalent for Binary Data

I'd use std::vector<unsigned char>. Most operations you need can be done using the STL with iterator ranges. Also, remember that if you really need the raw data &v[0] is guaranteed to give a pointer to the underlying array.

C++ binary file I/O to/from containers (other than char *) using STL algorithms

For the question 1) You need to specify a separator (for example a space). The non-decimal part was stuck to the decimal part of the previous number. Casting and using NULL is generally wrong in C++. Should have been a hint ;)

copy (vd.begin(), vd.end(), oi_t(output, " ")); 

For the question 2)

#include <iomanip>
output << setprecision(9);

How to store binary data when you only care about speed?

Locality of reference will likely be the driving force. So it's fairly obvious that you represent the D coordinates of a single point as a contiguous bitvector. std::bitset<D> would be a logical choice.

However, the next important thing to realize is that you see locality benefits easily up to 4KB. This means that you should not pick a single point and compare it against all other N-1 points. Instead, group points in sets of 4KB each, and compare those groups. Both ways are O(N*N), but the second will be much faster.

You may be able to beat O(N*N) by use of the triangle inequality - Hamming(a,b)+Hamming(b,c) >= Hamming (a,c). I'm just wondering how. It probably depends on how you want your output. The naive output would be a N*N set of distances, and that's unavoidably O(N*N).

Output binary buffer with STL

What does "binary data" mean? Just memory buffers? Do you want to be able push/pop one buffer at a time? Then you should wrap a buffer into a class, or use std::vector<char>, and push/pop them onto std::deque.

Data parsing of binary files in C++ : and the right STL for it?

One way is to use STL's map to achieve what you want.
Since each client's response will be two lines of strings, you can create a struct containing two variables to store it. Then you can insert the struct into the map, with "CLIENT-X" as the index element. Lastly, use the index to retrieve the client's data. Below is an example:

#include <sstream>
#include <fstream>
#include <map>
#include <string>

using namespace std;

struct data
{
    string firstLine, secondLine;
    data(){ }
};

int main()
{
    ifstream file("input.txt");

    std::map<string,data> mymap;    
    stringstream ss;
    string index;
    data buffer;
    int client = 1;

    if(file.is_open())
    {
        string server[4];

        for(int i = 0; i < 4; i++) // read the first 4 lines for the server
            getline(file, server[i]);

        while(getline(file, buffer.firstLine))
        {
            getline(file, buffer.secondLine);

            // define the index value for retrieval
            ss.str("");
            ss << "CLIENT-" << client;

            // insert the client's data into the map
            mymap.insert(pair<string,data>(ss.str(), buffer));
            client++;
        }

        // retrieve the client's data
        buffer = mymap["CLIENT-1"]; // example on how to access

        // here you can do what you want, like sending to the client
        //

        file.close();
    }

    return 0;
}   

Writing data structure with vector members to binary file

C++ STL does not provide native interface to store STL object to disk.

You can check this thread for help.

---

Related Topics