Serialization of Struct

how to serialize a struct in c?

This answer is besides the problems with your malloc.

Unfortunately, you cannot find a nice trick that would still be compatible with the standard. The only way of properly serializing a structure is to separately dissect each element into bytes, write them to an unsigned char array, send them over the network and put the pieces back together on the other end. In short, you would need a lot of shifting and bitwise operations.

In certain cases you would need to define a kind of protocol. In your case for example, you need to be sure you always put the object p is pointing to right after struct A, so once recovered, you can set the pointer properly. Did everyone say enough already that you can't send pointers through network?

Another protocolish thing you may want to do is to write the size allocated for the flexible array member s in struct B. Whatever layout for your serialized data you choose, obviously both sides should respect.

It is important to note that you cannot rely on anything machine specific such as order of bytes, structure paddings or size of basic types. This means that you should serialize each field of the element separately and assign them fixed number of bytes.

Serialize struct with a pointer to a struc

I read the article a little bit more, and seems like the data they're receiving in the stream object, comes from the machine, that's why the pp and ppend, probably having the "before and after" state saved there. Plus, the functions seems to be made only for that purpose.

Too bad as I thought it was kind of good code to learn from, probably I got frustrated over it and miss understood the concept.

Sorry to waste your time guys!, but I won't give up :)

Any books or sites to learn more about serialization that you know of? I don't want to use any external libraries at the moment because I want to learn more and practice doing my own tools. Using external libraries feels to me like cheating for some reason and makes me feel I'm not going through the right path.

thank you, greetings!

Serialization/Deserialization of a struct to a char* in C

since this is to be sent over a network, i strongly advise you to convert those data into network byte order before transmitting, and back into host byte order when receiving. this is because the byte ordering is not the same everywhere, and once your bytes are not in the right order, it may become very difficult to reverse them (depending on the programming language used on the receiving side). byte ordering functions are defined along with sockets, and are named htons(), htonl(), ntohs() and ntohl(). (in those name: h means 'host' or your computer, n means 'network', s means 'short' or 16bit value, l means 'long' or 32 bit value).

then you are on your own with serialization, C and C++ have no automatic way to perform it. some softwares can generate code to do it for you, like the ASN.1 implementation asn1c, but they are difficult to use because they involve much more than just copying data over the network.

C# - struct serialization

1. This will not serialize references correctly (i.e. any field of a non-primitive and non-struct type).
2. It doesn't include type data
   
   • Which makes it unsuitable for cases when not all types are known beforehand or some types have changed slightly since serialization time.
3. (Minor) ToStructure doesn't validate the binary data

Why not learn about the existing solutions before reinventing the wheel? The Serialization Guidelines article says there are 3 serialization techniques implemented in the .NET framework that have different characteristics and are tailored for different purposes.

Here's an example of the simplest, 3rd technique given as example in the Object Serialization in .NET article. It exists to reconstruct an object with exactly the same type and internal data as the original one (which means, serialization includes the objects it references).

_{(the code is in IronPython yet I hope it's readable enough to understand what's going on)}

>>> l=System.Collections.Generic.List[System.Drawing.Point]\
([System.Drawing.Point(*(random.randint(1,1000) for _ in range(2))) for _ in range(5)])
>>> l
List[Point]([<System.Drawing.Point object at 0x0000000000000233 [{X=491,Y=874}]>
, <System.Drawing.Point object at 0x0000000000000234 [{X=819,Y=595}]>, <System.D
rawing.Point object at 0x0000000000000235 [{X=456,Y=625}]>, <System.Drawing.Poin
t object at 0x0000000000000236 [{X=583,Y=29}]>, <System.Drawing.Point object at
0x0000000000000237 [{X=329,Y=212}]>])
>>> szr=System.Runtime.Serialization.Formatters.Binary.BinaryFormatter()
>>> stm=System.IO.MemoryStream()
>>> szr.Serialize(stm,l)
>>> stm.Length
481L
>>> bytes=stm.GetBuffer()
>>> s=''.join(chr(b) for b in bytes)
>>> s
u'\x00\x01\x00\x00\x00\xff\xff\xff\xff\x01\x00\x00\x00\x00\x00\x00\x00\x0c\x02\x
00\x00\x00QSystem.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f
5f7f11d50a3a\x04\x01\x00\x00\x00\x8c\x01System.Collections.Generic.List`1[[Syste
m.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToke
n=b03f5f7f11d50a3a]]\x03\x00\x00\x00\x06_items\x05_size\x08_version\x04\x00\x00\
x16System.Drawing.Point[]\x02\x00\x00\x00\x08\x08\t\x03\x00\x00\x00\x05\x00\x00\
x00\x00\x00\x00\x00\x07\x03\x00\x00\x00\x00\x01\x00\x00\x00\x05\x00\x00\x00\x04\
x14System.Drawing.Point\x02\x00\x00\x00\x05\xfc\xff\xff\xff\x14System.Drawing.Po
int\x02\x00\x00\x00\x01x\x01y\x00\x00\x08\x08\x02\x00\x00\x00\xeb\x01\x00\x00j\x
03\x00\x00\x01\xfb\xff\xff\xff\xfc\xff\xff\xff3\x03\x00\x00S\x02\x00\x00\x01\xfa
\xff\xff\xff\xfc\xff\xff\xff\xc8\x01\x00\x00q\x02\x00\x00\x01\xf9\xff\xff\xff\xf
c\xff\xff\xffG\x02\x00\x00\x1d\x00\x00\x00\x01\xf8\xff\xff\xff\xfc\xff\xff\xffI\
x01\x00\x00\xd4\x00\x00\x00\x0b\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

Serializing struct to file, and deserializing it again with string

Try to avoid using memcpy-ing to a pointer for serialization.. Instead, make use of streams.. You can even make a stream class that serializes any type of data. Next you'll just have to add << and >> operators to any class that needs to be serialized.

#include <iostream>
#include <fstream>
#include <string>
#include <cstring>

struct ArchiveTableEntry
{
    uint32_t id;
    std::string filename;
    uint32_t filename_crc32;
    uint32_t real_data_size;
    uint32_t block_data_size;

    uint32_t data_crc32;
    char data_md5_checksum[16];

    uint64_t data_offset;
    uint8_t flag;
};

std::ostream& operator << (std::ostream& os, const ArchiveTableEntry entry)
{
    return os << entry.id << entry.filename.size() << entry.filename
              << entry.filename_crc32 << entry.real_data_size << entry.block_data_size
              << entry.data_crc32 << entry.data_md5_checksum << entry.data_offset << entry.flag;
}

std::istream& operator >> (std::istream& os, ArchiveTableEntry& entry)
{
    std::size_t filesize = 0;
    os >> entry.id;
    os >> filesize;
    entry.filename.resize(filesize);
    os.read(&entry.filename[0], filesize);
    os >> entry.filename_crc32 >> entry.real_data_size >> entry.block_data_size >> entry.data_crc32;
    os >> entry.data_md5_checksum >> entry.data_offset >> entry.flag;
    return os;
}

int main()
{
    const char* md5_checksum = "some checksum";

    ArchiveTableEntry data;
    data.id = 1;
    data.filename = "file.txt";
    data.filename_crc32 = 10434235;
    data.real_data_size = 1024;
    data.block_data_size = 256;
    data.data_crc32 = 324225252;
    data.data_md5_checksum = 1;

    std::memset(data.data_md5_checksum, 0, sizeof(data.data_md5_checksum) / sizeof(char));
    strcpy(data.data_md5_checksum, md5_checksum);

    data.data_offset = 512;
    data.flag = 1;

    std::fstream out("C:/Users/School/Desktop/Test.bin", std::ios::out);
    if (out.is_open())
    {
        out << data;
        out.close();
    }

    std::fstream in("C:/Users/School/Desktop/Test.bin", std::ios::in);
    if (in.is_open())
    {
        in >> data;
        in.close();
    }

    std::cout<<data.id<<" "<<data.filename<<" "<<data.filename_crc32<<" "<<data.real_data_size<<" "<<data.block_data_size<<" ";
    std::cout<<data.data_crc32<<" "<<data.data_md5_checksum<<" ";
}

---

Related Topics