Fast and Compact Object Serialization in .Net

Fast and compact object serialization in .NET

You can use Protocol Buffers. I'm changing all my serialization code from BinaryFormatter with compression to Protocol Buffers and obtaining very good results. It's more efficient in both time and space.

There are two .NET implementations by Jon Skeet and Marc Gravell.

Update: Official .NET implementation can be found here.

Fastest way to serialize and deserialize .NET objects

Here's your model (with invented CT and TE) using protobuf-net (yet retaining the ability to use XmlSerializer, which can be useful - in particular for migration); I humbly submit (with lots of evidence if you need it) that this is the fastest (or certainly one of the fastest) general purpose serializer in .NET.

If you need strings, just base-64 encode the binary.

[XmlType]
public class CT {
    [XmlElement(Order = 1)]
    public int Foo { get; set; }
}
[XmlType]
public class TE {
    [XmlElement(Order = 1)]
    public int Bar { get; set; }
}
[XmlType]
public class TD {
    [XmlElement(Order=1)]
    public List<CT> CTs { get; set; }
    [XmlElement(Order=2)]
    public List<TE> TEs { get; set; }
    [XmlElement(Order = 3)]
    public string Code { get; set; }
    [XmlElement(Order = 4)]
    public string Message { get; set; }
    [XmlElement(Order = 5)]
    public DateTime StartDate { get; set; }
    [XmlElement(Order = 6)]
    public DateTime EndDate { get; set; }

    public static byte[] Serialize(List<TD> tData) {
        using (var ms = new MemoryStream()) {
            ProtoBuf.Serializer.Serialize(ms, tData);
            return ms.ToArray();
        }            
    }

    public static List<TD> Deserialize(byte[] tData) {
        using (var ms = new MemoryStream(tData)) {
            return ProtoBuf.Serializer.Deserialize<List<TD>>(ms);
        }
    }
}

Best practice to serialize and deserialize .net objects across versions

This all depends on your application. Is it a distributed rich application where old applications may encounter new data objects from a central database or other source? (Much like older versions of office applications need to have some ways of dealing with newer document formats.)

If so, custom serialization and deserialization with explicit schema version numbering, I'd say. I'd put explicit metadata on each element and attribute, stating whether a reader must understand the element/attribute (and default values if not). This can of course consume quite a lot of space and increase code complexity...

But the answer really depends on why you are serializing to a database. You're not interested in using the database for its relational capabiities? Otherwise, an O/R mapping solution might be of interest.

Mixing custom and basic serialization?

The first thing I'd try is adding a property for serialization, but hiding it from the UI:

[Browsable(false)] // hide in UI
public SomeType ABC {
    get {return GetABC();}
    set {SetABC(value);}
}

You can't really mix and match serialization unfortunately; once you implement IXmlSerializable, you own everything. If you were using WCF, then DataContractSerialier supports non-public properties for serialization, so you could use:

[DataMember]
private SomeType ABC {
    get {return GetABC();}
    set {SetABC(value);}
}

but this doesn't apply for "asmx" web-services via XmlSerializer.

Does the [Browsable] trick work at all? Assuming the custom grid uses TypeDescriptor, another option might be to hide it via ICustomTypeDescriptor, but that is a lot of work just to hide a property...

How to optimize this further (speed first, then memory)

The problem is that BinaryFormatter is using reflection to read the fields of your objects. Assume you have a simple class with 1 field:

[Serializable]
public class Test
{
    public int A;
}

If you serialize an array of those using BinaryFormatter, it will do something like that for each instance of Test:

int val = (int)typeof(Test).GetField("A").GetValue(obj);
var bytes = BitConverter.GetBytes(val);
stream.Write(bytes, 0, bytes.Length);

The calls to GetField() will consume quite a lot of time.
You can significantly improve the speed using 3 ways:

1. Serialize everything manually. Something similar to this code:

   void SimpleSerialize(Stream stream, Test[] arr)
   {
       foreach (var obj in arr)
       {
           var bytes = BitConverter.GetBytes(obj.A);
           stream.Write(bytes, 0, bytes.Length);
       }
   }
   

2. Generate a custom serialization class on-the-fly using Reflection.Emit functionality. This is more generic and "clean", but requires a lot of effort.

3. If you're fine with it, use some third-party serializer that suits your needs.

---

Related Topics