Fastest Way to Serialize and Deserialize .Net Objects

Fastest way to serialize and deserialize .NET objects

Here's your model (with invented CT and TE) using protobuf-net (yet retaining the ability to use XmlSerializer, which can be useful - in particular for migration); I humbly submit (with lots of evidence if you need it) that this is the fastest (or certainly one of the fastest) general purpose serializer in .NET.

If you need strings, just base-64 encode the binary.

[XmlType]
public class CT {
    [XmlElement(Order = 1)]
    public int Foo { get; set; }
}
[XmlType]
public class TE {
    [XmlElement(Order = 1)]
    public int Bar { get; set; }
}
[XmlType]
public class TD {
    [XmlElement(Order=1)]
    public List<CT> CTs { get; set; }
    [XmlElement(Order=2)]
    public List<TE> TEs { get; set; }
    [XmlElement(Order = 3)]
    public string Code { get; set; }
    [XmlElement(Order = 4)]
    public string Message { get; set; }
    [XmlElement(Order = 5)]
    public DateTime StartDate { get; set; }
    [XmlElement(Order = 6)]
    public DateTime EndDate { get; set; }

    public static byte[] Serialize(List<TD> tData) {
        using (var ms = new MemoryStream()) {
            ProtoBuf.Serializer.Serialize(ms, tData);
            return ms.ToArray();
        }            
    }

    public static List<TD> Deserialize(byte[] tData) {
        using (var ms = new MemoryStream(tData)) {
            return ProtoBuf.Serializer.Deserialize<List<TD>>(ms);
        }
    }
}

Fast and compact object serialization in .NET

You can use Protocol Buffers. I'm changing all my serialization code from BinaryFormatter with compression to Protocol Buffers and obtaining very good results. It's more efficient in both time and space.

There are two .NET implementations by Jon Skeet and Marc Gravell.

Update: Official .NET implementation can be found here.

Using JsonDerivedType attribute to serialize and deserialize Polymorphic objects in .NET 7

Sorry I was not paying attention to the serialized object. It contains: "$type": "foo1",, "$type": "foo2", etc..

That's why the deserializer knows how to deserialize the object.

Fastest serializer and deserializer with lowest memory footprint in C#?

It sounds like Protocol Buffers might be what you're looking for.

There are three .NET implementations that I'm aware of: protobuf-net, protobuf-csharp-port and Proto#.

The performance comparisons show that Protocol Buffers outperform the built-in serializers in terms of both size and serialization/deserialization speed.

Fast serialization/deserialization of structs

BinarySerializer is a very general serializer. It will not perform as well as a custom implementation.

Fortunately for your, your data consists of structs only. This means that you will be able to fix a structlayout for Child and just bit-copy the children array using unsafe code from a byte[] you have read from disk.

For the parents it is not that easy because you need to treat the children separately. I recommend you use unsafe code to copy the bit-copyable fields from the byte[] you read and deserialize the children separately.

Did you consider mapping all the children into memory using memory mapped files? You could then re-use the operating systems cache facility and not deal with reading and writing at all.

Zero-copy-deserializing a Child[] looks like this:

byte[] bytes = GetFromDisk();
fixed (byte* bytePtr = bytes) {
 Child* childPtr = (Child*)bytePtr;
 //now treat the childPtr as an array:
 var x123 = childPtr[123].X;

 //if we need a real array that can be passed around, we need to copy:
 var childArray = new Child[GetLengthOfDeserializedData()];
 for (i = [0..length]) {
  childArray[i] = childPtr[i];
 }
}

How to improve JSON deserialization speed in .Net? (JSON.net or other?)

I have spent a little bit more time reading about JSON.NET internals, and my conclusion is that the slowness is caused mostly by reflection.

On the JSON.NET site i have found some nice performance tips, and i tried pretty much everything (JObject.Parse, Custom Converters etc.) but i couldn't squeeze out any significant performance improvement. Then i read the most important note on the whole site:

If performance is important and you don't mind more code to get it then this is your best choice. Read more about using JsonReader/JsonWriter here

So i listened to the advice and i implemented a basic version of a JsonReader to read the string efficiently:

var reader = new JsonTextReader(new StringReader(jsonString));

var response = new GetRoomListResponse();
var currentProperty = string.Empty;

while (reader.Read())
{
    if (reader.Value != null)
    {
        if (reader.TokenType == JsonToken.PropertyName)
            currentProperty = reader.Value.ToString();

        if (reader.TokenType == JsonToken.Integer && currentProperty == "Acknowledge")
            response.Acknowledge = (AcknowledgeType)Int32.Parse(reader.Value.ToString());

        if (reader.TokenType == JsonToken.Integer && currentProperty == "Code")
            response.Code = Int32.Parse(reader.Value.ToString());

        if (reader.TokenType == JsonToken.String && currentProperty == "Message")
            response.Message = reader.Value.ToString();

        if (reader.TokenType == JsonToken.String && currentProperty == "Exception")
            response.Exception = reader.Value.ToString();

        // Process Rooms and other stuff
    }
    else
    {
        // Process tracking the current nested element
    }
}

I think the exercise is clear, and without doubt this is the best performance you can get out of JSON.NET.

Just this limited code is 12x faster than the Deserialize version on my box with 500 rooms, but of course the mapping is not completed. However, i am pretty sure it will be at least 5x faster than deserialization in the worst-case.

Check out this link for more info about the JsonReader and how to use it:

http://james.newtonking.com/json/help/html/ReadingWritingJSON.htm

---

Related Topics