How to Deserialize JSON with Duplicate Property Names in the Same Object

How to deserialize JSON with duplicate property names in the same object

Interesting question. I played around with this for a while and discovered that while a JObject cannot contain properties with duplicate names, the JsonTextReader used to populate it during deserialization does not have such a restriction. (This makes sense if you think about it: it's a forward-only reader; it is not concerned with what it has read in the past). Armed with this knowledge, I took a shot at writing some code that will populate a hierarchy of JTokens, converting property values to JArrays as necessary if a duplicate property name is encountered in a particular JObject. Since I don't know your actual JSON and requirements, you may need to make some adjustments to it, but it's something to start with at least.

Here's the code:

public static JToken DeserializeAndCombineDuplicates(JsonTextReader reader)
{
    if (reader.TokenType == JsonToken.None)
    {
        reader.Read();
    }

    if (reader.TokenType == JsonToken.StartObject)
    {
        reader.Read();
        JObject obj = new JObject();
        while (reader.TokenType != JsonToken.EndObject)
        {
            string propName = (string)reader.Value;
            reader.Read();
            JToken newValue = DeserializeAndCombineDuplicates(reader);

            JToken existingValue = obj[propName];
            if (existingValue == null)
            {
                obj.Add(new JProperty(propName, newValue));
            }
            else if (existingValue.Type == JTokenType.Array)
            {
                CombineWithArray((JArray)existingValue, newValue);
            }
            else // Convert existing non-array property value to an array
            {
                JProperty prop = (JProperty)existingValue.Parent;
                JArray array = new JArray();
                prop.Value = array;
                array.Add(existingValue);
                CombineWithArray(array, newValue);
            }

            reader.Read();
        }
        return obj;
    }

    if (reader.TokenType == JsonToken.StartArray)
    {
        reader.Read();
        JArray array = new JArray();
        while (reader.TokenType != JsonToken.EndArray)
        {
            array.Add(DeserializeAndCombineDuplicates(reader));
            reader.Read();
        }
        return array;
    }

    return new JValue(reader.Value);
}

private static void CombineWithArray(JArray array, JToken value)
{
    if (value.Type == JTokenType.Array)
    {
        foreach (JToken child in value.Children())
            array.Add(child);
    }
    else
    {
        array.Add(value);
    }
}

And here's a demo:

class Program
{
    static void Main(string[] args)
    {
        string json = @"
        {
            ""Foo"" : 1,
            ""Foo"" : [2],
            ""Foo"" : [3, 4],
            ""Bar"" : { ""X"" : [ ""A"", ""B"" ] },
            ""Bar"" : { ""X"" : ""C"", ""X"" : ""D"" },
        }";

        using (StringReader sr = new StringReader(json))
        using (JsonTextReader reader = new JsonTextReader(sr))
        {
            JToken token = DeserializeAndCombineDuplicates(reader);
            Dump(token, "");
        }
    }

    private static void Dump(JToken token, string indent)
    {
        Console.Write(indent);
        if (token == null)
        {
            Console.WriteLine("null");
            return;
        }
        Console.Write(token.Type);

        if (token is JProperty)
            Console.Write(" (name=" + ((JProperty)token).Name + ")");
        else if (token is JValue)
            Console.Write(" (value=" + token.ToString() + ")");

        Console.WriteLine();

        if (token.HasValues)
            foreach (JToken child in token.Children())
                Dump(child, indent + "  ");
    }
}

Output:

Object
  Property (name=Foo)
    Array
      Integer (value=1)
      Integer (value=2)
      Integer (value=3)
      Integer (value=4)
  Property (name=Bar)
    Array
      Object
        Property (name=X)
          Array
            String (value=A)
            String (value=B)
      Object
        Property (name=X)
          Array
            String (value=C)
            String (value=D)

How to serialize and deserialize a JSON object with duplicate property names in a specific order?

While not technically malformed, JSON objects with duplicated property names are not recommended by most recent JSON RFC, RFC 8259:

An object structure is represented as a pair of curly brackets surrounding zero or more name/value pairs (or members)...

An object whose names are all unique is interoperable in the sense that all software implementations receiving that object will agree on the name-value mappings. When the names within an object are not unique, the behavior of software that receives such an object is unpredictable. Many implementations report the last name/value pair only. Other implementations report an error or fail to parse the object, and some implementations report all of the name/value pairs, including duplicates.

You may wish to suggest an alternate JSON format to your customer, such as an array of single key-value pair objects.

That being said, if you must serialize and deserialize objects with duplicate properties, Json.NET will not do this out of the box, and you will have to create a custom JsonConverter to do it manually. Since the JSON object's values are all of the same type (here strings), you could bind to a List<KeyValuePair<string, string>>.

First, define the following model:

public class Model
{
    [JsonConverter(typeof(KeyValueListAsObjectConverter<string>))]
    public List<KeyValuePair<string, string>> data { get; } = new ();
}

And the following generic JsonConverter<List<KeyValuePair<string, TValue>>>:

public class KeyValueListAsObjectConverter<TValue> : JsonConverter<List<KeyValuePair<string, TValue>>>
{
    public override List<KeyValuePair<string, TValue>> ReadJson(JsonReader reader, Type objectType, List<KeyValuePair<string, TValue>> existingValue, bool hasExistingValue, JsonSerializer serializer)
    {
        if (reader.MoveToContentAndAssert().TokenType == JsonToken.Null)
            return null;
        reader.AssertTokenType(JsonToken.StartObject);
        var list = existingValue ?? (List<KeyValuePair<string, TValue>>)serializer.ContractResolver.ResolveContract(objectType).DefaultCreator();
        while (reader.ReadToContentAndAssert().TokenType != JsonToken.EndObject)
        {
            var name = (string)reader.AssertTokenType(JsonToken.PropertyName).Value;
            var value = serializer.Deserialize<TValue>(reader.ReadToContentAndAssert());
            list.Add(new KeyValuePair<string, TValue>(name, value));
        }
        return list;
    }

    public override void WriteJson(JsonWriter writer, List<KeyValuePair<string, TValue>> value, JsonSerializer serializer)
    {
        writer.WriteStartObject();
        foreach (var pair in value)
        {
            writer.WritePropertyName(pair.Key);
            serializer.Serialize(writer, pair.Value);
        }
        writer.WriteEndObject();
    }
}

public static partial class JsonExtensions
{
    public static JsonReader AssertTokenType(this JsonReader reader, JsonToken tokenType) => 
        reader.TokenType == tokenType ? reader : throw new JsonSerializationException(string.Format("Unexpected token {0}, expected {1}", reader.TokenType, tokenType));
    
    public static JsonReader ReadToContentAndAssert(this JsonReader reader) =>
        reader.ReadAndAssert().MoveToContentAndAssert();

    public static JsonReader MoveToContentAndAssert(this JsonReader reader)
    {
        if (reader == null)
            throw new ArgumentNullException();
        if (reader.TokenType == JsonToken.None)       // Skip past beginning of stream.
            reader.ReadAndAssert();
        while (reader.TokenType == JsonToken.Comment) // Skip past comments.
            reader.ReadAndAssert();
        return reader;
    }

    public static JsonReader ReadAndAssert(this JsonReader reader)
    {
        if (reader == null)
            throw new ArgumentNullException();
        if (!reader.Read())
            throw new JsonReaderException("Unexpected end of JSON stream.");
        return reader;
    }
}

And you will be able to deserialize and re-serialize the JSON shown in your question.

Demo fiddle here.

Json.NET deserialization: handle varying property names

You could deserialize it to a Dictionary and then iterate over the keys.

(an example in this fiddle).

How to deserialize a json document that does not have property names using C#?

You dont have a json with objects, simply a list that contains another list of strings.

You should deserialize like this,

var obj = JsonConvert.DeserializeObject<List<List<string>>>(myJsonResponse);

How to throw an exception when member comes twice on Deserializing with JsonConvert

You need to added DuplicatePropertyNameHandling = DuplicatePropertyNameHandling.Error in your JsonLoadSettings.

You can dig in details following this answer.

There is also a thread from Newtonsoft.json that cover this topic.

---

Related Topics