External JSON Vulnerable Because of JSON.Net Typenamehandling Auto

External json vulnerable because of Json.Net TypeNameHandling auto?

TL/DR: In the absence of any obvious object or dynamic members, you may well be safe, but you are not guaranteed to be safe. To further decrease your risk you should follow the recommendations from the Newtonsoft documentation:

TypeNameHandling should be used with caution when your application deserializes JSON from an external source. Incoming types should be validated with a custom SerializationBinder when deserializing with a value other than None.

Full Answer

The attacks described in How to configure Json.NET to create a vulnerable web API, TypeNameHandling caution in Newtonsoft Json and Alvaro Muñoz & Oleksandr Mirosh's blackhat paper all depend on using the TypeNameHandling setting of Json.NET to trick the receiver into constructing an attack gadget - a instance of a type that when constructed, populated or disposed effects an attack on the receiving system.

Json.NET does two things that help protect against such attacks. Firstly, it ignores unknown properties. Thus simply adding an additional, unknown property to a JSON payload whose value contains a "$type" property should do no harm. Secondly, during deserialization of a polymorphic value, when resolving the "$type" property, it checks to see whether the resolved type is compatible with the expected type in JsonSerializerInternalReader.ResolveTypeName():

    if (objectType != null
#if HAVE_DYNAMIC
        && objectType != typeof(IDynamicMetaObjectProvider)
#endif
        && !objectType.IsAssignableFrom(specifiedType))
    {
        throw JsonSerializationException.Create(reader, "Type specified in JSON '{0}' is not compatible with '{1}'.".FormatWith(CultureInfo.InvariantCulture, specifiedType.AssemblyQualifiedName, objectType.AssemblyQualifiedName));
    }

If the expected type of the polymorphic value is not compatible with any attack gadget type, the attack will fail. Provided you have no serializable members of type object, dynamic or IDynamicMetaObjectProvider, this is likely to be true. But not certain!

Cases in which an attack gadget might get constructed even without any obvious untyped members in your data model include:

• Deserialization of untyped collections. If you are deserializing any sort of untyped collection or dictionary such as ArrayList, List<object>, Dictionary<string, dynamic> or HashTable, then your system is vulnerable to attack gadgets contained in the collection's items.

• Deserialization of any of the dozens of collections inheriting from CollectionBase. This type predates the introduction of generics in .Net and represents a "semi-typed" collection, in which the types of the items are validated in runtime as they are added. Since the validation occurs after construction, there is a window in which an attack gadget might get constructed.

  Sample fiddle showing just this.

• Deserialization of values that share a common base type or interface with an attack gadget other than just object. TempFileCollection implements ICollection and IDisposable. ObjectDataProvider implements INotifyPropertyChanged and ISupportInitialize. If you have any polymorphic members or values that are declared to be any of these interfaces, you are vulnerable.

• Deserialization of types that implement ISerializable. Json.NET supports this interface by default, and it is possible that a seemingly-harmless type in some external library is deserializing untyped members inside its streaming constructor without your knowledge.

  One obvious example is Sytem.Exception (or any of its subtypes) which deserializes an untyped dictionary "Data" inside its streaming constructor which corresponds to the untyped dictionary Exception.Data. If you are deserializing an Exception (contained in a log file for example, which is very common), the following JSON should effect an attack:

  {
    "$type": "System.Exception, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089",
    "ClassName": "System.Exception",
    "Message": "naughty exception",
    "Data": {
      "$type": "System.Collections.ListDictionaryInternal, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089",
      "data": {
        "$type": "System.IO.FileInfo, System.IO.FileSystem",
        "fileName": "rce-test.txt",
        "IsReadOnly": true    
      }
    },
  }
  

  The attack can be mitigated without creation of a custom serialization binder by setting DefaultContractResolver.IgnoreSerializableInterface = true. Of course, this may cause problems with serialization of certain .Net class library types.

• Deserializing types marked with [Serializable] can have a similar problem if you set DefaultContractResolver.IgnoreSerializableAttribute = false. However, the default is true, so you should be OK if you don't change this setting.

• Deserializing types with members that you think are not serialized -- but will be deserialized if present. E.g. consider the following type:

  public MyType
  {
      public object tempData;
      public bool ShouldSerializeTempData() { return false; }
  }
  

  Thanks to Json.NET's conditional serialization functionality, the tempData member will never be serialized, so you might think you're in the clear. But it will be deserialized if present! An attacker who decompiles your code and notices such a member will be able to craft an attack gadget payload for MyType.

And that's just what I was able to think of off the top of my head. As you can see, verifying that, in a large object graph, there is never an attempt to deserialize a polymorphic type that is compatible with some attack gadget is substantially nontrivial. Thus I'd strongly recommend the additional protection of a custom SerializationBinder that ensures that no unexpected types are deserialized.

TypeNameHandling caution in Newtonsoft Json

When deserialize with TypeNameHandling.All and without a SerializationBinder checks json.net will try to create a instace of the type that comes as metadata in the JSON.

public class Car
{
    public string Maker { get; set; }
    public string Model { get; set; }
}

{
   "$type": "Car",
   "Maker": "Ford",
   "Model": "Explorer"
} //create a Car and set property values

But an attacker could send you dangerous types that exist in your code or in the framework.

i.e. from here System.CodeDom.Compiler.TempFileCollection is a serializable class whose purpose is to maintain a list of temporary files which resulted from a compilation process and delete them when they are no longer needed. To ensure that the files are deleted the class implements a finalizer that will be called when the object is being cleaned up by the Garbage Collector. An attacker would be able to construct a serialized version of this class which pointed its internal file collection to any file on a victims system. This will be deleted at some point after deserialization without any interaction from the deserializing application.

    [Serializable]
    public class TempFileCollection
    {
       private Hashtable files;
       // Other stuff...

       ~TempFileCollection()
       {
         if (KeepFiles) {return}
         foreach (string file in files.Keys)
         {
            File.Delete(file);
         }
       }
    }

   {
       "$type": "System.CodeDom.Compiler.TempFileCollection",
       "BasePath": "%SYSTEMDRIVE",
       "KeepFiles": "False",
       "TempDir": "%SYSTEMROOT%"
    } // or something like this, I just guessing but you got the idea

NewtonSoft SerializeObject ignores TypeNameHandling

TypeNameHandling works by adding a special, reserved "$type" property to JSON objects specifying the .Net type that was serialized. In addition, as explained in Newtonsoft's Serialization Guide, arrays will get nested inside a wrapper object that specifies the type:

Note that if TypeNameHandling or PreserveReferencesHandling has been enabled for JSON arrays on the serializer, then JSON arrays are wrapped in a containing object. The object will have the type name/reference properties and a $values property, which will have the collection data.

However, there is no such special case implemented when serializing JSON primitives. A .Net object that is serialized as a primitive will be emitted as a primitive without any wrapper added, even if TypeNameHandling is specified.

Thus, if you want to specify .Net type information for JSON primitives, you must create a wrapper object yourself. Some examples can be found in this answer to Deserialize Dictionary<string, object> with enum values in C# or this answer to JSON.net (de)serialize untyped property.

Following those answers, create the following wrapper:

public abstract class TypeWrapper
{
    // Taken from this answer https://stackoverflow.com/a/38340375/3744182
    // To https://stackoverflow.com/questions/38336390/deserialize-dictionarystring-object-with-enum-values-in-c-sharp
    // By https://stackoverflow.com/users/3744182/dbc
    protected TypeWrapper() { }

    [JsonIgnore]
    public abstract object ObjectValue { get; }

    public static TypeWrapper CreateWrapper<T>(T value)
    {
        if (value == null)
            return new TypeWrapper<T>();
        var type = value.GetType();
        if (type == typeof(T))
            return new TypeWrapper<T>(value);
        // Return actual type of subclass
        return (TypeWrapper)Activator.CreateInstance(typeof(TypeWrapper<>).MakeGenericType(type), value);
    }
}

public sealed class TypeWrapper<T> : TypeWrapper
{
    public TypeWrapper() : base() { }

    public TypeWrapper(T value)
        : base()
    {
        this.Value = value;
    }

    public override object ObjectValue { get { return Value; } }

    public T Value { get; set; }
}

Then modify your Convert<T>() as follows:

public static partial class JsonExtensions
{
    static readonly IContractResolver globalResolver = new JsonSerializer().ContractResolver;
    
    public static string Convert<T>(T cacheObject)
    {
        return JsonConvert.SerializeObject(ToTypeWrapperIfRequired(cacheObject), Formatting.Indented, new JsonSerializerSettings
        {
            TypeNameHandling = TypeNameHandling.All
        });
    }
    
    public static T UnConvert<T>(string json)
    {
        var obj = JsonConvert.DeserializeObject<object>(json, new JsonSerializerSettings
        {
            TypeNameHandling = TypeNameHandling.All
        });
        if ((obj is TypeWrapper wrapper))
            return (T)wrapper.ObjectValue;
        return (T)obj;
    }
    
    static object ToTypeWrapperIfRequired<T>(T obj, IContractResolver resolver = null)
    {
        resolver = resolver ?? globalResolver;
        if (obj == null)
            return null;
        // Type information is redundant for string or bool
        if (obj is bool || obj is string)
            return obj;
        var contract = resolver.ResolveContract(obj.GetType());
        if (contract is JsonPrimitiveContract)
            return TypeWrapper.CreateWrapper(obj);
        return obj;
    }
}

And now you can convert to JSON and back as follows:

var json = JsonExtensions.Convert(obj);

var objBack = JsonExtensions.UnConvert<T>(json);

Notes:

• A wrapper will be added to the object being serialized only if necessary -- i.e. if the .Net object is going to be serialized as a JSON primitive rather than a JSON object or array. In addition I don't add a wrapper if the incoming object is a string or a bool as these types can be inferred unambiguously from the JSON.

• To do:

  • Decide whether the type of null-valued nullable structs needs to be preserved. If so they will need wrapping also.
  • Decide what should be done when round-tripping a JValue.
  • Serializing and deserializing an already-wrapped object result in it being unwrapped on deserialization, so you may want to throw an exception from Convert<T>() if the incoming object is a wrapper.

• Do be aware that using TypeNameHandling can introduce security vulnerabilities into your application. For details see TypeNameHandling caution in Newtonsoft Json and External json vulnerable because of Json.Net TypeNameHandling auto?.

• Also be aware that type name information isn't always portable between different .Net frameworks. See for example Serializing and deserializing in different frameworks #1378. You may need to write a custom ISerializationBinder in such situations.

Demo fiddle here.

is Newtonsoft TypeNameHandling.all with a basic namespace check safe?

The risk with TypeNameHandling is that an attacker may trick the receiver into constructing an attack gadget - an instance of a type that when constructed, populated or disposed effects an attack on the receiving system. For an overview see

• TypeNameHandling caution in Newtonsoft Json
• External json vulnerable because of Json.Net TypeNameHandling auto?

If you are going to protect against such attacks by requiring all deserialized types to be in your own company's .Net namespace, be aware that, when serializing with TypeNameHandling.All, "$type" information will appear throughout the JSON token hierarchy, for all arrays and objects (including for .Net types such as List<T>). As such you must needs apply your "$type" check everywhere type information might occur. The easiest way to do this is with a custom serialization binder such as the following:

public class MySerializationBinder : DefaultSerializationBinder
{
    const string MyNamespace = "foo"; //'foo' is our company name, all our namespaces start with foo

    public override Type BindToType(string assemblyName, string typeName)
    {
        if (!typeName.StartsWith(MyNamespace, StringComparison.OrdinalIgnoreCase))
            throw new JsonSerializationException($"Request type {typeName} not supported, please use an IFoo");
        var type = base.BindToType(assemblyName, typeName);
        return type;
    }
}

Which can be used as follows:

var settings = new JsonSerializerSettings
{
    SerializationBinder = new MySerializationBinder(),
    TypeNameHandling = TypeNameHandling.All,
};

This has the added advantage of being more performant than your solution since pre-loading into a JObject is no longer required.

However, having done so, you may encounter the following issues:

1. Even if the root object is always from your company's namespace, the "$type" properties for nested values may not necessarily be in your companies namespace. Specifically, type information for harmless generic system collections such as List<T> and Dictionary<TKey, Value> as well as arrays will be included. You may need to enhance BindToType() to whitelist such types.

   Serializing with TypeNameHandling.Objects or TypeNameHandling.Auto can ameliorate the need to whitelist such harmless system types, as type information for such system types is less likely to get included during serialization as compared to TypeNameHandling.All.

   To further simplify the type checking as well as to reduce your attack surface overall, you might consider only allowing type information on the root object. To do that, see json.net - how to add property $type ONLY on root object. SuppressItemTypeNameContractResolver from the accepted answer can be used on the receiving side as well as the sending side, to ignore type information on non-root objects.

   Alternatively, you could serialize and deserialize with TypeNameHandling.None globally and wrap your root object in a container marked with [JsonProperty(TypeNameHandling = TypeNameHandling.Auto)] like so:

   public class Root<TBase>
   {
       [JsonProperty(TypeNameHandling = TypeNameHandling.Auto)]
       public TBase Data { get; set; }
   }
   

   Since your root objects all seem to implement some interface IFoo you would serialize and deserialize a Root<IFoo> which would restrict the space of possible attack gadgets to classes implementing IFoo -- a much smaller attack surface.

   Demo fiddle here.

2. When deserializing generics, both the outer generic and the inner generic parameter types may need to be sanitized recursively. For instance, if your namespace contains a Generic<T> then checking that the typeName begins with your company's namespace will not protect against an attack via a Generic<SomeAttackGadget>.

3. Even if you only allow types from your own namespace, it's hard to say that's enough to be sufficiently safe, because we don't know whether any of the classes in your own namespace might be repurposed as attack gadgets.

Why does Json.NET not include $type for the root object when TypeNameHandling is Auto?

Short answer: it doesn't because it can't.

As you stated in your question, setting TypeNameHandling to Auto directs Json.Net to include the .NET type name when the actual (run-time) type of the object being serialized is not the same as its declared (compile-time) type. In order to do that, Json.Net needs to know both types for every object.

For everything inside the root object, this is straightforward: just get the runtime type of the root object via GetType(), then use reflection to get all of its declared properties and their types, and for each one compare the declared type to the actual type to see if they differ. If they do, output the type name.

But for the root object itself, Json.Net doesn't have access to both types. All the information it has is the object referenced by fido, whose runtime type is Dog. There's no way for Json.Net to discover that the fido variable was declared as Animal, unless you provide that context somehow. And that is exactly why Json.Net provides overloads of SerializeObject which allow you to specify the compile-time type of the object being serialized. You must use one of these overloads if you want the TypeNameHandling.Auto setting to work for the root object.

Is it possible to set Json.Net to ignore $type?

"$type" information is only written when TypeNameHandling is modified to something other than TypeNameHandling.None -- which is the default. If you never change the value, "$type" information is never emitted.

Similarly "$type" properties are ignored on deserialization when TypeNameHandling = TypeNameHandling.None (which is, again, the default), as is stated in the docs:

// for security TypeNameHandling is required when deserializing
Stockholder newStockholder =
  JsonConvert.DeserializeObject<Stockholder>(jsonTypeNameAuto, new JsonSerializerSettings
{
    TypeNameHandling = TypeNameHandling.Auto
});

If nothing in your code (or in class libraries used by your code) ever modifies TypeNameHandling to something other than TypeNameHandling.None (either via settings or attributes such as JsonPropertyAttribute.TypeNameHandling) then that code execution attack cannot work. (For more precise details on usages of Json.NET's serializer that are and are not vulnerable to this attack, see Alvaro Muñoz & Oleksandr Mirosh's blackhat paper.

Also note that, if you are parsing with JToken.Parse() (or some similar static method like JObject.Parse()) rather than deserializing with JsonSerializer.Deserialize<T>() then the presence of "$type" properties will simply result in such properties getting populated into the JToken hierarchy, since JToken.Parse() never invokes the serializer. If you nevertheless want to strip those"$type" properties after parsing, you can use JsonExtensions.RemoveTypeMetadata(this JToken root) from Deserialize string that was serialized with TypeNameHandling.All to do just that.

That being said, if a collection was serialized by another application using TypeNameHandling.Arrays or TypeNameHandling.All then there will be an extra level of nesting in the JSON. To strip it when deserializing, see IgnoreCollectionTypeConverter from Strategies for migrating serialized Json.NET document between versions/formats or IgnoreArrayTypeConverter from Make Json.NET ignore $type if it's incompatible.

Finally, if you are working with a 3rd party library that sets TypeNameHandling in attributes, you can disable that with a custom contract resolver as shown in How to disable TypeNameHandling when specified in attributes by using JsonSerializerSettings in Json.NET?.

And if you're really concerned that somebody else in your team might enable TypeNameHandling, you could create a custom ISerializationBinder that throws an exception whenever an attempt is made to resolve a type or type name:

public class DisallowSerializationBindingBinder : ISerializationBinder
{
 #region ISerializationBinder Members

 public void BindToName(Type serializedType, out string assemblyName, out string typeName)
 {
  throw new JsonSerializationException("Binding of subtypes has been disabled");
 }

 public Type BindToType(string assemblyName, string typeName)
 {
  throw new JsonSerializationException("Binding of subtypes has been disabled");
 }

  #endregion
}

Then set it in JsonSerializerSettings as follows:

var settings = new JsonSerializerSettings
{
    SerializationBinder = new DisallowSerializationBindingBinder(),
};

And modify the settings globally as shown in Set default global json serializer settings (for a console app), How to set custom JsonSerializerSettings for Json.NET in MVC 4 Web API? (for ASP.NET Web API) or JsonSerializerSettings and Asp.Net Core (for asp.net core).

JsonConvert serialize/deserialize jagged array with TypeNameHandling enabled

This appears to be a bug with TypeNameHandling.Arrays and multidimensional arrays of rank > 2.

I can reproduce the problem more easily by serializing a 3d double array using TypeNameHandling.Arrays:

var root = new double[,,] { { { 1 } } };

var settings = new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.Arrays };
var json = JsonConvert.SerializeObject(root, Formatting.Indented, settings);

// Try to deserialize to the same type as root
// but get an exception instead:
var root2 = JsonConvert.DeserializeAnonymousType(json, root, settings);

The JSON generated by the code above is:

{
  "$type": "System.Double[,], mscorlib",
  "$values": [ [ [ 1.0 ] ] ]
}

The presence of the "$type" property is to be expected, and is documented in TypeNameHandling setting, but as you note it looks wrong: it should have an extra dimension in the array type like so:

  "$type": "System.Double[,,], mscorlib",

And in fact I can deserialize the JSON successfully if I manually replace the [,] with [,,] like so:

// No exception!
JsonConvert.DeserializeAnonymousType(json.Replace("[,]", "[,,]"), root, settings)

Finally, if I try the same test with a 2d array instead of a 3d array, the test passes. Demo fiddle here.

The cause appears to be a bug in the routine ReflectionUtils.RemoveAssemblyDetails when called at the following traceback:

Newtonsoft.Json.Utilities.ReflectionUtils.RemoveAssemblyDetails(string) C#
Newtonsoft.Json.Utilities.ReflectionUtils.GetTypeName(System.Type, Newtonsoft.Json.TypeNameAssemblyFormatHandling, Newtonsoft.Json.Serialization.ISerializationBinder)  C#
Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.WriteTypeProperty(Newtonsoft.Json.JsonWriter, System.Type)   C#
Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.WriteStartArray(Newtonsoft.Json.JsonWriter, object, Newtonsoft.Json.Serialization.JsonArrayContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty)    C#

When called, the input parameter has the value

System.Double[,,], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

But the returned value is

System.Double[,], mscorlib

which is clearly wrong.

An issue could be reported to Newtonsoft here if desired.

Update: a similar issue was opened today: Type of multi-dimensional array is incorrect #1918.

As a workaround, you should limit the scope of properties for which you output type information to situations where a given JSON object might, in practice, be polymorphic. Possibilities include:

1. You could serialize your object graph with TypeNameHandling.None but mark your polymorphic collections with JsonPropertyAttribute.ItemTypeNameHandling = TypeNameHandling.Auto like so:

   public class Baz
   {
       [JsonProperty(ItemTypeNameHandling = TypeNameHandling.Auto)]
       public readonly List<Foo> Foos;
   
       public Baz()
       {
           Foos = new List<Foo>();
       }
   }
   

   This solution results in less bloated JSON and also minimizes the security risks of using TypeNameHandling that are described in TypeNameHandling caution in Newtonsoft Json and External json vulnerable because of Json.Net TypeNameHandling auto? and thus is the preferable solution.

2. You could serialize your object graph with TypeNameHandling.None and use a custom contract resolver to set JsonArrayContract.ItemTypeNameHandling to TypeNameHandling.Auto for collections with potentially polymorphic items, by overriding DefaultContractResolver.CreateArrayContract.

   This would be the solution to use if you cannot add Json.NET attributes to your types.

3. You could serialize your object graph with TypeNameHandling.Auto or TypeNameHandling.Objects.

   Either option will avoid the bug and also reduce bloat in your JSON, but will not reduce your security risks.

4. You could serialize your object graph with JsonSerializerSettings.TypeNameAssemblyFormatHandling = TypeNameAssemblyFormatHandling.Full.

   This avoids the call to RemoveAssemblyDetails() but results in even more bloated JSON and does not avoid the possible security risks.

Serializing a dictionary with IConvertible values using Json.NET with the TypeNameHandling-flag

You actually have several problems here:

1. You seem to have encountered an odd problem with Json.NET when deserializing to a target type of IConvertible. When deserializing a convertible primitive type, it calls the system routine Convert.ChangeType() to convert the primitive to the target type (e.g. long to int). And, for some reason, this system routine throws an exception when asked to convert a primitive to type IConvertible, even though that primitive is already of that type.

2. You are using TypeNameHandling.Objects to serialize your dictionary of convertible values, however this setting is only documented to work work serializing to a JSON object. Your values, however, will be serialized as JSON primitives, so the setting does not apply.

   To preserve type information for a dictionary of polymorphic primitives, you need to manually wrap the values in a container object, such as the one shown in this answer to Deserialize Dictionary<string, object> with enum values in C#. (That answer does not work here, however, because of problem #1.)

3. Unless you write a custom serialization binder, TypeNameHandling is insecure and vulnerable to attack gadget injection attacks such as the ones shown in TypeNameHandling caution in Newtonsoft Json and External json vulnerable because of Json.Net TypeNameHandling auto?.

4. You are not using the same settings to deserialize as you are to serialize.

The above problems can be resolved with the use of the following custom JsonConverter:

public class ConvertibleDictionaryConverter : JsonConverter
{
    [JsonDictionary(ItemTypeNameHandling = TypeNameHandling.Auto)]
    class ConvertibleDictionaryDTO : Dictionary<string, ConvertibleWrapper>
    {
        public ConvertibleDictionaryDTO() : base() { }

        public ConvertibleDictionaryDTO(int count) : base(count) { }
    }

    public override bool CanConvert(Type objectType)
    {
        return typeof(IDictionary<string, IConvertible>).IsAssignableFrom(objectType);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        var dto = serializer.Deserialize<ConvertibleDictionaryDTO>(reader);
        if (dto == null)
            return null;
        var dictionary = (IDictionary<string, IConvertible>)(existingValue ?? serializer.ContractResolver.ResolveContract(objectType).DefaultCreator());
}

---

Related Topics