JSONvalueproviderfactory Throws "Request Too Large"

JsonValueProviderFactory throws request too large

If you use JSON.NET for serialization/deserialization, you could substitute the default JsonValueProviderFactory with a custom one as shown in this blog post:

public sealed class JsonDotNetValueProviderFactory : ValueProviderFactory
{
   public override IValueProvider GetValueProvider(ControllerContext controllerContext)
   {
        if (controllerContext == null)
            throw new ArgumentNullException("controllerContext");

        if (!controllerContext.HttpContext.Request.ContentType.StartsWith("application/json", StringComparison.OrdinalIgnoreCase))
            return null;

        var reader = new StreamReader(controllerContext.HttpContext.Request.InputStream);
        var bodyText = reader.ReadToEnd();

        return String.IsNullOrEmpty(bodyText) ? null : new DictionaryValueProvider<object>(JsonConvert.DeserializeObject<ExpandoObject>(bodyText, new ExpandoObjectConverter()) , CultureInfo.CurrentCulture);
    }
}

and in your Application_Start:

ValueProviderFactories.Factories.Remove(ValueProviderFactories.Factories.OfType<JsonValueProviderFactory>().FirstOrDefault());
ValueProviderFactories.Factories.Add(new JsonDotNetValueProviderFactory());

and if you want to stick with the default factory which uses the JavaScriptSerializer class you could adjust the maxJsonLength property in your web.config:

<system.web.extensions>
    <scripting>
        <webServices>
            <jsonSerialization maxJsonLength="2147483644"/>
        </webServices>
    </scripting>
</system.web.extensions>

JsonValueProvider throwing error from MVC App

When your controller method receives json, the DefaultModelBinder uses the JsonValueProviderFactory, which uses JavaScriptSerializer to deserialize the json string and adds the name/value pairs to a dictionary which is passed to the DictionaryValueProvider which in turn provides the values for binding.

Your error is caused because your json contains an object with duplicate property names, for example { name: someValue, NAME: anotherValue } so once name: someValue has been added to the dictionary, adding NAME: anotherValue throws an exception because of the duplicate (case-insensitive) key. This makes sense because a model cannot include 2 properties with the same name.

While the best solution will be to track down and correct the source of the 'invalid' json, you can write your own ValueProvideFactory to bind only the first value and ignore subsequent values with the same key. To save reinventing the wheel use the following steps:

1. Copy the source code of the JsonValueProviderFactory.cs source
   code into your project
2. Rename the namespace (say yourProject.ValueProviderFactories) and
   add a using System.Web.Mvc; statement
3. Rename the class (say InvalidJsonValueProviderFactory)

4. Modify the following code inside the private class EntryLimitedDictionary

   public void Add(string key, object value)
   {
       if (++_itemCount > _maximumDepth)
       {
           // throw new InvalidOperationException(MvcResources.JsonValueProviderFactory_RequestTooLarge);
       }
       // Add the following if block
       if (_innerDictionary.ContainsKey(key))
       {
           return;
       }
       _innerDictionary.Add(key, value);
   }
   

5. Add the following to your global.asax.cs file

   ValueProviderFactories.Factories.Insert(0, new yourProject.ValueProviderFactories.InvalidJsonValueProviderFactory());
   

Note that it needs to be added first because the JsonValueProviderFactory has already been added to the collection of factories.

This will prevent your exception, but of course means the duplicate property will be throw away. If its important that you get that duplicate, then you could consider more modifications to create a new dictionary entry with a different unique key containing indexers, for example

private int _index = 0;
....
if (_innerDictionary.ContainsKey(key))
{
    string invalidKey = string.Format("duplicates[{0}]", _index++);
    string invalidValue = string.Format("{0}|{1}", key, value);
    _innerDictionary.Add(invalidKey, invalidValue);
    return;
}

where index would start at zero and be incremented each time. Then you could add an additional parameter in the method string[] duplicates which would be populated with the duplicate values. At the very least it would aid in debugging the source of the problem.

Getting The JSON request was too large to be deserialized

You have to adjust the maxJsonLength property to a higher value in web.config to resolve the issue.

<system.web.extensions>
    <scripting>
        <webServices>
            <jsonSerialization maxJsonLength="2147483644"/>
        </webServices>
    </scripting>
</system.web.extensions>

Set a higher value for aspnet:MaxJsonDeserializerMembers in the appSettings:

<appSettings>
  <add key="aspnet:MaxJsonDeserializerMembers" value="150000" />
</appSettings>

If those options are not working you could try creating a custom json value provider factory using JSON.NET as specified in this thread.

JsonValueProviderFactory: System.ArgumentException: An item with the same key has already been added

I think there is nothing wrong with your design, but one of your class may have duplicated property which will cause runtime exception.

for example

public int storefrontId {get; set;}
public int StorefrontId {get; set;}

And you need to configure log4net to log your action calls.
for ex:

2021-02-16 10:24:17.5632|2|INFO|Microsoft.AspNetCore.Hosting.Diagnostics|Request finished in 141.7419ms 200  |url: http://myapp/OrderUpdated|action:

EDIT
Here is how you can do request log using DelegatingHandler

public class RequestLogHandler : DelegatingHandler
{
    private static readonly ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
    
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        if (request.Content != null)
        {
            string requestBody = await request.Content.ReadAsStringAsync();
            log.Info($"url {request.RequestUri} body = {requestBody}");
        }
        //// let other handlers process the request
        var result = await base.SendAsync(request, cancellationToken);

        return result;
    }
}

Register handler in config

config.MessageHandlers.Add(new RequestLogHandler());

This will give you something like below.

Furthermore, I will tell steps to override JsonValueProviderFactory AddToBackingStore method. You use that for find what property causing this issue.

1. Get source code from here.

2. Add Class MyJsonValueProviderFactory.cs

3. Register your new class before JsonValueProviderFactoruy in Global.asax.cs

   ValueProviderFactories.Factories.Insert(0, new MyJsonValueProviderFactory());

or remove original first and use yours.

ValueProviderFactories.Factories.Remove(ValueProviderFactories.Factories.OfType<JsonValueProviderFactory>().FirstOrDefault());
ValueProviderFactories.Factories.Add(new MyJsonValueProviderFactory());

Play with this class with exception catching, you will be able to find where is the problem, you may start with Add method in EntryLimitedDictionary class.

Again use below link to register error handling globally.
https://learn.microsoft.com/en-us/aspnet/web-api/overview/error-handling/exception-handling

JsonMaxLength exception on deserializing large json objects

The built-in JsonValueProviderFactory ignores the <jsonSerialization maxJsonLength="50000000"/> setting. So you could write a custom factory by using the built-in implementation:

public sealed class MyJsonValueProviderFactory : ValueProviderFactory
{
    private static void AddToBackingStore(Dictionary<string, object> backingStore, string prefix, object value)
    {
        IDictionary<string, object> d = value as IDictionary<string, object>;
        if (d != null)
        {
            foreach (KeyValuePair<string, object> entry in d)
            {
                AddToBackingStore(backingStore, MakePropertyKey(prefix, entry.Key), entry.Value);
            }
            return;
        }

        IList l = value as IList;
        if (l != null)
        {
            for (int i = 0; i < l.Count; i++)
            {
                AddToBackingStore(backingStore, MakeArrayKey(prefix, i), l[i]);
            }
            return;
        }

        // primitive
        backingStore[prefix] = value;
    }

    private static object GetDeserializedObject(ControllerContext controllerContext)
    {
        if (!controllerContext.HttpContext.Request.ContentType.StartsWith("application/json", StringComparison.OrdinalIgnoreCase))
        {
            // not JSON request
            return null;
        }

        StreamReader reader = new StreamReader(controllerContext.HttpContext.Request.InputStream);
        string bodyText = reader.ReadToEnd();
        if (String.IsNullOrEmpty(bodyText))
        {
            // no JSON data
            return null;
        }

        JavaScriptSerializer serializer = new JavaScriptSerializer();
        serializer.MaxJsonLength = 2147483647;
        object jsonData = serializer.DeserializeObject(bodyText);
        return jsonData;
    }

    public override IValueProvider GetValueProvider(ControllerContext controllerContext)
    {
        if (controllerContext == null)
        {
            throw new ArgumentNullException("controllerContext");
        }

        object jsonData = GetDeserializedObject(controllerContext);
        if (jsonData == null)
        {
            return null;
        }

        Dictionary<string, object> backingStore = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
        AddToBackingStore(backingStore, String.Empty, jsonData);
        return new DictionaryValueProvider<object>(backingStore, CultureInfo.CurrentCulture);
    }

    private static string MakeArrayKey(string prefix, int index)
    {
        return prefix + "[" + index.ToString(CultureInfo.InvariantCulture) + "]";
    }

    private static string MakePropertyKey(string prefix, string propertyName)
    {
        return (String.IsNullOrEmpty(prefix)) ? propertyName : prefix + "." + propertyName;
    }
}

The only modification I did compared to the default factory is adding the following line:

serializer.MaxJsonLength = 2147483647;

Unfortunately this factory is not extensible at all, sealed stuff so I had to recreate it.

and in your Application_Start:

ValueProviderFactories.Factories.Remove(ValueProviderFactories.Factories.OfType<System.Web.Mvc.JsonValueProviderFactory>().FirstOrDefault());
ValueProviderFactories.Factories.Add(new MyJsonValueProviderFactory());

Solution: The length of the string exceeds the value set on the maxJsonLength property for POST action

Create a JsonDotNetValueProviderFactory.csand write following code:

public sealed class JsonDotNetValueProviderFactory : ValueProviderFactory
{
  public override IValueProvider GetValueProvider(ControllerContext controllerContext)
 {
    if (controllerContext == null)
        throw new ArgumentNullException("controllerContext");

    if (!controllerContext.HttpContext.Request.ContentType.StartsWith("application/json", StringComparison.OrdinalIgnoreCase))
        return null;

    var reader = new StreamReader(controllerContext.HttpContext.Request.InputStream);
    var bodyText = reader.ReadToEnd();

    return String.IsNullOrEmpty(bodyText) ? null : new DictionaryValueProvider<object>(JsonConvert.DeserializeObject<ExpandoObject>(bodyText, new ExpandoObjectConverter()) , CultureInfo.CurrentCulture);
 }
}

and add following two lines in Global.asax.cs in Application_Start()

        ValueProviderFactories.Factories.Remove(ValueProviderFactories.Factories.OfType<JsonValueProviderFactory>().FirstOrDefault());
        ValueProviderFactories.Factories.Add(new JsonDotNetValueProviderFactory());

Source: Link

Thanks to Darin Dimitrov !!

If above code fails: Follow This might helps a lot

Thanks to dkinchen !!

---

Related Topics