Remove Keys from Anywhere in a Json String Without Deserialization

Remove keys from anywhere in a JSON string without deserialization

One way to solve this is with streaming token-parsing with Gson. This method handles extremely large JSON strings with ease.

Example before and after:

{"a":{"removeme":"unwanted","b":{"c":{"removeme":{"x":1},"d":{"e":123}}}}}
{"a":{"b":{"c":{"d":{"e":123}}}}}

Essential test harness:

String rawJson = "{\"a\":{\"removeme\":\"unwanted\",\"b\":{\"c\":{\"removeme\":{\"x\":1},\"d\":{\"e\":123}}}}}";

final Gson gson = new GsonBuilder().create();
JsonReader reader = gson.newJsonReader( new StringReader( rawJson ) );

StringWriter outWriter = new StringWriter();
JsonWriter writer = gson.newJsonWriter( outWriter );

JsonStreamFilter.streamFilter( reader, writer, Arrays.asList( "removeme" ) );

System.out.println( rawJson );
System.out.println( outWriter.toString() );

Tokenizing and filtering magic:

public class JsonStreamFilter
{
    /**
     * Filter out all properties with names included in the `propertiesToRemove` list.
     * 
     * @param reader JsonReader to read in the JSON token
     * @param writer JsonWriter to accept modified JSON tokens
     * @param propertiesToRemove List of property names to remove
     * @throws IOException
     * @see Gson docs at https://sites.google.com/site/gson/streaming
     */
    public static void streamFilter(
        final JsonReader reader,
        final JsonWriter writer,
        final List<String> propertiesToRemove
    ) throws IOException
    {
        while ( true )
        {
            JsonToken token = reader.peek();
            switch ( token )
            {
                case BEGIN_ARRAY:
                    reader.beginArray();
                    writer.beginArray();
                    break;
                case END_ARRAY:
                    reader.endArray();
                    writer.endArray();
                    break;
                case BEGIN_OBJECT:
                    reader.beginObject();
                    writer.beginObject();
                    break;
                case END_OBJECT:
                    reader.endObject();
                    writer.endObject();
                    break;
                case NAME:
                    String name = reader.nextName();

                    // Skip all nested structures stemming from this property
                    if ( propertiesToRemove.contains( name ) )
                    {
                        reader.skipValue();
                        break;
                    }

                    writer.name( name );
                    break;
                case STRING:
                    String s = reader.nextString();
                    writer.value( s );
                    break;
                case NUMBER:
                    String n = reader.nextString();
                    writer.value( new BigDecimal( n ) );
                    break;
                case BOOLEAN:
                    boolean b = reader.nextBoolean();
                    writer.value( b );
                    break;
                case NULL:
                    reader.nextNull();
                    writer.nullValue();
                    break;
                case END_DOCUMENT:
                    return;
            }
        }
    }
}

Remove the key-value pairs (nested at any level) from a Json String using LinkedTreeMap

I did it by traversing recursively inside the Nested LinkedTreeMap till I find the field and remove it if it exists. The full path of Key needs to be provide to get the exact key-value location inside the Object (like "objects.desc" in the below Json Sample to remove desc from the Json String)

Json Sample:

{
    "message": "MSG",
    "code": "COD001",
    "objects": [
        {
            "resource": "Student",
            "field": "StudentId",
            "desc": "Student Description"
        }
    ]
}

Code Sample:

public MapDifference<String, Object> getMapDifference(String jsonString1, String jsonString2) {
    MapDifference<String, Object> mapDifference = null;
    Gson gson = new Gson();
    Type mapType = new TypeToken<Map<String, Object>>() {
    }.getType();
    Map<String, Object> firstMap = gson.fromJson(jsonString1, mapType);
    Map<String, Object> secondMap = gson.fromJson(jsonString2, mapType);
    firstMap = CollectionUtils.isEmpty(firstMap) ? new HashMap<>() : firstMap;
    secondMap = CollectionUtils.isEmpty(secondMap) ? new HashMap<>() : secondMap;
    //This contains the List of keys that is required to be filtered out from Json Strings before comparision like {"message", "objects.desc"}
    List<String> firstIgnoreList = getIgnoreList1();
    List<String> secondIgnoreList = getIgnoreList2();

    filterKeys(firstMap, firstIgnoreList);
    filterKeys(secondMap, secondIgnoreList);

    mapDifference = Maps.difference(firstMap, secondMap);
    return mapDifference;
  }


private void filterKeys(Map<String, Object> keyMap, List<String> ignoreList) {
    if (!(CollectionUtils.isEmpty(keyMap) || CollectionUtils.isEmpty(ignoreList))) {
        ignoreList.stream().parallel().forEach(key -> recursiveRemove(keyMap, key));
    }
  }

private static void recursiveRemove(Map<String, Object> keyMap, String key) {
    List<String> path = Arrays.asList(StringUtils.split(key.trim(), "."));
    int size = path.size();
    int index = 0;
    List<LinkedTreeMap> treeMapList = new ArrayList<LinkedTreeMap>();
    treeMapList.add((LinkedTreeMap) keyMap);
    while (index != size - 1) {
      int i = index++;
      List<LinkedTreeMap> treeMapListTemp = new ArrayList<LinkedTreeMap>();
      treeMapList.stream().parallel().forEach(treeMap -> {
        Object obj = treeMap.get(path.get(i));
        if (obj instanceof List) {
          treeMapListTemp.addAll((List<LinkedTreeMap>) obj);
        } else if (obj instanceof LinkedTreeMap) {
          treeMapListTemp.add((LinkedTreeMap) obj);
        }
      });
      treeMapList = treeMapListTemp;
    }
    treeMapList.stream().parallel().forEach(treeMap -> treeMap.remove(path.get(size - 1)));
  }

Serialize a Dictionary<string,Type> and remove key/value from JSON string

It's not fun, but try this:

[Test]
public void Json()
{
         var input = @"{
   ""Description"":""test"",
   ""RoomTypes"":[
      {
         ""Key"":""A"",
         ""Value"":{
            ""Name"":""Delux""
         }
      },
      {
         ""Key"":""B"",
         ""Value"":{
            ""Name"":""Non delux""
         }
      }
   ],
   ""Url"":""http:\/\/test.com""
}";

    var temp = JsonConvert.DeserializeObject<Temp>(input);
    var transform = new Transform
        {
            Description = temp.Description,
            Url = temp.Url,
            RoomTypes = new List<IDictionary<string, Temp.NameObj>>()
        };

    foreach (var group in temp.RoomTypes)
    {
        var dic = new Dictionary<string, Temp.NameObj> {{@group.Key, @group.Value}};
        transform.RoomTypes.Add(dic);
    }

    Console.WriteLine(JsonConvert.SerializeObject(transform));
}

public class Transform
{
    public string Description { get; set; }
    public IList<IDictionary<string, Temp.NameObj>> RoomTypes { get; set; }
    public string Url { get; set; }
}


public class Temp
{
    public string Description { get; set; }
    public IList<GroupObj> RoomTypes { get; set; }
    public string Url { get; set; }

    public class GroupObj
    {
        public string Key { get; set; }
        public NameObj Value { get; set; }
    }

    public class NameObj
    {
        public string Name { get; set; }
    }
}    

The idea is to use Json.Net's dictionary serialization to achieve the structure you want.

how to remove json object key and value.?

delete operator is used to remove an object property.

delete operator does not returns the new object, only returns a boolean: true or false.

In the other hand, after interpreter executes var updatedjsonobj = delete myjsonobj['otherIndustry']; , updatedjsonobj variable will store a boolean
value.

How to remove Json object specific key and its value ?

You just need to know the property name in order to delete it from the object's properties.

delete myjsonobj['otherIndustry'];

let myjsonobj = {
  "employeeid": "160915848",
  "firstName": "tet",
  "lastName": "test",
  "email": "test@email.com",
  "country": "Brasil",
  "currentIndustry": "Remove Keys from Anywhere in a Json String Without Deserializationaaaaa",
  "otherIndustry": "Remove Keys from Anywhere in a Json String Without Deserializationaaaaa",
  "currentOrganization": "test",
  "salary": "1234567"
}
delete myjsonobj['otherIndustry'];
console.log(myjsonobj);

string JSONstring = JsonConvert.SerializeObject(dt, new 
JsonSerializerSettings()
{
            NullValueHandling = NullValueHandling.Ignore,
});

var temp = JArray.Parse(JSONstring);
temp.Descendants()
    .OfType<JProperty>()
    .Where(attr => attr.Value.ToString() == "")
    .ToList() // you should call ToList because you're about to changing the result, which is not possible if it is IEnumerable
    .ForEach(attr => attr.Remove()); // removing unwanted attributes

JSONstring = temp.ToString();

$ jq 'del(.pages)' myfile.json
{
  "actions": {
    "pages": {
      "stuff": "..."
    }
  }
}

$ jq 'del(.. | .pages?)' myfile.json
{
  "actions": {}
}

var json = config.Formatters.JsonFormatter;
json.SerializerSettings.PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.None;

public class Rates
{
    public double ABC { get; set; }
    public double DEF { get; set; }
    public int GHI { get; set; }
    public bool ShouldSerializeGHI()
    {
        if (this.GHI == 0)
            return false;
        return true;
    }
}

public class RootObject
{
    public string Date { get; set; }
    public string Location { get; set; }
    public Rates rates { get; set; }
}

var test = JsonConvert.DeserializeObject<RootObject>(json);
var newJson = JsonConvert.SerializeObject(test);