How to Deal With Json That Is Encoded Twice

How to deal with JSON that is encoded twice?

Your $test string in the question isn't an accurate reflection of the data you're trying to parse. You've said you copied and pasted it from a log, but when you paste text including backslashes into a string literal, those backslashes are no longer backslashes, they're escape characters. You have to escape them with another backslash. That is, if you have testing\foo in text, and you drop it into a string literal, you need to duplicate that backslash to have the literal create a string with the same text in it: 'testing\\foo'.

So to accurately reflect the text you've copied from the log, the $test literal should be:

$test = 
'{
"username":"sophia",
"event":{
"failure":"unreset",
"answers":{
"question1":"{\\"answer\\":\\"{\\\\\\"pass\\\\\\":true,\\\\\\"mark\\\\\\":9,\\\\\\"totalmark\\\\\\":9,\\\\\\"value\\\\\\":\\\\\\"get full attention|establish classroom rules|teach key words & concepts|use visual aids|demonstrate|check understanding|introduce point system|give handouts|play\\\\\\"}\\",\\"state\\":\\"{\\\\\\"result\\\\\\":{\\\\\\"pass\\\\\\":true,\\\\\\"mark\\\\\\":9,\\\\\\"totalmark\\\\\\":9,\\\\\\"value\\\\\\":\\\\\\"get full attention|establish classroom rules|teach key words & concepts|use visual aids|demonstrate|check understanding|introduce point system|give handouts|play\\\\\\"}}\\"}"
}
},
"event_source":"server"
}';

As you said, question1 is double-encoded. Even worse, when you unencode it, you find it has properties called answer and state that are double-encoded again.

To unencode all of it, you have to first unencode the main bit, then question1, and then its answer and state:

$obj = json_decode($test);
$obj->event->answers->question1 = json_decode($obj->event->answers->question1);
$obj->event->answers->question1->answer = json_decode($obj->event->answers->question1->answer);
$obj->event->answers->question1->state = json_decode($obj->event->answers->question1->state);

If you do that on the text you dumped to the log, it will work. The result (from var_dump) is:


object(stdClass)#1 (3) {
["username"]=>
string(6) "sophia"
["event"]=>
object(stdClass)#2 (2) {
["failure"]=>
string(7) "unreset"
["answers"]=>
object(stdClass)#3 (1) {
["question1"]=>
object(stdClass)#4 (2) {
["answer"]=>
object(stdClass)#5 (4) {
["pass"]=>
bool(true)
["mark"]=>
int(9)
["totalmark"]=>
int(9)
["value"]=>
string(161) "get full attention|establish classroom rules|teach key words & concepts|use visual aids|demonstrate|check understanding|introduce point system|give handouts|play"
}
["state"]=>
object(stdClass)#6 (1) {
["result"]=>
object(stdClass)#7 (4) {
["pass"]=>
bool(true)
["mark"]=>
int(9)
["totalmark"]=>
int(9)
["value"]=>
string(161) "get full attention|establish classroom rules|teach key words & concepts|use visual aids|demonstrate|check understanding|introduce point system|give handouts|play"
}
}
}
}
}
["event_source"]=>
string(6) "server"
}

Live Copy

JSON string encoded twice?

I think you should try JsonConvert.DeserializeObject to deserialize the JSON:

public class Sport
{
// Dummy "Sport" class as it was not mentioned by OP.
public int SportID { get; set; }
public string SportName { get; set; }
}

I get serialized JSON as:

enter image description here

Deserialized it:

string json = JSONTest();
var obj = JsonConvert.DeserializeObject<List<Sport>>(json);

Output:

enter image description here

UPDATE:

As per OP's shared JSON (which is being received from server), encoding can be removed by using:

private string RemoveEncoding(string encodedJson)
{
var sb = new StringBuilder(encodedJson);
sb.Replace("\\", string.Empty);
sb.Replace("\"[", "[");
sb.Replace("]\"", "]");
return sb.ToString();
}

Deserialize it by:

string js = "\"[{\\\"SportID\\\":1,\\\"SportName\\\":\"Tennis\\\"},{\"SportID\\\":2,\\\"SportName\\\":\\\"Footbal\\\"},{\"SportID\\\":3,\"SportName\":\\\"Swimming\\\"}]\"";

string res = RemoveEncoding(js);
var obj = JsonConvert.DeserializeObject<List<Sport>>(res);

Fixing json data accidentally double encoded

You can use JSON_UNQUOTE for this:

update yourtable set yourcolumn=json_unquote(yourcolumn) where yourcolumn like '"%';

https://dbfiddle.uk/?rdbms=mysql_5.7&fiddle=54e11ad53fb30d967c325445d0e08874

Double encode for JSON

The basic problem is that you are trying to construct JSON by mashing together strings (instead of using a JSON library) … and encoding parts of those strings (instead of encoding the resulting JSON string (i.e. the non-URI data that you want to put into the URI) … which jQuery will do for you).

If you want to submit JSON as part of a form encoded data request, then you would:

$.ajax({
data: {
dataString: JSON.stringify({
"mode": "UPDATE_INTNOTE",
"jobid": jobId,
"note": $('#adminNote').val()
})
},

Since you have a flat data structure here (i.e. no objects or arrays in the data) then you don't need to use JSON at all.

$.ajax({
        data: { 
            "mode": "UPDATE_INTNOTE", 
            "jobid": jobId, 
            "note": $('#adminNote').val() 
            }, 

(Then, in your PHP, just access $_POST['mode'] etc, instead of parsing the JSON and then getting the data from the resulting object)

Avoid double encoding json

Doing some quick research on pandas to_json, that function returns a string representation of your json object, not the actual json object.

If your intent is to send it as part of the actual object instead of just a string, I would parse it before sending it, aka, the equivalent of: job.result = JSON.parse(result.to_json(date_format='iso')).

Edit: because I kind of answered in the context of javascript instead of python, json.loads(result.to_json(date_format='iso')) is the Python solution the OP ended up using.

php json_encode is encoding the same data twice

This is because you are using fetch_array() (emphasis mine):

mysqli_fetch_array() is an extended version of the mysqli_fetch_row() function. In addition to storing the data in the numeric indices of the result array, the mysqli_fetch_array() function can also store the data in associative indices, using the field names of the result set as keys.

Use fetch_assoc() instead.

How to avoid double encoding of raw JSON output by a view

You end up double-escaping/encoding the JSON. The Razor Helper @ under most circumstances is going to try and encode the input and return HTML entities for things like quotes, ampersands, less/greater signs etc.

You can use the @Html.Raw method to indicate that the input is already an encoded string.

var stores = '@Html.Raw(ViewBag.nearbyStores)';
var geojson = JSON.parse(stores);

But note I'd be careful. You never know what might be in the source variable that could cause some sort of injection attack (an unescaped single quote for example).

You should be able to avoid the extra parsing altogether:

var stores = @Html.Raw(ViewBag.nearbyStores);

But that's with the understanding the nearbyStores is in fact valid JSON.

Rather than serializing in your controller, you could make this the responsibility of the view using JsonHelper.Serialize

//controller 
ViewBag.nearbyStores = mapFeatureCollection;
// view
var stores = @Html.Raw(Json.Serialize(ViewBag.nearbyStores));


Related Topics



Leave a reply



Submit