Nsjsonserialization Serialization of a String Containing Forward Slashes/And HTML Is Escaped Incorrectly

NSJSONSerialization serialization of a string containing forward slashes / and HTML is escaped incorrectly

I believeNSJSONSerialization is behaving as designed in regards to encoding HTML.

If you look at some questions (1, 2) on encoding HTML in JSON you'll see the answers always mention escaping the forward slashes.

JSON doesn't require forward slashes to be escaped, but HTML doesn't allow a javascript string to contain </ as it can be confused with the end of the <SCRIPT> tag.

See the answers here, here and most directly the w3.org HTML4 Appendix which states in B.3.2 Specifying non-HTML data

ILLEGAL EXAMPLE: 
The following script data incorrectly contains a "</" sequence (as part of "</EM>") before the SCRIPT end tag:

<SCRIPT type="text/javascript">
  document.write ("<EM>This won't work</EM>")
</SCRIPT>

Although this behaviour may cause issues for you NSJSONSerialisation is just playing by the age old rules of encoding HTML data for use in <SCRIPT> tags.

how to prevent NSJSONSerialization from adding extra escapes in URL

Yeah, this is quite irritating and even more so because it seems there's no "quick" fix to this (i.e. for NSJSONSerialization)

source:

http://www.blogosfera.co.uk/2013/04/nsjsonserialization-serialization-of-a-string-containing-forward-slashes-and-html-is-escaped-incorrectly/

or

NSJSONSerialization serialization of a string containing forward slashes / and HTML is escaped incorrectly

---

(just shooting in the dark here so bear with me)

If, you're making your own JSON then simply make an NSData object out of a string and send it to the server.

No need to go via NSJSONSerialization.

Something like:

NSString *strPolicy = [info description];
NSData *policyData = [strPolicy dataUsingEncoding:NSUTF8StringEncoding];

i know it won't be so simple but... hm... anyways

json.dumps(): escaping forward slashes

Only escape forward slashes when encode_html_chars=True

Check out this-
https://github.com/esnme/ultrajson/pull/114

The JSON spec says forward slashes shall be escaped implicitly.

Here is a solution to do it in JSONEncoder itself. Its just that you create an ESCAPE DICTIONARY and do computation before hand and do the encoding later.

https://chromium.googlesource.com/external/googleappengine/python/+/dc33addea2da464ca07e869cb11832e1ae82da9d/lib/django/django/utils/simplejson/encoder.py

Hope it helps.

-

Adding to the above solution, there is another reason to escape the characters. As kay said, it gives us some extra sleep. It prevents the attack. So the solution above takes care of all issues.

ESCAPE_DCT = {
    # escape all forward slashes to prevent </script> attack
    '/': '\\/',
    '\\': '\\\\',
    '"': '\\"',
    '\b': '\\b',
    '\f': '\\f',
    '\n': '\\n',
    '\r': '\\r',
    '\t': '\\t',
}

Jackson serialization: String field containing regex expression de-escaping to get actual literal

Thanks for @Bohemian and @NyamiouTheGaleanthrope

Indeed as you said, I think I have found the problem: the deserializer should be org.springframework.kafka.support.serializer.JsonDeserializer in the Consumer, and in the producer, the serializer should be org.springframework.kafka.support.serializer.JsonSerializer. Then all is good, I can see in the log that the regex has no extra escape char. I was using String(De)Serializer before.

Configs for both sides put together:

application.yml:

spring:
  kafka:
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
    consumer: # would be picked if you autowired the consumer
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
      properties.spring.json.trusted.packages: '*'

Producer:

ProducerRecord<String, OrderMessageDto> record = new ProducerRecord<>(
    kafkaTopicProperties.getTopic(),
    String.valueOf(orderDto.getId()),
    orderDto
);
kafkaTemplateOrder.send(record).get(kafkaTopicProperties.getTimeout(), TimeUnit.MILLISECONDS);

Consumer(I only have a consumer in test, so I have to configure by hand):

    @Autowired
    private EmbeddedKafkaBroker kafkaBroker;

    ...

    private ConsumerRecords consumeRecords(String topic) {
        Map<String, Object> consumerProps = KafkaTestUtils.consumerProps(topic, "true", kafkaBroker);
        consumerProps.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
        consumerProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        consumerProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
        JsonDeserializer<OrderPaymentUrlDto> valueDeserializer = new JsonDeserializer<>();
        valueDeserializer.addTrustedPackages("*"); // necessary for include DTO as trusted type
        ConsumerFactory<String, OrderPaymentUrlDto> factory = new DefaultKafkaConsumerFactory<>(
                consumerProps,
                new StringDeserializer(),
                valueDeserializer
        );
        Consumer consumer = factory.createConsumer();
        kafkaBroker.consumeFromAnEmbeddedTopic(consumer, topic);
        return KafkaTestUtils.getRecords(consumer, 2000);
    }

escaping json string with a forward slash?

You shouldn't need to do anything special to represent a / in JSON - a string can contain any character except a " or (when not used to start an escape sequence) \.

The problem is possibly therefore in:

• the way you parse the JSON server side
• the way your parse the HTTP data to get the JSON string
• the way you encode the string before making the HTTP request

(I'd bet on it being the last of those options).

I would start by using a tool such as LiveHttpHeaders or Charles Proxy to see exactly what data is sent to the server.

(I'd also expand the question with the code you use to make the request, and the code you use to parse it at the other end).

Is there a way to tell the json-smart parser not to escape forward slashes?

myJSONObject.toJSONString(new JSONStyle(JSONStyle.FLAG_PROTECT_4WEB)) passes slashes through undisturbed.

---

Related Topics