How to Handle Newlines in Json

How do I handle newlines in JSON?

This is what you want:

var data = '{"count" : 1, "stack" : "sometext\\n\\n"}';

You need to escape the \ in your string (turning it into a double-\), otherwise it will become a newline in the JSON source, not the JSON data.

How to parse a newline delimited JSON

As doubleunary said, you are likely missing .getContentText()

Summary: That the response from:

let response = UrlFetchApp.fetch(url, {'muteHttpExceptions': true});

Is an HTTPResponseObject, so trying to parse it as a string or JSON won't work. To fix it you need to add in a call to getContentText().

Except doubleunary's answer won't work because the result is BigQuery newline delimited JSON, so you will still need to split it by newlines as you have suggested.

So just adding that step:

function myFunction() {

let url = 'url_address';
let storedData = [];
let response = UrlFetchApp.fetch(url, {'muteHttpExceptions': true});
let contentText = response.getContentText();
let storedResponse = contentText.split("\n");

storedResponse.forEach((data) => {
   storedData.push(JSON.parse(data));
});
}

Should solve your issue, give it a try (seeing as I can't test it).

Reference

• HTTPResponseObject

Edit:

Its possible that you may get a few JSON lines being invalid due to the presence of a newline character that is within the JSON.

A way to minimize this would be to split by a closing curly brace and a newline
"}\n". Though in theory, a JSON line could also contain this.

function myFunction() {

let url = 'url_address';
let storedData = [];
let response = UrlFetchApp.fetch(url, {'muteHttpExceptions': true});
let contentText = response.getContentText();
let storedResponse = contentText.split("}\n");

storedResponse.forEach((data) => {
   storedData.push(JSON.parse(data + "}")); // you would need to add the ending brace again to parse it
});
}

Another approach you could take, if the JSON structure is always the same, is to use a regex like this one:

^{.+"issues":\[.+\]}$/gm

Which looks for chunks of text that start with a newline, and { with a variable amount of content inbetween, and then having "issues":[ with another variable about of data and finally ]}\n. This should catch most instances of possible newlines within a JSON that are causing the malformed JSONs coming out the split. The gm is the global flag allowing multiple results and the m is the multiline flag that allows you to identify line starts and ends with the ^ and $ characters.

You could implement it with string.match:

function myFunction() {

let url = 'url_address';
let storedData = [];
let response = UrlFetchApp.fetch(url, {'muteHttpExceptions': true});
let contentText = response.getContentText();
let storedResponse = contentText.match(/^{.+"issues":\[\]}$/gm);

storedResponse.forEach((data) => {
   storedData.push(JSON.parse(data));
});
}

JSON.parse with newline

JSON.parse('{"x": "\n"}') fails because '{"x": "\n"}' is not a valid JSON string due to the unescaped slash symbol.

JSON.parse() requires a valid JSON string to work.

'{"x": "\\n"}' is a valid JSON string as the slash is now escaped, so JSON.parse('{"x": "\\n"}') will work.

JSON.parse(JSON.stringify({"x" : "\n"})) works because JSON.stringify internally escapes the slash character.

The result of JSON.stringify({"x" : "\n"}) is {"x":"\n"} but if you try to parse this using JSON.parse('{"x":"\n"})' it will FAIL, as it is not escaped. As JSON.stringify returns an escaped character, JSON.parse(JSON.stringify()) will work.

Use newline in string by json

You can modify your received JSON string to replace all of the \n with a real newline character.

Based on your current output you have raw separate \ and n characters next to each other. So to fix this we just need to find all of those instances and replace them with what we want.

We first have to search for instances of \\\\n, which may seem complicated, but once you consider the escape characters, it turns into a raw \\n, which is what's really currently in your json. When the json decoder sees this, it doesn't see a newline character since you escape it with the backslash at the beginning, leading to a literal \n in the output.

Once we find the undesirable instances, we need to replace it with what we really want, \\n. This turns into a raw \n as explained before. The json decoder then see this as a newline character and creates it as such in the decoded output leading to your desired result when you show it in the Text widget.

_respserver = await cl.get('datacontents.json');
String jsonRaw = utf8.decode(_respserver.bodyBytes);
jsonRaw = jsonRaw.replaceAll("\\\\n","\\n");//Find and replace undesirable instances here
_analyzed = json.decode(jsonRaw);

Text(_analyzed['description'])

---

To do it after decoding, do the following:

_respserver = await cl.get('datacontents.json');
_analyzed = json.decode(utf8.decode(_respserver.bodyBytes));

_analyzed['description'] = _analyzed['description'].replaceAll("\\n" ,"\n");

Text(_analyzed['description'])

---

Related Topics