How to Solve Uncaught Rangeerror When Download Large Size JSON

How to solve Uncaught RangeError when download large size json

You can use fetch(), Response.body.getReader() which returns a ReadableStream, TextDecoder(), Blob(), URL.createObjectURL().

Note, at device having limited RAM or low available disk space, after clicking Save at Save File dialog four minutes and twenty seconds 4:20 elapsed before the Save File dialog closed, followed by an additional one minute and thirty seconds 1:30 before the .crdownload extension was removed from the file at file manager GUI. During the first 4:20 period where the file is downloading to filesystem and the Save File dialog is visible the mouse pointer is movable, though the UI is temporarily unresponsive to clicks or attempts to change tabs. When the Save File dialog closes and the the file is still downloading to filesystem, having extension .crdownload the UI returns to normal functionality.

At the conclusion of the process described above the file was successfully downloaded to local filesystem having a total size of 189.8 MB (189,778,220 bytes).

<!DOCTYPE html><html>
<head>  <style>    code {      color:navy;      background-color:#eee;      padding:2px;    }  </style></head>
<body>  <button>Request File</button><br>  <progress min="0" max="189778220" value="0"></progress>  <output></output>  <br><br>  <label></label>  <script>    var url = "https://raw.githubusercontent.com/zemirco/sf-city-lots-json/master/citylots.json";    var button = document.querySelector("button");    var progress = document.querySelector("progress");    var label = document.querySelector("label");    var output = document.querySelector("output");        var request = (url) => {            label.innerHTML = `Requesting <code>${url}</code> at ${new Date()}.<br><br>`;            return fetch(url)      .then(response => response.body.getReader())      .then(reader => {        var decoder = new TextDecoder();        var json = "";        label.innerHTML += `Request successful.<br><br>Reading request body at ${new Date()}.<br><br>`;        return reader.read().then(function processData(result) {            if (result.done) {              // do stuff when `reader` is `closed`              return reader.closed.then(function() {                // return `json` string                return json;              });            };            json += decoder.decode(result.value);            output.innerHTML = ` ${json.length} of ${progress.max} bytes read`;            progress.value = json.length;            return reader.read().then(processData)          })          .then(function(data) {            var message = `Reading of <code>${url}</code> complete at ${new Date()}. <br><br>`                                + `${data.length} total bytes read. `                               + `Please allow up to 4 minutes for file to download `                               + `to filesystem after clicking <code>Save</code>.<br><br>`;            label.innerHTML += message;                        var blob = new Blob([data], {              type: "application/json"            });            var file = URL.createObjectURL(blob);            var a = document.createElement("a");            a.download = "citylots.json";            a.href = file;            document.body.appendChild(a);            a.click();            var closeBlob = (e) => {              window.removeEventListener("focus", closeBlob);              blob.close();              URL.revokeObjectURL(file);            };            window.addEventListener("focus", closeBlob);            return message.replace(/<[^>]*>/g, "");          })          .catch(function(err) {            console.log("err", err)          })      });    }        var handleRequest = (e) => {      button.setAttribute("disabled", "disabled");      request(url).then(function(message) {        console.log(message);        button.removeAttribute("disabled");      })    };        button.addEventListener("click", handleRequest);  </script></body></html>

javascript program stopped working for large values

The real bottleneck here would the browser rendering. Don't forget that you are adding a huge amount of data to the DOM that all has to be handled.

If you just want to try out loops and play around in Javascript, maybe the better choice would be to use NodeJS where you can run your program outside of the browser environment

var timerLabel = 'running_loop';function loop() {  var num = 5;  while (++num < 20000000) {  }}
// starts a timerconsole.time(timerLabel);loop();// marks in ms how long the operation tookconsole.timeEnd(timerLabel);

Triggering a large file download from a fully-formed response

I've decided to go with Approach #7 for now.

javascript or jquery download large file as URI

You passed too long URL in XAMPP. XAMPP stands for Apache. In Apache,
maximum URL length is about 4,000 characters, after which Apache produces a "413 Entity Too Large" error.

I agree with @PatrickEvans it's better to use URL.createObjectURL.
URL.createObjectURL() can be used to construct and parse URLs. URL.createObjectURL() specifically, can be used to create a reference to a File or a Blob. As opposed to a base64-encoded data URL, it doesn’t contain the actual data of the object – instead it holds a reference.

The nice thing about this is that it’s really fast. Previously, we’ve had to instantiate a FileReader instance and read the whole file as a base64 data URL, which takes time and a lot of memory. With createObjectURL(), the result is available straight away, allowing us to do things like reading image data to a canvas.

As you can see in following demo. Two link are same.But if you inspect on Without createObjectURL link href attribute has too large to edit but in With createObjectURL link you can edit it because for creating it I used URL.createObjectURL().

Online demo (jsFiddle)

JavaScript heap out of memory while streaming large file

Posting my comment as an answer, since it solved the issue and might be useful to others having difficulting using the xml-stream package in this way.

In question, the collect method is causing the issue as it is forcing the parser to collect all the instances of the processed node in an array as they are parsed. collect should only be used to collect children items of a certain type from each node that is being parsed. The default behaviour is not to do that (due to the streaming nature of the parser that lets you process multi gigabyte files with ease).

So solution was to remove that line of code and just use the endElement event.

Converting array buffer to string - Maximum call stack size exceeded

I got my answer from another question

btoa(new Uint8Array(blob).reduce(function (data, byte) {
    return data + String.fromCharCode(byte);
}, ''));

Source

How to Solve Uncaught Rangeerror When Download Large Size JSON