How to Programmatically Detect Size Limit for Data Url

Is it possible to programmatically detect size limit for data url?

No, there doesn't seem to be any event letting you know if an anchor with the download attribute actually succeeded to download the resource.

But, the limitation you are facing seems to only concern this situation : an anchor element <a> with the download attribute.

The browser can handle way longer dataURI (I think that in most browsers the limitation is the same as the one for strings length). E.g, it can load it to <img> element, and more importantly in your case, it can process the dataURI string.

This is important because it allows you to convert this dataURI to a blob, then to create an object URL from this blob. object URL's URI are small and don't face this length limitation in anchors with download attribute.

So the best in your case is probably to directly use the canvas.toBlob method (a polyfill is available on mdn), and to pass the object URL's URI as the href of your anchor.

var img = new Image();
img.crossOrigin = "anonymous";

img.onload = function(){
  var ctx = document.createElement('canvas').getContext('2d');
  ctx.canvas.width = this.width*10;
  ctx.canvas.height = this.height*10;
  ctx.drawImage(this,0,0,ctx.canvas.width, ctx.canvas.height);
  // convert our canvas to a png blob 
  //  (for type use the second parameter, third is for quality if type is image/jpeg or image/webp)
  ctx.canvas.toBlob(function(blob){
     myAnchor.href = URL.createObjectURL(blob);
    });
  // since we are dealing with use files, we absolutely need to revoke the ObjectURL as soon as possible
  var revokeURL = function(){
    // but we have to wait that the browser actually prepared the file to download
    requestAnimationFrame(function(){
      // we've waited one frame, it's seems to be enough
      URL.revokeObjectURL(this.href);
      this.href=null;
      });
    this.removeEventListener('click', revokeURL);
    };

  myAnchor.addEventListener('click', revokeURL);
  };

img.src = 'http://lorempixel.com/200/200/';

<a id="myAnchor" download="anAwesomeImage.png">download</a>

[ Live Demo ] _{(since download attribute is blocked by some UA in SE-Snippets)}

But note that even if the string representation (URI) of the object URL is short, it will actually take the size of your file in browser's memory, and so until you hard-refresh the page (clearing cache), or close the tab where you created this object URL. So you'll absolutely need to call URL.revokeObjectURL() to clear this space.

But, since there is no event to know if the downloading actually succeeded, you're stuck with the onclick event, which will fire before the file downloading occurred. From my tests, waiting for a single frame with requestAnimationFrame is enough, but I may be wrong.

---

And for the ones coming here with an other source than a canvas for their dataURI, there are already a lot of posts in SO about converting a dataURI to a blob, and you can just check the mdn polyfill provided above, they're doing it too.

Data protocol URL size limitations

Regarding limitations in web browsers, MSIE 6/7 do not support the data url scheme...
More info on wikipedia

The length limits are different per browser - i believe IE8 allows up to 32KB and opera is 4KB, but can't really tell about other browsers...

canvas.toDataURL() download size limit

Check out toBlob https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toBlob

canvas.toBlob(function(blob) {
    exportPNGElement.href = URL.createObjectURL(blob);
});

browser support is not as awesome as toDataURL though. But Chrome and Firefox have it, so it solves your biggest issue. The mdn link above also has a polyfill based on toDataURL, so you get the best possible support.

Just in case you didn't know, you can also dramatically reduce the size using jpeg compression

exportPNGElement.href = canvas.toDataURL('image/jpeg', 0.7);

Node.js HTTP Get URL Length limitation

There is a built-in request size limit enforced by Node. Requested headers + URI should not be more than 80 kb.

As it is defined in http_parser.h#L55:

/* Maximium header size allowed */
#define HTTP_MAX_HEADER_SIZE (80*1024)

Asuming that UTF-8 character can be between 1 and 4 bytes, the size of a string with 50 000 characters would be from 50 000 to 200 000 bytes, or from ~48kb to 195kb.

How to calculate a file size from URL in java

Using a HEAD request, you can do something like this:

private static int getFileSize(URL url) {
    URLConnection conn = null;
    try {
        conn = url.openConnection();
        if(conn instanceof HttpURLConnection) {
            ((HttpURLConnection)conn).setRequestMethod("HEAD");
        }
        conn.getInputStream();
        return conn.getContentLength();
    } catch (IOException e) {
        throw new RuntimeException(e);
    } finally {
        if(conn instanceof HttpURLConnection) {
            ((HttpURLConnection)conn).disconnect();
        }
    }
}

How do you detect memory limits in JavaScript?

While it might be possible on some browsers, the right approach should be to decide what limit is acceptable for the typical customer and optionally provide a UI to define their limit.

Most heavy web apps get away with about 10MB JavaScript heap size. There does not seem to be a guideline. But I would imagine consuming more than 100MB on desktop and 20MB on mobile is not really nice. For everything after that look into local storage, e.g. FileSystem API (and you can totally make it PERSISTENT)

UPDATE

The reasoning behind this answer is the following. It is next to never user runs only one application. More so with counting on the browser having only one tab open. Ultimately, consuming all available memory is never a good option. Hence determining the upper boundary is not necessary.

Reasonable amount of memory user would like to allocate to the web app is a guess work. E.g. highly interactive data analytics tool is quite possible in JS and might need millions of data points. One option is to default to less resolution (say, daily instead of each second measurements) or smaller window (one day vs. a decade of seconds). But as user keeps exploring the data set, more and more data will be needed, potentially crippling the underlying OS on the agent side.

Good solution is to go with some reasonable initial assumption. Let's open some popular web applications and go to dev tools - profiles - heap snapshots to take a look:

• FB: 18.2 MB
• GMail: 33 MB
• Google+: 53.4 MB
• YouTube: 54 MB
• Bing Maps: 55 MB

Note: these numbers include DOM nodes and JS Objects on the heap.

It seems to be then, people come to accept 50MB of RAM for a useful web site. (Update 2022: nowadays averaging closer to 100MB.) Once you build your DOM Tree, fill your data structures with test data and see how much is OK to keep in RAM.

Using similar measurements while turning device emulation in Chrome, one can see the consumption of the same sites on tablets and phones, BTW.

This is how I arrived at 100 MB on desktop and 20 MB on mobile numbers. Seemed to be reasonable too. Of course, for occasional heavy user it would be nice to have an option to bump max heap up to 2 GB.

Now, what do you do if pumping all this data from the server every time is too costly?

One thing is to use Application Cache. It does create mild version management headaches but allows you to store around 5 MB of data. Rather than storing data though, it is more useful to keep app code and resources in it.

Beyond that we have three choices:

• SQLite - support was limited and it seems to be abandoned
• IndexDB - better option but support is not universal yet (can I use it?)
• FileSystem API

Of them, FileSystem is most supported and can use sizable chunk of storage.

---

Related Topics