Javascript: Getting Imagedata Without Canvas

JavaScript: getting ImageData without canvas

You have to create an in memory canvas and then draw on this canvas the image :

var canvas = document.createElement('canvas');
var context = canvas.getContext('2d');
var img = document.getElementById('myimg');
canvas.width = img.width;
canvas.height = img.height;
context.drawImage(img, 0, 0 );
var myData = context.getImageData(0, 0, img.width, img.height);

But this won't work if the image comes from another domain. This is a security restriction you can't get around if you have no control of the server (be careful that if you open your html file with file:// you'll have a lot of additional restrictions, use http://)

HTML5 getImageData without canvas

No, you can't.

But getting the imageData can be done with an in-memory canvas, that's fast and easy :

var canvas = document.createElement('canvas');
var context = canvas.getContext('2d');
var img = document.getElementById('someImageId');
context.drawImage(img, 0, 0 );
var theData = context.getImageData(0, 0, img.width, img.height);

You may keep the theData variable so that you don't have to build it at each click.

Note that this won't work if the image comes from another domain (and thus it won't work if you open your html file using file:// instead of http://).

JavaScript image processing without canvas

I solved my problems changing couple things here:

1. We do not need last 2 steps (6 and 7) from the list above:

   
   
   5. export data from canvas as base64 image
   6. create final image from exported base64 image

Since we add data again into the final canvas, we can send current canvas with processed image, since canvas can accept another canvas to add as image inside.

2. We separate streaming process and processing data to another working thread (web worker) and sending already prepared data to main thread.
   Also very important part is that we send data to main thread only if main thread is ready since we do not want to overflow main thread with bunch of data. Main thread became really slow if we send him data which cannot process. In meanwhile in the streaming thread we put all processed data into queue until main thread does not send notification that it is ready and wants more data.
   How to set up signalR streaming to another thread you can find here: Is it possible to create signalR streaming connection in web worker

3. We changed image data gained from the server from base64 image to array of lines. For example: [3, 5, 34, 3, 6, 34, 3, 7, 35] where every 3 values present [x, y, line length].

   Data gained from the server in this format are smaller and much easier to process since we do not need to load image into the canvas to get pixel data, we immidetly can create image data and process them.

With those 3 steps of improvement we achieved needed performances.

Can I use fabric.js to get image data without HTML?

According to the docs here fabric.js is (just) a wrapper for HTML canvas elements. In the end the browser does the drawing.

You can't use it without an HTML canvas element that is created by the browser in the DOM.

let canvas = new fabric.Canvas('c');

The value 'c' has to be the ID of an HTML canvas element that is available in the DOM.

Therefore a test running in node.js won't work this way.

Convert a byte array to image data without canvas

canvas.getImageData returns an ImageData object that looks like this:

interface ImageData {
  readonly attribute unsigned long width;
  readonly attribute unsigned long height;
  readonly attribute Uint8ClampedArray data;
};

(See the specs: http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#imagedata)

I guess you could box your data fitting that interface and try it out.

If you try, let me know how it turns out :)

Alternatively, you could create an in-memory canvas of your desired width/height--document.createElement("canvas"), Then grab its imagedata and plug in your own array. I know this works. Yes...that goes contrary to your question, but you're not adding the canvas to your page.

Is it possible to get pixels from ImageBitMap web object without a canvas?

No, there is no way currently.

We are working on adding a new getImageData() method on the ImageBitmap interface specs issue but this still needs implementer's interest, someones to write the specs and probably a good load of discussion as everything in there is not that easy.

(ImageBitmaps can represent a lot of different sources which are internally stored in various places with various tradeoffs for readback).

So for the time being, the only solution is indeed to go through a canvas.

In a near future, probably closer than ImageBitmap.getImageData, you will be able to extract VideoFrames from your MediaStream and paint it directly to a canvas as shown in this answer of mine, thus avoiding the ImageBitmap step, or even read the YUV planes of each frame as demonstrated in this answer of mine.

Get pixels' colors from HTML Canvas WITHOUT getImageData()?

There is. Since you're running from an extension your extension will have privileged access to cross-origin sources but only if loaded via fetch() and XMLHttpRequest() from a content script (or background script) - excerpt from that link:

Content scripts get the same cross-domain privileges as the rest of
the extension: so if the extension has requested cross-domain access
for a domain using the permissions key in manifest.json, then its
content scripts get access that domain as well.

This is accomplished by exposing more privileged XHR and fetch
instances in the content script [...]

Please note that these calls when called from a content script will not set origin and referer headers which sometimes can cause problems if the cross-origin site expects these to be set - for those cases you will need to use the non-privileged content.XMLHttpRequest or content.fetch() which will bring you back to square one.

The permissions in the manifest file (or if set permissions dynamically) must also allow access to these cross-origin sites.

This means however that you will have to "reload" the image source separately via these calls. You can do this the following way by first obtaining the original URL to the image you want to load, say, from a content script:

// example loading all images in current tab
let images = document.querySelectorAll("img");
for(let image of images) loadAsBitmap(image.src); // some sub-call using the url

Then load that source via the content script's fetch():

fetch(src).then(resp => {  // load from original source
  return resp.blob();      // obtain a blob
}).then(blob => {          // convert blob, see below
  // ...
};

When the blob is obtained you can convert it to an Object-URL and set that as source for an image and be able to go around the cross-origin restriction we otherwise face. In the content script, next steps would be:

let url = URL.createObjectURL(blob);         // attach Object-URL to blob
let img = new Image();                       // create image element *
img.onload = () => {                         // attach handler
  let c = document.createElement("canvas");  // create canvas
  let ctx = c.getContext("2d");              // get context
  c.width = img.width;                       // canvas size = image
  c.height = img.height;
  ctx.drawImage(img, 0, 0);                  // draw in image
  URL.revokeObjectURL(url);                  // remove reference.
  let imageData = 
    ctx.getImageData(0,0,c.width,c.height);  // get image data
  // .. callback to a function that handles the image data
};
img.src = url;                               // start loading blob

---

Related Topics