Image Manipulation and Texture Mapping Using HTML5 Canvas

Image manipulation and texture mapping using HTML5 Canvas?

I think you will never get an accurate result... I spent some time investigating how to do 3d graphics using canvas 2d context and I found it viable to do texture mapping gouraud shading by computing appropriate 2d gradients and matrices:

• Solid polygons are of course easy
• Gouraud filling is possible only on one component (i.e. you cannot have a triangle where every vertex is an arbitrary RGB filled with bilinear interpolation, but you can do that filling using for example three arbitrary shades of a single color)
• Linear texture mapping can be done using clipping and image drawing

I would implement perspective-correct texture mapping using mesh subdivision (like on PS1).

However I found many problems... for example image drawing with a matrix transform (needed for texture mapping) is quite inaccurate on chrome and IMO it's impossible to get a pixel-accurate result; in general there is no way to turn off antialiasing when drawing on a canvas and this means you will get visible see-through lines when subdividing in triangles. I also found multipass rendering working really bad on chrome (probably because of how hw-accellerated rendering is implemented).

In general this kind of rendering is surely a stress for web browsers and apparently these use cases (strange matrices for example) are not tested very well. I was even able to get Firefox crashing so bad that it took down the whole X susbsystem on my Ubuntu.

You can see the results of my efforts here or as a video here... IMO is surely impressing that this can be done in a browser without using 3D extensions, but I don't think current problems will be fixed in the future.

Anyway the basic idea used to draw an image so that the 4 corners ends up in specific pixels position is to draw two triangles, each of which will use bilinear interpolation.

In the following code I assume you have a picture object texture and 4 corners each of which is an object with fields x,y,u,v where x,y are pixel coordinates on the target canvas and u,v are pixel coordinates on texture:

function textureMap(ctx, texture, pts) {
    var tris = [[0, 1, 2], [2, 3, 0]]; // Split in two triangles
    for (var t=0; t<2; t++) {
        var pp = tris[t];
        var x0 = pts[pp[0]].x, x1 = pts[pp[1]].x, x2 = pts[pp[2]].x;
        var y0 = pts[pp[0]].y, y1 = pts[pp[1]].y, y2 = pts[pp[2]].y;
        var u0 = pts[pp[0]].u, u1 = pts[pp[1]].u, u2 = pts[pp[2]].u;
        var v0 = pts[pp[0]].v, v1 = pts[pp[1]].v, v2 = pts[pp[2]].v;

        // Set clipping area so that only pixels inside the triangle will
        // be affected by the image drawing operation
        ctx.save(); ctx.beginPath(); ctx.moveTo(x0, y0); ctx.lineTo(x1, y1);
        ctx.lineTo(x2, y2); ctx.closePath(); ctx.clip();

        // Compute matrix transform
        var delta = u0*v1 + v0*u2 + u1*v2 - v1*u2 - v0*u1 - u0*v2;
        var delta_a = x0*v1 + v0*x2 + x1*v2 - v1*x2 - v0*x1 - x0*v2;
        var delta_b = u0*x1 + x0*u2 + u1*x2 - x1*u2 - x0*u1 - u0*x2;
        var delta_c = u0*v1*x2 + v0*x1*u2 + x0*u1*v2 - x0*v1*u2
                      - v0*u1*x2 - u0*x1*v2;
        var delta_d = y0*v1 + v0*y2 + y1*v2 - v1*y2 - v0*y1 - y0*v2;
        var delta_e = u0*y1 + y0*u2 + u1*y2 - y1*u2 - y0*u1 - u0*y2;
        var delta_f = u0*v1*y2 + v0*y1*u2 + y0*u1*v2 - y0*v1*u2
                      - v0*u1*y2 - u0*y1*v2;

        // Draw the transformed image
        ctx.transform(delta_a/delta, delta_d/delta,
                      delta_b/delta, delta_e/delta,
                      delta_c/delta, delta_f/delta);
        ctx.drawImage(texture, 0, 0);
        ctx.restore();
    }
}

Those ugly strange formulas for all those "delta" variables are used to solve two linear systems of three equations in three unknowns using Cramer's method and Sarrus scheme for 3x3 determinants.

More specifically we are looking for the values of a, b, ... f so that the following equations are satisfied

a*u0 + b*v0 + c = x0
a*u1 + b*v1 + c = x1
a*u2 + b*v2 + c = x2

d*u0 + e*v0 + f = y0
d*u1 + e*v1 + f = y1
d*u2 + e*v2 + f = y2

delta is the determinant of the matrix

u0  v0  1
u1  v1  1
u2  v2  1

and for example delta_a is the determinant of the same matrix when you replace the first column with x0, x1, x2. With these you can compute a = delta_a / delta.

HTML5 Canvas: Rotated Pattern Fill

To achieve what is in the image from tiny wings using the shape(texture) you supplied.

1. draw your texture-shape vertically to screen (it looks like it has been skew'ed not rotated)
2. apply a few semi-transparent hill shaped lines with a wide stroke width to create the phong shading effect.
3. clip the texture-shape with the shape of the hill.
4. apply a semi-transparent grunge texture to the whole canvas.

Canvas fast texture mapping

First off, that's an awesome demo! Kudos for getting that much working on it.

For a usage like this WebGL will absolutely give you better performance, and given that all it looks like you need is some textured cubes, I don't think that it would be terribly difficult for you either. Yes, 3D has more of a learning curve than 2D canvas work, but considering that you're already doing a lot of the math to simulate 3D anyway you'll probably find "real" 3D to be easier!

A good place to start would be the lessons at LearningWebGL.com. They'll run you through the basics, and texture mapping (the bit you want) is only 5 lessons in.

Of course, you could also get a little bit crazy and get the effect that you're looking from with pure CSS! There's a pretty cool demo of somebody actually building a city scene with it, so it's certainly possible. That feels like a bit of a stretch, though, and you'll probably be much better of from a feature perspective with straight WebGL.

html5 canvas pixel manipulation problems on mobile devices when setting the alpha channel

It is possible that your problem comes from a bug in the default browser of android, when it draws a pixel that contains an alpha value different from 0 or 255 it alters its color. You are not the only one experiencing this issue: https://code.google.com/p/android/issues/detail?id=17565

I guess the only chance to get it solved is to report the bug. Also, it seems that the bug was partly fixed in android 4.1 (while 4.0 still has it).

How to eliminate the seam between UV map triangles (HTML5 Canvas)?

It has to do with the way different browsers choose to take on anti-aliasing (or not).

It looks good on Chrome 15.

Clipping less should solve your problem. I'd suggest you try expanding your clipping region for each object by 0.5 or 1 pixel and see how it does. (alternatively you could try shifting everything by 0.5 pixels and seeing if the anti-aliasing changes, you might be surprised)

---

Related Topics