Yuv to Rgb Conversion by Fragment Shader

YUV to RGB conversion by fragment shader

Not sure if you have already fixed this problem.My answer

1. By default Camera output is NV12, but in fragment shader YUV to RGB you are using YV12 -> RGB.
   You will have to do
   setPreviewFormat(ImageFormat.YV12);, or may be use some other shader

2. There are 3 textures , make sure you do

   GLES20.glActiveTexture(GLES20.GL_TEXTURE2);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, muTextureName)

   before call to any glTexImage2D. and glTexSubImage2D

3. You can also use glTexSubImage2D with every frame and glTexImage2D once.

4. size of U and V is same , atleast for YV12,

   System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4 * 2);

   should be
   System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4);
   change the size accordingly in the code.

Converting YUV (yCbCr420p) to RGB in GLSL fragment shader?

I have just solved this, silly error on my part when setting the shader uniforms!

This is what I was doing;

GL20.glActiveTexture(GL20.GL_TEXTURE0+2);
GL20.glBindTexture(GL20.GL_TEXTURE_2D, cbTexHandle);
program.setUniformi("u_texture_cb", GL20.GL_TEXTURE0+2);

GL20.glActiveTexture(GL20.GL_TEXTURE0+1);
GL20.glBindTexture(GL20.GL_TEXTURE_2D, crTexHandle);
program.setUniformi("u_texture_cr", GL20.GL_TEXTURE0+1);

It should actually be;

program.setUniformi("u_texture_cb", 2);
program.setUniformi("u_texture_cr", 1);

So if you end up getting a green and pink display from a YUV shader, I believe it's just down to not actually getting any UV values (they are probably just all zeros)

Where did this YUV420P to RGB shader conversion come from?

Why take [...] 0.5 and 0.5 in the first part?

U and V are stored in the green and blue color channel of the texture. The values in the color channels are stored in the range [0.0, 1.0]. For the computations the values have to be in mapped to the range [-0.5, 0.5]:

yuv.g = texture(tex_u, TexCoord).r - 0.5;
yuv.b = texture(tex_v, TexCoord).r - 0.5;

Subtracting 0.0625 from the red color channel is just an optimization. Thus, It does not have to be subtracted separately in each expression later.

The algorithm is the same as in How to convert RGB -> YUV -> RGB (both ways) or various books.

YUV Conversion via a fragment shader

We figured it out. :) Basically the problem was that while our YUV to RGB equations were correct, we weren't properly sampling the V data! So no amount of futzing with the equations would have helped!

In the end, I would recommend the following strategy for anyone attempting to do this:

1) Set R, G, and B to the value from Y. You should get a grayscale image (as Y contains just luminance).
2) Next, set R, G, and B to U. You should get funny colors!
3) Finally set R, G, and B to V. Again, you should get funny colors.

Also, properly normalizing the values is critical. Check our fourcc.org for a discussion of proper YUV normalization.

Understanding this shader for YUV to RGB conversion

The first conversion is a transformation that normalize the limited YUV data. In limited YUV formation, Y in limited in 16 to 240. If we use 16 to divide 256, the answer is 0.0625. So, it is a normalization step.

---

Related Topics