Modifying Camera Output Using Surfacetexture and Opengl

Modifying camera output using SurfaceTexture and OpenGL

mDirectVideo = new DirectVideo(texture);
texture = createTexture();

should be

texture = createTexture();
mDirectVideo = new DirectVideo(texture);

Shader

private final String vertexShaderCode =
        "attribute vec4 position;" +
        "attribute vec2 inputTextureCoordinate;" +
        "varying vec2 textureCoordinate;" +
        "void main()" +
        "{"+
            "gl_Position = position;"+
            "textureCoordinate = inputTextureCoordinate;" +
        "}";

    private final String fragmentShaderCode =
        "#extension GL_OES_EGL_image_external : require\n"+
        "precision mediump float;" +
        "varying vec2 textureCoordinate;                            \n" +
        "uniform samplerExternalOES s_texture;               \n" +
        "void main() {" +
        "  gl_FragColor = texture2D( s_texture, textureCoordinate );\n" +
        "}";

mColorHandle = GLES20.glGetUniformLocation(mProgram, "vColor");

should be

mColorHandle = GLES20.glGetAttribLocation(mProgram, "s_texture");

remove initialization stuff from DirectVideo draw.glVertexAttribPointer etc. Put it in some init function.

public void draw()
{
    GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, texture);
    GLES20.glDrawElements(GLES20.GL_TRIANGLES, drawOrder.length,
            GLES20.GL_UNSIGNED_SHORT, drawListBuffer);
}

Displaying the camera stream on a GLSurfaceView via SurfaceTexture

I have used the SurfaceTexture succesfully to draw camera frames on a custom opengl texture without using the transformation matrix provided by android.

Just try defining your indices vertices and textures the way you would do it for a normal texture draw. Like this for example.

const GLfloat Vertices[] = {0.5, -0.5, 0,
                            0.5, 0.5, 0,
                           -0.5, 0.5, 0,
                           -0.5, -0.5, 0};

const GLubyte Indices[] = { 0, 1, 2,
                            2, 3, 0 };

const GLfloat Textures[] = { 1.,0.,
                             0.,0.,
                             0.,1.,
                             1.,1. };

You should be able to use the surfacetexture the way you use a normal texture.

In case you want to perform some sort of 3D projection, this is a good post on how to generate a proper MVP matrix. Which you could use to multiply with the position in the vertex shader.

Modify and update camera frame via GLSurfaceView

As fadden explained, you cannot change the preview buffer that is connected to SurfaceTexture. But you can obtain the preview buffers with onPreviewFrame(), modify it and push the result to OpenGL via glTexSubImage2D(). There are two pitfalls: you should hide the actual preview (probably connecting it to a texture that will not be visible on your GL surface), and you should do all processing fast enough (20 FPS at least for the "preview" to look natural).

---

Related Topics