Converting Data from Glreadpixels() to Opencv::Mat

Converting data from glReadPixels() to OpenCV::Mat

First we create an empty (or unititialized) cv::Mat for our data to be read into directly. This can be done once at startup, but on the other hand cv::Mat::create doesn't really cost much when the image already has matching size and type. The type depends on your needs, usually it's something like CV_8UC3 for a 24-bit color image.

cv::Mat img(height, width, CV_8UC3);

or

img.create(height, width, CV_8UC3);

Then you have to account for cv::Mat not neccessarily storing image rows contiguously. There might be a small padding value at the end of each row to make rows 4-byte aligned (or 8?). So you need to mess with the pixel storage modes:

//use fast 4-byte alignment (default anyway) if possible
glPixelStorei(GL_PACK_ALIGNMENT, (img.step & 3) ? 1 : 4);

//set length of one complete row in destination data (doesn't need to equal img.cols)
glPixelStorei(GL_PACK_ROW_LENGTH, img.step/img.elemSize());

Next, the type of the matrix influences the format and type parameters of glReadPixels. If you want color images you have to keep in mind that OpenCV usually stores color values in BGR order, so you need to use GL_BGR(A) (which were added with OpenGL 1.2) instead of GL_RGB(A). For one component images use either GL_LUMINANCE (which sums the individual color components) or GL_RED, GL_GREEN, ... (to get an individual component). So for our CV_8UC3 image the final call to read it directly into the cv::Mat would be:

glReadPixels(0, 0, img.cols, img.rows, GL_BGR, GL_UNSIGNED_BYTE, img.data);

Finally, OpenCV stores images from top to bottom. So you may need to either flip them after getting them or render them flipped in OpenGL in the first place (this can be done by adjusting the projection matrix, but keep an eye on triangle orientation in this case). To flip a cv::Mat vertically, you can use cv::flip:

cv::flip(img, flipped, 0);

So to keep in mind OpenCV:

• stores images from top to bottom, left to right
• stores color images in BGR order
• might not store image rows tightly packed

Convering OpenGL texture to OpenCV matrix

I've tested the code below and it seems to work. (Note the usage of GL_BGR in glGetTexImage()).

cv::Mat get_ocv_img_from_gl_img(GLuint ogl_texture_id)
{
    glBindTexture(GL_TEXTURE_2D, ogl_texture_id);
    GLenum gl_texture_width, gl_texture_height;

    glGetTexLevelParameteriv(GL_TEXTURE_2D, 0, GL_TEXTURE_WIDTH, (GLint*)&gl_texture_width);
    glGetTexLevelParameteriv(GL_TEXTURE_2D, 0, GL_TEXTURE_HEIGHT, (GLint*)&gl_texture_height);

    unsigned char* gl_texture_bytes = (unsigned char*) malloc(sizeof(unsigned char)*gl_texture_width*gl_texture_height*3);
    glGetTexImage(GL_TEXTURE_2D, 0 /* mipmap level */, GL_BGR, GL_UNSIGNED_BYTE, gl_texture_bytes);

    return cv::Mat(gl_texture_height, gl_texture_width, CV_8UC3, gl_texture_bytes);
}

Reading from framebuffer GLSL to OpenCV

textureID looks like an incomplete texture.

Set GL_TEXTURE_MIN_FILTER to GL_NEAREST or GL_LINEAR.

Or supply a complete set of mipmaps.

---

Related Topics