Getting Frames from Video Image in Android

Getting frames from Video Image in Android

In API 17+, you can do conversion to RGBA888 from NV21 with the 'ScriptIntrinsicYuvToRGB' RenderScript. This allows you to easily process preview frames without manually encoding/decoding frames:

@Override 
public void onPreviewFrame(byte[] data, Camera camera) { 
   Bitmap bitmap = Bitmap.createBitmap(r.width(), r.height(), Bitmap.Config.ARGB_8888);
    Allocation bmData = renderScriptNV21ToRGBA888(
        mContext,
        r.width(),
        r.height(),
        data);
    bmData.copyTo(bitmap);
}

public Allocation renderScriptNV21ToRGBA888(Context context, int width, int height, byte[] nv21) {
  RenderScript rs = RenderScript.create(context);
  ScriptIntrinsicYuvToRGB yuvToRgbIntrinsic = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs));

  Type.Builder yuvType = new Type.Builder(rs, Element.U8(rs)).setX(nv21.length);
  Allocation in = Allocation.createTyped(rs, yuvType.create(), Allocation.USAGE_SCRIPT);

  Type.Builder rgbaType = new Type.Builder(rs, Element.RGBA_8888(rs)).setX(width).setY(height);
  Allocation out = Allocation.createTyped(rs, rgbaType.create(), Allocation.USAGE_SCRIPT);

  in.copyFrom(nv21);

  yuvToRgbIntrinsic.setInput(in);
  yuvToRgbIntrinsic.forEach(out);
  return out;
}

Extract frames from a video in real-time android

Using the camera object you can override a function setPreviewCallback. from that function you would get an array of byte data. Which you can convert to bitmap or you can save them in an array of array.

Assuming that you were extends SurfaceView and Implements SurfaceHolder.Callback

Code looks like,

mCamera.setPreviewCallback(new PreviewCallback() {
    public void onPreviewFrame(byte[] data, Camera camera) {

        //do some operations using data

    }
});

If you want to get the RGB value you can use this function,

static public void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width,
            int height) {
        final int frameSize = width * height;

        for (int j = 0, yp = 0; j < height; j++) {
            int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
            for (int i = 0; i < width; i++, yp++) {
                int y = (0xff & ((int) yuv420sp[yp])) - 16;
                if (y < 0)
                    y = 0;
                if ((i & 1) == 0) {
                    v = (0xff & yuv420sp[uvp++]) - 128;
                    u = (0xff & yuv420sp[uvp++]) - 128;
                }

                int y1192 = 1192 * y;
                int r = (y1192 + 1634 * v);
                int g = (y1192 - 833 * v - 400 * u);
                int b = (y1192 + 2066 * u);

                if (r < 0)
                    r = 0;
                else if (r > 262143)
                    r = 262143;
                if (g < 0)
                    g = 0;
                else if (g > 262143)
                    g = 262143;
                if (b < 0)
                    b = 0;
                else if (b > 262143)
                    b = 262143;

                rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000)
                        | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
            }
        }
    }

I think this is what you want as an answer. If i am wrong, please let me know whats is your actual requirement

For getting the green value, you can use the below code,

static public void calculateColor(int[] rgb, int[] color,
            int width, int height, int component) {
        for (int bin = 0; bin < 256; bin++) {
            color[bin] = 0;
        } // bin

        int start = 0;
        int end = width * height;

        if (component == 0) // red
        {
            for (int pix = start; pix < end; pix += 3) {
                int pixVal = (rgb[pix] >> 16) & 0xff;
                color[pixVal]++;
            } // pix
        } else if (component == 1) // green
        {
            for (int pix = start; pix < end; pix += 3) {
                int pixVal = (rgb[pix] >> 8) & 0xff;
                color[pixVal]++;
            } // pix
        } else // blue
        {
            for (int pix = start; pix < end; pix += 3) {
                int pixVal = rgb[pix] & 0xff;
                color[pixVal]++;
            } // pix
        }
    }

Calling this function would return the value of green for every pixel. If you wants the green value for a particular pixel, you have to modify this code.

How to capture frame by frame images from Android video recording in real time

One can get frames by using ImageReader for camera2 API. And preview callback for camera1 API.

Here is how to do it with camera2 API.

One can also explore this library for working example with both camera2 and camera1 API.

Capture video frames with opencv in android

Here is how to do it:

 public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
    //Do your processing here, for example:
    Mat image = inputFrame.rgba();
    Mat ret_mat=new Mat();
    Core.add(image, new Scalar(40, 40, 40, 0), ret_mat); //change brightness of video frame

    return ret_mat;
}

OpenCv functions operate with Mat object, which represents matrix of pixels of your image/frame. The code above makes each frame brighter by 40 units.

---

Related Topics