Opencv: Process Every Frame

Reading every nth frame from VideoCapture in OpenCV

I'm afraid there's not much you can do and it's not just a shortcoming of OpenCV. You see, modern video codecs, are, generally, complex beasts. To get a higher compression rate the encoding of a frame is often dependent on previous and sometimes even successive frames.

So, most of the time you have to decode frames before the desired one even if you don't need them.

There are rather non-trivial tricks to specifically encode a video file, so that it would be cheap to get every Nth frame, but it's not feasible in general case.

That said, you can try the seeking functionality OpenCV provides (see OpenCV Seek Function/Rewind). It may (as well as may not) work faster depending on the circumstances. However, personally, I wouldn't bet on it.

How to process images of a video, frame by frame, in video streaming using OpenCV and Python

After reading the documentation of VideoCapture. I figured out that you can tell VideoCapture, which frame to process next time we call VideoCapture.read() (or VideoCapture.grab()).

The problem is that when you want to read() a frame which is not ready, the VideoCapture object stuck on that frame and never proceed. So you have to force it to start again from the previous frame.

Here is the code

import cv2

cap = cv2.VideoCapture("./out.mp4")
while not cap.isOpened():
    cap = cv2.VideoCapture("./out.mp4")
    cv2.waitKey(1000)
    print "Wait for the header"

pos_frame = cap.get(cv2.cv.CV_CAP_PROP_POS_FRAMES)
while True:
    flag, frame = cap.read()
    if flag:
        # The frame is ready and already captured
        cv2.imshow('video', frame)
        pos_frame = cap.get(cv2.cv.CV_CAP_PROP_POS_FRAMES)
        print str(pos_frame)+" frames"
    else:
        # The next frame is not ready, so we try to read it again
        cap.set(cv2.cv.CV_CAP_PROP_POS_FRAMES, pos_frame-1)
        print "frame is not ready"
        # It is better to wait for a while for the next frame to be ready
        cv2.waitKey(1000)

    if cv2.waitKey(10) == 27:
        break
    if cap.get(cv2.cv.CV_CAP_PROP_POS_FRAMES) == cap.get(cv2.cv.CV_CAP_PROP_FRAME_COUNT):
        # If the number of captured frames is equal to the total number of frames,
        # we stop
        break

Does the VideoCapture method of the OpenCV library (in Python) process every frame, or does it also skip frames when doing hefty processing?

Even though VideoCapture has a buffer to store images, in a heavy process your loop will skip some frames. By standard, your VideoCaptureProperties has the property CAP_PROP_BUFFERSIZE = 38, meaning it will store 38 frames. The read() method uses grab() method that reads the next frame from the buffer.

You can test it yourself, below is a simple example with a time delay to simulate a heavy process.

import numpy as np
import cv2
import time
cap = cv2.VideoCapture(0)

while(True):
    # Capture frame-by-frame
    ret, frame = cap.read()

    # Our operations on the frame come here
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Display the resulting frame
    cv2.imshow('frame',gray)

    # Introduce a delay to simulate heavy process
    time.sleep(1) 
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

You will see the image skips frames (and does not create a "slow-motion" effect that we would expect in a slow sequence of images). Therefore, if your process is fast enough, you can match the camera FPS.

---

Related Topics