How to stabilize video with OpenCV?
If you wish to do this from OpenCV, this article should be helpful. It includes code samples.
Doug
opencv video stabilization algorithm
Here is possible sequence of steps:
Step 1. Read Frames from a Movie File
Step 2. Collect Salient Points from Each Frame
Step 3. Select Correspondences Between Points
Step 4. Estimating Transform from Noisy Correspondences
Step 5. Transform Approximation and Smoothing
Step 6. Run on the Full Video
More details on each step you can find here:
http://www.mathworks.com/help/vision/examples/video-stabilization-using-point-feature-matching.html
I think you can follow the same steps in OpenCV.
Video Stabilization with OpenCV
I can suggest one of the following solutions:
- Using local high level features: OpenCV includes SURF, so: for each frame, extract SURF features. Then build feature Kd-Tree (also in OpenCV), then match each two consecutive frames to find pairs of corresponding features. Feed those pairs into cvFindHomography to compute the homography between those frames. Warp frames according to (combined..) homographies to stabilize. This is, to my knowledge, a very robust and sophisticated approach, however SURF extraction and matching can be quite slow
- You can try to do the above with "less robust" features, if you expect only minor movement between two frames, e.g. use Harris corner detection and build pairs of corners closest to each other in both frames, feed to cvFindHomography then as above. Probably faster but less robust.
- If you restrict movement to translation, you might be able to replace cvFindHomography with something more...simple, to just get the translation between feature-pairs (e.g. average)
- Use phase-correlation (ref. http://en.wikipedia.org/wiki/Phase_correlation), if you expect only translation between two frames. OpenCV includes DFT/FFT and IFFT, see the linked wikipedia article on formulas and explanation.
EDIT
Three remarks I should better mention explicitly, just in case:
- The homography based approach is likely very exact, so stationary object will remain stationary. However, homographies include perspective distortion and zoom as well so the result might look a bit..uncommon (or even distorted for some fast movements). Although exact, this might be less visually pleasing; so use this rather for further processing or, like, forensics. But you should try it out, could be super-pleasing for some scenes/movements as well.
- To my knowledge, at least several free video-stabilization tools use the phase-correlation. If you just want to "un-shake" the camera, this might be preferable.
- There is quite some research going on in this field. You'll find some a lot more sophisticated approaches in some papers (although they likely require more than just OpenCV).
video stabilization using opencv
You can use my code snippet as a start point (not very stable but seems it works):
#include "opencv2/opencv.hpp"
#include <iostream>
#include <vector>
#include <stdio.h>
using namespace cv;
using namespace std;
int main(int ac, char** av)
{
VideoCapture capture(0);
namedWindow("Cam");
namedWindow("Camw");
Mat frame;
Mat frame_edg;
Mat prev_frame;
int k=0;
Mat Transform;
Mat Transform_avg=Mat::eye(2,3,CV_64FC1);
Mat warped;
while(k!=27)
{
capture >> frame;
cv::cvtColor(frame,frame,cv::COLOR_BGR2GRAY);
cv::equalizeHist(frame,frame);
cv::Canny(frame,frame_edg,64,64);
//frame=frame_edg.clone();
imshow("Cam_e",frame_edg);
imshow("Cam",frame);
if(!prev_frame.empty())
{
Transform=estimateRigidTransform(frame,prev_frame,0);
Transform(Range(0,2),Range(0,2))=Mat::eye(2,2,CV_64FC1);
Transform_avg+=(Transform-Transform_avg)/2.0;
warpAffine(frame,warped,Transform_avg,Size( frame.cols, frame.rows));
imshow("Camw",warped);
}
if(prev_frame.empty())
{
prev_frame=frame.clone();
}
k=waitKey(20);
}
cv::destroyAllWindows();
return 0;
}
You can also look for paper: Chen_Halawa_Pang_FastVideoStabilization.pdf as I remeber there was MATLAB source code supplied.
Related Topics
Boost Zip_Iterator and Std::Sort
Rewinding Std::Cout to Go Back to the Beginning of a Line
Opengl Gl_Polygon Concave Polygon Doesn't Color In
When to Use Shared_Ptr and When to Use Raw Pointers
How to Get a "Codesigned" Gdb on Osx
Multithreaded Memory Allocators for C/C++
How to Create a Dll with Swig from Visual Studio 2010
Checking Value Exist in a Std::Map - C++
Is There Any Way a C/C++ Program Can Crash Before Main()
How to Copy/Paste from the Clipboard in C++
In C++ What Causes an Assignment to Evaluate as True or False When Used in a Control Structure
At What Point Does Dereferencing the Null Pointer Become Undefined Behavior
Visual C++ 2008 Express Download Link Dead
Why Does a C/C++ Compiler Need Know the Size of an Array at Compile Time
Semicolon After Class Declaration Braces