Introduction Marvin Smith
to OpenCV
Introduction
• OpenCV is an Image Processing library created by Intel
and maintained by Willow Garage.
• Available for C, C++, and Python
• Newest update is version 2.2
• Open Source and free
• Easy to use and install
Installation Instructions
• Do not use synaptic on Linux to install OpenCV.
• It is version 1.2.
• For Linux and Windows, follow the installation guide at
http://opencv.willowgarage.com/wiki/InstallGuide
• Linux users can come to me for help. I have built it on
Ubuntu dozens of times. I have built it successfully on
Windows once.
• Make sure to read the beginning as it gives you
precise commands to install ffmpeg, libavformat-dev,
libswscale-dev, and other required libraries.
BASIC OPENCV
STRUCTURES
• Point, Point2f - 2D Point
• Size - 2D size structure
• Rect - 2D rectangle object
• RotatedRect - Rect object with angle
• Mat - image object
Point • Math operators, you
• 2D Point Object
may use
- int x, y;
- Point operator +
• Functions - Point operator +=
- Point.dot(<Point>) - computes dot
- Point operator -
product
- Point operator -=
- Point.inside(<Rect>) - returns true if
point is inside - Point operator *
- Point operator *=
- bool operator ==
- bool operator != double
norm
Size
• 2D Size Structure
- int width, height;
• Functions
- Point.area() - returns (width * height)
RECT
• 2D Rectangle Structure
- int x, y, width, height;
• Functions
- Point.tl() - return top left point
- Point.br() - return bottom right point
• The primary data structure in
OpenCV is the Mat object. It
cv::Mat
stores images and their
components.
• Main items
• rows, cols - length and
width(int)
• channels - 1: grayscale, 3:
BGR
• depth: CV_<depth>C<num
chan>
• See the manuals for more
information
cv::Mat
• Functions
- Mat.at<datatype>(row, col)[channel] - returns pointer to image
location
- Mat.channels() - returns the number of channels
- Mat.clone() - returns a deep copy of the image
- Mat.create( rows, cols, TYPE) - re-allocates new memory to
matrix
- Mat.cross(<Mat>) - computes cross product of two matricies
- Mat.depth() - returns data type of matrix
- Mat.dot(<Mat>) - computes the dot product of two matrices
cv::Mat
• Functions
- Mat(Range(xmin,xmax),Range(ymin,ymax)) - returns sub image
- Mat.type() - returns the TYPE of a matrix
• Iterator Usage
- Mat.begin() - moves Mat iterator to beginning of image
- Mat.end() - moves Mat iterator to end of image
Image TYPES
• The TYPE is a very important aspect of OpenCV
• Represented as CV_<Datatype>C<# Channels>
• Example Datatypes/ Depths
Pixeltypes
• PixelTypes shows how the image is represented in data
• BGR - The default color of imread(). Normal 3 channel color
• HSV - Hue is color, Saturation is amount, Value is lightness. 3
channels
• GRAYSCALE - Gray values, Single channel
• OpenCV requires that images be in BGR or Grayscale
in order to be shown or saved. Otherwise, undesirable
effects may appear.
HELLO WORLD
• Example Code This program will load and show
//Loads image and displays an image
//call by ./a.out image.jpg
//
#include <cv.h>
#include <cvaux.h>
#include <highgui.h>
using namespace cv;
int main(int argc, char* argv[ ]){
Mat image = imread(argv[1]);
namedWindow(“Sample Window”);
imshow(“Sample Window”,image);
waitKey(0);
return 0;
}
Starting Out in OpenCV
• OpenCV uses the cv namespace.
• cv::Mat object replaces the original C standard IplImage and
CvMat classes.
• All original functions and classes of the C standard OpenCV
components in the Bradski book are still available and
current. However you will need to read that book for it.
Image I/O
• OpenCV provides simple and useful • Examples
ways to read and write images. //Read an image
Mat image = imread( <string>, <0 -gray, 1 -BGR>)
//Note 1 is default
• Note that there are many extra
options to these commands which //Write an image
are available on the wiki. imwrite( <string filename> , image );
//Create window for output
• waitKey( int x ) has two main namedWindow( <window name> );
features.
//Output image to window
- if x > 0, then waitKey will wait x
imshow( <window name> , <image Mat to show> );
milliseconds
- if x = 0, then waitKey will not move //pause program for input
until key is pressed key = waitKey( 0 );
• Sometimes it is necessary to draw stuff onto the image.
DRAWING STUFF
Instead of using complicated functions, why not just call a
simple function?
• Here are some simple examples...
• void circle(image, Point(x,y),int rad, CV_BGR(b,g,r), int
thickness=1)
• void ellipse(image, RotatedRect box, CV_BGR(b,g,r), int
thickness=1)
• void line(image, Point(x,y), Point(x,y), CV_BGR(b,g,r), int
thickness= 1)
• void rectangle(img, Point(x,y), Point(x,y), CV_BGR(b,g,r), int
thickness)
• NOTE: negative thickness will fill in the rectangle
Drawing stuff
Using the Mouse
• OpenCV allows you
to use the mouse to
interact with the
screen. Note that this
feature is from
OpenCV 1.0 and is
compatible with Mat
objects.
• This program allows
you to draw dots on
the image.
USING THE MOUSE
Converting colorspaces
• cvtColor( image, image, code)
• Codes
• CV_<colorspace>2<colorspace>
• Examples
• CV_BGR2GRAY
• CV_BGR2HSV
• CV_BGR2LUV
• normalize(imagein, imageout, low, high,
Image Normalization
method);
• Image normalization is the process of
stretching the range of an image from
[a, b] to [c, d].
• This is incredibly important for
visualization because if the image is
beyond [0,255] it will cause truncation or
unsightly effects.
Thresholding
• threshold( image, image, thresh, maxVal, CODE);
• CODE - this is the method of thresholding. Different actions will be
taken depending on this code.
• Sobel Edge Detection
• Edge Detection
void cv::Sobel(image in, image out, CV_DEPTH, dx, dy);
• Scharr Edge Detection
• void cv::Scharr(image in, image out, CV_DEPTH, dx, dy);
• Laplacian Edge Detection
• void cv::Laplacian( image in, image out, CV_DEPTH);
of edges and detail in an image.
• OpenCV includes most of the commonly used
methods. Image Smoothing
• void GaussianBlur(imagein, imageout, Size ksize, sig);
• Note that there are more options, however this
should keep things simple
• void medianBlur (imagein, imageout, Size ksize);
• Other functions include generic convolution,
separable convolution, dilate, and erode.
Original Gaussian Blur Median Blur
STOP!
•This is not relevent until the last
part of the class.
•Beware!
Linear Algebra
• OpenCV contains many • Operators
useful and simple functions
for applying linear algebra on
images. • given: Mat image;
• Most major operators are
allowed. • image.inv(); //inverse
• image.t(); //transpose
• operator * performs matrix
multiplication, NOT • image.clone(); //creates deep copy
elementwise multiplication. • image.diag(int d=0) //returns diagonal
•image.mul(mat, double); //performs
elementwise multiplication.
•image.cross(mat); //performs cross
product
•image.dot(mat); //performs dot product
•
Singular Value Decomposition
Singular Value Decomposition is a
vital part of any computer vision
based system. Luckily, OpenCV
Example
makes this a trivial task. given:
-11x + 2y = 0
• To solve a least-squares problem, 2x + 3y = 7
simply call the solve command. 2x - y = 5
• bool solve(src1, src2, dst, int flags);
• Usually, src1 is A, src2 is b, and dst is x.
Remember flags is method...
• DECOMP_LU - Fast but cannot solve over-
determined systems.
• DECOMP_SVD - SVD, can solve just about
anything
SVD Results
• Using OpenCV
• Using GNU Octave
Principle Component Analysis
• Since you will need to learn this, I will include it. Although you will
undoubtably will have to create your own PCA program, OpenCV
covers it very nicely.
• PCA( Mat data, Mat mean, int FLAG, int numcomp=0)
• FLAG: PCA_DATA_AS_ROW / PCA_DATA_AS_COL
• numcomp is the k value, 0 means all values retained
• in general, just pass the vectors into data and the mean will be
returned.
• PCA.project( Mat vector)
• projects the vector into the built eigenspace and returns the result
• PCA.backproject( Mat vector)
Important Tips
• Remember that images are read from file as 8-bit
unsigned integers. In order to do complicated math
operations, convert to 32-bit floating point type. Then
convert back to write to file.
• Always remember that rows is your y coordinate and
that cols is your x coordinate. Size objects are called
X,Y while images are referenced row, col. There are
many subtle things that will ruin good code.
