ETHZ CV2012: Tutorial openCV

ETHZ CV2012: Tutorial openCV

• 1.
  OpenCV Basics andCamera Calibration Computer Vision Lab Tutorial 5 October 2012 Lorenz Meier, Kevin Koeser, Kalin Kolev Institute of Visual Computing
• 2.
  What is OpenCV? The most popular library in Computer Vision Released under the liberal BSD license It has C++, C, Python and Java interfaces and supports Windows, Linux, Android and Mac OS Offers optimized implementation of more than 2500 algorithms New official website: http://code.opencv.org OpenCV4Android: http://opencv.org/android.html Institute of Visual Computing
• 3.
  OpenCV Overview Instituteof Visual Computing
• 4.
  Core Functionality Each imageis represented as a matrix Basic class Mat for storing general n-dimensional arrays (e.g. grayscale images, color images, voxel volumes etc.) Fast access to its elements via pointers, e.g. Mat mat; mat.ptr<float>(i)[j] = … Direct access to user-allocated data (e.g. video frames) Mat image(height, width, CV_8UC3, pixels, step); Institute of Visual Computing
• 5.
  Hints for AndroidDevelopers If you use the camera device, make sure that you have required permission in your AndroidManifest.xml <uses-feature android:name="android.hardware.camera" /> <uses-permission android:name="android.permission.CAMERA" /> If you want to write images to the storage of the mobile device via imwrite(), make sure that you have required permission in your AndroidManifest.xml <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> Institute of Visual Computing
• 6.
  Image Segmentation Interactive imagesegmentation C. Rother, V. Kolmogorov, A. Blake, "GrabCut — Interactive Foreground Extraction using Iterated Graph Cuts" OpenCV implementation void grabCut(img, mask, rect, bgdModel, fgdModel, iterCount, mode); Institute of Visual Computing
• 7.
  Pinhole Camera 𝑃 = 𝐾[𝑅|𝑡] Pinhole camera model projection matrix 𝑓𝑥 𝑠 𝑝𝑥 calibration matrix 𝐾= 0 𝑓𝑦 𝑝𝑦 0 0 1 𝑅 ∊ ℝ3𝑥𝑥 extrinsic parameters 𝑡 ∊ ℝ3 Institute of Visual Computing
• 8.
  Camera Calibration OpenCV implementation bool findChessboardCorners(image, patternSize, corners, flags); double calibrateCamera(objectPoints, imgPoints, imgSize, camMatrix, distCoeffs, rvecs, tvecs, flags, criteria); Mat initCameraMatrix2D(objPoints, imgPoints, imgSize, aspectRatio); void getOptimalNewCameraMatrix(camMatrix, distCoeffs, imgSize, alpha, newImgSize, PixROI, cPP); void undistort(src, dst, cameraMatrix, distCoeffs, newCameraMatrix); Institute of Visual Computing
• 9.
  Structure from Motion Structure-from-motionpipeline feature point feature point camera pose extraction matching/tracking estimation Institute of Visual Computing
• 10.
  Feature Point Extraction Abstractbase class for 2D image feature detectors FeatureDetector void FeatureDetector::detect(image, keypoints, mask); Ptr<FeatureDetector> FeatureDetector::create(detectorType); The following detector types are supported: FAST, STAR, SIFT (nonfree module), SURF (nonfree module), ORB, MSER, GFTT, HARRIS, Dense, SimpleBlob Institute of Visual Computing
• 11.
  Feature Point Matching Abstractbase class for computing descriptors for image keypoints DescriptorExtractor void DescriptorExtractor::compute(image, keypoints, descriptors); Ptr<DescriptorExtractor> DescriptorExtractor::create(descriptorType); The following types of descriptor extractors are supported: SIFT, SURF, ORB, BRIEF Alternative in case of video input: KLT tracker void calcOpticalFlowPyrLK(prevImg, nextImg, prevPts, nextPts, …); int buildOpticalFlowPyramid(img, pyramid, …); Institute of Visual Computing
• 12.
  Camera Pose Estimation Generalapproach: triangulation + bundle adjustment X x x' Special case: ground plane estimation 𝐻 ∊ ℝ3𝑥𝑥 homography matrix 𝑥 ′ = 𝐻𝐻 Institute of Visual Computing
• 13.
  Hints to PlanarHomography Estimation Don’t use OpenCV’s findHomography() as it estimates a general homography Note that a general homography has 8 degrees of freedeom while a plane is determined by only 3 degrees of freedom (=> use additional constraints) Reference: R. Hartley, A. Zisserman, "Multiple View Geometry in Computer Vision" Institute of Visual Computing
• 14.
  Qualcomm`s Vuforia The Vuforiatoolkit estimates camera intrinsics and poses by means of markers Camera intrinsics: QCAR::CameraCalibration Camera poses: QCAR::Trackable -> getPose() Note that in the current implementation no distortion parameters are being estimated Institute of Visual Computing