Feature Matching using SIFT algorithm

Feature Matching using SIFT algorithm

• 1.
  DETECTING LEVELLING RODS USINGSIFT FEATURE MATCHING GROUP 1 MSc Course 2006-08 25TH June 2007 Sajid Pareeth Sonam Tashi Gabriel Vincent Sanya Michael Mutale PHOTOGRAMMETRY STUDIO
• 2.
  Objective Introduction SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo To develop a MATLAB procedure for the detection of levelling rods.  To carry out matching based on Algorithms proposed by David Lowe  Object recognition using invariant features Transformation Examples
• 3.
  Summary of Steps Images SIFTKey Feature Extraction Keypoint Matching Removing Outliers - RANSAC Transformation Introduction SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples
• 4.
  Background - SIFT Published by David Lowe et al. 1999-2004  Algorithm to extract features that are invariant to rotation, scaling and  partially invariant to changes in illumination and camera viewpoint  Resulting features are highly distinctive Consists of (Major stages of computation):  Scale-space extrema detection-the Gaussian Widow  Keypoint localization- 4x4 samples per window in 8 directions  Orientation assignment  Keypoint descriptor-feature vector is modified to reduce the effects of illumination change. The vector is normalized to unit length. large gradient magnitudes by thresholding the values in the unit feature vector to each be no larger than 0.2, and then renormalizing to unit length. SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Introduction
• 5.
  A keypoint descriptoris created by first computing the gradient magnitude and orientation at each image sample point in a region around the keypoint location, These are weighted by a Gaussian window, indicated by the overlaid circle. These samples are then accumulated into orientation histograms summarizing the contents over 4x4 subregions, The length of each arrow corresponding to the sum of the gradient magnitudes near that direction within the region.
• 6.
  Scale Invariant Detection Consider regions (e.g. circles) of different sizes around a point  Regions of corresponding sizes will look the same in both images SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
• 7.
   The problem:how do we choose corresponding circles independently in each image? Scale Invariant Detection SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
• 8.
   Solution:  Designa function on the region (circle), which is “scale invariant” (the same for corresponding regions, even if they are at different scales) Example: average intensity. For corresponding regions (even of different sizes) it will be the same.  For a point in one image, we can consider it as a function of region size (circle radius) scale = 1/2 f region size Image 1 f region size Image 2 Scale Invariant Detection SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
• 9.
   Common approach scale= 1/2 f region size Image 1 f region size Image 2 Take a local maximum of this function Observation: region size, for which the maximum is achieved, should be invariant to image scale. s1 s2 Important: this scale invariant region size is found in each image independently! Scale Invariant Detection SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
• 10.
  scale x y ← DoG → ←DoG→ SIFT(Lowe) Maximaand minima of the difference-of-Gaussian images detected by comparing a pixel (marked with X) to its 26 neighbors in 3x3 regions at the current and adjacent scales (marked with circles). SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
• 11.
  Gaussian pyramids Scale First Octave Scale Next Octave Differenceof Gaussians Difference of Gaussian(DoG) SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
• 12.
  Keypoint Descriptors Feature Vectors Thresholded image gradients sampled over:  16x16 array of locations in scale space  Histogram of 4x4 samples per window in 8 directions  Gaussian weighting around center  8 orientations x 4 x 4 histogram array = 128 dimensional feature vector  4x4 Gradient window SIFT Algorithm Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT-Keypoints Extraction Introduction Gaussian Gradient Window
• 13.
  Criteria for Selection Prominent  Distinguishable-from neighborhood  Invariant  Stable to disturbances  Rare (exceptional) - from other selected points  Meaningful - with respect to image interpretation SIFT Algorithm Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT-Keypoints Extraction Introduction
• 14.
  SIFT keypoints  DetectedKeypoints in reference and candidate image  Keypoints in both images will be matched Candidate image with keypointsReference image with keypoints SIFT Algorithm Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT-Keypoints Extraction Introduction
• 15.
  Keypoint Matching  Fundamentalaspect in computer vision.  Based on Euclidean distance of the feature vectors.  Nearest neighbor algorithms.  Product between descriptors is calculated.  Inverse cosine of the products gives the Euclidean distance  Matches with Ratio of vector angles from the nearest to second nearest neighbor less than distRatio value are selected. SIFT Algorithm Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT-Keypoints Extraction Introduction
• 16.
  90% of thefalse matches are removed. False matches due to ambiguos features or features arise from background clutter. Reliable object recognition with few best Matches Remove outliers Matched points SIFT Algorithm SIFT-Keypoints Extraction Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Keypoints Matching Introduction
• 17.
  RANSAC: Algorithm RANdom SAmpleConsensus  Estimate parameters of a mathematical model from a set of observed data which contains outliers.  A model is fitted using hypothetical inliers.  If other data fits to the model, added to the inliers.  Reestimated the model with the new set of inliers.  We used ransac-fit-homography which Robustly fits a homography to a set of matched points. SIFT Algorithm SIFT-Keypoints Extraction Work Flow Advantages/ Limitations Demo Transformation Examples Keypoints Matching RANSAC Introduction
• 18.
  Ransac RANSAC: Result SIFT Algorithm SIFT-Keypoints Extraction WorkFlow RANSAC Advantages/ Limitations Demo Transformation Examples Keypoints Matching Introduction
• 19.
  Make the shortestimage the same height as the other image. Append Images Transformation tform = cp2tform(M1,M2,’affine') SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow Advantages/ Limitations Demo Transformation Examples RANSAC Introduction TStructure
• 20.
  2 3 Rod vs. OpenShrubs 1 Matches:9 2 Rod: 626 keypoints found Image: 28958 keypoints found Matches: 16 …Good Matches SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Examples Transformation Introduction
• 21.
  2 Rod Vs HumanOcclusion?? 1 Rod: 626 keypoints found Image: 2347 keypoints found Matches: 155 Rod: 626 keypoints found Image: 2347 keypoints found Matches: 144 2 3 …Good Matches SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Introduction
• 22.
  Limitations 1 Rod: 626 keypointsfound Image: 17927 keypoints found Matches: 16 Matches: 4 2 Rod: 626 keypoints found Image: 30505 keypoints found 3 SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Introduction
• 23.
  Advantages  Locality: featuresare local, so robust to occlusion and clutter (no prior segmentation)  Distinctiveness: individual features can be matched to a large database of objects  Quantity: many features can be generated for even small objects  Efficiency: close to real-time performance  Extensibility: can easily be extended to wide range of differing feature types, with each adding robustness SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Introduction
• 24.
  Problems/Enhancement  Only invariantto affine transformations to a certain degree  Best performance on highly textured images  Use Principal components analysis PCA instead of Gaussian weighting for gradients Demo SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Transformation Examples Introduction
• 25.
  Demo Demo SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Transformation Examples Introduction THANKYOU!!