EECE 5639 Computer Vision I
Lecture 1
Edge detection, Corner
Hw2 has been poste
Next Clas
Descriptors, Model Fitting
1
s
Modern Edge Detection
P. Dollar and C. Zitnick
“Structured Forests for Fast Edge Detection,” ICCV 2013.
2
:
Let’s look at Canny’s output
Task: Find the cows in the image
3
Canny Edges
Image Gradients + NMS Canny’s Edges
4
Canny Edges
Image Gradients + NMS Canny’s Edges
5
Canny Edges
Image Gradients + NMS Canny’s Edges
Many distractors, missing edges!
6
Can we do better?
Imagine … that someone goes and ANNOTATES which edges matter
7
Annotate
Some people have done that!
The Berkeley Segmentation Dataset and Benchmark
by D. Martin, C. Fowlkes, D. Tal and J. Malik
8
Using Annotations
How can we use annotations to improve (task oriented) edge detection?
We can use Machine Learning!
9
Very over-simpli ed explanation
Supervised Classi cation Problem:
Samples are represented by “features”
Class 1
Class -1
We have “training” labeled data from 2 classes.
10
fi
fi
Very over-simpli ed explanation
Supervised Classi cation Problem:
Class 1
?
Class -1
We have “training” labeled data from 2 classes.
Given a new sample, we want to assign to it a label.
11
fi
fi
Very over-simpli ed explanation
Supervised Classi cation Problem:
class boundary
Class 1
?
Class -1
We have “training” labeled data from 2 classes.
Given a new sample, we want to assign to it a label.
12
fi
fi
Very over-simpli ed explanation
Supervised Classi cation Problem
class boundary
Class 1
?
Class -1
We have “training” labeled data from 2 classes.
Given a new sample, we want to assign to it a label.
13
fi
fi
Supervised Classi cation
There are many strategies to learn classi ers
Nearest neighbor
Support Vector Machin
Classi cation Tree
Classi cation Forest
Neural networks (deep learning
Classi ers provide
a class “label” for new sample
a “score” of how well the label ts the new sampl
14
fi
fi
fi
s
fi
)
fi
fi
e
Training an Edge Detector
15
Training an Edge Detector
Labeled Samples!
16
Recipe for Training an Edge Detector
• Extract LOTS of image patche
• Assign labels to the samples using annotation
• Represent each patch using features (measurements) of the patc
• Simplest possibility would be to vectorize the patch:
17
s
Recipe for Training an Edge Detector
• Extract LOTS of image patche
• Assign labels to the samples using annotation
• Represent each patch using features (measurements) of the patc
• Simplest possibility would be to vectorize the patc
• Better: extract meaningful features such as gradients, color histogram, etc
18
s
Recipe for Training an Edge Detector
• Extract LOTS of image patche
• Assign labels to the samples using annotation
• Represent each patch using features (measurements) of the patc
• Simplest possibility would be to vectorize the patc
• Better: extract meaningful features such as gradients, color histogram, etc
• Train the classi er
19
fi
s
Using the Classi er
20
fi
Using the Classi er
21
fi
Using the Classi er
score
22
fi
Canny vs Structured Edge Detector
23
Canny vs Structured Edge Detector
24
Canny vs Structured Edge Detector
25
Canny vs Structured Edge Detector
26
Canny vs Structured Edge Detector
27
Evaluating Results
28
Evaluation: Recall & Precision
29
Evaluation: Recall & Precision
(green + red)
30
Evaluation
31
Point Features
Corners
Corners are useful
Which city?
33
Corners are useful
Which city?
34
Corners are useful
Which city?
35
Corners are useful
Where can I nd this pattern?
36
fi
Corners are useful
Textureless patches are nearly impossible to localize.
37
Corners are useful
Patches with large contrast are easier to nd.
38
fi
Corners are useful
Patches with large contrast are easier to nd, but edges cannot be localized.
39
fi
Corners are useful
Corners are easier!!
40
Detection of Corner Features
Need two strong edges
Example:
Create the following matrix:
Either Ex or Ey but not both are large in a neighborhood of
corner
If min(λ1,λ2) > T
There is a corner!
41
Detection of Corner Features
What if the corner is not aligned with the image coordinate
system?
But this is also true
for a slanted edge!
Both, Ex or Ey are large in the neighborhood of the corner
42
Detection of Corner Features
Solution:
“Rotate” the corner to align it with the image coordinate system!
43
Detection of Corner Features
How do we do this rotation
Since C is symmetric, it can be diagonalized;
the diagonalization is done by the rotation we need!
44
?
Harris Corner Detector
The Harris corner detector gives a mathematical approach for
determining the amount of changes when we move in all directions a
small window.
45
Harris Corner Detector
Change of intensity for the shift [u,v]:
Change
Window In
Function Intensity
Window w(x,y):
46
First Order Approximation of f(x,y):
47
Harris Corner Detector
48
Harris Corner Detector
Rewrite as a Matrix:
49
Harris Detector
M is computed from the
gradient components
50
Harris Detector
Intensity change in shifting window eigenvalue analysis:
51
Classi cation via Eigenvalues
52
fi
Corner Response Measure
53
Corner Response Measure
R = det M - k trace2(M)
54
Corner Response Example
R score; Gradient computed with Sobel mask
Window Gaussian, sigma=1
55
Corner Response Example: Edges
R < - 10000
56
Corner Response Example: Corners
R > 10000
57
Corner Response Example:
-10000 < R < 10000
Neither edges nor corners
58
Harris Corner Detection Algorithm
Compute the Image Gradient
Ix = Gx,s * I and Iy = Gy,s *
Compute products of derivatives at each pixe
I2x = Ix . Ix and I2y = Iy. Iy and Ixy = Ix . I
Compute the sums of the products at each pixel using a window
averaging
S2x = Gs’ * I2x S2y = Gs’ * I2y Sxy = Gs’ * Ix
De ne the Matrix at each pixel M = [S2x Sxy ; Sxy S2y]
Compute the response R = det M - k trace(M)2
Threshold
Compute Nonmax suppression
59
fi
:
