Lectures 9-10: Imaging Geometry and Camera
Model
Dr. V Masilamani
masila@iiitdm.ac.in
Department of Computer Science and Engineering
IIITDM Kancheepuram
Chennai-127
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 1 / 37
Overview
1 Multiple View Imaging Geometry
2 Single View Imaging Geometry
3 Geometric Transformation in 3D
Translation in 3D
Scaling in 3D
Rotation in 3D
Shearing in 3D
Perspective Transformation in 3D
4 Camera Modeling
5 Camera Calibration
6 Acknowledgements
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 2 / 37
Multiple View Imaging Geometry
I What is multiple view imaging
geometry
• Geometry of images of the
same scene taken by many
camera
I Objective of multiple view
imaging geometry
• Given the multiple images
of the same scene, find the
geometry of 3D object
• The geometry of 3D object
means: shape, volume,
surface area, length,
position etc.
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 3 / 37
Multiple View Imaging Geometry (cont.)
Some Applications of Multiple
View Geometry
I Animating the real actor
I Depth computation
I Image Stitching
I 3D Model construction
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 4 / 37
Single View Imaging Geometry
I Coordinate System: • p can be written as a linear
(O, (X , Y , Z ) combination of those
orthonormal basis vectors
• O is a point in the space
called origin • Example:
(a, b, c) = a(1, 0, 0) +
• X , Y , Z are ortho-normal
b(0, 1, 0) + c(0, 0, 1)
basis vectors
I Example of a Coordinate
system
• O = (0, 0, 0)
• X = (1, 0, 0), Y = (0, 1, 0),
Z = (0, 0, 1)
I How to represent a point p in
a coordinate system
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 5 / 37
Single View Imaging Geometry (cont.)
How to relate a point on an object and the corresponding point on
its image
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 6 / 37
Single View Imaging Geometry (cont.)
The coordinate systems involved in Imaging geometry
I World Coordinate system(3D):
• The coordinate system used to locate the points of the object
• The coordinates (x, y , z) ∈ R 3
I Image Coordinate system (2D)
• Origin: The center of the image plane
• X , Y axes are the X , Y axes of the image plane in the camera
• The coordinates (x, y , z) ∈ R 2
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 7 / 37
Single View Imaging Geometry (cont.)
I Camera Coordinate system(3D):
• Origin: The center of the lens (also called as projection center)
• X , Y axes are the X , Y axes of the image plane in the camera
• The coordinates (x, y , z) ∈ R 3
I Pixel Coordinate system (2D)
• Origin: Top most-Left most point in the image plane
• X , Y axes are the X , Y axes of the image plane in the camera
• The coordinates (x, y , z) ∈ A × B, where A = {0, 1, 2, ..., (M − 1)
and B = {0, 1, 2, ..., (N − 1)
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 8 / 37
Geometric Transformation in 3D
I Definition of Geometric Transformation:
• Transformation that may change the geometry of the objects
I The geometry of object refers to
• Shape, size, distance, positions etc.
I Some Geometric Transformations
• Translation
• Scaling
• Rotation
• Shearing
• Perspective
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 9 / 37
Translation in 3D
x1 x2
I Translation of a 3D Point: y 1 → y 2
z1 z2
where
x2 = x1 + ∆x
y 2 = y 1 + ∆y
z2 = z1 + ∆z
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 10 / 37
Translation in 3D (cont.)
I HW: Prove that Translation is not linear in 3D Cartesian Coordinate
system
I To make translation linear, transform the Cartesian coordinate
system to homogeneous coordinate system
I Homogeneous Coordinatesystem:
kx
ky
Coordinates of the form
kz
k
I Given Cartesian coordinate system, homogeneous coordinates can be
obtained
I Given homogeneous coordinates, Cartesian coordinate system can be
obtained
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 11 / 37
Translation in 3D (cont.)
I To make 3D translation linear in homogeneous system(4D),
x
x y
transform 3D point y to z
z
1
I 3D translation in homogeneous
coordinate
system(4D):
x2 1 0 0 ∆x x1
y 2 0 1 0 ∆y y 1
=
z2 0 0 1 ∆z z1
1 0 0 0 1 1
I Translation
matrix:
1 0 0 ∆x
0 1 0 ∆y
T= 0 0 1 ∆z
0 0 0 1
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 12 / 37
Translation in 3D (cont.)
1 0 0 −∆x
I T −1
0 1 0 −∆y
=
0 0 1 −∆z
0 0 0 1
I Observation: Given the translation vector, T −1 can be computed
directly(without using algorithm for computing inverse)
I Where is translation used
• To model moving object
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 13 / 37
Scaling in 3D
I Scaling of 3D point:
x1 ax1
y1 → by1
z1 cz1
I In
Matrix
equation
form:
x2 a 0 0 x1
y 2 = 0 b 0 y 1
z2 0 0 c z1
I In hom. coord. system:
x2 a 0 0 0 x1
y 2 0 b 0 0 y 1
=
z2 0 0 c 0 z1
1 0 0 0 1 1
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 14 / 37
Scaling in 3D (cont.)
0
I x x
= S
y0 y
a 0
where, scaling S =
0 b
1
I S −1 = a 01
0 b
I S −1 S = S −1 S = I
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 15 / 37
Rotation in 3D
I Defn. of Rotation of a point
about Z axis
• P1(x1, y 1, z1) is said to be
rotated about Z axis with
angle θ to result
inP2(x2, y 2, z2) if
z2 = z1, and x2, y 2 are
obtained using 2d Rotation
on XY plane in anti-clock
wise direction
I Similarly rotations of a point
about X and Y axes can be
defined
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 16 / 37
Rotation in 3D (cont.)
Rotations of objects about X, Y and Z directions
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 17 / 37
Rotation in 3D (cont.)
Is the rotation of a point about Z axis in 3D Cartesian coordinate
linear?
I Since the rotation is along Z axis, z1 = z2.
I Let (r , θ) represents P1 on XY plane, and hence, (r , θ + φ) will
represent P2 on XY plane
I sinΦ = y1
r ⇒ y 1 = rsinΦ
I cosΦ = x1
r ⇒ x1 = rcosΦ
I sin(Θ + Φ) = y2
r ⇒ y 2 = rsin(Θ + Φ)
I similarly x2 = rcos(Θ + Φ) x2 = r (cosΘcosΦ − sinΘsinΦ)
x2 = (rcosΦ)cosΘ − (rsinΦ)sinΘ
x2 = x1cosΘ − y 1sinΘ
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 18 / 37
Rotation in 3D (cont.)
I similarly y 2 = rsin(Θ + Φ) y 2 = r (sinΘcosΦ + cosΘsinΦ)
y 2 = (rcosΦ)sinΘ + (rsinΦ)cosΘ
y 2 = x1sinΘ + y 1cosΘ
I Rotation of P1 = (x1, y 1, z1) about Z axis which results in
P2 = (x2, y 2, z2) can be defined as a matrix equation
I
In 3DCartesian
coordinates:
x2 cosΘ −sinΘ 0 x1
y 2 = sinΘ cosΘ 0 y 1
z2 0 0 1 z1
I In homogeneous coord. system:
P2 = RΘz P1 where
cosΘ −sinΘ 0 0
Z
sinΘ cosΘ 0 0
RΘ = 0
0 1 0
0 0 0 1
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 19 / 37
Rotation in 3D (cont.)
I Similarly, rotation about X axis:
P2 = RΘX P1 where
1 0 0 0
X
0 cosΘ −sinΘ 0
RΘ =
0 sinΘ cosΘ 0
0 0 0 1
I Rotation about Y axis:
Y
P2 = RΘ P1 where
cosΘ 0 −sinΘ 0
0 1 0 0
RΘY = sinΘ 0 cosΘ 0
0 0 0 1
I Hw: Derive transformation matrices for rotations about Y and Z
axes
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 20 / 37
Shearing in 2D
I Shearing in x Dir:
x 0 = x + sy ; y 0 = y ;
I Shearing in y Dir:
x 0 = x; y 0 = sx + y ;
I Shearing in x-y Directions:
x 0 = x + sy ; y 0 = tx + y ;
I Application: When the film in
the camera not fixed properly
-Shearing effect may happen
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 21 / 37
Shearing in 3D
I 3D-Shearing in x Dir: opposite faces
x 0 = x + by + cz;
y0 = y; z0 = z
I 3D-Shearing in y Dir:
x 0 = x; z 0 = z
y 0 = dx + y + fz;
I 3D-Shearing in Z Dir:
x 0 = x; y 0 = y ;
z 0 = gx + iy + z
I Application: To synthesize
funny cartoon characters
-3D sheering pulls the
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 22 / 37
Perspective Transformation in 3D
where
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 23 / 37
Perspective Transformation in 3D (cont.)
f - is the focal length
Yc - height of the object
-y - height of the image
I Camera coordinate system is considered
I Image plane and XY plane of camera are the same
I Using the property of similar triangles,
h2 f
=
h1 Zc
−y f
Sub value of h1 and h2, =
Yc Zc
−Yc f
Therefore, y =
Zc
−Xc f
Similarly , x =
Zc
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 24 / 37
Perspective Transformation in 3D (cont.)
I Hw: Prove that perspective transformation is not linear in Cartesian
coord. system.
I The perspective transformation in homogeneous coord. system
kXc 1 1 0 0 kXc
kYc 0 1 0 0 kYc
kZc = 0 0 1 0 kZc
−kZc
f
0 0 −1 f 1 k
I To convert perspective transformed points in homogeneous coord.
system to Cartesian coord. sysem
kXc −Xc f
x= −kZc
⇒x =
f
Zc
kYc −Yc f
y= −kZc
⇒y =
f
Zc
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 25 / 37
Camera Modeling
I Observation: Any two coordinate systems can be aligned using 1
translation and 3 rotations.
I Therefore, Camera Coordinate System(CCS) can be obtain from
World Coordinate System(WCS) by translating with (∆x, ∆y , ∆z)
followed by the rotating with Θ1, Θ2, Θ3 about X,Y and Z axes
respectively.
I Let Pw and Pc be the representations of p in the WCS and CCS
respectively
Z −1 Y −1 X −1
Pc = ((RΘ3 ) (RΘ2 ) (RΘ1 ) T −1 ) Pw
Z
Pc = ((R−Θ3 Y
) (R−Θ2 X
) (R−Θ1 ) T −1 ) Pw
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 26 / 37
Camera Modeling (cont.)
I Show that (RΘZ )−1 = R−Θ
Z
cosΘ −sinΘ 0 0
sinΘ cosΘ 0 0
RΘZ =
0
0 1 0
0 0 0 1
cosΘ sinΘ 0 0
−sinΘ cosΘ 0 0
(RΘZ )−1 =
0
0 1 0
0 0 0 1
Z
= R−Θ
I Let Pi be the output of perspective transformation of pc
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 27 / 37
Camera Modeling (cont.)
I X,Y axes of CCS and ICS are the same, Pi can be considered as a
point in ICS
Ih1 Ch1
Ih2 Ch2
Pi =
Ih3 P = Ch3
Ih4 Ch4
in hom. coord. system
I To obtain Cartesian coordinates Pp in PCS from Pi in ICS,
Ih1 Ih2
x= y=
Ih4 Ih4
I To obtain Cartesian coordinates of Pi in PCS
• Step 1: Translate Pi so that Left Most and Top Most point is origin
(call the transform as T1 )
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 28 / 37
Camera Modeling (cont.)
• Step 2: Scale the resultant of Step 1 so that coordinates are
Non-negative integers(call the transform as S)
Z
I Pc = ((R−Θ3 Y
) (R−Θ2 X
) (R−Θ1 ) T −1 ) Pw
Z
I Pi = PPc = (P((R−Θ3 Y
) (R−Θ2 X
) (R−Θ1 ) T −1 )) Pw
Z
I Pp = ST1 Pi = (ST1 P((R−Θ3 Y
) (R−Θ2 X
) (R−Θ1 ) T −1 )) Pw
I
Ih10
Ih20 Z Y X −1
Let Pp =
Ih30 = APw ; A = (ST1 P((R−Θ3 ) (R−Θ2 ) (R−Θ1 ) T ))
Ih40
I The Cartesian coord. of Pp :
Ih10 Ih20
x0 = y0 =
Ih40 Ih40
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 29 / 37
Camera Calibration
I What is camera calibration?
• Camera calibration means finding a matrix A such that Pp = APw
• A is called as camera matrix
I Camera calibration is also called as computaion of camera matrix
How to find the camera matrix A:
I In Pp = APw , we can substitute the values for Pp and Pw
I As Pp = (x 0 , y 0 )T where
Ih10 Ih20
x0 = y0 =
Ih40 Ih40
Ih10 − x 0 Ih40 = 0
Ih20 − y 0 Ih40 = 0
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 30 / 37
Camera Calibration (cont.)
I Sub values of Ih10 , Ih40 , Ih20 , we get
a11 X + a12 Y + a13 Z + a14 − x 0 (a41 X + a42 Y + a43 Z + a44 ) = 0
a21 X + a22 Y + a23 Z + a24 − y 0 (a41 X + a42 Y + a43 Z + a44 ) = 0
I for each pair (Pw , Pp ), two equations are formed
a11 a12 a13 a14 X
a21 a22 a23 a24 Y
I A = Ph =
− − − − Z
a41 a42 a43 a44 1
I Since A is a 4*4 matrix, we have 16 unknowns requiring 16 equations,
thus we need 8 points.
I since Z is not required, we can use 6 points to form 12 equations for 12
unknowns
I But all the equations should be linearly independent which might not be
the case. Hence more points will be required.
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 31 / 37
Camera Calibration (cont.)
I Use n pairs of points, to get 2n equations as follows
a11 X1 + a12 Y1 + a13 Z1 + a14 − x1 (a41 X1 + a42 Y1 + a43 Z1 + a44 ) = 0
.
.
.
a11 Xn + a12 Yn + a13 Zn + a14 − xn (a41 Xn + a42 Yn + a43 Zn + a44 ) = 0
a21 X1 + a22 Y1 + a23 Z1 + a24 − y1 (a41 X1 + a42 Y1 + a43 Z1 + a44 ) = 0
.
.
.
a21 Xn + a22 Yn + a23 Zn + a24 − yn (a41 Xn + a42 Yn + a43 Zn + a44 ) = 0
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 32 / 37
Camera Calibration (cont.)
I To Convert homogeneous system of linear equations to
non-homogeneous system of linear equations
• Divide the equation by a44 on both sides
• Then the aij ← aij /a44
• rearrange the matrix equation as shown in the next slide
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 33 / 37
Camera Calibration (cont.)
I By solving the updated non-homogeneous system of linear
equations(as shown below), we shall get 11 parameters, the 12th
parameter namely a44 = 1
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 34 / 37
Camera Calibration (cont.)
C D = U
C T (C D) = C T U
(C T C ) D = C T U
D = (C T C ) C T U
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 35 / 37
Acknowledgements
I Images are downloaded from internet sources
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 36 / 37
Thank You! :)
Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 37 / 37
