01/20/11

Projective Geometry and Camera Models

Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem

Administrative Stuff
Office hours
Derek: Wed 4-5pm + drop by Ian: Mon 3-4pm, Thurs 3:30-4:30pm

HW 1: out Monday

Prob1: Geometry, today and Tues Prob2: Lighting, next Thurs Prob3: Filters, following week

Next Thurs: Im out, David Forsyth will cover

Last class: intro

Overview of vision, examples of state of art Logistics

Next two classes: Single-view Geometry

How tall is this woman? How high is the camera? What is the camera rotation? What is the focal length of the camera?

Which ball is closer?

Todays class
Mapping between image and world coordinates
Pinhole camera model Projective geometry
Vanishing points and lines

Projection matrix

Image formation

Lets design a camera

Idea 1: put a piece of film in front of an object Do we get a reasonable image?

Slide source: Seitz

Pinhole camera

Idea 2: add a barrier to block off most of the rays

This reduces blurring The opening known as the aperture
Slide source: Seitz

Pinhole camera
f

f = focal length c = center of the camera

Figure from Forsyth

Camera obscura: the pre-camera

First idea: Mo-Ti, China (470BC to 390BC) First built: Alhacen, Iraq/Egypt (965 to 1039AD)

Illustration of Camera Obscura

Freestanding camera obscura at UNC Chapel Hill

Photo by Seth Ilys

Camera Obscura used for Tracing

Lens Based Camera Obscura, 1568

First Photograph
Oldest surviving photograph
Took 8 hours on pewter plate

Photograph of the first photograph

Joseph Niepce, 1826

Stored at UT Austin

Niepce later teamed up with Daguerre, who eventually created Daguerrotypes

Dimensionality Reduction Machine (3D to 2D)

3D world 2D image

Point of observation

Figures Stephen E. Palmer, 2002

Projection can be tricky

Slide source: Seitz

Projection can be tricky

Slide source: Seitz

Projective Geometry
What is lost?
Length

Who is taller?

Which is closer?

Length is not preserved

A C B

Figure by David Forsyth

Projective Geometry
What is lost?
Length Angles
Parallel?

Perpendicular?

Projective Geometry
What is preserved?
Straight lines are still straight

Vanishing points and lines

Parallel lines in the world intersect in the image at a vanishing point

Vanishing points and lines

Vanishing Point Vanishing Line

Vanishing Point

Vanishing points and lines

Vertical vanishing point (at infinity) Vanishing line

Vanishing point

Vanishing point

Slide from Efros, Photo from Criminisi

Vanishing points and lines

Photo from online Tate collection

Note on estimating vanishing points

Use multiple lines for better accuracy but lines will not intersect at exactly the same point in practice One solution: take mean of intersecting pairs bad idea! Instead, minimize angular differences

Vanishing objects

Projection: world coordinatesimage coordinates

Optical Center (u0, v0)

.
f

.
Camera Center (tx, ty, tz)

X P = Y Z

u p= v

Homogeneous coordinates
Conversion
Converting to homogeneous coordinates

homogeneous image coordinates

homogeneous scene coordinates

Converting from homogeneous coordinates

Homogeneous coordinates
Invariant to scaling
x kx kx x kw w k y = ky ky = y kw w w kw
Homogeneous Coordinates Cartesian Coordinates

Point in Cartesian is ray in Homogeneous

Basic geometry in homogeneous coordinates

ai Line equation: ax + by + c = 0 linei = b i ci Append 1 to pixel coordinate to get ui homogeneous coordinate pi = v i 1

Line given by cross product of two points

lineij = pi p j

Intersection of two lines given by cross product of the lines qij = linei line j

Another problem solved by homogeneous coordinates

Intersection of parallel lines

Cartesian: (Inf, Inf) Homogeneous: (1, 1, 0) Cartesian: (Inf, Inf) Homogeneous: (1, 2, 0)

Projection matrix
R,T

Slide Credit: Saverese

jw kw Ow iw

x = K[R t ] X

x: Image Coordinates: (u,v,1) K: Intrinsic Matrix (3x3) R: Rotation (3x3) t: Translation (3x1) X: World Coordinates: (X,Y,Z,1)

Interlude: when have I used this stuff?

When have I used this stuff?

Object Recognition (CVPR 2006)

When have I used this stuff?

Single-view reconstruction (SIGGRAPH 2005)

When have I used this stuff?

Getting spatial layout in indoor scenes (ICCV 2009)

When have I used this stuff?

Inserting photographed objects into images (SIGGRAPH 2007)

Original

Created

When have I used this stuff?

Inserting synthetic objects into images

Projection matrix

Intrinsic Assumptions Extrinsic Assumptions No rotation Unit aspect ratio

Optical center at (0,0) No skew Camera at (0,0,0)
K

x = K [I 0] X
Slide Credit: Saverese

u f wv = 0 1 0

0 f 0

x 0 0 y 0 0 z 1 0 1

Remove assumption: known optical center

Intrinsic Assumptions Extrinsic Assumptions No rotation Unit aspect ratio
No skew Camera at (0,0,0)

x = K [I 0] X

u f wv = 0 1 0

0 f 0

u0 v0 1

x 0 y 0 z 0 1

Remove assumption: square pixels

Intrinsic Assumptions Extrinsic Assumptions
No skew No rotation Camera at (0,0,0)

x = K [I 0] X

u wv = 0 1 0

u0 v0 1

x 0 y 0 z 0 1

Remove assumption: non-skewed pixels

Intrinsic Assumptions Extrinsic Assumptions
No rotation Camera at (0,0,0)

x = K [I 0] X

u wv = 0 1 0

u0 v0 1

x 0 y 0 z 0 1

Note: different books use different notation for parameters

Oriented and Translated Camera

R
jw

t
Ow iw

kw

Allow camera translation

Intrinsic Assumptions Extrinsic Assumptions
No rotation

x = K [I t ] X

u wv = 0 1 0

x u0 1 0 0 t x y v0 0 1 0 t y z 1 0 0 1 t z 1

3D Rotation of Points

Slide Credit: Saverese

Rotation around the coordinate axes, counter-clockwise:

0 1 Rx ( ) = 0 cos 0 sin sin cos 0 sin 1 0 0 cos 0 0 0 1

y
p

cos Ry ( ) = 0 sin cos sin Rz ( ) = sin cos 0 0

Allow camera rotation

x = K [R t ] X
u wv = 0 1 0 s u0 r11 v0 r21 1 r31 r12 r22 r32 r13 r23 r33 x tx y ty z tz 1

Degrees of freedom
x = K [R t ] X
5 6

u wv = 0 1 0

u0 r11 v0 r21 1 r31

r12 r22 r32

r13 r23 r33

x tx y ty z tz 1

Vanishing Point = Projection from Infinity

x xR x y p = K[R t ] p = KR y p = K y R z z zR 0
u f wv = 0 1 0 0 f 0 u0 xR v0 y R 1 zR

fxR u= + u0 zR fyR v= + v0 zR

Orthographic Projection
Special case of perspective projection
Distance from the COP to the image plane is infinite
Image World

Also called parallel projection Whats the projection matrix?

x u 1 0 0 0 y w v = 0 1 0 0 z 1 0 0 0 1 1
Slide by Steve Seitz

Scaled Orthographic Projection

Special case of perspective projection
Object dimensions are small compared to distance to camera
Image World

u f Also called weak perspective w v = 0 Whats the projection matrix? 1 0

0 f 0

x 0 0 y 0 0 z 0 s 1
Slide by Steve Seitz

Suppose we have two 3D cubes on the ground facing the viewer, one near, one far.
1. What would they look like in perspective? 2. What would they look like in weak perspective?

Photo credit: GazetteLive.co.uk

Beyond Pinholes: Radial Distortion

Corrected Barrel Distortion

Image from Martin Habbecke

Things to remember
Vanishing line

Vertical vanishing point (at infinity)

Vanishing points and vanishing lines Pinhole camera model and camera projection matrix Homogeneous coordinates

Vanishing point

Vanishing point

x = K [R t ] X

Next class
Applications of camera model and projective geometry
Recovering the camera intrinsic and extrinsic parameters from an image Recovering size in the world Projecting from one plane to another

Questions

What about focus, aperture, DOF, FOV, etc?

Adding a lens

circle of confusion

A lens focuses light onto the film

There is a specific distance at which objects are in focus
other points project to a circle of confusion in the image

Changing the shape of the lens changes this distance

Focal length, aperture, depth of field

F focal point optical center (Center Of Projection)

A lens focuses parallel rays onto a single focal point

focal point at a distance f beyond the plane of the lens Aperture of diameter D restricts the range of rays

Slide source: Seitz

The eye

The human eye is a camera

Whats the film?

Iris - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris
photoreceptor cells (rods and cones) in the retina

Depth of field

Slide source: Seitz

f / 5.6

f / 32

Changing the aperture size or focal length affects depth of field

Flower images from Wikipedia
http://en.wikipedia.org/wiki/Depth_of_field

Varying the aperture

Slide from Efros

Large aperture = small DOF

Small aperture = large DOF

Shrinking the aperture

Why not make the aperture as small as possible?

Less light gets through Diffraction effects
Slide by Steve Seitz

Shrinking the aperture

Slide by Steve Seitz

Relation between field of view and focal length

Field of view (angle width)

d fov = tan 2f
1

Film/Sensor Width

Focal length

Dolly Zoom or Vertigo Effect

http://www.youtube.com/watch?v=Y48R6-iIYHs
How is this done?

Zoom in while moving away

http://en.wikipedia.org/wiki/Focal_length