Pix4D Introduction Guide
October 2013
Plan
Introduction
Stereoscopy/calibration theory
Processing steps
Step1: Calibration + exterior orientation
Step2: Point densification
Step3: DSM and orthomosaic
Output files usage
Software quick guide
Introduction
Pix4D converts thousands of images into
3D Point Cloud
3D Digital Surface Model
Orthomosaic
Plan
Intro
Stereoscopy/calibration theory
Processing steps
Step1: calibration + exterior orientation
Step2: point densification
Step3: DSM and orthomosaic
Output files usage
From 2D to 3D
From 3D to 2D
From 2D to 3D
One camera
Far appears smaller
than near on image
Two images
Triangulate to get
sense of depth
From 2D to 3D: Stereoscopy
3D point
2D Keypoint
2D Keypoint
Camera model
(focal length,
principal point,
lense distortion)
External orientation
(camera position &
rotation)
Camera model
(focal length,
principal point,
lense distortion)
Relative exterior orientation
Pix4D is able to compute a precise camera calibration
and relative modelfrom image content only!
Relative Position Error
gps
above 1 meter
computed from pixel
below 1 centimeter
Plan
Intro
Stereoscopy/calibration theory
Processing steps
Step1: calibration + exterior orientation
Step2: point densification
Step3: DSM and orthomosaic
Output files usage
Software quick guide
3 main processing steps
Step 1: initial
processing
Images
=> calibrate cameras +
exterior orientation
Step 2: point
densification
Step 3: DSM and
orthomosaic
Calibrated cameras
=> point clouds
Point clouds
=> DSM and orthomosaic
n
n
s
n
tim
ing
ing
tio
CP
r
ing
tio
p
h
tio
ge
a
e
d
c
G
o
c
a
t
a
c
/
t
l
n
i
a
r
l
i
f
if
e
tr
ne
PS
ma
ble
nt
c im
ns
ex
e
od
i
i
G
s
c
e
t
g
o
i
t
a
m
d
n
a
P
in
os
oin
ra
tio
os
int
SM
po
p
m
e
a
o
y
y
D
m
o
c
m
P
Ke
Ke
rth
tho
olo
Ca
r
O
e
O
G
Plan
Intro
Stereoscopy/calibration theory
Processing steps
Step1: calibration + external orientation
Step2: point densification
Step3: DSM and orthomosaic
Output files usage
Software quick guide
Camera calibration + Exterior
Orientation
Step 1: initial processing
Images => calibration + exterior orientation
e
int
o
p
ing
n
tio
c
ra
xt
y
Ke
Example: 60'000 keypoints
per image
(14MP,
100
images)
ts
oin
h
atc
yp
Ke
Average of 6'000
matches per pair
of images
CP
pti
lo
od
m
a
me
a
C
600'000
matches =>
150'000 3D
Points for 100
images project
S/G
o
Ge
G
on
i
t
a
loc
Average of GPS
OR
Min 3 GCP
Getting enough matches
Enough matches:
>1000 matches
75% overlap
enough matches
<100 matches
20% overlap
not enough matches
Getting enough matches
Enough matches:
# matches > 1000 per image pair
Depends on overlap, image size and visual content
If small image size => more overlap required
If low visual content => more overlap required
Images of 12 MP => 75% overlap recommended in most cases
Extract Keypoints
Good image:
rocks,bushes,dirt
buildings, urban
more than 10MP
extract > 10'000 keypoints
Difficult images:
sand, snow, fog
blurry, out of focus
Overexposed, underexposed
less than 3MP
extract < 100 keypoints
Match keypoints
Easy to match
images with high # keypoints
images with high overlap
Hard to match
images with low # keypoints
trees at low altitude
extreme angle with no transition
low overlap
Impossible to match
Reflective surface like water
Moving objects
Flight plan
Ideal flight plan
regular grid flight plan
easy terrain: 75% frontal, 50% side
difficult terrain: 85% frontal, 60% side
Difficult flight plan
low overlap
multiple images at same location
corridor mapping
high difference in altitude (>2xGSD)
Initial Processing: quality report
Ensures quality of project
Issues with project
To solve 99% of issues: Increase overlap or fly again
Proposed workflow
Model Georeference
Project size, orientation and position unknown
GPS tags
approximate position,orientation,size
good synchronisation, high number of GPS tags => better approximation
~10 meter shift in random direction
GCP
If GCP, GPS tags are not used
Minimum 3 GCPs
Results in same coordinate system as GCPs
Geolocation: accuracy
GCPs
well distributed in dataset, visible on the images (GSD)
5-8 usually enough for 1000 images
Quality report shows reprojection error
Good calibration + GCP: error at most 1-2 times
the GSD
Verification points
Error is minimized at GCPs: not optimal to assess global
accuracy
Verification points are not used during optimization
Plan
Intro
Stereoscopy/calibration theory
Processing steps
Step1: calibration + exterior orientation
Step2: point densification
Step3: DSM and orthomosaic
Output files usage
Software quick guide
Point densification
Step 2: point densification
Calibrated cameras + exterior
orientation => cloud of points
on
ti
a
c
i
if
ns
td
oin
ng
ri
lte
i
f
t
in
Po
Example:
8 millions 3D
points
2 millions filtered
and triangulated
3D points
Point densification
Visual content
3D points computed where there is visual content
There may be "holes" in point cloud
No visual, no 3D (example: no ground under trees)
Roads, rooftops, may contain little visual content
Less visual => less accuracy/less points
Trees visual content only at lower resolution
Use "high tolerance" to model trees
Expect lower accuracy at trees
Point densification: filtering
Noise filtering
improves quality
2.5D only
Smoothing
removes redundant points
makes point cloud easier to manage (smaller files generated)
2.5D only
Point densification: accuracy
Point cloud accuracy
Good camera: error at most 1-2 times the GSD
up to 3 times the GSD at areas with low visual content
Global Accuracy
Calibration+Georeference+Point Cloud
Plan
Intro
Stereoscopy/calibration theory
Processing steps
Step1: calibration + exterior orientation
Step2: point densification
Step3: DSM and orthomosaic
Output files usage
Software quick guide
DSM and Orthomosaic
Step 3: DSM and orthomosaic
Point cloud
=> DSM and orthomosaic
es
ion
t
era
M
DS
ge
ing
ag
m
i
ic
sa
Or
mo
o
h
t
Or
th
ic
sa
o
om
nd
ble
DSM generation
Point cloud => Regular grid of elevation (2.5D)
Interpolation where no points => no holes
DSM accuracy
Interpolation at place with no visual information
Global Accuracy
Calibration+Georeference+Point
Cloud+Interpolation
Photo Stitching VS
Orthorectification
Glue images together
Undistort perspective using the
3D model and blend images
Works only if terrain perfectly flat
Error accumulation, only small
dataset
No good georeference support
without GCP
Most distances not preserved
Requires low number of
matches/keypoints (<100)
Handles any terrain type
Handles large dataset
Supports Georeference perfectly
Preserves distances and
becomes measurable
Requires high number of
matches/keypoints (>1000) to
generate the 3D model
Orthorectification
UAV image
with
perspective
UAV image with perspective
Facade
Roof not correct size
Orthorectify image
1) calibrate camera
2) project image on 3D model
3) generate ortho image by looking
from above
Computed 3D model
Undistorted orthoimage
Same as satellite
Roof correct size
Orthorectification artefacts
3D model not always perfect
Edges of buildings
Small details
trees (branches smaller than pixels)
lamppost, fences
Orthorectification depends on surface model =>
artifacts at 3D model errors
Original image tilted: area not
visible producing artefacts
3D model not as precise as
texture, leading to "wobbly"
edges
Blending and Editing
Default: automated blending
Scene editor:
choose image
correct brightness
selective use of orthorectification to generate
visually pleasing mosaics
Plan
Intro
Stereoscopy/calibration theory
Processing steps
Step1: calibration + exterior orientation
Step2: point densification
Step3: DSM and orthomosaic
Output files usage
Generated files
Quality report
Camera position & parameters
Raw point cloud
(ply,las)
Filtered point cloud (ply/las)
regular point cloud
DSM in tiff
individual ortho and
planar
orthomosaic in
tif, KML tiles
Step 1: initial
processing
Step 2: point
densification
Step 3: DSM and
orthomosaic
n
n
s
n
tim
ing
ing
tio
CP
r
ing
tio
p
h
tio
ge
a
e
d
c
G
o
c
a
t
a
c
/
t
l
n
i
a
r
l
i
f
if
e
tr
ne
PS
ma
ble
nt
c im
ns
ex
e
od
i
i
G
s
c
e
t
g
o
i
t
a
m
d
n
a
P
in
os
oin
ra
tio
os
int
SM
po
p
m
e
a
o
y
y
D
m
o
c
m
P
Ke
Ke
rth
tho
olo
Ca
r
O
e
O
G
Generated files
Pix4uav outputs
Format
Possible use
Examples of compatible
software
Raster orthomosaic
geoTIFF (.tiff)
KML tiles (.png/.kml)
Area overview
Digitize buildings
Annotate areas
Overlay in GIS package
Analyze spectral bands
ArcGIS
Global Mapper
QuantumGIS
AutoCAD
Google Earth
Undistorted images
TIFF (.tiff)
Stereo Viewing
LPS
Individual ortho/planar
geoTIFF (.tiff)
Seamline editing
OrthoVista
3D
point cloud
.las, .ply, .ascii
Visualization
Surface Editing
DSM generation
DTM generation
ArcGIS
Global Mapper
AutoCAD
Quick Terrain Reader
3D Reshaper
Trimble RealWorks Viewer
Raster digital surface
model (DSM)
geoTIFF (.tiff)
Analyze surface
Measure volumes
Generate contour lines
Draw breaklines
ArcGIS
Global Mapper
QuantumGIS
Quick Terrain Reader
3D mesh with texture
Wavefront (.obj)
Render in animation package
Visualize small projects
AutoCAD
Bentley Pointools View
CC Viewer
3D Reshaper
Use of generated files
Editing
Analyzing
Visualization
3D point cloud
Raster DSM
3D Mesh
best (VRMesh)
possible
possible
mainly transform to raster
DSM
best (ArcGIS/Global
Mapper)
small projects only
good (Quick Terrain
Reader)
good (ArcGIS/Global
Mapper)
small projects only
Plan
Intro
Stereoscopy/calibration theory
Processing steps
Step1: calibration + exterior orientation
Step2: point densification
Step3: DSM and orthomosaic
Output files usage
Software Demo
Software Demo: create project
3 Step wizard
1&2)
- create project, import images
- is also used to "merge" overlapping
project. Split and merge when computer
resources not sufficient, or when
calibration issues between areas of
project.
3)
geotags and camera model selection
- automatic geotag, or using text file
- automatic camera model selection,
or using database
- at least three valid geotag required
Software Demo: main interface
1) Processing
button
2) Start button
Steps in red if not yet
processed. Checkbox
allows run of individual
step.
127 images
12 MP Canon Ixus
quadcore laptop
Initial Processing:
Option: rapid,full
Point densification:
Option: density
Orthomosaic/DSM:
Option: resolution
rapid: 3 min, low
accuracy
full: 14 minutes
optimal: 28 min
Choose high
tolerance if forest
auto will select average
computed GSD: 15 min
Can be replaced by any
other number
Software Demo: GCP editor
1) Add GCP/control/verification
using file OR right click
3) Measure GCP in at least 2
images per GCP, 5
recommended
2) List of images
use top icons to sort
closest images
use right click to remove
measurements
4) Import/export
clicked points without
coordinates
5) If initial processing is
already done, bundle
with GCP allows for
faster re-processing
Software Demo: Scene Editor
4) Change/Add cells using
polygon/line/points tools
5) Blend mosaic
2) Select ortho/planar
to improve visual
aspect
3) Choose most
appropriate image in
the list (use keyboard
up/down arrow).
Good to remove
blurry images/ moving
cars
1) Select a cell by
clicking
Thank you
