DIGITAL IMAGE
PROCESSING
Lecture 1
Introduction
Tammy Riklin Raviv
Electrical and Computer Engineering
Ben-Gurion University of the Negev
2
Introduction to Digital Image Processing
¡Lecturer: Dr. Tammy Riklin Raviv
¡Teaching assistants: Assaf Arbelle and Boris Kodner
¡No.: 361-1-4751
¡Time: Wednesday 14:00-17:00
¡Location: Building 28 Room 204
¡Prerequisites: Digital Signal Processing
Introduction to Stochastic Processes
• Course website:
http://www.ee.bgu.ac.il/~rrtammy/DIP-2017/DIP
3
Course Objectives
¡The primary objective of this course is to provide the
students the necessary computational tools to:
l Understand the main principles of image processing and computer
vision.
l Be familiar with different classical and commonly used algorithms
and understand their mathematical foundation
l Implement (Matlab) and test commonly used image analysis
algorithms
l Develop critical reading of computer vision and digital signal
processing and analysis literature
4
Course Resources
• Szeliski, Richard. Computer vision: algorithms and
applications. Springer Science & Business Media, 2010.
• Gonzalez, Rafael C., and Richard E. Woods. "Image
processing." Digital image processing 2 (2007).
• Forsyth, David A., and Jean Ponce. "A modern
approach." Computer vision: a modern approach (2003).
• Duda, Richard O., Peter E. Hart, and David G.
Stork. Pattern classification. John Wiley & Sons, 2012.
• Bishop, C. "Pattern Recognition and Machine Learning
(Information Science and Statistics), 1st edn. 2006. corr.
2nd printing edn." Springer, New York(2007).
5
What should I do in order to succeed in
the course?
• 6 Matlab assignments assignments, each 5% -> 30%
• Bonus assignments
• Final project and presentation 70%
6
The instructor
• Tammy Riklin Raviv,
Research interests:
Signal processing: Biomedical Image Analysis, Computer
Vision, Machine Learning
Contact info:
Telephone: 086428812
Fax: 08-647 2949
E-mail: rrtammy@ee.bgu.ac.il
Office: 212/33
Reception hours:
please email
• Personal web page:
http://www.ee.bgu.ac.il/~rrtammy/
7
The Final Project
¡A list of possible projects will be distributed around Passover
¡Students can choose a subject out of this list or come up with
their own ideas.
¡It is the students responsibility to schedule a meeting with the
teaching assistants to discuss the project of their choice.
¡Students can work with a single or two team mates.
¡Students are expected to based their project on a scientific
publication, make sure they understand it and are able to
implement it using Matlab.
¡All students should present their final project. Date TBD.
8
List of topics (Tentative)
Overview on digital image processing,
Visual Perception
What is an image?
Sampling, quantizatiotion,
Histogram processing
Color image processing
Edge detection
Frequency domain analysis, Fourier transform
2D shape representation: Hough transform, pyramids, quad trees
Imaging geometry: Scaling, rotation, camera model, pose estimation
List of topics (tentative)
• Photometry, shape from shading
• Image segmentation
• representation and descriptors, SIFTs and Hogs
• Stereo and Motion
• Face detection
A graphical view on the syllabus
The Rest of Today’s Class
• Brief Overview
• Human Vision and Visual Perception
• What is an Image?
The Rest of Today’s Class
• Brief Overview
• Human Vision and Visual Perception
• What is an Image?
A brief overview: What is an Image?
Brown’s CV course
A Brief Overview: Image Formation
Output Image
Camera Sensor
Brown’s CV course
A Brief Overview: Image Formation
See: Introduction to Medical Imaging
Magnetic Resonance Imaging
A Brief Overview: Image Formation
Main Focus
A Brief Overview: Image Formation
Sensor Array
CMOS sensor
James Hays
A Brief Overview: Image Processing
Quantization
James Hays
A Brief Overview: Image Processing
Images as 2D Signals
Sampling
A Brief Overview: Image Processing
Images as 2D Signals
Sampling
Resolution – geometric vs. spatial
resolution
Both images are ~500x500 pixels
A Brief Overview: Image Representation
Grayscale Digital Image
Brightness
or intensity
x y
Danny Alexander
A Brief Overview: Light and Color
Danny Alexander
A Brief Overview: Edge detection
Danny Alexander
A Brief Overview: Frequency Analysis
Fourier domain Image domain
A Brief Overview: Frequency Analysis
Fourier domain Image domain
A Brief Overview: Frequency Analysis
Fourier domain Image domain
A Brief Overview: Image Segmentation
A Brief Overview: Image Segmentation
Texture
segmentation
A Brief Overview: Image Segmentation
Prior based
segmentation
Riklin Raviv et al, IJCV 2007
A Brief Overview: Image Segmentation
Symmetry based
segmentation
Riklin Raviv et al, TPAMI 2009
A Brief Overview: Feature Detection
A Brief Overview: Correspondences
A Brief Overview: Correspondences
A Brief Overview: Stereo and 3D
reconstruction
A Brief Overview: Object Detection and
Recognition
A Brief Overview: Motion
Optical Flow
A Brief Overview: Tracking
Hyun Tae Na Thesis
A Brief Overview: Tracking
RPE
Arbelle’s Thesis
The Rest of Today’s Class
• Brief Overview
• Human Vision and Visual Perception
• What is an Image?
Human Vision/ Visual Perception
• The human eye is a camera
• Iris - colored annulus with radial muscles
• Pupil - the hole (aperture) whose size is controlled by the iris
• What’s the sensor?
– photoreceptor cells (rods and cones) in the retina
Slide by Steve Seitz
Human Vision: the Retina
Cross-section of eye Cross section of retina
Pigmented
epithelium
Ganglion axons
Ganglion cell layer
Bipolar cell layer
Receptor layer
Human Vision: Retina up-close
Light
Human Vision: Retina up-close
Cones
cone-shaped
less sensitive
operate in high light
color vision
Rods
rod-shaped
highly sensitive
operate at night
gray-scale vision
Two types of light-sensitive receptors
© Stephen E. Palmer, 2002
James Hays
Rod / Cone sensitivity
Distribution of Rods and Cones
.
Blind
# Receptors/mm2
Fovea Spot
150,000 Rods Rods
100,000
50,000 Cones Cones
0
80 60 40 20 0 20 40 60 80
Visual Angle (degrees from fovea)
Night Sky: why are there more stars off-center?
Averted vision: http://en.wikipedia.org/wiki/Averted_vision
© Stephen E. Palmer, 2002
James Hays
Electromagnetic Spectrum
Human Luminance Sensitivity Function http://www.yorku.ca/eye/photopik.htm
Human Visual Perception
What can we learn from human visual perception?
Human Perception
Muller-Lyer illusion
Human Perception
brightness constancy
Ted Adelson, http://web.mit.edu/persci/
people/adelson/checkershadow illusion.html
Human Perception
Hermann grid illusion
Hany Farid, http://www.cs.dartmouth.edu/farid/illusions/hermann.html
Human Perception
Count the red X
pop-out effect (Treisman 1985)
The Rest of Today’s Class
• Brief Overview
• Human Vision and Visual Perception
• What is an Image?
What is an Image?
This Image
is taken from
Brown’s
Computer
Vision Course
How would it look through the
“computer’s eyes” ?
Why is this an image?
Hello Word !
Hello Word !
Hello Word !
Hello World J R G B
Let’s zoom in
Let’s Pixel
Pixel = Picture Element
Let’s Pixel
Let’s count
1 Pixel = 8 bits (UINT 8) = 1 Byte
1,205,592
myFirstImage worth maybe 1000 words
but costs much more ….
What is an Image?
An image I is a two dimensional (2D) function
that maps the image domain ⌦ to [0, 255]
I : ⌦ ! [0, 255]
or (for RGB)
I(x) = I(x, y) = ~v , v 2 [0, 255]3
and the value of a single pixel is:
I(x) = I(x, y) = v, v 2 [0, 255] Gray Level Pixel
I(x) = I(x, y) = (vR , vG , vB ) RGB pixel
Is this an image?
What’s the difference?
Building an histogram
Can we measure the differences?
A bit on information theory
(in the context of images) {0, 1}
Choose an arbitrary pixel in an image,
can you guess its value?
Well, we can build an histogram and gamble on the value
with the highest frequency
A bit on information theory {0, 1}
Choose an arbitrary pixel in an image,
can you guess its value?
Well, we can build an histogram and gamble on the value
with the highest frequency
A bit on information theory
(in the context of images)
By normalizing an histogram, one can get the
probability pi for the occurrence of the i-th value.
The Shannon entropy (measured in bits) is given by:
X
H= pi log2 (pi )
i
where log2 (pi ) is the self-information,
which is the entropy contribution of an individual pixel.
Entropy of an Image
What does it mean ? Does it mean anything?
Entropy = 7.98
Entropy = 6.98
Entropy of an Image
What does it mean ? Does it mean anything?
Entropy = 7.98
Entropy = 6.98
But Entropy = 0
Entropy of an Image
What does it mean ? Does it mean anything?
Entropy ( ) = Entropy ( )
Obtained by pixel permutation
Entropy of an Image
What does it mean ? Does it mean anything?
SmallI1
Obtained by pixel permutation
SmallI1im
Next-door neighbors
You have chosen a pixel and you know its value.
What can you say about the value of its next-door neighbor?
Next-door neighbors
Next-door neighbors
Next-door neighbors
Random permutation of the
red channel of myFirstImage
Mutual Information
(a little bit more on information theory)
• The Mutual Information of two random variables is a
measure of the variables’ mutual dependence.
• The most common unit of measurement of mutual
information is the bit.
Mutual Information
(a little bit more on information theory)
• The Mutual Information of two random variables is a
measure of the variables’ mutual dependence.
XX ⇣ p(x, y) ⌘
I(X; Y ) = p(x, y) log
p(x)p(y)
y2Y x2X
p(x, y) is the joint probability function of X and Y .
p(x), p(y) are the marginal probability distribution
functions of X and Y (respectively).
Mini-assignment #1 (bonus)
• Read an image (any image)
• Present one of its RGB channels – I1
• Permute I1 and present it.
• Present the histogram of I1.
• Calculate its entropy
• Calculate the Mutual Information between I1 pixels and their
respective left-neighbors
• Calculate the Mutual Information between the permutation
image’s pixels and their respective left-neighbors
Is it enough?
Is it enough?
Is it enough?
not yet there
Next Class
Sampling
Quantization
Histogram Processing
