25-02-2025

Digital Image Processing:

Introduction

Introduction

“One picture is worth more than ten

thousand words”

1
 25-02-2025

• An image is a two-dimensional array in which

color information is arranged along x and y
spatial axis. So, in order to understand how
the image is formed, we should first
understand how the signal is formed?

Signal
• Analog Signal
– a continuous signal that represents a physical
quantity, such as voltage, current, temperature, or
sound

• Digital Signal
– a signal that represents data as a series of
discrete values, usually in binary form (0s and
1s).

2
 25-02-2025

Discrete Time Signal

• The signals, which are defined at discrete times are called as
discrete time signals.
• Example: if you took the temperature reading of a room every
day at the same time, the result would be a discrete-time
signal.
• These signals are represented by x[n] where n indicates the
independent variable in the time domain
• Specifically, x[n] represents the sample value of the signal at
the nth instance.

Image Processing
• Analog Image Processing:
– the images are manipulated by electrical means by
varying the electrical signal.
– The common example the television image.

• Digital Image Processing:

– applied to digital images (a matrix of small pixels and
elements).
– color processing, image recognition, video
processing, etc

3
 25-02-2025

Image Formation on a Camera

• How Human Eye Works?

Image Formation on Analog Camera

• the image formation is due to the chemical reaction that takes place on the strip that is
used for image formation.

4
 25-02-2025

Image Formation on Digital Camera

Charge-coupled device (CCD)

Image Formation on Digital Camera

• acquiring an image from a digital camera involves transfer of a

signal from one part of the system to the other.
• capturing an image from a camera is a physical process. The
sunlight is used as a source of energy. A sensor array is used for the
acquisition of the image.
• So, when the sunlight falls upon the object, then the amount of
light reflected by that object is sensed by the sensors, and a
continuous voltage signal is generated by the amount of sensed
data.
• In order to create a digital image , we need to convert this data into
a digital form. This involves sampling and quantization.
• The result of sampling and quantization results in a two
dimensional array or matrix of numbers which are nothing but a
digital image.

10

5
 25-02-2025

11

What is a Digital Image?

A digital image is a representation of a two-

dimensional image as a finite set of digital
values, called picture elements or pixels

12

6
 25-02-2025

What is a Digital Image? (cont…)

Pixel values typically represent gray levels,
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

colours, heights, opacities etc

Remember digitization implies that a digital
image is an approximation of a real scene

1 pixel

13

What is Digital Image Processing?

Digital image processing focuses on two major
tasks
– Improvement of pictorial information for human
interpretation
– Processing of image data for storage, transmission
and representation for autonomous machine
perception

14

7
 25-02-2025

What is a Digital Image? (cont…)

Common image formats include:
– 1 sample per point (B&W or Grayscale)
– 3 samples per point (Red, Green, and Blue)
– 4 samples per point (Red, Green, Blue, and “Alpha”,
i.e. For Opacity)

For most of this course we will focus on grey-scale

images
15

16

8
 25-02-2025

What is DIP? (cont…)

The continuum from image processing to
computer vision can be broken up into low-,
mid- and high-level processes
Low Level Process Mid Level Process High Level Process
Input: Image Input: Image Input: Attributes
Output: Image Output: Attributes Output: Understanding
Examples: Noise Examples: Object Examples: Scene
removal, image recognition, understanding,
sharpening segmentation autonomous navigation

17

Examples: Image Enhancement

One of the most common uses of DIP techniques:
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

improve quality, remove noise etc

18

9
 25-02-2025

Examples: The Hubble Telescope

•Launched in 1990 the Hubble
The Hubble Space Telescope

telescope can take images of

very distant objects
•However, an incorrect mirror
made many of Hubble’s
images useless
•Image processing
techniques were
used to fix this

19

Examples: Artistic Effects

Artistic effects are
used to make
images more
visually appealing,
to add special
effects and to make
composite images

20

10
 25-02-2025

Examples: Medicine
Take slice from MRI scan of canine heart,
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

and find boundaries between types of tissue

– Image with gray levels representing tissue
density
– Use a suitable filter to highlight edges

Original MRI Image Edge Detection Image

21

Examples: GIS
Geographic Information Systems
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

– Digital image processing techniques are used

extensively to manipulate satellite imagery
– Terrain classification
– Meteorology

22

11
 25-02-2025

Examples: Law Enforcement

Image processing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

techniques are used

extensively by law
enforcers
– Number plate
recognition for speed
cameras/automated
toll systems
– Fingerprint recognition
– Enhancement of
CCTV images
25

Examples: HCI
Try to make human computer
interfaces more natural
– Face recognition
– Gesture recognition

26

12
 25-02-2025

Key Stages in Digital Image Processing

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

27

Key Stages in Digital Image Processing:

Image Aquisition
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

28

13
 25-02-2025

Key Stages in Digital Image Processing:

Image Enhancement
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

29

Key Stages in Digital Image Processing:

Image Restoration
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

30

14
 25-02-2025

Key Stages in Digital Image Processing:

Morphological Processing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

31

Key Stages in Digital Image Processing:

Segmentation
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

32

15
 25-02-2025

Key Stages in Digital Image Processing:

Object Recognition
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

33

Key Stages in Digital Image Processing:

Representation & Description
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

34

16
 25-02-2025

Key Stages in Digital Image Processing:

Image Compression

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

35

Key Stages in Digital Image Processing:

Colour Image Processing

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression

36

17
 25-02-2025

Components of an Image
Processing System

37

Color Image Processing

• Color is a powerful descriptor
• Descriptors greatly simplifies object segmentation
and identification
• Color image processing is divided into two major
areas
– Full-color processing
• Images are acquired and processed in full color
• Example: Color TV
– Pseudo-color processing
• Images are by nature grayscale and
• Converted to color images for visualization purposes

38

18
 25-02-2025

Color Fundamentals

• Primary Colors: Red (R ), Green (G ), Blue (B )

• Defined as one that subtracts or absorbs a primary color of light and
reflects or transmits the other two.

39

Visible Light Spectrum

40

19
 25-02-2025

Color Fundamentals
Primary Colors
• Red(R), Green(G) and Blue(B)
• Can produce all visible colors with proportion mixing

Secondary colors (Primary colors can be mixed to generate)

• Magenta (red+blue)
• Cyan( green + blue) and
• Yellow (red+green)

Primary Colors is known as RGB

Secondary Colors is known as CMYK

41

Characteristics used to distinguish

between colors
• Brightness: intensity – chromatic notation
• Hue: determined by the dominant wavelength of the visible
spectrum.
• Saturation: the amount of white light mixed with a hue.

42

20
 25-02-2025

COLOR MODELS

• RGB

• HSV

• CMYK

43

ADDITIVE AND SUBTRACTIVE COLOR

MODELS
• Additive Color Model
– These type of models use light which is emitted
directly from a source to display colors.
– These models mixes different amount of RED,
GREEN, and BLUE (primary colors) light to produce
rest of the colors.
– Adding these three primary colors results in WHITE
image.
– Example: RGB model is used for digital displays
such as laptops, TVs, tablets, etc.

44

21
 25-02-2025

ADDITIVE AND SUBTRACTIVE

COLOR MODELS
• Subtractive Color Model
– These type of models use printing inks to display
colors.
– Subtractive color starts with an object that reflects light
and uses colorants to subtract portions of the white
light illuminating an object to produce other colors.
– If an object reflects all the white light back to the
viewer, it appears white, and if it absorbs all the light
then it appears black.
– Example: Graphic designers used the CMYK model
for printing purpose.

45

46

22
 25-02-2025

RGB Color Model

47

CMY Color Model

• The CMY color model is a subtractive color
model in which cyan, magenta, and yellow
(secondary colors) pigments or dyes are mixed
in different ways to produce a broad range of
colors

48

23
 25-02-2025

49

Convert RGB (98,35,146) to CMY

50

24
 25-02-2025

HSV/ HSI Color Model

• Hue, Saturation, Value / Hue, Saturation, Intensity
• The RGB and CMY color models are ideally suited for hardware
implementations
• When humans view a color object, its hue, saturation, and
brightness are described.
– Hue: It is a color attribute that describes a pure color.

– Saturation: It measures the extent to which a pure color is diluted

by white light.

– Brightness: It depends upon color intensity, which is a key factor in

describing the color sensation. The intensity is easily measurable,
and the results are also easily interpretable.

51

• RGB is great for color generation, but HSI

is great for color description.

52

25
 25-02-2025

Calculation of Hue, Saturation, and

Intensity

53

Calculation of Hue, Saturation, and

Intensity

54

26
 25-02-2025

55

56

27
 25-02-2025

57

Color to Greyscale conversion

• Average Method
Grayscale = (R + G + B ) / 3

• Disadvantage
– human eyeballs react differently to RGB
– Eyes are most sensitive to green light, less
sensitive to red light, and the least sensitive to
blue light.

58

28
 25-02-2025

Color to Greyscale conversion

• Weighted Method/ Luminosity Method

– weighs red, green, and blue according to their
wavelengths. The improved formula is as
follows:

Grayscale = 0.299R + 0.587G + 0.114B

59

Color to Greyscale conversion

• RGB to HSV
– Read an RGB image
– Represent RGB image in the range [0,1]
– Find the HSI components

60

29
 25-02-2025

Color to Greyscale conversion

61

Example of RGB to HSI

• Covert RGB (29, 104, 215) to HSI.

62

30
 25-02-2025

Introduction to Computer Vision

63

Computer Vision and Image

Processing
• Image processing is the process of creating a
new image from an existing image, typically
simplifying or enhancing the content in some
way.
• is not concerned with understanding the content
of an image.
• A given computer vision system may require
image processing to be applied to raw input, e.g.
pre-processing images.

64

31
 25-02-2025

Common tasks that computer vision

systems can be used for

• Object classification.
• For example, the system can find a dog among all
objects in the image.
• Object identification.
• For example, the system can find a specific dog
among the dogs in the image.
• Object tracking.
• The system processes video, finds the object (or
objects) that match search criteria and track its
movement.

65

The goal of computer vision

• “Enhance” images

66

32
 25-02-2025

The goal of computer vision

• Forensics

Source: Nayar and Nishino, “Eyes for Relighting”

68

Source: Nayar and Nishino, “Eyes for Relighting”

69

33
 25-02-2025

Source: Nayar and Nishino, “Eyes for Relighting”

70

The goal of computer vision

• Improve photos (“Computational Photography”)

Super-resolution (source: 2d3)

Low-light photography
(credit: Hasinoff et al., SIGGRAPH ASIA 2016)

Depth of field on cell phone camera Inpainting / image completion

(source: Google Research Blog) (image credit: Hays and Efros)

71

34
 25-02-2025

Why study computer vision?

• Billions of images/videos captured per day

• Huge number of potential applications

• The next slides show the current state of the art

72

Optical character recognition (OCR)

• If you have a scanner, it probably came with OCR software

Digit recognition, AT&T labs (1990’s) License plate readers

http://yann.lecun.com/exdb/lenet/ http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Automatic check processing

73

35
 25-02-2025

Face detection

• Nearly all cameras detect

faces in real time

74

Face analysis and recognition

75

36
 25-02-2025

Vision-based biometrics

Who is she? Source: S. Seitz

76

Vision-based biometrics

“How the Afghan Girl was Identified by Her Iris Patterns” Read the story

Source: S. Seitz

77

37
 25-02-2025

Login without a password

Fingerprint scanners on Face unlock on Apple iPhone X

many new smartphones See also http://www.sensiblevision.com/
and other devices

78

Bird identification

Merlin Bird ID (based on Cornell Tech technology!)

79

38
 25-02-2025

3D face tracking with consumer cameras

Snapchat Lenses

Face2Face system (Thies et al.)

82

Image synthesis

Zhu, et al., Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, ICCV 2017

83

39
 25-02-2025

Smart cars

• Mobileye
• Tesla Autopilot
• Safety features in many cars

84

Self-driving cars

Waymo

85

40
 25-02-2025

Medical imaging

3D imaging
(MRI, CT) Skin cancer classification with deep learning
https://cs.stanford.edu/people/esteva/nature/

86

CV+AR

87

41
 25-02-2025

Agriculture
• Crop monitoring
• Detect disease: Computer vision can identify
signs of disease, such as changes in leaf color,
shape, and texture.
• Detect pest infestations: Computer vision can
identify signs of pest infestations.
• Detect nutrient deficiencies: Computer vision
can identify signs of nutrient deficiencies.

88

Current state of the art

• You just saw many examples of current systems.

– Many of these are less than 5 years old

• Computer vision is an active research area, and rapidly

changing
– Many new apps in the next 5 years
– Deep learning powering many modern applications

• Many startups across a dizzying array of areas

– Deep learning, robotics, autonomous vehicles, medical imaging,
construction, inspection, VR/AR, …

89

42
 25-02-2025

Why is computer vision difficult?

Viewpoint variation

Credit: Flickr user michaelpaul

Credit: Flickr user michaelpaul

Scale
Illumination

90

Why is computer vision difficult?

Motion (Source: S. Lazebnik)

Intra-class variation

Background clutter Occlusion

91

43