Spatial Filtering in intro image processingr

Spatial Filtering
CS474/674 - Prof. Bebis
Sections 3.4, 3.5, 3.6, 3.7, 3.8

Spatial Filtering Methods
output image

Spatial Filtering (cont’d)
• Spatial filters are defined by:
(1) Neighborhood (i.e., which pixels to process)
(2) Operation (i.e., how to process the pixels in the specified
neighborhood)
output image

Spatial Filtering – Neighborhood
• Usually, it has a square shape K x K (we call it a “window”).
– e.g., 3x3 or 5x5
center

Spatial filtering - Operation
output image
• A new value is obtained by processing the pixels in the window.
• Stored at the corresponding center location of the window
in the output image.
z’5 = 5z1 -3z2+z3-z4-2z5-3z6+z8-z9-9z7
Example:

Linear vs Non-Linear filters
• Depending on the operator used, a filter can be linear
or non-linear
output image
z’5 = 5z1 -3z2+z3-z4-2z5-3z6+z8-z9-9z7
Examples:
z’5 = max(z1,z2,z3,z4,z5,z6,z7,z8,z9)
linear
non-linear

Linear Operators
• Two common linear operators are:
– Correlation
– Convolution
• The output is a linear combination of the inputs.

Correlation (linear operator)
output image
• The output of correlation is a weighted sum of the input pixels.
The weights are defined by
a K x K mask (has the same
size as the window):

Correlation (cont’d)
Output
Image
w(i,j)
f(i,j)
/2 /2
/2 /2
( , ) ( , ) ( , ) ( , ) ( , )
K K
s K t K
g i j w i j f i j w s t f i s j t
 
    
 
g(i,j)
The output image is
generated by moving the
center of the mask at
each location of the input
image.
i,j=0,1,…,N-1

Handling Locations Close to Boundaries
Usually, we pad with zeroes
Alternatively, skip the first/last few rows/columns
0 0 0 ……………………….0
0
0
0
……………………….0

Correlation (cont’d)
Often used in applications where
we need to measure the similarity
between an image and a pattern
(e.g., template matching).
Simple template matching
does not work in most
practical cases.

Convolution (linear operator)
• Similar to correlation except that the mask is first flipped
both horizontally and vertically.
• Note: if w(i, j) is symmetric (i.e., w(i, j)=w(-i,-j)), then
convolution is equivalent to correlation!
/2 /2
/2 /2
( , ) ( , ) ( , ) ( , ) ( , )
K K
s K t K
g i j w i j f i j w s t f i s j t
 
    
 
i,j=0,1,…,N-1

Example
Correlation:
Convolution:

Filter Categories
• We will focus on two types of filters:
– Smoothing (also called “low-pass”) filters
• Remove details
– Sharpening (also called “high-pass”) filters
• Enhance details

Smoothing Filters (low-pass)
• Useful for reducing noise and removing small details.
– The elements of the mask must be positive.
– Mask elements sum to 1 assuming normalized weights (i.e.,
divide each weight by the sum of weights).

Smoothing filters – Example
smoothed image
input image

Sharpening Filters (high-pass)
• Useful for emphasizing fine details but can enhance noise.
– The elements of the mask contain both positive and negative
weights.
– Mask elements sum to 0.

Sharpening Filters - Example
• e.g., emphasize edges in an image

Sharpening Filters - Example
sharpened image
input image
Note: for better visualization, the
original image is typically added to
the sharpened image.

Common Smoothing Filters
• Averaging (linear)
• Gaussian (linear)
• Median filtering (non-linear)

Smoothing Filters: Averaging
The mask weights are all equal to 1

Smoothing Filters: Averaging (cont’d)
• Mask size determines degree of smoothing (i.e., loss of detail).
3x3 5x5 7x7
15x15 25x25
original

Smoothing Filters: Averaging (cont’d)
15 x 15 averaging After image thresholding
Example: extract largest, brightest objects

Smoothing filters: Gaussian
• The mask weights are the sampled values of a 2D Gaussian:

Smoothing filters: Gaussian (cont’d)
• Mask size depends on σ, e.g., it is usually chosen as
follows:

• In this case, σ controls the amount of smoothing
(since mask size depends on it)
σ = 3
σ = 1.4

Example

Averaging vs Gaussian Smoothing
Averaging
Gaussian

Smoothing Filters: Median Filtering
(non-linear)
• Very effective for removing “salt and pepper” noise (i.e.,
random occurrences of black and white pixels).
averaging
median
filtering

Smoothing Filters: Median Filtering (cont’d)
• Idea: replace each pixel by the median in a neighborhood
around the pixel.
• The size of the neighborhood controls the amount of
smoothing.

Sharpening Filters
• Common sharpening filters
– Unsharp masking
– High Boost filtering
– Gradient (1st derivative)
– Laplacian (2nd derivative)

Sharpening Filters: Unsharp Masking &
High Boost Filtering
• First, obtain a sharp image by subtracting a smoothed
image (i.e., low-passed (LP)) from the original image:
- =
Note: contrast enhancement
has been applied for
better visualization
( , ) ( , ) ( , )
mask LP
g x y f x y f x y
 
( , )
mask
g x y
( , )
LP
f x y
( , )
f x y

Sharpening Filters: Unsharp Masking &
High Boost Filtering (cont’d)
• Image sharpening emphasizes edges but other details are lost.
– Add a weighted portion of gmask back to the original image.
– When k=1, this process is known as high unsharp masking.
– When k>1, this process is known as high boost filtering.
+ k =
( , ) ( , ) ( , ), 0
mask
g x y f x y kg x y k
  
( , )
f x y ( , )
mask
g x y ( , )
g x y

High Boost Filtering - Example
original
Gaussian
smoothed
gmask
Unsharp
Masking
(k=1)
Highboost
Filtering
(k>1)
2D Example:

Sharpening Filters: Derivatives
• The derivative of an image results in a sharpened image.
• Image derivatives can be computed using the gradient:

Gradient
• The gradient is a vector which has magnitude and direction:

Gradient (cont’d)
• Gradient magnitude: provides information about edge
strength.
• Gradient direction: perpendicular to the direction of the
edge (useful for tracing object boundaries).

Gradient Computation
• Approximate partial derivatives using finite differences:
Δx
232 177 82 7
241 18 152 140
156 221 67 3
100 45 1 103
 
 
 
 
 
 
x
y
Notation:

Gradient Computation (cont’d)
sensitive to vertical edges
sensitive to horizontal edges
y2=y3+Dy, y3=y, x3=x, Dy=1

Example: visualize partial derivatives
f
x


f
y


• Image derivatives can be visualized as an image
by mapping the values to [0, 255]

Implement Gradient Using Masks
• We can implement and using masks:
(x+1/2,y)
(x,y+1/2)
*
*
good approximation
at (x+1/2,y)
good approximation
at (x,y+1/2)
Derivatives are not
computed at the same
location!

Implement Gradient Using Masks (cont’d)
• A different approximation of the gradient:
• We can implement and using the following masks:
*
(x+1/2,y+1/2)
good approximation
Derivatives are
computed at the
same location!

Implement Gradient Using Masks (cont’d)
• Other approximations
Sobel
Prewitt

Example: Visualize Gradient Magnitude
f
y


f
x


Gradient Magnitude
(isotropic, i.e., detects
edges in all directions)
• The gradient magnitude can be visualized
as an image by mapping the values to [0, 255]

Second Derivative – 1D case
( ) ( 1) ( )
( ) ( 1) ( ) ( 1) ( 1) 2 ( )
f x f x f x
f x f x f x f x f x f x
   
  
       

Second Derivative – 1D case (cont’d)
• Often, points that lie on an edge
can be detected by:
(1) Local maxima or
minima of the first derivative.
(2) Zero-crossings
of the second derivative (i.e., where
the second derivative changes sign). 2nd derivative
1st derivative

Second Derivative – 1D case (cont’d)
Example:

Second Derivative – 2D case: Laplacian
The Laplacian is defined as:
(dot product)
Approximate
2nd partial
derivatives:

(cont’d)
Laplacian Mask
Edges can
be found
by detecting
the zero-
crossings
input image output image
5
5
5
5 5
-5 -5
-10
-10
-10
10

(cont’d)
• Other realizations of the Laplacian mask found in practice.

Visualize the results of the Laplacian
• Add the results of the Laplacian to the original
image for better visualization
2
2
2
( , ) ( , ) ( , )
where,
( , ) is input image,
( , ) is sharpenend images,
-1 if ( , ) has a negative center value
and 1 if ( , ) has a positive center value.
g x y f x y c f x y
f x y
g x y
c f x y
c f x y
 
  
 
 
 

Visualize the results of the Laplacian
(cont’d)
no normalization,
negative values
clipped to zero
blurred image
scaled to [0, 255]
Example:

Laplacian vs Gradient
Laplacian Sobel
• Laplacian localizes edges better (zero-crossings).
• Higher order derivatives are typically more sensitive to noise.
• Laplacian is less computational expensive (i.e., one mask).
• Laplacian can provide edge magnitude information
but no information about edge direction.

Quiz #2
• When: October 2, 2023
• What: Arithmetic, Logical and Geometric Transformations,
Spatial Transformations

Spatial Filtering in intro image processingr

More Related Content

Similar to Spatial Filtering in intro image processingr

Recently uploaded

Spatial Filtering in intro image processingr