Chapter Three

Spatial Domain Image Processing

Computer Vision 3-Image Processing

3-1
 Reading

 Rich Szeliski, Computer Vision: Algorithms and Applications,

Chapter 3

 Some of the slides are adapted from Prof. Kristen Grauman

(UT-Austin)

Computer Vision 3-Image Processing

3-2
 Image processing

 Once an image has been formed and digitized, it can be

operated upon by various image processing operations.

 Image processing is a method to perform some operations

on an image, in order to get an enhanced image or to
extract some useful information from it.

Image to image transformation

It starts with an image and produces a modified

(enhanced) image

Computer Vision 3-Image Processing

3-3
 Image processing

 There are different classes of operations in image

processing. In this chapter we will see focus on
1) Point Operations

2) Neighborhood Operations
 Linear filtering / convolution operations

 Non-linear operations

Computer Vision 3-Image Processing

3-4
 Point operators
 The simplest kinds of image processing transforms
are point operators.
 Point operators:
 each output pixel’s value depends on only the
corresponding input pixel value.
 They cannot alter the spatial relationships of the image
 Two commonly used point processes are multiplication and
addition with a constant,
𝒈 𝒙, 𝒚 = 𝒂𝒇 𝒙, 𝒚 + 𝒃
 The parameters a and b are often called the gain and bias
parameters; sometimes these parameters are said to control
contrast and brightness, respectively.
 Point operators cannot accomplish tasks like smoothing.
Computer Vision 3-Image Processing
3-5
 Neighborhood Operations

 Generate an output pixel on the basis of the pixel at

the corresponding position in the input image and on
the basis of its neighboring pixel.

 The size of the neighborhood may vary :

3 * 3

5 * 5

 63 * 63 pixels

 Often referred to as Filtering Operations

Computer Vision 3-Image Processing

3-6
 Filtering

 Filtering is a fundamental signal processing operation, and

often a pre-processing operation before further processing.

 The term filtering refers to a process that forms a new image,

the pixel values of which are transformations of the original
pixel values.

 Applications:
 Enhance an image (denoise, resize, etc)

 Extract information (texture, edges, etc)

 Detect patterns (template matching)

Computer Vision 3-Image Processing

3-7
 Noise reduction

 Salt and pepper noise:

 random occurrences of black and white pixels

 Gaussian noise:
 variations in intensity drawn from a Gaussian normal
distribution

Computer Vision 3-Image Processing

3-8
 Matlab Salt and pepper noise

>> im=imread('peppers.png');
>> grayim= rgb2gray(im);
>> subplot(1,2,1), subimage(im);
>> out=imnoise(grayim, 'salt & pepper');
>> subplot(1,2,1), imshow(grayim);
>> title('Orginal', 'color', 'r' );
>> subplot(1,2,2), imshow(out);
>> title('Salt and Pepper noise’,
'color', 'r');

Computer Vision 3-Image Processing

3-9
 Salt and pepper noise
 Salt and pepper noise:

Computer Vision 3-Image Processing

3-10
 Gaussian noise

>> im=imread('peppers.png');
>> grayim= rgb2gray(im);
>> subplot(1,2,1), subimage(im);
>> out=imnoise(grayim, ‘Gaussian');
>> subplot(1,2,1), imshow(grayim);
>> title('Orginal', 'color', 'r' );
>> subplot(1,2,2), imshow(out);
>> title(‘Gaussian noise’, 'color', 'r');

Computer Vision 3-Image Processing

3-11
 Gaussian noise

Computer Vision 3-Image Processing

3-12
 Filters

 Filters are examples of systems, which are units that

convert an input function f(x,y) to an output (or response)
function g(x, y) where x and y are the independent
variables.

 When dealing with images, x and y are the representation

of a spatial position in the image.

 Image filtering is modifying the pixels in an image based

on some function of a local neighborhood of each pixel.

Computer Vision 3-Image Processing

3-13
 Linear filter

 The most commonly used type of neighborhood operator is a

linear filter, in which an output pixel’s value is determined as
a weighted sum of input pixel values.

 Linear filter:

Replace each pixel by a linear combination of its

neighbors.

Computer Vision 3-Image Processing

3-14
 Smoothing an image by averaging

 Replace each pixel by the average of its neighboring pixels .

 Assume a 3x3 neighborhood:

Computer Vision 3-Image Processing

3-15
 Smoothing an image by averaging

Computer Vision 3-Image Processing

3-16
 Smoothing an image by averaging

Computer Vision 3-Image Processing

3-17
 Smoothing an image by averaging

Computer Vision 3-Image Processing

3-18
 Smoothing an image by averaging

Computer Vision 3-Image Processing

3-19
 Moving average

Computer Vision 3-Image Processing

3-20
 Moving average

Computer Vision 3-Image Processing

3-21
 Moving average
 The pervious equation can be extended for average
window size (2k+1)x(2k+1) as:

𝟏
 is the uniform weight for each pixel.
𝟐𝒌+𝟏 𝟐
 The above equation can be generalized to allow non
uniform weights depending on neighboring pixel’s
relative position as:

Computer Vision 3-Image Processing

3-22
 Correlation filtering

 This operator is the correlation operator.

 The prescription for the linear combination is called

the kernel (mask or filter).

 The entries in the weight kernel H(u,v) are often called

the filter coefficients.
Computer Vision 3-Image Processing
3-23
 Averaging filter

 What values belong in the kernel H for the moving average

example?

0 0 0 0 0 0 0 0 0 0

0
0

0
0

0
0

90
0

90
0

90
0

90
0

90
0

0
0

0
1 1 1 0 10 20 30 30

0
0

0
0

0
90

90
90

90
90

90
90

90
90

90
0

0
0

0
1 1 1 ?
0 0 0 90 0 90 90 90 0 0
1 1 1
0 0 0 90 90 90 90 90 0 0

“box filter”
0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Computer Vision 3-Image Processing

3-24
 Matlab-Smoothing by averaging (Box filter)
>> im=imread('C:\Users\PC\Desktop\do.png');
>> img=rgb2gray(im);
>> h=fspecial('average')
>> hsize=[5,5];
>> h5=fspecial('average',hsize)
>> out3=imfilter(img,h);
>> out5=imfilter(img,h5);
>> subplot(2,2,1),subimage(img);
>> subplot(2,2,2),subimage(out3)
>> subplot(2,2,3),subimage(out5)

Computer Vision 3-Image Processing

3-25
 Matlab-Smoothing by averaging (Box filter)

Computer Vision 3-Image Processing

3-26
 Matlab-noise removing by box filter
>> ch=imread('C:\Users\PC\Desktop\cheker.png');
>> nn=imnoise(rgb2gray(ch), 'salt & pepper');
>> imshow(nn)
>> h=fspecial('average');
>> hsize=[5,5];
>> h5=fspecial('average',hsize);
>> ch3=imfilter(nn,h);
>> imshow(ch3)
>> ch5=imfilter(nn,h5);
>> imshow(ch5)

Computer Vision 3-Image Processing

3-27
 Box filter Application: removing noise

a) Original image
b) Corrupted with salt and pepper noise
c) Filtered with 3x3 box filter
d) Filtered with 5x5 box filter

Computer Vision 3-Image Processing

3-28
 Gaussian filter

 What if we want nearest neighboring pixels to have

the most influence on the output?
This kernel is an
approximation of a 2d
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Gaussian function:
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0 1 2 1
0 0 0 90 90 90 90 90 0 0
2 4 2
0 0 0 90 0 90 90 90 0 0
1 2 1
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0

Computer Vision 3-Image Processing

3-29
 Matlab-Smoothing by Gaussian filter
>> im=imread('C:\Users\PC\Desktop\do.png');
>> img=rgb2gray(im);
>> hsize=10;
>> sigma=5;
>> hG=fspecial('gaussian',hsize, sigma);
>> outG=imfilter(img,hG);
>> subplot(1,2,1),subimage(img);
>> subplot(1,2,2),subimage(outG);

Computer Vision 3-Image Processing

3-30
 Smoothing with a Gaussian

 Parameter σ is the width(spread) of the Gaussian

kernel, and controls the amount of smoothing.

Computer Vision 3-Image Processing

3-31
 Matlab-Smoothing by Gaussian filter
>> im=imread('C:\Users\PC\Desktop\do.png');
>> img=rgb2gray(im);
>> hsize=10;
>> sigma=1;
>> hG1=fspecial('gaussian',hsize, sigma);
>> sigma=5;
>> hG5=fspecial('gaussian',hsize, sigma);
>> sigma=10;
>> hG10=fspecial('gaussian',hsize, sigma);
>> outG1=imfilter(img,hG1);
>> outG5=imfilter(img,hG5);
>> outG10=imfilter(img,hG10);
>> subplot(2,2,1),subimage(img);
>> subplot(2,2,2),subimage(outG1);
>> subplot(2,2,3),subimage(outG5);
>> subplot(2,2,4),subimage(outG10);

Computer Vision 3-Image Processing

3-32
 Convolution

 Convolution:
 Flip the filter in both dimensions (bottom to top, right to left)
 Then apply cross-correlation

Notation for
convolution
operator

Computer Vision 3-Image Processing

3-33
 Convolution
 Note that the mask is first rotated by 180° since
 f(i-1, j-1) must be multiplied by h(1, 1)
 f(i-1, j) must be multiplied by h(1,0)
 – ……. ,
 – and f(i+1, j+1) must be multiplied by h(-1,-1)

F
H

Computer Vision 3-Image Processing

3-34
 Properties of convolution
 Superposition:
h * (f1 + f2) = (h * f1) + (h * f2)
 Commutative:
f*g=g*f
 Associative
(f * g) * h = f * (g * h)
 Distributes over addition
f * (g + h) = (f * g) + (f * h)
 Scalars factor out
kf * g = f * kg = k(f * g)
 Identity:
unit impulse e = […, 0, 0, 1, 0, 0, …]. f * e = f
 Differentiation property :

Computer Vision 3-Image Processing

3-35
 Boundary issues
 What is the size of the output?
 MATLAB: output size options
 shape = ‘full’: output size is sum of sizes of f and g
 shape = ‘same’: output size is same as f

full same
g g g g

f f

g g
g g
Computer Vision 3-Image Processing
3-36
 Boundary issues

 What about near the edge?

 the filter window falls off the edge of the image

 need to extrapolate
1
432
 methods:
1- clip filter (black)

2- wrap around

3- copy edge

4- reflect across edge

Computer Vision 3-Image Processing

3-37
 Matlab-Boundary issues

 methods (MATLAB):
 clip filter (black): imfilter(f, g, 0)

 wrap around: imfilter(f, g, ‘circular’)

 copy edge: imfilter(f, g, ‘replicate’)

 reflect across edge: imfilter(f, g, ‘symmetric’)

Computer Vision 3-Image Processing

3-38
 Median Filter

 Replace each pixel value with the median of the gray values in
the region of the pixel:
 Take a 3 x 3 (or 5 x 5 etc.) region centered around pixel (i,j)

 Sort the intensity values of the pixels in the region into ascending order

 Select the middle value as the new value of pixel (i,j)

 No new pixel values introduced

 Removes spikes: good for impulse, salt & pepper noise

 Non-linear filter

Computer Vision 3-Image Processing

3-39
 Median Filter

Computer Vision 3-Image Processing

3-40
 Matlab-Median filter

Salt and
Median
pepper
filtered
noise

Plots of a row of the image

Computer Vision
Matlab: output 3-Image
im =Processing
medfilt2(im, [h w]);
3-41
 Median filter

 Median filter is edge preserving

Computer Vision 3-Image Processing

3-42
 Application of filtering: Template matching

 Filters as templates: filters look like the effects they are

intended to find.

 Use normalized cross-correlation score to find a given pattern

(template) in the image.

 Normalization needed to control for relative brightness.

Computer Vision 3-Image Processing

3-43
 Matlab_Template matching
function [yindex xindex]=find_template(template,img)
c=normxcorr2(template,img);
[yRaw xRaw]=find(c==max(c(:)));
yindex=yRaw -size(template,1) ;
xindex=xRaw -size(template,2) ;
end
>>im=imread('C:\Users\PC\Desktop\do.png');
>>imm=rgb2gray(im);
>>t=imread('C:\Users\PC\Desktop\temp.png');
>>tt=rgb2gray(t);
>>[y x]=find_template(tt,imm);
>>disp([y x]); % displays the top left pixel where matching starts
>>[n,m]=size(tt);
>>NewTemplate=imm(y:y+n, x:x+m);
>>imshow(NewTemplate);
Computer Vision 3-Image Processing
3-44
 Matlab_Template matching
>>subplot(2,2,1); subimage(imm)
>> title('Orginal')
>> subplot(2,2,2); subimage(tt)
>> title('Template')
>> subplot(2,2,3); subimage(c)
>> title('Correlated')
>> subplot(2,2,4); subimage(NewTemplate)
>> title('Template Match')

Computer Vision 3-Image Processing

3-45