Introduction To DSP 2
Introduction To DSP 2
Introduction to DSP 2
Summary
This document is an introduction to linear filters and dynamical systems. Filters have later a dedicated
lecture. Dynamical systems are an important tool that is used in control theory and systems modelling.
Instead of always going through rigor mathematical definitions and proofs, many things are introduced in
the practical level and with animations for a quick glance and grasping the basic idea.
Table of contents
1. Introduction..........................................................................................................................................2
2. Convolution and filtering.....................................................................................................................3
2.1. Filtering.......................................................................................................................................5
3. Dynamical systems.............................................................................................................................11
3.1. State space model......................................................................................................................15
1. Introduction
Filters are used to modify signals; usually, to remove unwanted components from it. Filters belong to the
body of knowledge as for signal processing. Dynamical systems are an even broader concept. Namely, we
will see that linear filters can be implemented as a linear dynamical model.
The basic filter theory deals with linear filters (see also Figure 1). A linear system H is such that for
signals x and y:
Figure 1. The output of a linear system is the same, no matter if the signals are scaled (by a and b) and
summed before or after the system.
Example 1:
Let H compute the average of two consecutive samples. Let a = 2 and b = 3. Let x = (1, 3, 5) and
y = (2, 2, 1). Test if the linearity equation holds for these (the edges of the signals are now omitted).
H ( x )= ( 1+32 , 3+52 )= ( 2, 4)
H ( y )= ( 2+22 , 2+12 )=( 2 , 1. 5)
aH(x) = (22, 24) = (4, 8)
The left and right-hand sides are equal. Averaging may obey linearity.
Linearity makes it possible to use more or less simple mathematics to analyze the properties of the filter.
Important mathematical tools are:
Z transform: transfer function, black-box model, of the filter; e.g., stability issues
Fourier transform: steady-state frequency response of the filter; which frequencies are
attenuated/amplified
More precisely, the filter h is such that computes a sliding weighted average of the input, and that the
weights of the average do not change over time. The output for digital signals is computed as follows:
N−1
y [ n ] =x [ n ]∗h [ n ] = ∑ x [ n−m ] h [ m ]
m=0 ,
where N is the number of weights (coefficient) in filter h. Here 0 stands for the first sample of the signal.
Example 2:
Let a temperature signal be x[n] = (1, 3, 1, 3, 1, 5, 1, 5, 1). Filter x by h[n] = (0.5, 0.5).
1
y [ 0 ] = ∑ x [ 0−m ] h [ m ] =x [ 0−0 ] h [ 0 ] +x [ 0−1 ] h [ 1 ] =
m=0
x [ 0 ] h [ 0 ] +x [−1 ] h [ 1 ]=1⋅0 .5+0⋅0 .5=0 . 5.
For n = 1 the output is:
1
y [ 1 ] = ∑ x [ 1−m ] h [ m ] =x [ 1−0 ] h [ 0 ] +x [ 1−1 ] h [ 1 ] =
m=0
x [ 1 ] h [ 0 ] +x [ 0 ] h [ 1 ]=3⋅0 .5+1⋅0 .5=2.
For n = 2 the output is:
1
y [ 2 ] = ∑ x [ 2−m ] h [ m ] =x [ 2 ] h [ 0 ] + x [ 1 ] h [ 1 ] =1⋅0. 5+3⋅0 .5=2.
m=0
1
y [ 3 ] = ∑ x [ 3−m ] h [ m ]=x [ 3 ] h [ 0 ] + x [ 2 ] h [ 1 ] =3⋅0 . 5+1⋅0 . 5=2 .
m=0
In Moodle, there is a Matlab script file that animates this filter. In order to display it start Matlab,
download and open the m file, highlight the code and press F9.
2.1. Filtering
Filters are used to remove noise from signals. Filters are also used to separate different frequencies from
the signal in order change their mutual volume, balance. The latter is probably familiar from music
players where the amount of bass and treble can be adjusted using an equalizer.
When we are dealing with filters that compute convolution, the eye is on frequencies. With sliding
weighted average one can realize such filters as low-pass, high-pass, band-pass, and band-stop. Moreover,
it is possible to computationally obtain a derivative of the signal, among others.
Let us test the filter from example 2 to a signal that has two sines at different frequencies ( f1 = 5 Hz and
f2 = 400 Hz). The effect of the filter is illustrated by computing the Fourier transform spectrum before and
after filtering.
Figure 3 shows the results. It can be concluded that the filter with h = (0.5, 0.5) attenuates the higher of
the two frequency components. So, the filter is a low-pass filter. The filtered signal looks less noisy,
because usually noise occurs at the high frequencies.
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
Figure 3. Left column: original signal (red) and the filtered signal (green).
Right column: spectra of the original (red) and the filtered signal (green).
N−1
y [ n ] =x [ n ]∗h [ n ] = ∑ x [ n−m ] h [ m ]
m=0 .
Figure 4 gives an FIR example with three coefficients. Each coefficient in this case is 1/3, which implies
that the output is the sliding average of every three input samples:
The output of the FIR in Figure 4 is computed given that the current input x[n] = 5, the previous input
x[n–1] = –2, and the input before that x[n–1] = 2:
Figure 4. An FIR filter with three coefficients, b = (1/3, 1/3, 1/3), and the computation of a single output.
FIR filters will be discussed later in detail. However, it is beneficial to check an introductory video now:
https://www.youtube.com/watch?
v=aV7cM9nmv0A&feature=iv&src_vid=mCzoxw4j4uA&annotation_id=annotation_103492
Example 3:
Let the image X with 5 5 pixels and the image filter mask/kernel H be, respectively:
0 0 0 1 0 H
0 2 4 4 0
1 3 4 3 0 –1 0 1
1 3 3 3 1 –1 0 1
1 0 0 0 0 –1 0 1
Next we can start computing the filtered image Y by sliding the filter over the image. The first position
when the whole filter fist inside the image is as follows (the bold pixels show the location of H):
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
X Y
Position of H in bold
0 0 0 1 0
0 2 4 4 0 7
1 3 4 3 0
1 3 3 3 1
1 0 0 0 0
Y(1, 1) = X * H =
In image processing, the result of convolution is usually inserted to the center position of the filter mask
(see above). Then the mask is shifted to the next position to obtain a new output pixel:
X Y
Position of H in bold
0 0 0 1 0
0 2 4 4 0 7 3
1 3 4 3 0
1 3 3 3 1
1 0 0 0 0
Y(1, 2) = X * H =
As a last example, the image filter is applied to a real-life image. The results in Figure 5 show that this
image filter picks up vertical edges in the image.
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
P Q
y (n )=∑ b i x( n−i)−∑ a j y (n− j )
i=0 j=1 ,
where n is discrete time, x(n–i) and y(n–j) refer to the previous inputs and outputs, respectively. Note that
i starts from 0 (i.e., the current input is also used) while j starts from 1 (i.e., the previous output is the first
term in the feedback sum). a0 is actually assumed to be 1.
2 2
y (n )=∑ b i x( n−i)−∑ a j y (n− j )=b 0 x( n)+b 1 x (n−1)+b 2 x (n−2 )−a 1 y (n−1 )−a 2 y (n−2 )
i=0 j=1
.
Figure 6. IIR filter with three feedforward coefficients bi and two feedback coefficients aj. (By Akilaa
(Own work) [GFDL (http://www.gnu.org/copyleft/fdl.html), CC BY-SA 3.0
(http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via
Wikimedia Commons) IIR filter is a recursive filter (https://en.wikipedia.org/wiki/Recursive_filter).
Example 4:
Let us say that the coefficients of an IIR filter are b = (1/3, 1/3, 1/3) and a = (1, –0.5, –0.55). Simulate the
output using the following Matlab code:
% Coefficients
b = [1/3, 1/3, 1/3];
a = [1, -0.5, -0.55];
% Unit impulse as the input
x = [1 zeros(1, 49)];
% Output of IIR
y = filter(b, a, x);
figure
subplot(2, 1, 1)
hold on
stem(x)
xlabel('n')
ylabel('input, x')
subplot(2, 1, 2)
hold on
stem(y)
xlabel('n')
ylabel('output, y')
Figure 7 shows the input (a unit impulse) and the output of this IIR filter. This is an example of an
unstable IIR filter.
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
Figure 7. An unstable IIR. The output to a unit impulse input diverges towards infinity.
3. Dynamical systems
Dynamical systems are used to describe time dependent behavior of signals and systems. The main
components of dynamical system models include the state vector of the system and an evolution rule that
tells what is the future state given the current state. The evolution rule can be deterministic or stochastic.
Moreover, it is important to define the initial state.
Dynamical systems can be used to model continuous systems, for instance, mechanical systems, quantum
mechanics, and control systems. Now we mainly restrict ourselves to discrete-time systems. It means that
state is changed only at discrete time instances. Moreover, often the state space is also discrete, and the
respective system is called a quantized-data systems.
Example 5:
Let us do a Matlab example using discrete-time system with two continuous state variables:
x1[k] = the balance of your bank account [euros], where k is discrete time (1, 2, 3, …), and
(1) After each time step k, your money increases due to interest and decreases due to some shopping. The
amount of shopping depends on your ‘shopping mood’.
(2) After each time step k, your ‘shopping mood’ increases if your balance is over 1000 euros and
decreases otherwise. Shopping mood is always between 0 and 100 points, inclusively.
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
(1)
x 1 [ k +1 ] =(1+ r ) x1 [ k ] −sx 2 [ k ] ,
where r is the interest rate multiplier and s is your shopping rate [euros/mood point].
where dm is your rate of change of the shopping mood, and restrict mood to [0, 100]:
When k = 2: you get 0.051001 euros = 50.05 euros as interests and you will do shopping by 250 euros
= 100 euros. Consequently, x1[2] = 1001 + 50.05 – 100 = 951.05 (euros). Your bank account had over
1000 euros, and therefore your mood is increased by 10 points to 60 points.
% Constants
r = 0.05;
s = 2;
dm = 10;
% Initialize vectors
N = 300;
x1 = zeros(N, 1);
x2 = zeros(N, 1);
% Initial conditions
x1(1) = 1001;
x2(1) = 50;
figure
plot(x1, x2)
ylim([0 100])
xlabel('balance [eur]')
ylabel('mood')
Using different initial values lead to different behavior. Figure 8 and Figure 10 show two examples with
different parameters but equal initial conditions. It can be seen that the range of the balance is almost the
same in both cases. However, it would be difficult to predict when there will be an especially low plunge.
It is typical to dynamical systems that there are some orbits like in Figure 9.
When the interest rate is increased, finally the earned interests exceed the spent money. Consequently, the
balance starts to increase infinitely. However, the system does not necessarily immediately show its
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
instability (Figure 11), because there is random-like variation, even though the system is totally
deterministic!
The canonical theory of dynamical systems assumes linear systems. In linear cases, the system equation
can be presented by matrices and thus analyzed by matrix algebra.
In the case of linear continuous-time systems, the dynamics of the system can be written by differential
functions.
Example 6:
According to Newton’s second law mass times acceleration is force, i.e., a force causes acceleration:
m ÿ ( t )=∑ F (t) .
Now three forces are present: the time-variant external force u(t), spring force ky(t), and viscous friction
force that is linearly dependent on the speed of the object b ẏ ( t ).
Be careful with the signs (directions) of the forces: Positive direction of y is set to the right as well as
positive external force. In that coordinate frame, if y is positive, spring force should be negative (i.e., to
the left). Similarly, if movement is to the right, viscous friction should cause a force to the left.
This is a second-order differential equation. Usually, dynamical systems are modelled using first order
differential equations. The trick is to use a many state variables as the order of the system is; now two. Let
x1(t) and x2(t) be the position and velocity of the object, respectively. Then the dynamic model can be re-
written as a first-order differential equation:
⇔
u ( t ) b x2 ( t ) k x1 ( t )
m ẋ2 ( t )=u ( t ) −b x 2 ( t )−k x 1 ( t )¿ m ẋ 2 ( t )= − −
m m m
This is actually the evolution rule for state variable x2(t). The respective rule for x1(t) is now simply:
ẋ 1 ( t )=x 2 ( t ) ,
These two evolution rules can be written as a set of equations (notice rearranging and added zero terms):
{
ẋ 1 ( t )=0 x 1 ( t )+ x 2 ( t ) +0 u ( t )
−k b 1
ẋ 2 ( t )= x 1 ( t )− x 2 ( t ) + u ( t )
m m m
[ ][ ][ ] [ ]
0 1 0
ẋ1 ( t ) x1 ( t )
= −k −b + 1 u(t) ,
ẋ2 ( t ) m m x 2 ( t ) m
where x1 and x2 are the state variables position and velocity, respectively.
The output of this system is the position y(t). The output is also presented by matrices:
y (t )= [ 1 0 ]
[ ]
x 1 (t )
x 2 (t )
+0 u(t )
.
{[ ][ ][ ]
ẋ 1 (t) x 1 (t) u1 (t)
ẋ 2 (t) x (t) u (t)
=A 2 +B 2
⋮ ⋮ ⋮
ẋ n (t) x n (t) u p (t)
[ ][ ][ ]
y 1 (t) x1 (t ) u 1(t)
y 2 (t) =C x2 (t ) + D u 2(t) .
⋮ ⋮ ⋮
y q (t ) x n (t) u p (t)
x (k +1 )= Ax(k )+Bu(k )
y (k +1 )=Cx (k )+Du(k ).
However, the A, B, C, and D matrices are not equivalent to the continuous-time case!
% Constants
k = 10;
b = 0.5;
m = 1;
% System matrix
A = [0 1; -k/m -b/m];
% Input matrix
B = [0;1/m];
% Output matrix
C = [1 0];
% Feedthrough matrix
D = 0;
% Noise coefficient matrix
K = [0;0];
% External force u:
samples = 300;
u = zeros(samples, 1);
u(1, 1) = 100;
u(2, 1) = 100;
u(40, 1) = -200;
u(41, 1) = -200;
u(100, 1) = 300;
u(101, 1) = 300;
u(140, 1) = -400;
u(141, 1) = -400;
figure
plot(data)
xlim([0 samples*data.ts])
figure
% Initial condition: position = 1, velocity = 0
sim(SSM, data, [1;0]);
% Save: y = output, ysd = output deviation, x = states
[y, ysd, x] = sim(SSM, data, [1;0]);
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
Figure 14. The output of the simulation, the position of the moving object.
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
Figure 15. The state vector trajectories of the moving object. In free ride, the orbit is a spiral. External
force causes the discontinuities.
Controllability means that it is possible – within finite time – to steer the states from any initial condition
to desired values. In the case of the moving object, it would possible to have such an external force
function that a desired location and velocity were reached.
test = zeros(size(A));
for column=1:size(A, 2)
test(:, column) = A^(column-1)*B;
end
disp(strcat('rank = ', num2str(rank(test)) ))
if rank(test) == size(A, 1)
disp('Is controllable')
else
disp('Is not controllable')
end
The output of the Matlab code shows that it is controllable. But how to obtain a desired state is not an
easy question in the general case.
A dual of controllability is observability. Observability implies the internal states can be inferred from the
output trajectory.
[ ]
C
CA
rank C A2 =n
⋮
n−1
CA
Example 6…
test = zeros(size(A));
for row=1:size(A, 2)
test(row, :) = C*A^(row-1);
end
disp(strcat('rank = ', num2str(rank(test)) ))
if rank(test) == size(A, 1)
disp('Is observable')
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
else
disp('Is not observable')
end
The output of the Matlab code shows that the moving object state model is observable.
Stability can be assessed from the eigenvalues of the state matrix A. Actually, those eigenvalues are the
poles of the transfer function.
poles = eig(A)
poles =
-0.2500 + 2.2220i
-0.2500 - 2.2220i
A continuous time-invariant system is stable if the real parts of the poles are negative. Now they are
(–0.25), and thus the moving object system is stable.
y (k )=b 0 u(k )+b1 u (k−1 )+b2 u (k−2 )−a 1 y (k−1 )−a 2 y (k −2) ,
where u(k) is now the input as x is reserved to the state variables. The order of this equation is N = 2. It
can be replaced by a set of N first-order state update equations:
{[ ] [ ][ ] [ ]
x1(k+1) −a1 −a2 x1(k) 1
=
x2(k+1) 1 0 x2(k) 0
+ u (k) ¿ ¿¿¿
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
https://ccrma.stanford.edu/~jos/fp/Converting_State_Space_Form_Hand.html.
Example 7:
Let the coefficients of an IIR filter be b = (1, 0.5, 0.25) and a = (1, –0.5, 0.1). What is the state space
model of this filter? Is the filter stable?
Outputs:
A = 0.5000 -0.1000
1.0000 0
B = 1
0
C = 1.0000 0.1500
D = 1
An IIR filter is stable if all its poles are inside the unit circle. Now the absolute values of the poles are <
1, i.e., they are inside the unite circle. The zero-pole plot shows this, too (Figure 17). We will later return
the zero-plots and other filter analysis.
© Janne Koljonen
University of Vaasa/School of Technology and Innovation Updated: February 3, 2020
ICAT3180 Sound Processing web course
Lecture 3: Introduction to DSP 2
Figure 17. The zero-pole plot of the IIR filter. o = zeros, = poles.