Detection and Estimation

Detection and Estimation in Digital Communications

Detection Theory:
Goal: Identify which symbol m (from M possible) was transmitted.
i

Input: Received signal corrupted by additive noise.

Processor: Uses decision rules to guess the transmitted symbol.
Application: Symbol-by-symbol decisions, e.g., decoding binary/quaternary
data.
Estimation Theory:
Goal: Extract physical parameters or waveform estimates from the received
noisy signal.
Focus: Continuous-valued parameters (e.g., phase, amplitude) or signals.
Tool: Often uses Wiener filter or maximum likelihood estimation (MLE).
Application: Channel parameter estimation, waveform reconstruction.
Common Aspects:
Both operate under additive noise.
Errors are inevitable → objective is to minimize and control them for
acceptable system performance.

Model of a Digital Communication System (Fig. 3.1)

Message Source:
Emits one symbol every T seconds.
Symbols belong to a set {m 1
.
, m2 , … , mM }

Examples:
Binary communication: Alphabet = {0, 1}.
Quaternary signaling: Alphabet = {00, 01, 10, 11}.
Assumption: All M symbols are equally likely, simplifying analysis and system
design.
A Priori Probability of Message Symbols

Definition:

Assumption:

Modulator:

Signal s i (t)

One signal s
≤ M
p i = P (m i emitted)

⇒ Uniform probability across all M symbols.

Vector Representation and Modulation

Vector Transmitter:

Maps vector s to a real-valued signal s

i
pi =

Converts symbol m into a vector of real numbers:

Dimension: N
i

si =

must have finite energy:

i (t)
Ei = ∫
0
M

T
1

⎢⎥
⎡

2
f or all i

s i1

s i2

s iN

s i (t) dt < ∞

is transmitted every T seconds.

Model of Digital Communication System (with AWGN Channel)

Channel Assumptions:
1. Linear with sufficient bandwidth ⇒ transmits s
2. Additive White Gaussian Noise (AWGN):
Additive: Noise W (t) adds to the signal.
White: Power is uniformly spread across all frequencies.
Gaussian: Noise samples are normally distributed.
⎤

i (t)
(3.1)

(3.2)

of duration T .

i (t)
(3.3)

without distortion.
 Zero-mean, stationary: E[W (t)] = 0, statistical properties are time-invariant.

Received Signal Model:

x(t) = s i (t) + W (t) (3.4)

s i (t) : Transmitted signal

W (t) : Sample of AWGN
x(t) : Received signal (sample of the random process X(t))
Receiver Structure:
1. Detector:
Operates on x(t) to produce an observation vector x.
2. Vector Receiver:
Uses x, prior knowledge of all s , and a priori probabilities p to estimate
i i

the transmitted symbol m

^.

Decision is statistical due to presence of noise.

Objective:
Minimize average probability of symbol error:

P e = P (m
^ ≠ mi )

A receiver achieving this is optimum in the minimum error probability sense.

Gram-Schmidt Orthogonalization Procedure (GSOP)

In digital communication, each message m is converted into a signal s

i .
i (t)

Using GSOP, we can represent each s i

(t) as a sum of basis functions, which
makes signal processing easier.

Signal Representation

Any real-valued energy signal s , defined over 0 ≤ t ≤ T , can be written as:

i (t)

s i (t) = ∑ s ij ϕ j (t) (3.6)

j=1

ϕ j (t) : orthonormal basis functions

s ij : scalar coefficients that tell us how much of each basis function is in s i (t)

N ≤ M: number of required basis functions is less than or equal to the

number of signals
How to Calculate the Coefficients

Each coefficient s is computed using:

ij

s ij = ∫ s i (t) ϕ j (t) dt (3.7)

0

This gives the projection of s (t) onto ϕ i j (t)

Properties of the Basis Functions

The basis functions ϕ j (t) satisfy:
T
1 if i = j
∫ ϕ i (t) ϕ j (t) dt = { (3.8)
0 if i ≠ j
0

They are orthonormal, meaning:

Orthogonal: they are uncorrelated (no overlap)
Normal: each has unit energy
Modulation (Fig. 3.3a)
To generate s , use:
i (t)

A bank of multipliers (each multiplies s with ϕ ij j (t))

A summer to add all products
This is how the modulator in Fig. 3.1 works.

Demodulation (Fig. 3.3b)

To recover s from s
ij , use:
i (t)

A bank of correlators (integrators that compute each s using Eq. 3.7) ij

 This is part of the receiver.

Why Use GSOP?

It converts waveforms s i (t) into vectors s i = [s i1 , . . . , s iN ]

Makes detection easier (e.g., minimum distance detection)

Used to build optimal receivers

Proof of the Gram-Schmidt Orthogonalization Procedure

The proof has two main stages:
Stage 1: Linear Independence

We start with M signals:

s 1 (t), s 2 (t), … , s M (t)

First, we check for linear independence. If the signals are not independent, then
there exists a set of constants a , a , . . . , a , not all zero, such that:
1 2 M

a 1 s 1 (t) + a 2 s 2 (t) + ⋯ + a M s M (t) = 0 f or 0 ≤ t ≤ T (3.9)

Suppose a M ≠ 0 , then we can isolate s M (t) as:

a1 a2 a M −1
s M (t) = − s 1 (t) − s 2 (t) − ⋯ − s M −1 (t) (3.10)
aM aM aM

⇒ This shows s M
(t) is a linear combination of the others.
We continue checking the remaining signals recursively:

b 1 s 1 (t) + b 2 s 2 (t) + ⋯ + b M −1 s M −1 (t) = 0 (if not independent) (3.11)

M −2
bi
s M −1 (t) = − ∑ s i (t) (3.12)
b M −1
i=1

Eventually, we find a set of N signals (where N ≤ M ) that are linearly

independent:
 s 1 (t), s 2 (t), … , s N (t)

All original signals can now be written as combinations of these N signals.

Stage 2: Constructing Orthonormal Basis Functions
We now build N orthonormal basis functions ϕ (t), ϕ 1 2
(t), . . . , ϕ N (t) from the linearly
independent signals.
Step 1: Define the first basis function
T
s 1 (t)
2
ϕ 1 (t) = where E 1 = ∫ s 1 (t) dt
√E 1 0

This means:

s 1 (t) = √ E 1 ⋅ ϕ 1 (t), with coef f icient s 11 = √ E 1

Step 2: Construct the second basis function

First, compute the projection of s 2 (t) onto ϕ 1 (t) :
T

s 21 = ∫ s 2 (t)ϕ 1 (t) dt (3.14)

0

Remove this projection from s 2 (t) to form an orthogonal component:

g 2 (t) = s 2 (t) − s 21 ϕ 1 (t) (3.16)

Normalize g 2
(t) to get ϕ 2
(t) :
T
g 2 (t)
2
ϕ 2 (t) = where E 2 = ∫ g 2 (t) dt (3.17)
√E 2 0

Substitute to write:

s 2 (t) − s 21 ϕ 1 (t)
ϕ 2 (t) = (3.18)
√E 2

⇒ϕ 1 (t), ϕ 2 (t) are now orthonormal:

T T
2
∫ ϕ 1 (t)ϕ 2 (t) dt = 0, ∫ ϕ 2 (t) dt = 1
0 0

Step 3: General Case (Recursive Step)

To get ϕ (t) from s (t), subtract all projections onto previous ϕ
i i j
:
(t)

i−1

g i (t) = s i (t) − ∑ s ij ϕ j (t) (3.19)

j=1

Where:
T

s ij = ∫ s i (t)ϕ j (t) dt (3.20)

0

Then normalize:
 T
g i (t) 2
ϕ i (t) = with E i = ∫ g i (t) dt (3.21)
√E i 0

Final Result
Now we have a set of orthonormal basis functions ϕ 1 (t), .
. . . , ϕ N (t)

Every original signal s i (t) can now be expressed as:

N

s i (t) = ∑ s ij ϕ j (t) (3.6 revisited)

j=1

Remarks

1. Unlike Fourier series, the basis functions ϕ j

(t) are not pre-defined — they are
built from the signal set itself.
2. The expansion is exact — not an approximation — and uses exactly N terms.

Example

Given signals: s 1 (t), s 2 (t), s 3 (t), s 4 (t) (from Fig. 3.4a)

Goal: Use Gram-Schmidt to find an orthonormal basis from this subset.

Observation: s 4 (t) = s 1 (t) + s 3 (t) → signals are not linearly independent.

To proceed, select the linearly independent subset: s 1
(t), s 2 (t), s 3 (t).
Step 1:
Calculate the energy of s 1
:
(t)

7
2
E1 = ∫ s 1 (t)dt =
3
From Eq. 3.13 (Gram-Schmidt basis function formula), the first orthonormal basis
function is:

s 1 (t) 3
φ 1 (t) = = √ s 1 (t)
√E 1 7

3 7
√ s 1 (t) if 0 < t <
φ 1 (t) = { 7 3

φ 1 (t) = 0 elsewhere

Step 2: Constructing the Second Basis Function ϕ 2 (t)

Coefficient s : 21

T T /3
3 T
s 21 = ∫ s 2 (t)ϕ 1 (t) dt = ∫ (1) (√ )dt = √
0 0
T 3

Energy of s 2 (t) :
T 2T /3
2 2
2T
E2 = ∫ s 2 (t) dt = ∫ (1) dt =
0 0
3

Second Basis Function ϕ 2 (t) :

3 T 2T
s 2 (t) − s 21 ϕ 1 (t) √ , ≤ t ≤
ϕ 2 (t) = = { T 3 3

√E 2 − s 2 0, elsewhere
21

Step 3: Constructing the Third Basis Function ϕ 3

(t)

Coefficient s : 31

s 31 = ∫ s 3 (t)ϕ 1 (t) dt = 0
0

Coefficient s : 32

T 2T /3
3 T
s 32 = ∫ s 3 (t)ϕ 2 (t) dt = ∫ (1) (√ )dt = √
0 T /3
T 3

Intermediate Function g 3 (t) :

2T
1, ≤ t ≤ T
3
g 3 (t) = s 3 (t) − s 31 ϕ 1 (t) − s 32 ϕ 2 (t) = {
0, elsewhere

Third Basis Function ϕ 3

(t) :

3 2T
g 3 (t) √ , ≤ t ≤ T
ϕ 3 (t) = = { T 3 $$ ∗ ∗Conclusion : ∗∗
T
√∫ 2 0, elsewhere
g (t) dt
0 3

The orthonormal basis functions are:

: non-zero on [0, T /3]
ϕ 1 (t)
 ϕ 2 (t) : non-zero on [T /3, 2T /3]
ϕ 3 (t) : non-zero on [2T /3, T ]
Each original signal s (t), s (t), s (t), s (t) can now be expressed as a linear
1 2 3 4

combination of these three basis functions.

Geometric Interpretation of Signals

Signal Vector Representation

Signal in 2D:
S 11
S 1 (t) → S 1 = [ ]
S 12

2 2
∥S 1 ∥ = √ S + S
11 12

Generalized to N-Dimensions:
S 11
⎡ ⎤
S 12
S1 = ,
⋮

⎣ ⎦
S 1N
1/2
N 2
∥S 1 ∥ = (∑ S )
j=1 1j

⮕ Each signal becomes a vector in orthonormal signal space.

Basis: ϕ 1 (t), ϕ 2 (t)

Signals:
N N

S 1 (t) = ∑ S 1j ϕ j (t), S 2 (t) = ∑ S 2j ϕ j (t)

j=1 j=1
 Coordinates in 2D: (S 11
, S 12 ) and (S 21
, S 22 )

⮕ Signals are points in N-D space. Axes = basis functions.

Signal:
N
S i (t) = ∑ j=1 S ij ϕ j (t)

Energy:
2
T T N
2
Ei = ∫ S i (t)dt = ∫ (∑ S ij ϕ j (t)) dt
0 0
j=1

N N T

= ∑ ∑ S ij S ik ∫ ϕ j (t)ϕ k (t)dt
0
j=1 k=1

Since {ϕ j (t)} are orthonormal:

T N
1 j = k 2
∫ ϕ j (t)ϕ k (t)dt = { ⇒ E i = ∑ S ij
0 j ≠ k
0
j=1

⮕ Energy = sum of squared components.

Norm:
1/2
N 2
∥S i ∥ = (∑ S )
j=1 ij

Energy:
2 T
E i = ∥S i ∥ = S Si
i

Matrix form:

S i1
⎡ ⎤
N
S i2
T 2
S i S i = [S i1 S i2 … S iN ] = ∑ S ij
⋮ j=1

⎣ ⎦
S iN

⮕ Energy is dot product of signal vector with itself.

Inner Product of Two Signals

Signals:
T T
S i = [S i1 , … , S iN ] , S k = [S k1 , … , S kN ]

Inner product:

T N
T
∫ S i (t)S k (t)dt = S i S k = ∑ S ij S kj
0
j=1
 ⮕ This measures similarity (projection) between signals.

Geometric Distance Between Signals

Signal vectors as points:
S 1 = (x 1 , y 1 ), S 2 = (x 2 , y 2 )

Distance in 2D:
2 2
d = √ (x 1 − x 2 ) + (y 1 − y 2 )

Generalized to N-D:
N

2 2
d = ∑(S ij − S kj )

j=1

⮕ Distance = Euclidean norm of difference vector ∥S i − Sk ∥

Final Distance Derivation from Signals

Distance in time-domain:

T N

2 2 2 2
∫ [S i (t) − S k (t)] dt = ∑(S ij − S kj ) = ∥S i − S k ∥ = d
0
j=1

⮕ Time-domain signal difference maps directly to geometric distance.

Detection of Known Signals in Noise

Digital Communication in the Presence of Noise

In a digital communication system, in each time slot of duration T seconds, one out of
M possible signals, s (t), s (t), … , s (t), is transmitted. These signals are assumed to
1 2 M

be transmitted with equal probability, i.e., 1/M .

Received Signal Model

When these signals are transmitted through an Additive White Gaussian Noise
(AWGN) channel, the received signal, x(t), is a sample function of a random process
X(t) . It can be described as:

x(t) = s i (t) + w(t) f or i = 1, 2, … , M (Eq. 3.51)

Here:

s i (t) is the transmitted signal corresponding to the symbol m . i

w(t) is a sample function of the white Gaussian noise process W (t), which is
assumed to have:
Zero mean
Power spectral density N 0 /2

The primary task of the receiver is to observe the received signal x(t) and make
the "best estimate" of which signal s (t) (and thus which symbol m ) was originally
i i

transmitted.

Signal, Noise, and Observation Vectors in Signal Space

1. Transmitted Signal Point (Message Point):

When a transmitted signal s i (t) is passed through a bank of N correlators (each
correlating with one of the N orthonormal basis functions), the outputs define the
signal vector s (as per Eq. 3.7). Since knowing s is equivalent to knowing s (t),
i i i

s i (t) can be represented as a point in an N -dimensional Euclidean space. This

point is called the transmitted signal point or message point.
2. Signal Constellation:
The entire collection of all M possible message points {s , s , … , s
1 2 M
} in this N -
dimensional Euclidean space is known as a signal constellation.
3. Received Signal Point (Observation Vector):
The presence of additive noise w(t) complicates the representation of the received
signal x(t). When x(t) is fed into the same bank of N correlators, the outputs
define a new vector, x, called the observation vector or received vector. This
vector x differs from the original signal vector s due to the presence of a noise
i

vector w:

x = si + w f or i = 1, 2, … , M (Eq. 3.52)

Here, x and w are sample values of random vectors X and W, respectively. The
orientation of the noise vector w is completely random.

The observation vector x (representing the received signal x(t)) can also be
represented as a received signal point in the same Euclidean space as the
transmitted signals.

Receiver's Decision Problem

Given the observed vector x, the receiver must perform a mapping from x to an
estimate m
^ of the transmitted symbol m . The goal is to perform this mapping in a way
i
that minimizes the average probability of symbol error in the decision.

The maximum-likelihood detector is a solution designed to address this fundamental

problem.

Maximum-Likelihood Detector
Decision Rule Goal
Given an observation vector x, choose the message m that was most likely i

transmitted.

1. Error Probability
Average probability of symbol error if decision M = mi is made:

P e (m i , x) = 1 − P (m i sent |x) (3.53)

To minimize this error, the optimal decision is:

Decide M = m i if P (m i sent |x) >= P (m k sent |x) f or all k ≠ i (3.54)

2. MAP Rule (via Bayes' Rule)

Using Bayes’ theorem:

p i f X|m i (x|m i )
P (m i sent |x) = (3.55)
f X (x)

So the MAP (Maximum A Posteriori) rule becomes:

p k f x (x|m k )
Decide M = m i if is maximum f or all k = i (3.55)
f x (x)

3. Maximum Likelihood (ML) Rule

If all messages are equally likely, p k = pi =
1

M
, then:

Decide M = m i if f (x|m i ) is maximum f or k = i (3.56)

Use log-likelihood for practical computation:

Decide M = m i if ln f (x|m i ) is maximum f or k = i (3.57)

4. Decision Regions
Observation space Z is partitioned into regions Z 1, Z2 , . . . , ZM , where:

x ∈ Z i if ln f x (x|m k ) is maximum f or k = i (3.58)

Maximum-Likelihood (ML) Decision Rule for AWGN Channels

In an AWGN channel, the ML detector chooses the transmitted signal s that is i

closest to the received vector x in Euclidean distance.

Decision Rule Formulations:
The ML detector selects region Z if: i
 N
1 2
− ∑(x j − s kj ) is maximum f or k = i (3.61)
N0
j=1

Since minimizing a squared distance is equivalent to maximizing its negative,

this simplifies to:
N
2
∑(x j − s kj ) is minimum f or k = i (3.62)

j=1

Recognizing this sum as the squared Euclidean distance:

N

2 2
∑(x j − s kj ) = ∥x − s k ∥ (3.63)

j=1

Therefore, the decision rule becomes:

x ∈ Z i if ∥x − s k ∥ is minimum f or k = i (3.64)

Interpretation:

The ML rule selects the nearest signal vector in R . N

The observation space is divided into Voronoi regions, where each region
contains all points closest to one s . i

Probability of Error in ML Detection

1. Setup

The observation space Z ⊆ R is partitioned into M decision regions:

N

{Z 1 , Z 2 , … , Z M }

If symbol m (signal s ) is transmitted, but the received vector x ∉ Z , then an error

i i i

occurs.
2. Average Symbol Error Probability
 M

P e = ∑ P (m i sent) ⋅ P (x ∉ Z i ∣ m i sent) (3.68)

i=1

With equal probability for all M symbols:

1
P (m i sent) =
M

So:
M M
1 1
Pe = ∑ P (x ∉ Z i ∣ m i sent) = 1 − ∑ P (x ∈ Z i ∣ m i sent)
M M
i=1 i=1

3. Likelihood Form
The conditional probability that x ∈ Z given m was sent is:
i i

P (x ∈ Z i ∣ m i sent) = ∫ f (x ∣ m i ) dx
Zi

where f (x ∣ m ) is the likelihood function.

i

Hence, the error probability becomes:

M
1
Pe = 1 − ∑∫ f (x ∣ m i ) dx (3.69)
M Zi
i=1

Interpretation
Correct decision probability for m : integral of the likelihood over the correct
i

region Z .
i

Error probability: 1 minus the average correct decision probability.

In higher dimensions (N-dimensional), the integral becomes over an N -
dimensional region Z ⊂ R .
i
N
Correlation Receiver
For an AWGN channel with equally likely signals s 1 (t), s 2 (t), … , s M (t) , the optimum
receiver consists of two main subsystems:

1. Detector: Correlator Bank (Fig. 3.10a)

Comprises M product-integrators (correlators).

Uses locally generated orthonormal basis functions:

ϕ 1 (t), ϕ 2 (t), … , ϕ N (t)

Operates on received signal x(t), 0 ≤ t ≤ T , to produce the observation vector:

T

x = [x 1 , x 2 , … , x N ], where x j = ∫ x(t)ϕ j (t) dt

0

2. Vector Receiver: Maximum-Likelihood Detector (Fig. 3.10b)

The vector receiver acts as a maximum-likelihood detector. Its primary goal is to
estimate the transmitted symbol (M) from the received observation vector (x) in a way
that minimizes the average probability of symbol error.

Each signal vector s k = [s k1 , s k2 , … , s kN ]

Computes inner products:

N

⟨x, s k ⟩ = ∑ x j s kj

j=1

Applies energy correction (from Eq. 3.66):

 N N
1 2
Metric k = ∑ x j s kj − Ek , E k = ∑ s kj
2
j=1 j=1

Selects the k for which the metric is maximum, and decides M = mk

Conclusion:
This structure is called a correlation receiver.
It is optimum in the maximum-likelihood sense for AWGN channels.
Minimizes average probability of symbol error.

Matched Filter and Correlator Equivalence

Given orthonormal basis functions ϕ 1
(t), ϕ 2 (t), … , ϕ N (t) , each zero outside
0 ≤ t ≤ T .
This allows avoiding analog multipliers, which are difficult to build.

Linear Filter Approach

Let h j
(t) be the impulse response of a linear filter.
If input is received signal x(t), output is:
∞

y j (t) = ∫ x(τ )h j (t − τ )dτ (3.76)

−∞

Matched Filter Definition

Set impulse response to:

h j (t) = ϕ j (T − t) (3.77)

Then, output becomes:

∞

y j (t) = ∫ x(τ )ϕ j (T − t + τ )dτ (3.78)

−∞

Sampling at t = T
Sample filter output at t = T :
∞

y j (T ) = ∫ x(τ )ϕ j (τ )dτ (3.79)

−∞

Since ϕ j (t) is zero outside [0, T ], it simplifies to:

T

y j (T ) = ∫ x(τ )ϕ j (τ )dτ (3.80)

0

Hence, y j (T ) = xj , the jth correlator output from Fig. 3.10a.

Matched Filter Receiver

The filter with impulse response ϕ j (T − t) is matched to ϕ j (t).

 The corresponding system is called a matched filter receiver (Fig. 3.11).

Causality of Matched Filter

For real-time implementation, matched filter must be causal:

Impulse response h(t) must be 0 for t < 0
Since ϕ(t) is defined only on 0 < t < T , this causality condition is satisfied.

Estimation Problem Setup

Given a received signal:

x(t) = s(t, a) + w(t), 0 ≤ t ≤ T (3.110)

s(t, a): signal with unknown parameter a (e.g., amplitude or phase)

w(t) : white Gaussian noise, zero mean, power spectral density N 0 /2

Goal: Estimate unknown parameter a from noisy observation x(t)

Estimate: Assigned value of a
Estimator: Algorithm that produces the estimate
Depends on optimality criterion

1. Minimum Mean-Square Estimate (MMSE)

Estimation error: ϵ = a − a
^

Use a quadratic cost function: C = (a − a

^)
2

Average cost:

2
E = ∫ (a − a
^) f A|X (a|x) da

: posterior PDF of a given observation vector x

f A|X (a|x)

MMSE estimate: Value of a

^ that minimizes E

Also called Bayes' estimate

2. Maximum Likelihood Estimate (ML)
Choose a
^ that maximizes likelihood of observation x:

a
^ M L = arg max f X|A (x|a)
a

: Likelihood function of a
f X|A (x|a)

ML estimate = MAP estimate if a is uniformly distributed

Popular due to strong statistical properties

2. Maximum Likelihood (ML) Estimation

The Maximum Likelihood (ML) estimate is a method to estimate an unknown

parameter α from an observed data vector x. It works by:
 1. Formulating the Likelihood Function: This is the conditional probability density
function f X
, treated as a function of α for the given observation x.
(x|α)

2. Maximization: The ML estimate α

^ ML is the value of α that maximizes this
likelihood function, effectively finding the parameter value that makes the
observed data most probable.
It is equivalent to the Maximum A Posteriori (MAP) estimate when the parameter's
prior distribution is uniform.

3. Minimum Mean-Square Estimate (MMSE) / Bayes' Minimum Mean-

Square Estimate

The Minimum Mean-Square Estimate (MMSE), also known as the Bayes' MMSE, aims
to estimate an unknown parameter α by minimizing the average squared error of the
estimate α
^.

1. Cost Function: The average cost C is defined as the expected value of the
1

squared error (α − α)
^ , conditioned on the observed data x:
2

2
^
C 1 = ∫ (α − α) f A (α|x)dα

where f A
(α|x) is the a posteriori PDF of α given x.
2. Minimization: The MMSE estimate is the α
^ that minimizes this average squared

cost. This estimate is the conditional mean of α given x.

Wiener Filter for Waveform Estimation

The Wiener filter is a linear filter designed to estimate a desired signal D from a noisy
n

observation X by minimizing the mean squared error (MSE) between the desired
n

signal and the filter output.

1. Input Samples:

Discrete samples from the noisy process:

X n , X n−1 , … , X n−M
2. Filter Structure:

A tapped delay line filter with:

Delay elements (each delays by T seconds),

Multipliers (scale inputs by h ), k

Adders (sum weighted inputs).

3. Filter Output:

The filter output is a linear combination of inputs:

M

Y n = ∑ h k X n−k

k=0

4. Estimation Error:

The difference between desired signal and output:

εn = Dn − Yn

5. Mean Squared Error (MSE):

Objective is to minimize:
2
ζ = E[ε n ]

Expanding the expression:

2 2
ζ = E[D n ] − 2E[D n Y n ] + E[Y n ]

Substituting Y :
n

M M M

2
ζ = σ D − 2 ∑ h k R DX (k) + ∑ ∑ h k h m R X (m − k)

k=0 k=0 m=0

Where:
 σ
2
D
2
= E[D n ] : Power of desired signal
R DX (k) = E[D n X n−k ] : Cross-correlation
R X (m − k) = E[X n−k X n−m ] : Autocorrelation

6. Optimality Condition:

Set gradient to zero for minimization:

M
∂ζ
= −2R DX (k) + 2 ∑ h m R X (m − k) = 0
∂h k
m=0

7. Wiener-Hopf Equations:

Linear equations to solve for optimal coefficients h : k

∑ h m R X (m − k) = R DX (k), k = 0, 1, … , M

m=0

8. Minimum MSE:

After substituting optimal h values: k

M
2
ζ min = σ D − ∑ h k R DX (k)

k=0

Questions
(b) Explain Wiener filter for waveform estimation. (4)
3(a) Draw block diagrams of the sub-systems of a matched filter receiver. (2)
(b) Derive the discrete-time version of Wiener-Hopf equations for a transversal Wiener
filter. (4)
(c) Write down the steps involved in the application of a Least-Mean-Square (LMS)
algorithm. (2.75)

(c) Describe mathematically the Maximum-likelihood detector.

3. (a) What are the criteria for estimation in digital communication? Discuss briefly.
(b) Draw and explain prediction-error process.
(c) Write down the characteristics of adaptive filter.
4. (a) Why are detection and estimation considered fundamental issues in
determining the optimum design of a receiver?
State and prove the Gram-Schmidt orthogonalization procedure.
How can you define a matched filter receiver? Write down the properties of a
matched filter.
5. (a) What is adaptive filter? Write down the characteristics of adaptive filter. 3
(b) With suitable illustration, explain a classical model for the speech production
 process. 5
(c) Define matched filter receiver. Describe the detector part of matched filter
receiver with necessary figure(s). 3 2/3

Q3.
a) Why are detection and estimation considered fundamental issues in determining the
optimum design of a receiver? (2%)
b) How can you characterize a matched filter? Write down the properties of matched
filters. (3%)
c) Describe the subsystems of a typical optimum digital communication receiver with
suitable block diagrams. (4%)
d) What do you mean by the average probability of symbol error? (2%)

Answers
(c) Write down the steps involved in the application of a Least-Mean-Square (LMS)
algorithm.

The Least-Mean-Square (LMS) algorithm is an adaptive filtering algorithm used to

minimize the mean squared error between the desired signal and the filter output. The
steps involved in the application of the LMS algorithm are as follows:

1. Initialization:
Initialize the filter coefficients h to zero or small random values.
k

Choose the step size μ (a small positive constant).

2. Filter Output Calculation:
Compute the filter output Y as a linear combination of the input samples:
n

Y n = ∑ h k X n−k

k=0

3. Error Calculation:
Calculate the estimation error ε as the difference between the desired signal
n

Dn and the filter output Y :

n

εn = Dn − Yn

4. Coefficient Update:
Update the filter coefficients using the LMS update rule:

h k (n + 1) = h k (n) + με n X n−k , k = 0, 1, … , M

Where μ is the step size controlling the rate of adaptation.

5. Iteration:
Repeat steps 2-4 for each new set of input samples to adaptively update the
filter coefficients.
(a) Adaptive Filter (Short Answer):

An adaptive filter is a filter whose parameters adjust automatically based on the input
signal to minimize an error criterion, such as mean square error.

Characteristics of Adaptive Filter:

Self-learning: Updates coefficients in real-time.

Error-driven: Minimizes error between desired and actual output.
Handles time-varying signals.
Uses algorithms like LMS, RLS for adaptation.
Widely used in noise cancellation, echo suppression, channel equalization.

(c) Define matched filter receiver. Describe the detector part of matched filter
receiver with necessary figure(s).

A matched filter receiver is a linear filter designed to maximize the signal-to-noise

ratio (SNR) at the output of the filter. It is matched to the shape of the transmitted
signal.

Detector Part of Matched Filter Receiver:

The detector part of the matched filter receiver consists of a bank of matched filters,
each matched to one of the possible transmitted signals. The output of each matched
filter is sampled at the end of the signal interval, and the sampled values are used to
make a decision about which signal was transmitted.

Q3.

(a) Why are detection and estimation considered fundamental issues in determining
the optimum design of a receiver?
Detection determines transmitted symbols (e.g., "0" or "1"), while estimation recovers
parameters (e.g., phase, amplitude). Both are fundamental because:

1. Noise Mitigation: They optimize performance in noisy channels.

2. Error Minimization: Ensure accurate symbol decisions (detection) and parameter
recovery (estimation).

(b) How can you characterize a matched filter? Write down the properties of
matched filters.

A matched filter is a linear filter designed to maximize the signal-to-noise ratio (SNR)
at the output of the filter. It is matched to the shape of the transmitted signal.

1. Maximizes SNR: Gives best possible signal-to-noise ratio at the output

2. Time-Reversed Copy: Its shape matches the transmitted signal backwards
3. Works Like a Correlator: Does the same job as comparing signals point-by-point
 4. Best for Gaussian Noise: Most effective for common white noise types
5. Delay Doesn't Matter: Works equally well if the signal arrives slightly late
Matched Filter Properties:
6. Maximizes SNR
Output SNR at t=T:

2E
SNR max =
N0

(E = signal energy, N₀ = noise power)

7. Impulse Response
Time-reversed signal:
∗
h(t) = s (T − t)

8. Correlator Equivalence
Output at t=T equals correlation:
T
∗
y(T ) = ∫ x(τ )s (τ )dτ
0

9. Optimal for AWGN

Minimizes error probability in Gaussian noise
10. Frequency Response
∗ −j2πf T
H (f ) = S (f )e

11. Delay Invariant

Performance unchanged by constant time shifts

(d) What do you mean by the average probability of symbol error?

The average probability of symbol error is the probability that the receiver makes an
incorrect decision about which symbol was transmitted. It is given by:

P e = P (m
^ ≠ mi )

where m
^ is the estimated symbol and m is the transmitted symbol.
i

The average probability of symbol error is a measure of the performance of the

receiver. The goal of the receiver is to minimize the average probability of symbol
error.

3(a) Criteria for Estimation in Digital Communication (Brief Discussion)

In digital communication systems, estimation techniques are used to determine

unknown parameters (e.g., amplitude, phase, delay) from noisy received signals. The
main estimation criteria are:
 1. Minimum Mean-Square Error (MMSE)
Minimizes the average squared difference between the true parameter and its
estimate:

^ 2
E[(θ − θ) ]

Optimal for Gaussian noise; equivalent to the conditional mean of the posterior
distribution.
2. Maximum Likelihood (ML)
Finds the parameter value that maximizes the likelihood function f (x|θ).
Asymptotically efficient (achieves Cramér-Rao lower bound for large data).
3. Maximum A Posteriori (MAP)
Incorporates prior knowledge of the parameter via f (θ).
Maximizes the posterior distribution f (θ|x).
4. Unbiasedness
Ensures E[θ]
^ = θ
(no systematic error).
5. Consistency
The estimate converges to the true value as sample size increases.

Draw and explain prediction-error process.

1. Define the Prediction Error:

The prediction error e is calculated as the difference between the actual
n

value X and the predicted value X

n
^
. n

^
en = Xn − X n

M
^
X = ∑ a k X n−k
n k=1

2. Prediction-Error Filter:
Use the present and past samples of a stationary process X n, X n−1 , … , X n−M

and the predictor coefficients a 1, a2 , … , aM to compute the prediction error e . n

 3. Inverse Filter:
Rearrange the prediction error equation to reconstruct the original process.
M
Xn = ∑ a k X n−k + e n
k=1

This allows the computation of the present sample X using past samples and
n

the present prediction error.

4. Variance Reduction:
The prediction-error variance σ is less than the variance of the original
2
e

process σ . 2
X

This makes the prediction-error process a more efficient representation of the

original process.

3(c) Characteristics of Adaptive Filters

1. Self-Optimizing: Automatically adjusts coefficients to minimize error (e.g., LMS,

RLS algorithms).
2. Time-Varying: Coefficients evolve dynamically with input statistics.
3. Applications:
Noise cancellation, channel equalization, beamforming.
4. Key Properties:
Convergence Rate: Speed of adaptation (e.g., LMS trades speed for stability).
Steady-State Error: Residual error after convergence.
Computational Complexity: RLS is more complex but faster than LMS.
5. Robustness: Performance under non-stationary environments (e.g., varying
channel conditions).
A matched filter receiver maximizes the SNR at sampling time t = T for detecting
known signals in AWGN.

Impulse response: h(t) = s(T − t)

Detector Part

1. Input: Received signal x(t) enters the receiver.

2. Matched Filters: Signal passes through N filters, each matched to a basis function
ϕ (T − t).
i

3. Sampling: Filter outputs are sampled at t = T → gives x 1, x2 , . . . , xN .

4. Observation Vector: Outputs form a vector x = [x 1
, x2 , . . . , xN ]
T
.
5. Detection: The vector is compared with known signal templates to decide the
transmitted signal (e.g., using maximum likelihood rule).

Classical Model of Speech Production

The classical speech production model is a source–filter model:

Source:
Voiced sounds: periodic pulses from vocal cords (glottal excitation).
Unvoiced sounds: random noise source.
Filter:
Vocal tract shapes the sound by acting as a time-varying linear filter
(resonator).
Output:
Speech = Source signal convolved with vocal tract filter response.

Example:
For a voiced sound, the source block generates a periodic pulse train, and the
filter block shapes the spectrum of the pulse train to produce the desired speech
sound.
For an unvoiced sound, the source block generates a random noise signal, and the
filter block shapes the spectrum of the noise signal to produce the desired speech
sound.