'CHALMERS

Digital Filter Design Using

Optimization

M ATS V IBERG
Chalmers University of Technology
Department of Signals and Systems
SE-412 96 Göteborg, Sweden
Email: viberg@chalmers.se
Mini-project for the Applied Optimization Course, Spring 2008.

& %
Applied Optimization Slide 1
'CHALMERS
$

Outline

• Introduction to Digital Filters

– Sampling and digital signals
– Digital filters
– Frequency response
• Digital Filter Design
– Specifications
– Linear Phase FIR Filters
– Least-Squares Design
– Minimax Design
• Optimization Tasks
& %
Applied Optimization Slide 2
'CHALMERS
$

Sampling

EX analog signal x(t) sampled with period T = 5 s:

Sampling a continuous−time signal
3

2.5

2
x(t) and samples x(n)

1.5

0.5

0
0 5 10 15 20 25 30
time [s]

Digital signal x(n), n = . . . , −1, 0, 1, 2, . . . , represents x(t) for

t = . . . , −T, 0, T, 2T, . . . .
T sampling time [s], fsamp = 1/T sampling frequency [Hz].
& %
Applied Optimization Slide 3
'CHALMERS
$

Digital Filters

Analog filter = continuous-time LTI-system: differential equation

Digital filter = discrete-time LTI-system: difference equation

x(n) y(n)
Digital Filter

First-order filter:

y(n) = −a1 y(n − 1) + b0 x(n) + b1 x(n − 1)

Implemented in a computer, DSP, FPGA or digital ASIC

& %
Applied Optimization Slide 4
'CHALMERS
$

Time-Domain Characterization

First-order difference equation:

y(n) + a1 y(n − 1) = b0 x(n) + b1 x(n − 1)

Impulse response h(n) = filter output when x(n) = δ(n). I/O relation:
∞

y(n) = h(k)x(n − k)
k=−∞

• Causal filter if h(n) = 0, n < 0.

• Finite Impulse Response (FIR) if h(n) = 0, n > M (a1 = 0)

• Infinite Impulse Response (IIR) otherwise

& %
Applied Optimization Slide 5
'CHALMERS
$

Discrete-Time Fourier
Transform

Frequency content of digital signals? Need ≥ 2 samples per period ⇒ can

only represent frequencies ≤ fsamp /2 = Nyquist frequency.
DTFT (Discrete-Time Fourier Transform):
∞

X(ejωT ) = x(n)e−jωT n , ω = 2πf [rad/s]
n=−∞

Equal to continuous-time FT if x(t) contains no frequencies above Nyquist.

Then: x(t) can be perfectly reconstructed from samples x(n)!
If x(t) has frequency content above fsamp /2 ⇒ aliasing (folding
distortion).
& %
Applied Optimization Slide 6
'CHALMERS
$

Frequency Response

Frequency domain: DTFT{x(n − k)} = e−jωT k X(ejω ) ⇒

Y (ejωT ) + a1 e−jωT Y (ejωT ) = b0 X(ejωT ) + b1 e−jωT X(ejωT )

which implies
b0 + b1 e−jωT
Y (e jωT
)= X(ejωT
) = H(ejωT
)X(ejωT
)
1 + a1 e−jωT

H(ejωT ) = Y (ejωT )/X(ejωT ) = H(ejΩ ) is the frequency response of the

filter!
Note: H(ejΩ ) is the DTFT of the impulse response h(n)!
ω frequency in rad/s, Ω = ωT normalized frequency in rad/sample.
& %
Applied Optimization Slide 7
'CHALMERS
$

Stability

A critical feature of a filter: stability!

A filter is BIBO-stable (Bounded Input Bounded Output) if |x(n)| < Cx
implies |y(n)| < Cy . A filter is BIBO-stable if
∞

|h(k)| < ∞
k=−∞

The transfer function of a general (finite-dimensional) system is

b0 + b1 z −1 + · · · + bM z −M B(z)
H(z) = =
1 + a1 z −1 + · · · + aN z −N A(z)
so H(ejΩ ) = H(z)|z=ejΩ . The poles, pi , i = 1, . . . , N of an N th order
system are the zeros of A(z), and the system is BIBO-stable if |pi | < 1 ∀i.
& %
Applied Optimization Slide 8
'CHALMERS
$

Frequency Response, cont’d

Filter response to a single frequency:

x(n) = sin(Ωn)

Stationary output (assuming stable filter):

y(n) = |H(ejΩ )| sin(Ωn + Arg{H(ejΩ )})

The frequency response can obviously be interpreted as a complex

frequency-dependent gain!
• Amplitude characteristics: |H(ejΩ )|
• Phase characteristics: Arg{H(ejΩ )}
Choice of filter coefficients determines which frequencies to pass and which
ones to block.
& %
Applied Optimization Slide 9
'CHALMERS
$

Digital Filters, cont’d

The spectrum |X(ejΩ )|2 is the energy of x(n) per Hz.

The spectrum of y(n) is related to that of x(n) by

|Y (ejΩ )|2 = |H(ejΩ )|2 |X(ejΩ )|2

The filter changes the frequency content of a signal - we can suppress noise
and interference!
Filter design = choice of coefficients {ak }N
k=1 and {bk }k=1 so that
M

|H(ejΩ )| (and Arg{H(ejω )}) meets specifications.

Filter design tools in Matlab: sptool, fdatool (graphic), fir1, firls, firpm,
butter, cheby1, cheby2, etc.
& %
Applied Optimization Slide 10
'CHALMERS
$

Digital Filters, cont’d

EX LP-filter y(n) = 0.4y(n − 1) + 0.6x(n)

First order digital LP filter
1

0.9

0.8

0.7

0.6
|H|2

0.5

0.4

0.3

0.2

0.1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Normalized frequency F/Fs

& %
Applied Optimization Slide 11
'CHALMERS
$

Digital Filters, cont’d

EX HP-filter y(n) = −0.4y(n − 1) + 0.6x(n − 1)

First order digital HP filter
1

0.9

0.8

0.7

0.6
|H|2

0.5

0.4

0.3

0.2

0.1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Normalized frequency F/Fs

& %
Applied Optimization Slide 12
'CHALMERS
$

Digital Filters, summary

• Sampling x(n) ∼ x(t) , t = nT

∞
• DTFT X(e jωT
) = n=−∞ x(n)e−jωT n
• Digital filter (1st order): y(n) + a1 y(n − 1) = b0 x(n) + b1 x(n − 1)
b0 +b1 e−jΩ
• Frequency domain: Y (e ) = H(e )X(e ) , H(e ) =
jΩ jΩ jΩ jΩ
1+a1 e−jΩ

• Amplitude characteristics: |H(ejΩ )|

• Phase characteristics: Arg{H(ejΩ )}

• A digital filter is stable if all poles are inside the unit circle

& %
Applied Optimization Slide 13
'CHALMERS
$

Filter Specifications (LP Filter)

Low−pass filter, f = 48 kHz
samp
10

−10

−20

−30
Gain [dB]

−40

−50

−60

−70

−80

−90
0 0.5 1 1.5 2 2.5
Frequency [Hz] 4
x 10

Passband: |H(ej2πf T ) − 1| ≤ 1 for 0 ≤ f ≤ fp

Stopband: |H(ej2πf T )| < 2 , fs ≤ f ≤ fsamp /2 (Hz)
& %
Applied Optimization Slide 14
'CHALMERS
$

FIR vs IIR Filters

FIR or IIR filter? A causal FIR filter with M + 1 "taps" has:

H(ejΩ ) = b0 + b1 e−jΩ + · · · + bM e−jM Ω

so h(n) = bn , n = 0, 1, . . . , M , and h(n) = 0 for n < 0 or n > M .

Pros and cons of FIR filters vs IIR filters:
+ H(ejΩ ) and h(n) are linear functions of bi (but non-linear in ai )
+ FIR filters are always stable (all poles are in the origin)
+ FIR filters have linear phase if bi = ±bM −i (symmetry or
anti-symmetry)
- FIR filters need in general more coefficients to meet given
specifications on |H(ejΩ )|
In this project we focus on FIR filter design!
& %
Applied Optimization Slide 15
'CHALMERS
$

Linear Phase FIR Filters

Assume M is odd and bi = bM −i (symmetry). Then:

H(ejΩ ) = b0 (1 + e−jM Ω ) + b1 (e−jΩ + e−j(M −1)Ω ) + · · · = e−jM Ω/2 A(Ω)

where
MΩ (M − 2)Ω Ω
A(Ω) = 2b0 cos + 2b1 cos + · · · + 2b(M −1)/2 cos
2 2 2
is a real function! Thus, the phase Arg{H(ejΩ )} = −(M/2)Ω is linear in
Ω - corresponds to a time-delay by M/2 samples and no phase distortion!
Similar calculations when bi = −bM −i and/or M even - still linear phase,
but real function A(Ω) slightly different.

& %
Applied Optimization Slide 16
'CHALMERS
$

Least-Squares FIR Filter Design

An FIR filter with (anti-)symmetric coefficients has "perfect" (linear) phase.

We need only worry about the amplitude!
Natural and simple approach: specify desired response A(Ω, b) at a discrete
set of frequencies {Ωk }K
k=1

A(Ωk , b) = Ad (Ωk ), k = 1, 2, . . . , K

where Ad (Ωk ) is the desired response and

 
MΩ (M − 2)Ω Ω
A(Ω, b) = 2 b0 cos + b1 cos + · · · + b(M −1)/2 cos
2 2 2
(for the symmetric case with M odd). The dependency on the (M + 1)/2
filter parameters b = [b0 , . . . , b(M −1)/2 ]T has been stressed.
& %
Applied Optimization Slide 17
'CHALMERS
$

LS FIR Filter Design, cont’d

Weighted Least-Squares filter design:


K
bLS = arg min Wk |Ad (Ωk ) − A(Ωk , b)|2
b k1

where Wk is a set of weights, used to emphasize certain frequencies or

frequency bands.
Linear LS-problem ⇒ explicit solution!
To solve this in, e.g. Matlab, express the magnitude response as

A(Ω, b) = φT (Ω)b

where
MΩ (M − 2)Ω Ω
φT (Ω) = 2[cos , cos , . . . , cos ]
2 2 2
& %
Applied Optimization Slide 18
'CHALMERS
$

LS FIR Filter Design, cont’d

The LS problem can now be put in matrix-vector form as

bLS = arg min ad − Φb2W

b
where

ad = [Ad (Ω1 ), . . . , Ad (ΩK )]T (K × 1)

Φ = [φ(Ω1 ), . . . , φ(ΩK )]T (K × (M + 1)/2)

and W is a diagonal matrix with diagonal elements Wk .

The weighted norm is:
x2W = xT Wx

& %
Applied Optimization Slide 19
'CHALMERS
$

LS FIR Filter Design, cont’d

The normal equations are

ΦT WΦ bLS = ΦT Wad

and the LS-solution is

bLS = (ΦT WΦ)−1 ΦT Wad

Never type this in Matlab, but use:

bLS = (W1/2 Φ)\(W1/2 ad )

1/2 √
where W has Wk at the diagonal (generally a matrix square-root).
This is a numerically more stable implementation, using QR decomposition!
& %
Applied Optimization Slide 20
'CHALMERS
$

Minimax FIR Filter Design

LS approach simple, but the optimal (according to specifications) is a

minimax design! Given desired response Ad (Ω), the weighted error is

E(Ω, b) = W (Ω) (Ad (Ω) − A(Ω, b))

where W (Ω) controls passband ripple vs stopband damping.

Minimax (Chebyshev) filter design:

bP M = arg min max |E(Ω, b)|

b Ω∈O
where O is the set of frequencies where specifications exist, normally the
passband and stopband. "PM" stands for Parks and McClellan.

& %
Applied Optimization Slide 21
'CHALMERS
$

Minimax FIR Filter Design:

Equiripple Property

"Easy" to see that the error |E(Ω, bP M )| has equi-ripple - the maximum
|E(Ωi , bP M )| = δ is "generically" attained at (M + 3)/2 frequencies Ωi ,
with alternating sign of E(Ω). This is the "alternation theorem".
If the Ωi s were known, we could use this insight to find bP M by solving a
linear equation system ((M + 3)/2 equations for (M + 3)/2 unknowns,
including δ). Parks and McClellan’s algorithm iterates between solving for
b and δ, and updating the Ωi s by finding all local maxima in an efficient
way. In Matlab, this is called firpm. Your goal is to implement your own
algorithm, and compare with firpm!

& %
Applied Optimization Slide 22
'CHALMERS
$

Exercises: FIR Filter Design

The tasks consist of implementing Matlab functions for LS (with weighting)

and minimax FIR filter design. The functions should then be compared with
the existing ones in Matlab. As a test example we use the following lowpass
specifications (with fsamp = 1 Hz):
• Passband: 0 ≤ f ≤ 0.25, corresponding to 0 ≤ Ω ≤ π/2

• Passband ripple: 20 log10 (1/1 ) < 3 dB

• Transition band (don’t care region): 0.25 < f < 0.3

• Stopband: 0.3 ≤ f ≤ 0.5

• Damping: 20 log10 (1/2 ) > 40 dB

& %
Applied Optimization Slide 23
'CHALMERS
$
1. Warm-up. The simple averaging filter
M
1
y(n) = x(n − k)
M +1 k=0

corresponds to an FIR filter with bk = 1/(M + 1), k = 0, . . . , M . Compute and display the
corresponding frequency response, using the Matlab commands:
» M = 10; %%% Filter order
» h = ones(M+1,1)/(M+1); %%% Impulse response
» [H,w] = freqz(h,1,1024); %%% Frequency response
» plot(w/2/pi,20*log10(abs(H))); %%% Magnitude plot in dB
» xlabel(’Frequency [normalized]’)
» ylabel(’Magnitude [dB]’)
Hopefully, you will see that this is a lowpass filter with bandwidth approximately 1/M and stopband
damping −13 dB (which does not improve with increasing M !).
2. Least-squares design. Write a Matlab function that implements the weighted least-squares digital FIR
filter design. The input should be:
• M , the filter order (corresponding to M + 1 filter taps)
• F , a vector of frequency samples
• A, a vector of desired response values (of equal length as F )
• W , a vector of weights to be used in the LS solution
& %
Applied Optimization Slide 24
'CHALMERS
$
The output of the function is the filter coefficients b LS . Hint: the following commands are useful for
setting up the matrix Φ, avoiding for-loops that are slow in Matlab:
» MM = (M:-2:1)/2;
» F = [linspace(0,Fp,K/2) linspace(Fs,0.5,K/2)];
» Phi = 2*cos(2*pi*F’*MM);
3. Run your LS design for increasing filter orders M and different choices of passband and stopband
weights W . Use the following set of commands for the first run:
» M = 11; %%% Filter order
» K = 100; %%% Number of frequency samples
» Fp = 0.25; Fs = 0.3; %%% Passband and stopband edge
» F = [linspace(0,Fp,K/2) linspace(Fs,0.5,K/2)]; %%% Frequency
samples
» A = [ones(1,K/2) zeros(1,K/2)]; %%% Desired response
» Wpass = 1; Wstop = 1; %%% Passband and stopband weights
» W = [Wpass*ones(1,K/2) Wstop*ones(1,K/2)]; %%%
» bLS = LSdesign(M,F,A,W); %%% Name your function "LSdesign.m"
» [H,w] = freqz(bLS,1,1024); %%% Frequency response
» plot(w/2/pi,20*log10(abs(H))); %%% Check if the specifications
are met
» xlabel(’Frequency [normalized]’)
& %
Applied Optimization Slide 25
'CHALMERS
$
» ylabel(’Magnitude [dB]’)
Spend some time to select the input to the filter so you get something that almost meets the
specifications of the test problem. Compare your results with that of Matlab’s implementation:
firls. Note that the syntax is slightly different for firls. You should use the following command:
» bFIRLS = firls(M,2*[0 0.25 0.3 0.5],[1 1 0 0],[Wpass Wstop]);
4. Parks-McClellan design. Write a Matlab function that implements the Parks-McClellan digital FIR
filter design for the low-pass case. The input should be:
• M , the filter order (corresponding to M + 1 filter taps).
• F , a vector of frequencies used to separate the frequency bands, i.e. F = [0, F p, F s, 0.5]
(assume a lowpass filter!).
• A = [1, 1, 0, 0], a vector used to define that there are two bands, where the first is a passband
and the second a stopband (only this case needs to be implemented).
• W = [Wpass , Wstop ], specifies the weights for the passband and stopband respectively.
The output is the filter coefficients b P M , obtained by solving the minimax optimization problem:

bP M = arg min max |E(Ω, b)|

b Ω∈O
where E(Ω, b) is the same weighted error as used in the LS-design:

E(Ω, b) = W (Ω) (Ad (Ω) − A(Ω, b)) .

The simplest way to solve the optimization problem is to use Matlab’s fminimax routine. See help
& %
Applied Optimization Slide 26
'CHALMERS
$

fminimax for details. If you are really interested, you could also try to implement your own code
based on the iterative procedure described above. Contact viberg@chalmers.se in that case.
Regardless which implementation you have used, you should verify that your resulting filter is indeed
equi-ripple!

5. Spend some time to select the input to the filter (M and the weights) so that the resulting amplitude
response meets the specifications of the test problem. Compare your results with that of Matlab’s
implementation firpm. Use:
» bFIRPM = firpm(M,2*[0 0.25 0.3 0.5],[1 1 0 0],[Wpass Wstop]);
(there is also a command firpmord that estimates the necessary filter order to satisfy the given
specifications).

& %
Applied Optimization Slide 27