4

TIME DOMAIN REPRESENTATION OF LINEAR TIME-

INVARIANT SYSTEMS: CONVOLUTION

4.1 Introduction
Convolution is one of the major concepts of linear time-invariant system theory. Convolution relates an LTIs system's input to its
output thus it is a mathematical operation of fundamental importance in the theory of signals and systems. Convolution yields the
output of a relaxed (zero initial conditions) LTI system, given the input xn and the system's impulse response hn. A system's
impulse response is a unique property of the system that completely characterizes its dynamic behavior (under the assumption of
linearity). The impulse response, as the name suggests, is the output of a system given a unit impulse (sample) input.

4.2 Convolution integral

In an earlier chapter we presented methods of finding the impulse response of an LTI system. In this chapter we begin with a
known impulse response, denoted here ht, and we show that ht is sufficient to describe the system’s response to any input and
we derive the relationship between the input and output signals and the impulse response. Given an LTI system , the impulse

ht = dt
response of the system is the output to a unit impulse input.
(4.1)

For example, if the system is the RC filter, then it is well-known that the output voltage measured aross the capacitor resulting from
a unit impulse input at the source takes the form of a decaying exponential. The impulse response of an RC filter with R = C = 1
and zero initial conditions is therefore:
ht = ‰-t ut

xt in terms of the impulse function d t as follows.

Now consider an arbirtrary input xt. Using the sampling property of the impulse function we can write an integral expression for

xt = 
¶
xt dt - t „ t (4.2)
-¶

The output of an LTI system to input xt will then be given by

yt =  xt  =   xt dt - t „ t 

¶
(4.3)
-¶

Using the fact that xt is independent of, or constant with respect to the system , we get

yt =  xt  dt - t  „ t

¶
(4.4)
-¶

Now, from Eq. (1.1) and time-invariance we conclude that

yt = 
¶
xt ht - t „ t (4.5)
-¶
2 Chapter04_Convolution.nb

This last result shows that the output of any LTI system may be obtained by evaluating the integral in Eq. (1.4) given an input and
the impulse response of the system. The integral in Eq. (1.5) is known as the convolution integral. Convolution is then the opera-
tion of computing this integral for two signal, the input signal xt and the impulse response signal ht. Convolution is frequently
denoted by the symbol * as in yt = xt * ht. Convolution is known to satisfy the commutative, associative, and distributive laws
of algebra therefore xt * ht = ht * xt and therefore the convolution integral may be written in the following two equivalent ways.

yt =  xt ht - t „ t = 

¶ ¶
ht xt - t „ t (4.6)
-¶ -¶

The output of an LTI system at time t is the area of the intersection of the input with a time shifted and reversed copy of the
impulse response (or vice versa). Clearly, evaluating thi integral may be difficult, unless the signals xt and ht are “nice”, simple
functions. The simplest possible convolution problem involves two signals that are ideal pulses. This problem can be easily solved
by hand calculation. Here I define signals xt and ht as pulses of width 2 and 1, respectively.
In[9]:= Clearx, h, y;
xt_ : Piecewise1, 0  t  2;
ht_ : Piecewise1, 0  t  1;

xt ht
1.0 1.0
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2

-2 -1 1 2 3 4 -2 -1 1 2 3 4

It is typical to use a “divide-and-conquer” approach with the region of integration broken up into non-overlapping subregions. The
sum of the individual solutions in each of the subregions gives the total solution. The following plots will illustrate the mechanics of
evaluating equation (4.6).
This shows the shifted and reflected signal ht across the domain of the signal xt. In the first example, the two signals do not
overlap and therefore (1.6) evaluates to zero.

1.0

0.8

0.6

0.4

0.2

t
-4 -2 0 2 4

yt_ : 0 ; t  0

As the signal ht - t shifts to the right, it will begin to overlap signal xt. What is the value of t at which the two signals "touch"
each other?
 Chapter04_Convolution.nb 3

The integral in the region of partial overlap is defined as follows

 1 
t

t
Therefore, we define the output signal as
yt_ : t ; 0  t  1
Continuing to shift to the right, signal ht enters a region in which it is completely overlapped by signal xt. Clearly, at this stage
the integral evaluates to a constant value of 1 (find the area inside the red rectangle).

Thus,
yt_ : 1 ; 1  t  2
Continuing, a partial overlap occurs involving the trailing edge of ht.
4 Chapter04_Convolution.nb

This yields the following integral


2
1 
t1

3t
and
yt_ : 3  t ; 2  t  3
Finally, ht shifts beyond the leading edge of xt. Since the two signals do not overlap, the integral is again zero.

1.0

0.8

0.6

0.4

0.2

-2 -1 0 1 2 3 4

yt_ : 0 ; t  3
Having defined the output yt for all values of t, we plot the result.
Plotyt, t,  2, 6, Exclusions  None, ImageSize  Small
1.0

0.8

0.6

0.4

0.2

-2 2 4 6

The derivation shown above describes the traditional manual method of evaluating the convolution integral using paper-and-
pencil. Here, however, is the evaluation using Mathematica’s Integrate function.
 Chapter04_Convolution.nb 5

In[16]:= Cleary;

yt_ :  x ht    ; yt  OutputForm




Out[17]//OutputForm=
Piecewise[{{1, 1 < t <= 2}, {3 - t, 2 < t < 3}, {t, 0 < t <= 1}}]

Alternatively, the Convolve function may be used.

In[18]:= yt_ : Convolvex, h, , t; yt  OutputForm
Out[18]//OutputForm=
Piecewise[{{1, 1 < t <= 2}, {3 - t, 2 < t < 3}, {t, 0 < t <= 1}}]

Finally, here is an interactive demonstration of the convolution operation on the two pulses defined in the preceding example.

Here is a demonstration showing the response of the RC circuit to a sinusoidal source. Recall that the impulse response of the RC
circuit, the voltage across the capacitor is a decaying exponential, ht = ‰-t ut.

1. Obtain the output of an LTI system defined by the impulse response ht = ‰-t ut to a pulse input of duration 2 seconds, starting at time
t = 0.
2. Obtain the output of an LTI system defined by the impulse response ht = ‰-t ut to the sinusoidal source xt = sint ut, as

Verify your result by comparing to the solution of the differential equation y ' t + yt == sint ut with zero initial condition (y0 = 0).
demonstrated above, by identifying the relevant solution regions, writing and solving the defining convolution integrals for each of them.
6 Chapter04_Convolution.nb

4.3 Convolution sum

The general one-dimensional linear convolution sum formula has the following two equivalent forms:

yn =  hk xn - k =  xk hn - k

¶ ¶

(4.7)
k=-¶ k=-¶

where hn is the so-called impulse response, xn the input and yn the output of a discrete-time LTI system. Convolution
satisfies the commutative, associative and distributive laws of algebra. These formulas can be easily derived. Consider an
arbitrary input signal as a superposition of scaled and shifted unit samples.

xn = ... + x-1 dn + 1 + x0 dn + x1 dn - 1 + x2 dn - 2 + ... =  xk dn - k
¶

(4.8)
k=-¶

Now, the system response to a unit sample input is the sequence hn, namely, dn Ø hn. In a linear, time-invariant system, the
output of the system to input xn must be a superposition of the responses to each of the scaled and time-shifted impulses in
(2.2). We therefore get the following convolution formula.

yn = ... + x-1 hn + 1 + x0 hn + x1 hn - 1 + x2 hn - 2 + ... =  xk hn - k
¶

(4.9)
k=-¶

As an example, consider the evaluation of the output of an LTI system with the impulse response and input shown below.
1, n = 0
hn = - 1, n = 1
0, otherwise
1, n = 0
2, n = 1, 2
xn =
-1, n = 3
0, otherwise
The output yn, by virtue of linearity and shift-invariance, is given by the following sum of scaled and shifted impulse responses.
yn = hn + 2 hn - 1 + 2 hn - 2 - hn - 3
Substitution of the algebraic formula for the impulse response yields the following summation.
dn -dn - 1
+
2 dn - 1 -2 dn - 2
yn = +
2 dn - 2 -2 dn - 3
+
- dn - 3 dn - 4
Simplifying, we get the following output signal.
yn = dn + dn - 1 - 3 dn - 3 + dn - 4
Here is an interactive presentation of the flip-and-slide method of evaluating a discrete-time convolution. Observe how the impulse
response h slides across the sequence of numbers x. At each position n the sum of products of the two sequences is shown
confirming the result previously derived using an algebraic approach.
 Chapter04_Convolution.nb 7

0 1 2 2 1 0
 
1 1

1 1 0

3. Assume an LTI system with the following impulse response.

1
, n=0
hn =  1 , n = 1
2

2
0, otherwise
Determine the output of this system in response to the input
1, n = 0

xn = 
2, n = 1, 2
-1, n = 3
0, otherwise
Here is a second example of convolving two numerical sequences using (1.7):

hn =   un
2 n
3

and

xn = 
1, 0§ n§ 9
0, else
This defines the two sequences.
In[21]:= Clearh, x
2 n
In[22]:= hn_ : UnitStepn;
3
xn_ : UnitStepn  UnitStepn  10;
This example illustrate the so-called "flip-and-slide" operation which arises naturally as a consequence of formula (1.7). For each
value of n, the sum of the point-by-point products of the two sequences is evaluated yielding a single output value. When evalu-
ated for all values of n, we get the result shown below. An examination of the regions of support of the two sequences reveals that
there are three disjoint, but contiguous regions we need to consider when evaluating the convolution sum.
Region 1. For n < 0 there is no overlap between hm (blue) and xn - m (red).

n
8 Chapter04_Convolution.nb

So the result is
yn = 0, for n < 0
Region 2. For 0 § n < 9 there is a partial overlap between hm and xn - m.

Evaluating in Mathematica we get

 hm  OutputForm
n
In[24]:=

m0
Out[24]//OutputForm=
1 + n 1 + n
2 - 3
Piecewise[{{1, n == 0}}, -(---------------)]
--------------
n
3

Using a formula for the finite sum we get,

1- 
nm=0   =3-2 
2 n+1
2 m 2 n
yn = =
3

3 2 3
1-
3

Region 3. For n ¥ 9 we have full overlap between xm and hn - m.

The convolution sum now takes the form

 hm  Simplify
n

29n 3n 19 683 UnitStep 9  n 

mn9

2 6561 UnitStep 8  n  4374 UnitStep 7  n  2916 UnitStep 6  n 

1944 UnitStep 5  n  1296 UnitStep 4  n  864 UnitStep 3  n 
576 UnitStep 2  n  384 UnitStep 1  n  256 UnitStepn
 Chapter04_Convolution.nb 9

Using a formula for the finite sum we obtain the following result for n ¥ 9 :

  - 
yn = nm=n-9    
2 n-9 2 n+1
2 m 58 025 2 n
= =
3 3

3 2 512 3
1-
3

Putting together the results for all three regions we get the following piecewise function:

, 0  n  9, 
2 n 58 025 2 n
yn_ : Piecewise3  2 , n  9;
3 512 3
This shows the result.

This result can be confirmed by evaluating the convolution sum directly in Mathematica:
In[25]:= DiscreteConvolvexm, hm, m, n  Simplify  OutputForm
Out[25]//OutputForm=
-9 + n 1 + n
58025 2 2
Piecewise[{{-------------,
------------ n >= 10}}, 3 - ------]
-----
n n
3 3

Here is an interactive demonstration.

4. Repeat the convolution calculation for the sequences in (2.4) by flipping-and-sliding sequence hn.

FIR filters
An important category of discrete-time systems exhibits a finite impulse response (FIR), so hn has finite support. It is common to
define hn on the interval 0 § n § Lh - 1. Sequence hn is then called a causal FIR filter of length Lh . In practice, the sequence
xn is also finite in length, so that yn can be computed in finite time. Consider two sequences xn and hn, defined for positive
values of n only, of length Lx and Lh , respectively. Equation (1.7) then reduces to the following finite summation:

yn =  hi xn - i

Lh -1
(4.10)
i=0

with the output sequence yn supported on the finite domain 0 § n < Lx + Lh - 1. It is instructive to evaluate the sum for all
values of n in this domain. Begin by defining the sequences and their lengths.
10 Chapter04_Convolution.nb

Clearx, h, y;
Lx  8; Lh  3;
xn_ : 0 ; n  0  n  Lx ;
hn_ : 0 ; n  0  n  Lh ;
Here is a Mathematica implementation of (1.10).

yn_ :  hi xn  i

Lh 1

i0

This shows the result.

Tablen, yn, n, 0, Lx  Lh  2  TableForm
0 h0 x0
1 h1 x0  h0 x1
2 h2 x0  h1 x1  h0 x2
3 h2 x1  h1 x2  h0 x3
4 h2 x2  h1 x3  h0 x4
5 h2 x3  h1 x4  h0 x5
6 h2 x4  h1 x5  h0 x6
7 h2 x5  h1 x6  h0 x7
8 h2 x6  h1 x7
9 h2 x7
The output may be subdivided into three sections. The first and last Lh - 1 samples represent "transient" portions of the output
signal due to a partial overlap of nonzero values of hn and the reversed and shifted version of xn. The middle samples repre-
sent a "steady state" portion of the output sequence. This occurs whenever sequence hn and the reversed and shifted version

Tablen, yn, n, Lh  1, Lx  1  TableForm

of xn are fully overlapped.

2 h2 x0  h1 x1  h0 x2

3 h2 x1  h1 x2  h0 x3
4 h2 x2  h1 x3  h0 x4
5 h2 x3  h1 x4  h0 x5
6 h2 x4  h1 x5  h0 x6
7 h2 x5  h1 x6  h0 x7
Here is an interactive version of the result shown above.

0 0 x0 x1 x2 x3 x4 x5 x6 x7 0 0

  
h2 h1 h0

h2 x2  h1 x3  h0 x4

5. How many non-zero output values result from convolving sequences xn and hn in preceding example? How many multiplication and
addition operations are needed to obtain a single output value? All the output values?

Convolution sum: efficient evaluation

Convolution is such an important mathematical operator that most advanced computational systems available today have built-in
functions for computing the convolution of two finite-length numerical sequences. For example, the two sequences in Problem 1
can be represented by the following two lists of numbers.
 Chapter04_Convolution.nb 11

x  1, 2, 2,  1; h  1,  1;

Clearx, h;

Now, use the built-in convolution operator ListConvolve to obtain the result. By default only the steady-state values are
returned.
ListConvolveh, x
1, 0,  3
This returns all the non-zero values assuming zero-padding on both ends of the given lists of numbers.
ListConvolveh, x, 1,  1, 0
1, 1, 0,  3, 1

Alternatively, explicit padding of the input list x may be used.

ListConvolveh, ArrayPadx, 1
1, 1, 0,  3, 1

Example
As an example, we use convolution to smooth a noisy data sequence. The data values are monthly S&P 500 composite index

data  FinancialData"SP500", 1957, 2011, "Month"All, 2;

quotations for the period from 1957 to 2011. First we read and plot the data.

ListPlotdata, ImageSize  Small

1500

1000

500

100 200 300 400 500 600

A popular method of smoothing noisy data is to "locally" average the data. Such an operation may be conveniently expressed by
the following formula:

 xn - N + ... + xn - 1 + xn + xn + 1 + ... xn + N 

1
yn = (4.11)
2N +1
The unit sample response of a filter that performs this averaging operation (such a filter is commonly called a moving-average
filter) is defined

hn = 
1
, for - N § n § N
2 N+1 (4.12)
0, otherwise
6. Verify (1.12) by evaluating (1.11) for a unit sample input, namely, xn = dn.
Thus the operation of smoothing signal xn may be implemented by convolving with impulse response hn. Here we show the

fir  Table1, 11  11.;

result of smoothing with an 11-point moving average filter. First we define hn as a list.

This gives output of the moving average filter using the built-in function ListConvolve.
smooth  ListConvolvefir, data;
This plots the smoothed data. By default, Mathematica returns the steady-state values of the convolution sum.
12 Chapter04_Convolution.nb

ListPlotsmooth, ImageSize  Small

1400
1200
1000
800
600
400
200

100 200 300 400 500 600

This returns both the steady-state and transient samples for different lengths of moving average filter given by the parameter n.

filter length, n

1400
1200
1000
800
600
400
200

0 100 200 300 400 500 600

7. Discuss how the number of steady-state and transient samples changes with length of moving average filter.

4.4 Convolution and the impulse response

The impulse response of a LTI system is the system output when the input is a unit impulse: if xt = dt then yt = ht. In this
case, the output is the impulse response of the filter. Here is a validation of this statement using equation (1.6),

y t =  ht dt - t „ t = 

¶ ¶
dt ht - t „ t = ht (4.13)
-¶ -¶

Equation (4.13) illustrates a fundamental property of the unit impulse, namely that it is an identity element of convolution. This
property also holds for discrete convolution.

yn =  hi xn - i =  hi dn - i = hn

¶ ¶
(4.14)
i=-¶ i=-¶

It is also of interest to obtain formulas for the step response of a system, given that its impulse response is known. In the case of
continuous-time systems we get

y t =  ht ut - t „ t =  ut ht - t „ t = 

¶ ¶ ¶
ht - t „ t (4.15)
-¶ -¶ 0

We can now use the result derived in (1.15) to find the step response of a system with ht = ‰-t ut. We get the following well-
known result.

yt =  ht - t „ t = =  ‰-t-t ut - t „ t =  ‰-t-t „ t = ‰-t  ‰ t „ t = ‰-t  ‰ t t0 = ‰-t ‰t - 1 = 1 - ‰-t .
¶ ¶ t t

0 0 0 0

This shows a graphical interpretation of the result derived in Equation (1.15). The impulse response ht = ‰-t ut is used.
 Chapter04_Convolution.nb 13

8. Obtain the step response of a system with impulse response given by ht = 1 - t ut - ut - 1.
Following similar steps as in Equaton (1.15), here is the step response of a discrete-time system given its impulse (unit sample)
response.

yn =  hi un - i =  ui hn - i =  hn - i

¶ ¶ ¶
(4.16)
i=-¶ i=-¶ i=0

Using (1.16) we now find the step response of a system with hn =   un.
2 n
3

yn =  hn - i =  un - i =  

¶ ¶ 2 n-i n 2 n-i 2 n n 3 i 2 n
= =3-2 .
i=0 i=0 3 i=0 3 3 i=0 2 3

4.5 Solutions