ECE 6640

Digital Communications

Dr. Bradley J. Bazuin

Assistant Professor
Department of Electrical and Computer Engineering
College of Engineering and Applied Sciences
 Chapter 6
6. Channel Coding: Part 1.
1. Waveform Coding.
2. Types of Error Control.
3. Structured Sequences.
4. Linear Block Codes.
5. Error-Detecting and Correcting Capability.
6. Usefulness of the Standard Array.
7. Cyclic Codes.
8. Well-Known Block Codes.

ECE 6640 2
 Sklar’s Communications System

Notes and figures are based on or taken from materials in the course textbook:
ECE 6640 Bernard Sklar, Digital Communications, Fundamentals and Applications, 3
Prentice Hall PTR, Second Edition, 2001.
 Signal Processing Functions

Notes and figures are based on or taken from materials in the course textbook:
ECE 6640 Bernard Sklar, Digital Communications, Fundamentals and Applications, 4
Prentice Hall PTR, Second Edition, 2001.
 Encoding and Decoding

• Data block of k-bits is encoded using a defined algorithm

into an n-bit codeword
• Codeword transmitted
• Received n-bit codeword is decoded/detected using a
defined algorithm into a k-bit data block

Data
Encoder Codeword
Block

Data
Decoder Codeword
Block
ECE 6640 5
 Waveform Coding Structured Sequences

• Waveform Coding:
– Transforming waveforms in to “better” waveform representations
– Make signals antipodal or orthogonal
• Structured Sequences:
– Transforming waveforms in to “better” waveform representations
that contain redundant bits
– Use redundancy for error detection and correction

– Bit sets become longer bit sets (redundant bits) with better
“properties”
– The required bit rate for transmission increases.

ECE 6640 6
 Antipodal and Orthogonal Signals

• Antipodal
– Distance is twice “signal voltage”
– Only works for one-dimensional signals
1
T
1 for i  j
d  2  Eb z ij    s i t   s j t   dt  
E 0  1 for i  j
• Orthogonal
– Orthogonal symbol set
– Works for 2 to N dimensional signals
1
T
1 for i  j
d  2  Eb z ij    s i t   s j t   dt  
E 0 0 for i  j

ECE 6640 7
 M-ary Signals Waveform Coding

• Symbol represents k bits at a time

– Symbol selected based on k bits
– M waveforms may be transmitted M  2k

• Allow for the tradeoff of error probability for bandwidth

efficiency

• Orthogonality of k-bit symbols

– Number of bits that agree=Number of bits that disagree

 sumb   
K N
i
 b   sum b ik  b kj
j
k k
1 for i  j
z ij  k 1 k 1

K 0 for i  j
ECE 6640 8
 Hadamard Matrix Orthogonal Codes
H D 1 H D 1 
HD  
H D 1 H D 1 
• Start with the data set for one bit and generate a Hadamard
code for the data set
0  0 0 
1  H 1  0 1 
   
0 0 0 0 0 0
0 1 H1   0
  H1 1 0 1
 H2    
1 0  H1 H1  0 0 1 1
  0 1 1 0 
1 1

1 for i  j
z ij  
ECE 6640 0 for i  j 9
Hadamard Matrix Orthogonal Codes (2)
0 0 0 0 0 0 0 0 0 0 0
0 1 0
 0
 1 0 1 0 1 0 1
0 1 0 0 0 1 1 0 0 1 1
 
0 1 1 H H 2  0 1 1 0 0 1 1 0

Data Set Codeword
 H3   2  
1 0 0 H 2 H 2  0 0 0 0 1 1 1 1
  0
1 0 1 1 0 1 1 0 1 0
 
1 1 0 0 0 1 1 1 1 0 0
   
1 1 1 0 1 1 0 1 0 0 1

0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 1 0
 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
See Hadamard in 0

0
0 1 0
0 1 1


0

0 1 1 0 0 1 1 0 0 1 1 0 0 1 1

0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0
MATLAB 0

0
1 0 0
1 0 1
 0
0
0
1
0
0
0
1
1
1
1
0
1
1
1
0
0
0
0
1
0
0
0
1
1
1
1
0
1
1
1
0
0  
1 1 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0
 
H4=hadamard(16) 0
1
1 1 1
0 0 0
H
 H4   3
H3
H 3  0

H 3  0
1
0
1
0
0
0
1
0
0
0
0
0
1
0
0
1
1
1
1
1
0
1
1
1
0
1
0
1
1 
1
 
0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 0
H4xcor=H4'*H4=H4*H4'
1 0 0 1
   
1 0 1 0 0 0 1 1 0 0 1 1 1 1 0 0 1 1 0 0
1 0 1
0 1 1 
1 1 0 0 1 1 0 1 0 0 1 1 0 0

ans = 16*eye(16)
 
1 1 0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0
1 0 1 0 1 1 0 1 0 1 0 1 0 0 1 0 1
1 0 1  
 
1 1 1 0 0 0 1 1 1 1 0 0 1 1 0 0 0 0 1 1
0 0  10
ECE 6640 1
 1 1 1  1 1 0 1 0 0 1 1 0 0 1 0 1 1
 Hadamard Matrix Orthogonal Codes (2)

• The Hadamard Code Set generates orthogonal code for the

bit representations Letting
1 for i  j
z ij   0 = +1V
0 for i  j
1 = -1V
• For a D-bit pre-coded symbol a 2^D bit symbol is
generated from a 2^D x 2^D matrix
– Bit rate increases from D-bits/sec to 2^D bits/sec!
• For equally likely, equal-energy orthogonal signals, the
probability of codeword (symbol) error (PE) can be
bounded as
 Es  D
M2
PE M   M  1  Q 

 N0  Es  D  E b
ECE 6640 11
 Symbol error to Bit Error

• Discussed in chapter 4 (Equ 4.112) – k-bit set (codeword)

– 4.9.3 Bit error probability versus symbol error probability for
orthogonal signals
M
PB k  2 k 1 PB M  2
 k M  2k 
PE k  2  1 PE M  M  1
• Substituting bit error probability for symbol error
probability bounds previously stated
 Es 
PE M   M  1  Q 

 N0 
 k  Eb   M   E S 
 
PB k   2 k 1 
 Q 
 PB M      Q
 N0   2   N 0 
ECE 6640 12
 Derivation of Equ 6.6

• In orthogonal signaling, a conditional probability for a

message error can be stated as (1 of M-1 possible errors
where m=2:M)
Psm received | s1sent  
Pe Pe
 k
M 1 2 1
• If s1 is a data sequence of k bits with a 0 is the first bit,
there are 2k-1 possible sequences. The probability of the
single bit error causing a message error at this location
must be related to (for k large)
1
 PsM received | s1sent   2
k 1 k 1 Pe k 1 Pe
Pb 0  2  2  k   Pe
M 1 2 1 2
M k 1
2 P  2
PB   Pe
M 1 2 1
e k
ECE 6640 13
 Biorthogonal Codes

• Code sets with cross correlation of 0 or -1

1 for i  j


z ij   1 for i  j , i  j  M
2

0 for i  j , i  j  M
2

• Based on Hadamard codes (2-bit data set shown)

0 0 0 0
H D 1  0 1  1
BD      H1   0
H
 D 1   B2    
1 0   H1  1 1
   
1 1 1 0
ECE 6640 14
 Biorthogonal Codes (2)
0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
 0 0 1 0
 1 0 1 0 1 0 1
0 0 1 0 1 0 1
0

0 1 0

0

0 1 1 0 0 1 1

  0
0
0 1 1 0 1 1 0 0 1 1 0
1 0 0 0 0 0 0 1 1 1 1
0 1 0 0 0 1 1    
    0
0
1 0 1
1 1 0
0
0
1 0 1 1 0 1 0
0 1 1 1 1 0 0
0 1 1  H 2  0 1 1 0    
 B3     0 1 1 1  H 3  0 1 1 0 1 0 0 1
1 0 0   H 2  1 1 1 1 1

0 0 0

 B4     
 H 3  1 1 1 1 1 1 1 1

    1 0 0 1 1 0 1 0 1 0 1 0
1 0 1 1 0 1 0 
1 0 1 0
 
1 1 0 0 1 1 0 0


1 1 0 1 1 0 0
1

0 1 1

1

0 0 1 1 0 0 1

    1
1
1 0 0

1
1
1 1 1 0 0 0 0
0 1 0 0 1 0 1
1 1 1 1 0 0 1 
1 0 1
  
1 1 1 0 1 1 0 0 0 0 1 1
1 1 1 1 1 0 0 1 0 1 1 0
 

ECE 6640 15
 B3 Example
H2=hadamard(4)
B3=[H2;-H2]
B3*B3'
ans =
4 0 0 0 -4 0 0 0
0 4 0 0 0 -4 0 0
0 0 4 0 0 0 -4 0
0 0 0 4 0 0 0 -4
-4 0 0 0 4 0 0 0
0 -4 0 0 0 4 0 0
0 0 -4 0 0 0 4 0
0 0 0 -4 0 0 0 4
ECE 6640 16
 Biorthogonal Codes (3)

• Biorthogonal codes require half as many code bits per code

word as Hadamard.
– The required bandwidth would be one-half
– Due to some antipodal codes, they perform better
• For equally likely, equal-energy orthogonal signals, the
probability of codeword (symbol) error can be bounded as

 Es   2  Es  M  2 k 1
PE M   M  2   Q
 
  Q 

 N0   N0  Es  k  E b

 
PB M   PE M , for M  8
2

ECE 6640 17
 Biorthogonal Codes (4)

• Comparing to Hadamard to Biorthogonal

– Note that for D bits, M is ½ the size for Biorthogonal
M  2 D 1
 M  2   E s   2  Es 
Biorthogonal
PB M      Q   Q 
 M  2k
 2   N0   N0 
Es  k  E b

 M   E S  M  2k
Hadamard
PB M      Q
 2   N 0  Es  k  E b

ECE 6640 18
 Waveform Coding System Example

• For a k-bit precoded symbol, one of M generators creates

an encoded waveform.
• For a Hadamard Code, 2^k bits replace the k-bit input

ECE 6640 19
Waveform Coding System Example (2)

• This bit sequence can be PSK modulated (an antipodal

binary system) for transmission over M time periods of
length Tc. T  2 k  TC  M  TC

• The symbol rate is 1/M*Tc RS  1  1 k

T 2  TC

• For real-time transmission, the code word transmission rate

and the k-bit input data rate must be identical.
– Codeword bits shorter than original message bits
Rk  1  RS  1  1 k
 k  T
k  Tb T 2  TC k
Tb  2 C
ECE 6640
T  k  Tb  2  TC
k
20
Waveform Coding System Example (3)

• The coherent detector now works across multiple PSK bits

representing a symbol

Direct Sequence
Spread Spectrum
(DSSS) Related

ECE 6640 21
Waveform Coding System Example (4)

Stated Improvement Tb   k  T
2 k
C

• For k=5, detection can be accomplished with

about 2.9 dB less Eb/No!
– Required bandwidth is 32/5=6.4 times the initial data word rate.
• Homework problem 6.28
– Compare PB of BPSK to the PB for orthogonally encoded symbols
at a predefined BER. Solution set uses 10^-5.

ECE 6640 22
 Types of Error Control

• Error detection and retransmission

– Parity bits
– Cyclic redundancy checking
– Often used when two-way communication used. Also used when
low bit-error-rate channels are used.

• Forward Error Correction

– Redundant bits for detection and correction of bit errors
incorporated into the sequence
– Structured sequences.
– Often used for one-way, noisy channels. Also used for real-time
sequences that can not afford retransmission time.
ECE 6640 23
 Structured Sequence

• Encoded sequences with redundancy

• Types of codes
– Block Codes or Linear Block Codes (Chap. 6)
– Convolutional Codes (Chap. 7)
– Turbo Codes (Chap. 8)

ECE 6640 24
 Channel Models

• Discrete Memoryless Channel (DMC)

– A discrete input alphabet, a discrete output alphabet, and a set of
conditional probabilities of conditional outputs given the particular
input.
• Binary Symmetric Channel
– A special case of DMC where the input alphabet and output
alphabets consist of binary elements. The conditional probabilities
are symmetric p and 1-p.
– Hard decisions based on the binary output alphabet performed.
• Gaussian Channels
– A discrete input alphabet, a continuous output alphabet
– Soft decisions based on the continuous output alphabet performed.

ECE 6640 25
 Code Rate Redundancy

• For clock codes, source data is segmented into k bit data

blocks.
• The k-bits are encoded into larger n-bit blocks.
• The additional n-k bits are redundant bits, parity bits, or
check bits.
• The codes are referred to as (n,k) block codes.
– The ratio of redundant to data bits is (n-k)/k.
– The ratio of the data bits to total bits, k/n is referred to as the code
rate. Therefore a rate ½ code is double the length of the underlying
data.

ECE 6640 26
 Binary Bit Error Probability

• Defining the probability of j errors in a block of n

symbols/bits where p is the probability of an error.
n j
P j , n      p  1  p 
n j

 j
 n!  j
P j , n      p  1  p n  j
 j!n  j ! 

Binomial Probability Law

• The probability of j failures in n Bernoulli trials

E x   n  p  
E x 2  n  p  1  p 
ECE 6640 27
 Code Example Triple Redundancy

• Encode a 0 or 1 as 000 or 111

– Assume single bit errors detected and corrected
 3!  j
P j ,3     p  1  p 3 j
 j!3  j ! 
• Assume BER of p = 10-3
 3! 
P0,3     0.999   0.999   0.997
3 3

 0!3  0 ! 

 3! 
P1,3     0.001  0.999   3  0.001  0.999  0.002994
1 2 1 2

 1!3  1! 

 3! 
P2,3     0.001  0.999   3  0.001  0.999   0.000002997
2 1 2 1

 2!3  2 ! 

 3! 
P3,3     0.001  0.001  10 9
3 3

 3!3  3! 
 Code Example Triple Redundancy

• Probability of two or more errors

Pr  x  2   P2,3  P2,3  2.998 10 6  3 10 6
• Probability of one or no errors
Pr  x  2   P0,3  P1,3  1  Pr x  2   1  2.998 10 6  0.999997

• If the raw bit error rate of the environment is p=10-3 …

• The probability of detecting the correct transmitted bit
becomes 3*10-6
 Parity-Check Codes

• Simple parity – single parity check code

– Add a single bit to a defined set of bits to force the “sum of bits” to
be even (0) or odd (1).
– A rate k/k+1 code
– Useful for bit error rate detection
• Rectangular Code (also called a Product Code)
– Add row and column parity bits to a rectangular set of bits/symbols
and a row or column parity bit for the parity row or column
– For an m x n block, m+1 x n+1 bits are sent
– A rate mn/(m+1)(n+1) code
– Correct signal bit errors!

ECE 6640 30
 (4,3) Single-Parity-Code

• (4,3) even-parity code example

– 8 legal codewords, 8 “illegal” code words to detect single bit errors
– Note: an even number of bit error can not be detected, only odd
– The probability of an undetected message error
n / 2 ( n even ) n / 2 ( n even )
( n 1) / 2 ( n odd ) ( n 1) / 2 ( n odd )
 n!  2j
 P2 j, n       p  1  p n  2 j
j 1 j 1  2 j!n  2 j ! 
Message Parity Codeword
000 0 0 000
001 1 1 001
010 1 1 010
011 0 0 011
100 1 1 100
101 0 0 101
ECE 6640
110 0 0 110 31
111 1 1 111
 Matlab Results
p = 1e-3;
n=
4.00000000000000e+000
n=4
Perror =
0.00000000000000e+000
Perror = 0
jfactor =
for j=2:2:n
6.00000000000000e+000
jfactor = factorial(n)/(factorial(j)*factorial(n-j))
jPe = 2 bit errors
jPe = jfactor * p^j * (1-p)^(n-j)
5.98800600000000e-006
Perror = Perror + jPe
Perror =
end
5.98800600000000e-006
jfactor =
1.00000000000000e+000
jPe = 4 bit errors
1.00000000000000e-012
Perror =
5.98800700000000e-006

Probability of an undetected message error

ECE 6640 32
 Matlab Results

• Computing the probability of a message error

– Channel bit size the parity is added to 2 to 16
– Raw BER = 10^-3
– Compare with parity and without (n x p)
-1
Message Error Probability for BER = 10-3
10
no parity
parity
-2
10
Prob of Error Detection

-3
10

-4
10

-5
10

-6
ECE 6640 10 33
2 4 6 8 10 12 14 16
Channel Bits
 Rectangular Code

• Add row and column parity bits to a rectangular set of

bits/symbols
and a row or column parity bit for the parity row or column
• For an m x n block, m+1 x n+1 bits are sent
• A rate mn/(m+1)(n+1) code
• Correct signal bit errors!
– Identify parity error row and column, then fix the “wrong bit”

ECE 6640 34
 Rectangular Code Example

• 5 x 5 bit array  25 total bits

• 6 x 6 encoded array  36 total bits
• A (36,25) code

• Compute probability that there is an undetected error

message
– The block error or word error becomes: j errors in blocks of n
symbols and t the number of bit errors. (note t=1 corrected)
n
n j
PM       p  1  p n j

j t 1  j 

 n  t 1
PM     p  1  p n t 1
ECE 6640
 t  1 35
 Error-Correction Code Tradeoffs

• Improve message error/bit error rate performance

• Tradeoffs
– Error performance vs. bandwidth
• More bits per sec implies more bandwidth
– Power Output vs. bandwidth
• Lowering Eb/No changer BER, to get it back, encode and increase
bandwidth
• If communication need not be in real time … sending more
bits increases BER at the cost of latency!

ECE 6640 36
 Coding Gain

• For a given bit-error probability,

the relief or reduction in required Eb/No
 Eb   Eb 
CodeGaindB    dB    dB 
 No uncoded  N o  coded

Note:
Code gain exists for a given BER.
(CB @ 1e-4 or ED @ 1e-6 in the figure)
CG may be different at different BERs!

ECE 6640 37
 Data Rate vs. Bandwidth

• From previous SNR discussions

P W  1
 W  T   S   , R 
Eb PS

N 0 PN PN  R  T

Eb PS  W  PS  1 
     
N 0 N 0 W  R  N 0  R 

• The tradeoff between Eb/No and bit rate

ECE 6640 38
 Linear Block Codes

• Encodeing a “message space” k-tuple into a “code space”

n-tuple with an (n,k) linear block code
– maimize the distance between code words
– Use as efficient of an encoding as possible

– 2k message vectors are possible (for example M k-tuples to send)

– 2n message codes are possible (M n-tuples after encoding)

• Encoding methodology
– A giant table lookup
– Generators that can span the subspace to create the desired linear
independent sets can be defined
ECE 6640 39
 Linear Block Code Structure

• k-tuple encode into an n-tuple of an

n-dimensional space.
• Error probability based on distance
between selected encoded values.
• Distance to be maximized by design.

ECE 6640 40
 Generator Matrix

• We wish to define an n-tuple codeword U

U  m1  V1  m2  V2   mk  Vk

where mi are the message bits (k-tuple elements), and Vi

are k linearly independent n-tuples that span the space.
– This allows us to think of coding as a vector-matrix multiple
with m a 1 x k vector and V a k x n matrix
• If a generator matrix composed of the elements of V can be
defined
V1   v11 v12  v1n 
V  v v22  v2 n 
  
U m G m   2
 m   21

     
   
 k
V  vk 1 vk 2  vkn 
ECE 6640 41
 Table 6.1 Example
% Simple single bit parity
n = 6; k = 3; % Set codeword length and message length.
msg = [0 0 0; 0 0 1; 0 1 0; 0 1 1; ...
1 0 0; 1 0 1; 1 1 0; 1 1 1]; % Message is a binary matrix.

gen = [ 1 1 0 1 0 0; ...
0 1 1 0 1 0; ...
1 0 1 0 0 1];

code = rem(msg * gen, 2)

0 0 0 0 0 0
1 0 1 0 0 1
0 1 1 0 1 0
1 1 0 0 1 1
1 1 0 1 0 0
0 1 1 1 0 1
To Match:
1 0 1 1 1 0 swap c2 and c3
0 0 0 1 1 1
swap c4 and c6
A Systematic Code A Systematic Code with
ECE 6640 42
shifted message
 “Systematic” Linear Block Codes

• The generator preserves part of the message in the

resulting codeword
 p11 p12  p1,( n  k ) 1 0  0  v11 v12  v1n 
p 
p22  0 1  0 v21 v22  v2 n 
G  P I   
21 p2 ,( n  k )
              
  
 pk1 pk 2  pk , ( n  k ) 0 0  1 vk1 vk 2  vkn 
• P is the parity array and I regenerates the message bits
– Note the previous example uses this format
• Matlab generators
– hammgen or
– POL = cyclpoly(N, K, OPT) finds cyclic code generator polynomial(s) for a given
code word length N and message length K.
ECE 6640 43
 “Systematic” Code Vectors

• Because of the structure of the generators

G  P I k 

• The codeword becomes

U  m  G   p | m

where p are the parity bits generated for the message m

and m is the embedded message of a systematic code

ECE 6640 44
 Parity Check Matrix

• The parity check matrix allows the checking of a codeword

– It should be trivial if there are no errors …
but if there are?! Syndrome testing


H  I nk PT 
I 
U  H T  m  P | I k  H T  m  P | I k   n  k   0 mr ,n  k
 P 

ECE 6640 45
 Parity Check Matrix

• Let the message be I

I 
U  H T  I k  P | I k  H T  P | I k   n  k   P  P   0
 P 

• The parity check matrix results in a zero vector, representing the

“combination” of the parity bits.

[parmat,genmat] = hammgen(3)
rem(genmat * parmat', 2)

[parmat,genmat] = cyclgen(7,cyclpoly(7,4))
rem(genmat * parmat', 2)

ECE 6640 46
 Syndrome Testing

• Describing the received n-tuples of U

r U e
– where e are potential errors in the received matrix

• As one might expect, use the check matrix to generate a

the syndrome of r
S  r  H T  U  e   H T  U  H T  e  H T

• but from the check matrix

S  eHT

– if there are no bit errors, the result is a zero matrix!

ECE 6640 47
 Syndrome Testing

• If there are no bit error, the Syndrome results in a 0 matrix

• If there are errors, the requirement of a linear block code is
to have a one to one mapping between correctable errors
and non-zero syndrome results.

• Error correction requires the identification of the

corresponding syndrome for each of the possible errors.
– generate a 2^n x n array representing all possible received n-tuples
– this is called the standard array

ECE 6640 48
 Standard Array Format (n,k)

• The actual codewords are placed in the top row.

– The 1st code word is an all zeros codeword.
– It also defines the 1st coset that has zero errors
• Each row is described as a coset with a coset leader
describing a particular error.
– for an n-tuple, there will be n-k coset leaders, one of which is zero
errors.
ECE 6640 49
 Figure 6.11

ECE 6640 50
 Decoding with the Standard Array

• The n-tuple received can be located somewhere in the

standard array.
– If there is an error, the corrupted codeword is replaced by the
codeword at the top of the column.
– The standard array contains all possible n-tuples, it is 2^(n-k) x 2^k
in size; therefore, all n-tuples are represented.
– There are 2^(n-k) cosets

ECE 6640 51
 The Syndrome of a Coset

• Computing the syndrome of the jth coset for the ith

codeword
S  U i  e j  H T  U i  H T  e j  H T  e j  H T

– the syndrome is identical for the entire coset

– Thus, coset is really a name defined for
“a set of numbers having a common feature”
– The syndrome uniquely defines the error pattern

ECE 6640 52
 Error Correct

• Now that we know that the syndrome identifies a coset, we

can identify the coset and perform error correction
regardless of the n-tuple transmitted.
• The procedure
1. Calculate the Syndrome of r
2. Locate the coset leader whose syd4om is identified
3. This is the assumed corruption of a valid n-tuple
4. Form the corrected codeword by “adding” the coset to the
corrupted codeword n-tuple.

ECE 6640 53
 Error Correction Example

• see Table61_example
– each step of the process is shown in Matlab for the (6,3) code of
the text

ECE 6640 54
 Decoder Implementation

ECE 6640 55
 Hamming Weight and Distance

• The Hamming weight of a code/codeword defines the

performance
• The Hamming weight is defined as the number of nonzero
elements in U.
– for binary, count the number of ones.
• The Hamming difference between two codewords, d(U,V),
is defined as the number of elements that differ.
Ucode = w(U)= d(U0,U)
0 0 0 0 0 0 0
1 0 1 0 0 1 3
0 1 1 0 1 0 3
1 1 0 0 1 1 4
1 1 0 1 0 0 3
0 1 1 1 0 1 4
1 0 1 1 1 0 4
0 0 0 1 1 1 3
ECE 6640 56
 Minimum Distance

• The minimum Hamming distance is of interest

– dmin = 3 for the (6,3) example

• Similar to distances between symbols for symbol

constellations, the Hamming distance between codewords
defines a code effective performance in detecting bit errors
and correct them.
– maximizing dmin for a set number of redundant bits is desired.
– error-correction capability
– error-detection capability

ECE 6640 57
 Hamming Distance Capability

• Error-correcting capability, t bits

 d  1
t   min 
 2 
– for (6,3) dmin=3: t = 1

– Codes that correct all possible sequences of t or fewer errors may

also be able to correct for some t+1 errors.
• see the last coset of (6,3) – 2 bit errors!
– a t-error-correcting (n,k) linear code is capable of correcting 2(n-k)
error patterns!
– an upper bound on the proabaility of message error is
n
n j
PM       p  1  p n j

j t 1  j 
ECE 6640 58
 Hamming Distance Capability

• Error-detection capability
e  d min  1
– a block code with a minimum distance of dmin guarantees that all
error patterns of dmin-1 or fewer error can be detected.
– for (6,3) dmin=3: e = 2 bits

– The code may also be capable of detecting errors of dmin bits.

– An (n,k) code is capable of detecting 2n-2k error patterns of length n.
• there are 2k-1 error patterns that turn one codeword into another and
are thereby undetectable.
• all other pattersn should produce non-zero syndrom
• therefore we have the desired number of detectable error patterns
• (6,3)  64-8=56 detectable error patterns
ECE 6640 59
 Codeword Weight Distribution

• The weight distribution involves the number of codewords

with any particular Hamming weight
– for (6,3) 4 have dmin=3, 3 have d = 4
– If the code is used only for error detection, on a BSC, the
probability that the decoder does not detect an error can be
computed from the weight distribution of the code.
n
Pnd   A j  p j  1  p 
n j

j 1
– for (6,3) A(0)=1, A(3)=4, A(4)=3, all else A(i)=0
Pnd  4  p 3  1  p   3  p 4  1  p 
3 2

 4  1  p   3  p   p 3  1  p 
2

 4  p   p 3  1  p 
2

ECE 6640 60
– for p=0.001: Pnd ~ 3.9 x 10-6
 Simultaneous Error
Correction and Detection
• It is possible to trade capability from the maximum
guaranteed (t) for the ability to simultaneously detect a
class of errors.
• A code can be used for simultaneous correction of alpha
errors and detection of beta errors for beta>=alpha
provided

d min      1

ECE 6640 61
 Visualization of (6,3) Codeword
Spaces

ECE 6640 62
 MATLAB References
• see http://www.mathworks.com/help/comm/error-detection-and-
correction.html
– http://www.mathworks.com/help/comm/block-coding.html
– http://www.mathworks.com/help/comm/linear-block-codes.html
– http://www.mathworks.com/help/comm/bch-codes.html

ECE 6640 63
 (8,2) Block Code

see:
BlockCode82.m

A general script for

clock codes

ECE 6640 64
 Block Codes

• Now that you have seen how to work with block codes …

• How do we select or generate a code?

– from p. 348
– “We would like for the space to be filled with as many codewords
as possible (efficient utilization of the added redundancy), and we
would like the codewords to be as far from one another as possible.
Obviously, these goals conflict.”

ECE 6640 65
 Code Design: Hamming Bound

• The Hamming Bound is describes:

– the number of parity bits needed
 n n  n 
n  k  log 2 1           
 1  2  t 
– the number of cosets is then

nk
 n n  n 
2  1           
 1  2  t 
n-k
– Note that fewer cosets than 2 may be used, for example cosets
corresponding on 1 to t errors and not “additional” slots available
for t+1 errors. (LinBlkCodeTestXX.m examples)

ECE 6640 66
 Defining Code Length

• t described the bit corrections

d min  2  t  1
• Minimum non-trivial k = 2
– if selected an (n,2) code is needed
• Allow t=2   n   n 
– we will need n  k  log 2 1      
  1   2 
– Hamming bound might suggest n=7 (standard array size of 4 x 29 < 2^7
– But for small codes, the Plotkin bound must also be met

n  2 k 1
d min  k
2 1
ECE 6640 67
 Defining Code Length

• Using the Plotkin bound for n=7

n  2 k 1 7  2 14 2
d min  k   4
2 1 4 1 3 3
– But we need 5 or more …
• Therefore n=8 is the smallest possible code
n  2 k 1 8  2 16 1
d min  k   5
2 1 4 1 3 3
– as we saw, the (8,2) code corrects 1, 2 and some 3 bit errors

ECE 6640 68
 MATLAB Examples

• LinBlkCodeTestXX.m
– (3,1), (6,3), (7,4), and (8,2)
– code in linblkcoderxXX.m explicitly defines coset matrix “E”,
– but only 0:t bit error conditions are defined, there are coset errors
conditions that may not be included
(no extras for (3,1) or (7,4) … can you tell me why?)
• (3,1) t=1 … 3 single bit errors 2n-k = 4 cosets available
• (6,3) t=1 … 6 single bit errors 2n-k = 8 cosets available
• (7,4) t=1 … 7 single bit errors 2n-k = 8 cosets available
n-k
• (8,2) t=2 … 8 single bit and 28 two bit errors 2 = 64 cosets
• (n,k) t=z … n single bit errors … 2n-k cosets available

ECE 6640 69
 Coset Comments

• There is a syndrome created by the cosets

• The previous MATLAB examples defined the cosets and
performed an exhaustive search of the syndrome table
created from the coset to determine the error.

• However, the order of the coset is really arbitrary!

(see linblkcoderx31.m you can switch the order!)
• The syndrome has 2n-k possible values …
• Why not order the cosets so that the syndrome is a perfect
counting sequence from 0 to 2n-k-1 in binary?
– This is what MATLAB’s syndtable.m does!
ECE 6640
– The exhaustive table search is no longer necessary. 70
 Coset defined by syndtable

• When the coset errors are reorder so that the syndrome

table is a binary counting sequence,
• The coset used to correct for errors is simply the row
defined by the binary to decimal conversion of the
syndrome.
– 0 is still no errors and all zeros
– all others are
correction_code = coset(binary2decimal(syndrome),:);
• See LBC_Test.m
• Also BlockCode63.m, BlockCode63b.m, and
BlockCode82.m
– look at the cosets for the additional bit error patterns corrected!
ECE 6640 71
 What About BER

• See ChCode_MPSK_BJB
– uses linblkcoderxXX.m to decode!
– code the simulation do better?
– If the coset is only partially filled, yes … otherwise, no.

ECE 6640 72
 Cyclic Codes

• An important subclass of linear clock codes

• Easily implemented using a feedback shift register
– all cyclic shifts of a codeword form another codeword
• The components of the codeword can be treated like a
polynomial
U  X   u0  u1  X  u2  X 2    un 1  X n 1

ECE 6640 73
 Algebraic Structure of Cyclic Codes

• Based on
U  X   u0  u1  X  u2  X 2    un 1  X n 1
– U(X), an (n-1) degree polynomial code word with the property
X i U  X  U i   X 
 q X   n
X 1n
X 1
X i U  X   q X   X n  1  U i   X 

– where the final term is the remainder of the division process and it
is also a codeword
– The remainder can also be described in terms of modulo arithmetic
as
U i   X   X i U  X moduloX n  1

ECE 6640 74
 Example
U  X   u0  u1  X  u2  X 2    un 1  X n 1

• Multiplying a codeword by X and adding “two” un-1which

is equal to zero in a binary sense
X U  X   un 1  u0  X  u1  X 2  u2  X 2    un  2  X n 1  un 1  X n  un 1


X U  X   un 1  u0  X  u1  X 2  u2  X 2    un  2  X n 1  un 1  X n  1 
• The final term module Xn+1 results in a new codeword

X U  X   un 1  u0  X  u1  X 2  u2  X 2    un  2  X n 1

• Repeat for the rest of the codewords ….

ECE 6640 75
 Binary Cyclic Codes

• We can generate a cyclic code using a generator

polynomial in much the same way that we generated a
block code using a generator matrix.
– The generator polynomial g(x) for an (n,k) cyclic code is unique
and is of the form (with the requirement that g0=1 and gp=1)
g  X   g 0  g1  X  g 2  X 2    g p  X p
– Every codeword in the space is of the form
U  X   m X   g  X 
– with the message polynomial
m X   m0  m1  X  m2  X 2    mn  p 1  X n  p 1
– such that k-1=n-p-1 or the number of parity bits is

p  nk
ECE 6640 76
 Binary Cyclic Codes

• U is said to be a valid codeword of the subspace S if, and

only if, g(X) divides into U(X) without a remainder.
U  X   m X   g  X 
U i   X 
 m X 
gX 
• A generator polynomila g(X) of an (n,k) cyclic code is a
factor of Xn+1. (A result of the codewords being cyclic).
• Therefore some example generators are
  
X 7 1  X 3  X 1  X 4  X 2  X 1
– g(X) for (7,3) with p=3
– g(X) for (7,4) with p=4
ECE 6640 77
 Systematic Encoding

• Output message and feed message to LFSR

• When message done, the parity bits are in the encoder and
can be output.
• Initialize the encoder and restart for the next message.
• Syndrome computation is performed in a similar manner.
ECE 6640 78
 Cyclic Block Code

• With proper manipulation, the cyclic block codes can use

linear combinations of codes to recreate the generator and
parity matrices of block codes.

• Then processing could also be performed as previously

shown.

ECE 6640 79
 Well-Known Block Codes

• Hamming Codes
– minimum distance of 3, single error correction codes.
– known as perfect codes, codes where the standard array has all
patterns of t or fewer errors and no others as coset leaders (that is
no residual error-correcting capacity)
• Extended Golay Codes
• Bose-Chadhuri-Hocquenghem (BCH) Codes
– powerful class of cyclic codes
– generalization of Hamming codes that allow multiple error
correction

ECE 6640 80
 Hamming Codes

• Hamming codes are characterized by a single integer m

that creates codes of
n, k   2m  1,2m  1  m 
– (3,1) , (7,4), (15,11), (31,26), etc.
– dmin = 3, t=1, e=2 … a perfect code

(n-k) m
– 2 = 2 therefore 1 all zeros coset leader
m
and 2 -1=n single bit errors. Always a perfect code!

ECE 6640 81
 Code Generation Algorithm

• See Wikipedia: http://en.wikipedia.org/wiki/Hamming_code

• In MATLAB see the function hamgen
– Produce parity-check and generator matrices for Hamming code.
– H = hammgen(M) produces the parity-check matrix H for a given integer M,
M >= 3. The code length of a Hamming code is N=2^M-1. The message length is
K = 2^M - M - 1. The parity-check matrix is an M-by-N matrix.
– [H, G, N, K] = hammgen(...) produces the parity-check matrix H as well as the
generator matrix G, and provides the codeword length N and the message length K.

ECE 6640 82
 MATLAB hamgen
[H, G, N, K] = hammgen(3)
H=
1 0 0 1 0 1 1
0 1 0 1 1 1 0
0 0 1 0 1 1 1
G=
1 1 0 1 0 0 0
0 1 1 0 1 0 0
1 1 1 0 0 1 0
1 0 1 0 0 0 1
N=
7
K=
4

ECE 6640 83
 Performance Improvement

• If we assume BPSK transmission with coherent detection.

• The channel symbol error and bit error probability of
n
n j 1 n n j
PS      p  1  p  PB       p  1  p 
n j n j

j t 1  j  n j t 1  j 

• For performance over a Gaussian channel we have

 2 Ec  Ec k Eb
 
p  Q 
 N0 n N0
 N0 

– with the results plotted in on Figure 6.22 and the following slide.

ECE 6640 84
 Comparative Code Performance
Figure 6.22

ECE 6640 85
 Extended Golay Code

• The binary (24,12) code generated from the perfect (23,12)

Golay code.
– A rate ½ code. dmin = 8
– Wikipedia: http://en.wikipedia.org/wiki/Binary_Golay_code
– Correct 3 bit errors, detect 7 bit errors

1 24  24  j
     p  1  p 
24  j
PB 
24 j  4  j 

ECE 6640 86
 BCH Codes

• Bose-Chadhuri-Hocquenghem (BCH)
• From Wikipedia: http://en.wikipedia.org/wiki/BCH_code
– One of the key features of BCH codes is that during code design,
there is a precise control over the number of symbol errors
correctable by the code. In particular, it is possible to design binary
BCH codes that can correct multiple bit errors.
– BCH codes are used in applications such as satellite
communications,[2] compact disc players, DVDs, disk drives,
solid-state drives[3] and two-dimensional bar codes.

• Table 6.4 provides generators of primitive BCH Codes

based on the desired (n,k) and t

ECE 6640 87
 BCH Code Performance
Figure 6.23

ECE 6640 88
 Matlab Block Code Examples

• ChCode_MPSK_BJB
– BPSK, QPSK or 8-PSK Time Symbols at a fixed 1 Mbps channel
symbol bit rate (1 Msps for BPSK, ½ Msps for QPSK, and 1/3
Msps for 8-PSK)
– Linear Block Channel Encoding
• (3,1) with t=1, (6,3) with t=1, (7,4) with t=1, and (8,2) with t=2
– Comparison curve based on symbol bit error rate versus corrected
bit-error rates for the data stream

ECE 6640 89
 BPSK Simulation Results

(3,1) t=1, rate 1/3 (6,3) t=1, rate 1/2

0
Bit Error Rate, M=2 0
Bit Error Rate, M=2
10 10

-1 -1
10 10

-2 -2
10 10

-3 -3
10 10
BER

BER
-4 -4
10 10

-5 -5
10 10

MPSK Simulation MPSK Simulation

-6 -6
10 Encoded MPSK Simulation 10 Encoded MPSK Simulation
MPSK Theory MPSK Theory
Encoded MPSK Theory Encoded MPSK Theory
-7 -7
10 10
-8 -6 -4 -2 0 2 4 6 8 10 12 -8 -6 -4 -2 0 2 4 6 8 10 12
E b/N0 (dB) E b/N0 (dB)

ECE 6640 90
 BPSK Simulation Results (2)

(7,4) t=1, rate 4/7 (8,2) t=2, rate 1/4

0
Bit Error Rate, M=2 0
Bit Error Rate, M=2
10 10

-1
10 10
-1

-2
10 -2
10
-3
10
-3
10
-4
BER

BER
10
-4
10
-5
10
-5
-6 10
10
MPSK Simulation MPSK Simulation
-6
-7 Encoded MPSK Simulation 10 Encoded MPSK Simulation
10
MPSK Theory MPSK Theory
Encoded MPSK Theory Encoded MPSK Theory
-8 -7
10 10
-6 -4 -2 0 2 4 6 8 10 12 -10 -5 0 5 10 15
E b/N0 (dB) E b/N0 (dB)

ECE 6640 91