Information Theory and Coding: Linear Block Codes

Dr Jacqueline John George
Information Theory and Coding

Linear Block Codes
Lecture 3
Linear Block Code

The generation and detection of error correcting codes
make extensive use of Modulo-2 mathematics:

0 + 0 = 0
0 + 1 = 1
1 + 0 = 1
1 + 1 = 0

which is effectively the same as Exclusive-OR function

Note: + indicates addition, not Logical OR
Naming conventions

Lower case bold to denote a vector:

v = n x 1 vector, i.e. [v
0
, v
1
, v
2
, , v
n-1
]

Where v
i
= 0 or 1 for 0 i n

Upper case bold to denote a matrix:
G is a k x n
matrix:
(
(
(
(
(
(
(
(
=
(
(
(
(
(
(
(
(
1 1,n k 1,2 k 1,1 k 1,0 k

1 1,n 1,2 1,1 1,0
1 0,n 0,2 0,1 0,0
g . . . g g g
. . . . . . .
. . . . . . .
. . . . . . .
g . . . g g g
g . . . g g g
1 k
1
0
g
.
.
.
g
g
G
Example

Consider the following (7, 4) error correcting code v with
message bits v6, v5, v4, v3 and redundant parity check
bits v2, v1, v0 defined by the following (modulo 2 sums)
equations:

v
2
= v
4
+ v
5
+ v
6
v
1
= v
3
+ v
4
+ v
5
v
0
= v
3
+ v
5
+v
6

Form the codewords, what is the error detection and error
correction
Solution

Messages
Code
words
[v
0
- v
6
]
0000 0000000
1000 1101000
0100 0110100
1100 1011100
0010 1110010
1010 0011010
0110 1000110
1110 0101110

Messages
Code
words
[v
0
- v
6
]
0001 1010001
1001 0111001
0101 1100101
1101 0001101
0011 0100011
1011 1001011
0111 0010111
1111 1111111
Hamming distance = 3
Error detection capability = 2
Error correction capability = 1
Defining a Linear Block Code

What defines a linear block code?

A block code is linear if the modulo-2 sum of two code
words is also a code word


Messages
Code
words
[v
0
- v
6
]
0000 0000000
1000 1101000
0100 0110100
1100 1011100
0010 1110010
1010 0011010
0110 1000110
1110 0101110

Messages
Code
words
[v
0
- v
6
]
0001 1010001
1001 0111001
0101 1100101
1101 0001101
0011 0100011
1011 1001011
0111 0010111
1111 1111111
1011100
+ 1110010
0101110

What defines a linear block code?

A block code is linear if and only if the modulo-2 sum
of two code words is also a code word

What properties do linear block codes possess?

Simple mathematical structure
Simple structure can be exploited in design of real-
world encoders/decoders
Best described using a Generator Matrix (Encoder)
and a Parity check matrix (Decoder)
The Generator Matrix

For an (n, k) linear block code C it is possible to find k linearly
independent code words, g
0
, g
1
, . . ., g
k-1
such that every codeword
v in C is a linear combination of these k codewords. i.e.

v = u
0
g
0
+ u
1
g
1
+ . . . . + u
k-1
g
k-1
where u
i
= 0 or 1 for 0 i k

The codewords g
0
- g
k-1
can be represented by a k x n matrix
(
(
(
(
(
(
(
(
=
(
(
(
(
(
(
(
(
1 1,n k 1,2 k 1,1 k 1,0 k

1 1,n 1,2 1,1 1,0
1 0,n 0,2 0,1 0,0
g . . . g g g
. . . . . . .
. . . . . . .
. . . . . . .
g . . . g g g
g . . . g g g
1 k
1
0
g
.
.
.
g
g
G
G is called the Generator Matrix
n columns
k rows
Using the generator matrix to encode messages

If u = [u
0
, u
1
, . . ., u
k-1
] is the message to be encoded. The
corresponding codeword can be written as:
v = u.G

| |
(
(
(
(
(
(
(
(
1 k
1
0
g
.
.
.
g
g
v
1 k 1 0
u . . . u u
v = [ u
0
g
0
+ u
1
g
1
+ u
2
g
2
+ . . . + u
k-1
g
k-1
]
Generator matrix - notes

An (n, k) linear code is completely specified by the k rows
of its generator matrix G
It is not necessary to use all n rows -
Any k linearly independent code words can be used to
form G
Care must be taken to ensure linear independence
(code words must not be multiples of each other)
An encoder for such a code needs only to store the k
rows of the generator matrix G and to form linear
combinations of these k rows, based on the input
message u, to be able to produce a valid codeword v
Example

Given the following generator matrix encode the following message
u = (1 1 0 1)
.
(
(
(
(
=
(
(
(
(
=
1 0 0 0 1 0 1
0 1 0 0 1 1 1
0 0 1 0 1 1 0
0 0 0 1 0 1 1
3
2
1
0
g
g
g
g
G

If u = [ 1 1 0 1 ] then:
v = u.G
= 1.g
0
+ 1.g
1
+ 0.g
2
+ 1.g
3

= [ 1 1 0 1 0 0 0 ] + [ 0 1 1 0 1 0 0 ] + [ 1 0 1 0 0 0 1 ]
= [ 0 0 0 1 1 0 1 ]

Systematic codes
Systematic codes

Definition: Codes in which the codeword is divided into two
parts:
the message, and
the redundant checking part

For an (n, k) linear systematic code the code consists of k
information (or message) digits and n-k parity check digits
which are linear sums of the information digits.
Message Redundant checking bits
Valid codeword
n - k digits k digits
Simplifying the generator matrix
(
(
(
(
=
(
(
(
(
=
1 0 0 0 1 0 1
0 1 0 0 1 1 1
0 0 1 0 1 1 0
0 0 0 1 0 1 1
3
2
1
0
g
g
g
g
G
A linear systematic (n,k) code can be defined by a k x n matrix G of the
form:
(
(
(
(
(
(
=
(
(
(
(
(
(
1 . . 0 0 p . . . p p p
. . . . . . . . . . . .
. . . . . . . . . . . .
0 . . 1 0 p . . . p p p
0 . . 0 1 p . . . p p p
1 k 1,n k 1,2 k 1,1 k 1,0 k
1 k 1,n 1,2 1,1 1,0
1 k 0,n 0,2 0,1 0,0
1 k
1
0
g
g
g
G
.
.
If I
k
= k x k unity matrix then G = [ P I
k
]
Simplifying the generator matrix
If I
k
= k x k unity matrix then G = [ P I
k
]
and if u = [u
0
, u
1
, . . . ., u
k-1
]
then v = [v
1
, v
2
, . . . v
n-1
]
= [u
0
, u
1
, . . . u
k-1
] . G

therefore the components of v are:
v
n-k+i
= u
i
for 0 i k
and v
j
= u
0
p
0j
+ u
1
p
1j
+ . . . +u
k-1
p
k-1,j
for 0 j n-k

The n-k equations for v
j
are called the
parity check equations for the code
Impelmenting the Encoder
Encoding linear block codes generic form
p
0,0
p
1,0
p
k-1,0

p
0,
n-k-1

p
1,
n-k-1

p
k-1,
n-k-1

p
0,1
p
1,1
p
k-1,1

+ + +
u
0
u
1
u
k-1
u
0
u
1
u
k-1
u
0
u
1
u
k-1
v
0
v
1
v
n-k-1
To
channel

u
0
u
2
u
1
u
k-1
To
channel

Message register

Input u

Encoding circuit for previous example
+ + +
v
0
v
1
v
2
v
3
v
5
v
4
v
6
To
channel

Message register

Input u

v
0
= v
3
+ v
5
+v
6
v
1
= v
3
+ v
4
+v
5
v
2
= v
4
+ v
5
+v
6
[1101]

1
0
0 0
1
1
1 1
1
0 0 0
0 0 1
1
1 1 1 1
The parity check matrix
Parity check matrix
The mathematical properties of the generator matrix G
can be used to develop an alternative definition of a
linear code based on H, the parity check matrix.

For an n x k generator matrix of the form G there exists
an (n-k) x k matrix H such that:

G . H
T
= 0

Furthermore, it can be shown that an n-tuple
v = [v
0
, v
1
, . . . v
n-1
]
is a code word generated by G, if and only if
v . H
T
= 0
How do I generate the Parity Check Matrix?

For the (n x k) generator matrix G of the previous
example this parity check matrix has the form:
H = [I
n-k
P
T
]
(
(
(
(
(
(
1 k 1,n k 1 k 2,n 1 k 1,n 1 k 0,n

1,1 k 2,1 1,1 0,1
1,0 k 2,0 1,0 0,0
p . . . p p p . .
. . . . . . . . . . .
. . . . . . . . . . . .
p . . . p p p . .
p . . . p p p . .
1 0 0
.
0 1 0
0 0 1
H
(
(
(
=
1 1 1 0 1 0 0
0 1 1 1 0 1 0
1 1 0 1 0 0 1
H
Step 1: For an (n,k) code, generate a blank n-k x k matrix Step 3: Transpose rows of G into columns in H Step 2: Insert a n-k identity matrix in the left hand side of the matrix
How do I generate the Parity Check Matrix?
(
(
(
(
=
(
(
(
(
=
1 0 0 0 1 0 1
0 1 0 0 1 1 1
0 0 1 0 1 1 0
0 0 0 1 0 1 1
3
2
1
0
g
g
g
g
G
(
(
(
=
1 0 0
0 1 0
0 0 1
H
(
(
(
= H
Decoding linear
block codes
r = v + e
Parity check matrix practical implications
G
H
T
Encoder
Decoder
u
v = u.G

r is a valid codeword if r.H
T
= 0
v = u.[PI
k
]
e
r
Syndrome

Define a new vector s, such that s = r . H
T

From the definition of the parity check matrix H:

if s = 0 then r is a codeword
if s 0 then r is not a codeword

s is called the syndrome of the code r
Properties of the syndrome
The syndrome can not just tell us whether or not r is a
codeword. If there is an error, it can also give an
indication of where the error might occur:

s = r . H
T
= (v + e) . H
T
= v . H
T
+ e . H
T

But v.H
T
= 0

Therefore s = e.H
T

The syndrome of r is dependent only on the error
pattern e and not on the codeword v
Using the syndrome to perform error correction
Expanding s = e.H
T
gives n-k linear equations:

s
0
= e
0
+ e
n-k
p
0,0
+ e
n-k+1
p
1,0
+ . . + e
n-1
p
k-1,0
s
1
= e
1
+ e
n-k
p
0,1
+ e
n-k+1
p
1,1
+ . . + e
n-1
p
k-1,1
.
.
s
n-k-1
= e
n-k-1
+ e
n-k
p
0,n-k-1
+ e
n-k+1
p
1,n-k-1
+ . . + e
n-1
p
k-1,n-k-1

The syndrome digits are simply linear combinations of
the error digits and can therefore provide information
that can be used for error correction
A solution to these equations shown on the previous slide
provides a value for e, the error vector.

Since e and r are known and v = r + e

It is therefore possible to calculate v

But
There are 2
k
possible solutions to this set of linear
equations
This means that there are 2
k
error patterns with the same
syndrome

The error correction circuit must choose which vector is
the true error vector.

To minimise the probability of a decoding error the most
probable error is used

For a BSC the most probable error is the one with the
least number of non-zero digits - Maximum Likelihood

If r contains errors but r.H
T
= 0. The presence of errors is
not detected and a decoding error occurs. Such errors
are called undetectable errors

Example - Error correction using a
syndrome calculation
Syndrome Error Correction Example

If the transmitted code word v= [1 0 0 1 0 1 1]
the received vector r= [1 0 0 1 0 0 1] and
the generator matrix G=

Find the error vector e?

(
(
(
(
=
(
(
(
(
=
1 0 0 0 1 0 1
0 1 0 0 1 1 1
0 0 1 0 1 1 0
0 0 0 1 0 1 1
3
2
1
0
g
g
g
g
G
Solution
To calculate the syndromes of r
S = r . H
T

Calculating the syndromes of r gives

S = r . H
T
= [1 1 1]
S = e . H
T

The error digits are related to the syndrome digits by:
1 = e
0
+ e
3
+ e
5
+ e
6
1 = e
1
+ e
3
+ e
4
+ e
5
1 = e
2
+ e
4
+ e
5
+ e
6

Syndrome error correction example

There are 2
4
= 16 patterns which satisfy the simultaneous
equations:
[0 0 0 0 0 1 0], [1 0 1 0 0 1 1],
[1 1 0 1 0 1 0], [0 1 1 1 0 1 1],
[0 1 1 0 1 1 0], [1 1 0 0 1 1 1],
[1 0 1 1 1 1 0], [0 0 0 1 1 1 1],
[1 1 1 0 0 0 0], [0 1 0 0 0 0 1],
[0 0 1 1 0 0 0], [1 0 0 1 0 0 1],
[1 0 0 0 1 0 0], [0 0 1 0 1 0 1],
[0 1 0 1 1 0 0], [1 1 1 1 1 0 1],
Applying Maximum Likelihood to a BSC means that we
assume the error is that with the least number of non-
zero elements.
Syndrome error correction example

The error vector e = [0 0 0 0 0 1 0] has the smallest number
of nonzero components, therefore

v* = r + e
= [1 0 0 1 0 0 1]
+ [0 0 0 0 0 1 0]
= [1 0 0 1 0 1 1]

c.f. v = [1 0 0 1 0 1 1]

Information Theory and Coding: Linear Block Codes

Uploaded by

Copyright:

Available Formats

Information Theory and Coding: Linear Block Codes

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Information Theory and Coding: Linear Block Codes

Uploaded by

Copyright:

Available Formats

Dr Jacqueline John George

Information Theory and Coding

1 1,n k 1,2 k 1,1 k 1,0 k

1 1,n k 1,2 k 1,1 k 1,0 k

1 k 1,n k 1 k 2,n 1 k 1,n 1 k 0,n

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.