College of Engineering, Department of Mechanical Engineering

Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

Linear Algebraic Equations

The general form of a system of linear equations:
𝒂𝟏𝟏 𝒙𝟏 + 𝒂𝟏𝟐 𝒙𝟐 + 𝒂𝟏𝟑 𝒙𝟑 + ⋯ + 𝒂𝟏𝒏 𝒙𝒏 = 𝑪𝟏
𝒂𝟐𝟏 𝒙𝟏 + 𝒂𝟐𝟐 𝒙𝟐 + 𝒂𝟐𝟑 𝒙𝟑 + ⋯ + 𝒂𝟐𝒏 𝒙𝒏 = 𝑪𝟐
𝒂𝟑𝟏 𝒙𝟏 + 𝒂𝟑𝟐 𝒙𝟐 + 𝒂𝟑𝟑 𝒙𝟑 + ⋯ + 𝒂𝟑𝒏 𝒙𝒏 = 𝑪𝟑
.
.
.
𝒂𝒏𝟏 𝒙𝟏 + 𝒂𝒏𝟐 𝒙𝟐 + 𝒂𝒏𝟑 𝒙𝟑 + ⋯ + 𝒂𝒏𝒏 𝒙𝒏 = 𝑪𝒏

Where,
𝒂𝒊𝒋 are constant coefficients (𝒊 = 𝟏, 𝟐, 𝟑, … , 𝒏, 𝒋 = 𝟏, 𝟐, 𝟑, … , 𝒏)
𝑪𝒊 are the right hand side constants (𝒊 = 𝟏, 𝟐, 𝟑, … , 𝒏)

 Above system of equations is made up of n equations with n unknowns.

 For the above system of equations to have a solution, the n equations
must be linearly independent.

In matrix form, the above system of equations can be re-written as:

𝒂𝟏𝟏 𝒂𝟏𝟐 𝒂𝟏𝟑 𝒂𝟏𝟒 . . 𝒂𝟏𝒏 𝒙𝟏 𝑪𝟏

𝒂𝟐𝟏 𝒂𝟐𝟐 𝒂𝟐𝟑 𝒂𝟐𝟒 . . 𝒂𝟐𝒏 𝒙𝟐 𝑪𝟐
𝒂𝟑𝟏 𝒂𝟑𝟐 𝒂𝟑𝟑 𝒂𝟑𝟒 . . 𝒂𝟑𝒏 𝒙𝟑 𝑪𝟑
. . . . . . . . =
.
. . . . . . . . .
[𝒂𝒏𝟏 𝒂𝒏𝟐 𝒂𝒏𝟑 𝒂𝒏𝟒 . . 𝒂𝒏𝒏 ] [𝒙𝒏 ] [𝑪𝒏 ]

We can write above equation as:

𝑨. 𝒙 = 𝑪
Where,
𝒂𝟏𝟏 𝒂𝟏𝟐 𝒂𝟏𝟑 𝒂𝟏𝟒 . . 𝒂𝟏𝒏
𝒂𝟐𝟏 𝒂𝟐𝟐 𝒂𝟐𝟑 𝒂𝟐𝟒 . . 𝒂𝟐𝒏
𝒂𝟑𝟏 𝒂𝟑𝟐 𝒂𝟑𝟑 𝒂𝟑𝟒 . . 𝒂𝟑𝒏
𝑨= . . . . . . . ≡ 𝑻𝒉𝒆 𝒄𝒐𝒏𝒔𝒕𝒂𝒏𝒕 𝒄𝒐𝒆𝒇𝒇𝒊𝒄𝒊𝒆𝒏𝒕 𝒎𝒂𝒕𝒓𝒊𝒙
. . . . . . .
[𝒂𝒏𝟏 𝒂𝒏𝟐 𝒂𝒏𝟑 𝒂𝒏𝟒 . . 𝒂𝒏𝒏 ]

Page 1 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

𝒙𝟏
𝒙𝟐
𝒙𝟑
𝒙 = . ≡ 𝑽𝒆𝒄𝒕𝒐𝒓 𝒐𝒇 𝒖𝒏𝒌𝒏𝒐𝒘𝒏𝒔
.
[𝒙𝒏 ]

𝑪𝟏
𝑪𝟐
𝑪
𝑪 = 𝟑 ≡ 𝑹𝒊𝒈𝒉𝒕 − 𝒉𝒂𝒏𝒅 𝒔𝒊𝒅𝒆 𝒗𝒆𝒄𝒕𝒐𝒓 𝒐𝒇 𝒄𝒐𝒏𝒔𝒕𝒂𝒏𝒕𝒔
.
.
[𝑪𝒏 ]

Solution Methods:

1. Elimination Methods
2. Iteration Methods
3. Method of determinants
4. Matrix Inversion Method

Elimination Methods
 Elimination procedure: Performing a set of operations on a
system of equations until each equation is reduced to an identity
for a single unknown.
 Operations to be performed must be “permissible”; mathematical
operations that can be performed on a system of equations
without altering the solution.

Permissible operations:
1. Solution of a set of equations does not change if the sequence of
equations is changed.
Example 1:
𝒂 𝟏 𝒙𝟏 + 𝒂 𝟐 𝒙𝟐 = 𝒂 𝟑 𝒃 𝒙 + 𝒃 𝟐 𝒙𝟐 = 𝒃𝟑
{ } is same as { 𝟏 𝟏 }
𝒃𝟏 𝒙𝟏 + 𝒃𝟐 𝒙 𝟐 = 𝒃𝟑 𝒂 𝟏 𝒙𝟏 + 𝒂 𝟐 𝒙𝟐 = 𝒂 𝟑
Page 2 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

2. Any equation in the set can be multiplied or divided by a non-

zero constant without changing the solution
Example 2:
Multiply 1st equation above by 3, 
System of equations become:
𝟑𝒂𝟏 𝒙𝟏 + 𝟑𝒂𝟐 𝒙𝟐 = 𝟑𝒂𝟑
{ }  Same solution as above.
𝒃𝟏 𝒙𝟏 + 𝒃 𝟐 𝒙𝟐 = 𝒃𝟑
3. We can add two equations in a system and use the resulting
equation to replace either of the two original equations.
Example 3:
𝟐𝒙𝟏 + 𝟑𝒙𝟐 = 𝟏
{ }  Solution is: 𝒙𝟏 = −𝟏, 𝒙𝟐 = 𝟏
−𝟒𝒙𝟏 + 𝒙𝟐 = 𝟓
Now: Adding the two equations:
−𝟐𝒙𝟏 + 𝟒𝒙𝟐 = 𝟔

𝟐𝒙𝟏 + 𝟑𝒙𝟐 = 𝟏 −𝟒𝒙𝟏 + 𝒙𝟐 = 𝟓

→{ } OR: { } Both have the same
−𝟐𝒙𝟏 + 𝟒𝒙𝟐 = 𝟔 −𝟐𝒙𝟏 + 𝟒𝒙𝟐 = 𝟔
solution as the original system; 𝒙𝟏 = −𝟏, 𝒙𝟐 = 𝟏
4. Any combination of the previous three permissible operations is
also a permissible operation.
Example: we can multiply one equation by a constant, add the
resulting equation to another one in the system, and replace one
of the original equations by the new one. This won’t change the
solution of the system.

Page 3 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

Gauss Elimination Method

Successive application of permissible operations on the system of
equations until each equation is reduced to an identity for a single
unknown.
In matrix form, the system of n equations with n unknowns is:
𝒂𝟏𝟏 𝒂𝟏𝟐 𝒂𝟏𝟑 𝒂𝟏𝟒 . . 𝒂𝟏𝒏 𝒙𝟏 𝑪𝟏
𝒂𝟐𝟏 𝒂𝟐𝟐 𝒂𝟐𝟑 𝒂𝟐𝟒 . . 𝒂𝟐𝒏 𝒙𝟐 𝑪𝟐
𝒂𝟑𝟏 𝒂𝟑𝟐 𝒂𝟑𝟑 𝒂𝟑𝟒 . . 𝒂𝟑𝒏 𝒙𝟑 𝑪𝟑
. . . . . . . . =
.
. . . . . . . . .
[𝒂𝒏𝟏 𝒂𝒏𝟐 𝒂𝒏𝟑 𝒂𝒏𝟒 . . 𝒂𝒏𝒏 ] [𝒙𝒏 ] [𝑪𝒏 ]

OR:
𝑨. 𝒙 = 𝑪

Since the permissible operations are performed on both the constant coefficient
matrix A and the RHS vector of constants C, it is more convenient to re-write the
above system of equations as:

𝒂𝟏𝟏 𝒂𝟏𝟐 𝒂𝟏𝟑 𝒂𝟏𝟒 . . 𝒂𝟏𝒏 𝑪𝟏

𝒂𝟐𝟏 𝒂𝟐𝟐 𝒂𝟐𝟑 𝒂𝟐𝟒 . . 𝒂𝟐𝒏 𝑪𝟐
𝒂𝟑𝟏 𝒂𝟑𝟐 𝒂𝟑𝟑 𝒂𝟑𝟒 . . 𝒂𝟑𝒏 |𝑪𝟑
. . . . . . . . “Augmented” matrix
|
. . . . . . . .
[𝒂𝒏𝟏 𝒂𝒏𝟐 𝒂𝒏𝟑 𝒂𝒏𝟒 . . 𝒂𝒏𝒏 𝑪𝒏 ]
Permissible operations are performed on the “Augmented” matrix.
The Gaussian elimination method consists of two passes:
1. The forward pass: using permissible operations to reduce above system
to upper-triangular matrix
2. The backward pass: using permissible operations to reduce the “upper-
triangular” matrix to an identity matrix.

Page 4 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

The Forward Pass:

The resulting upper triangular matrix after finishing the forward pass
looks like:
𝒂𝟏𝟏 𝒂𝟏𝟐 𝒂𝟏𝟑 𝒂𝟏𝟒 . . 𝒂𝟏𝒏 𝑪𝟏 𝟏 𝒅𝟏𝟐 𝒅𝟏𝟑 𝒅𝟏𝟒 . . 𝒅𝟏𝒏 𝒆𝟏
𝒂𝟐𝟏 𝒂𝟐𝟐 𝒂𝟐𝟑 𝒂𝟐𝟒 . . 𝒂𝟐𝒏 𝑪𝟐 𝟎 𝟏 𝒅𝟐𝟑 𝒅𝟐𝟒 . . 𝒅𝟐𝒏 𝒆𝟐
𝒂𝟑𝟏 𝒂𝟑𝟐 𝒂𝟑𝟑 𝒂𝟑𝟒 . . |
𝒂𝟑𝒏 𝑪𝟑 |
𝟎 𝟎 𝟏 𝒅𝟑𝟒 . . 𝒅𝟑𝒏 𝒆𝟑
. . . . . . . . −→ . . . . . . . | .
|
. . . . . . . . . . . . . . . .
[𝒂𝒏𝟏 𝒂𝒏𝟐 𝒂𝒏𝟑 𝒂𝒏𝟒 . . 𝒂𝒏𝒏 𝑪𝒏 ] [𝟎 𝟎 𝟎 𝟎 . . 𝟏 𝒆𝒏 ]
Notes:
1. All diagonal elements are 1
2. All the elements below the main diagonal are zeros
The resulting system of equations can be written as:
𝒙𝟏 + 𝒅𝟏𝟐 𝒙𝟐 + 𝒅𝟏𝟑 𝒙𝟑 + ⋯ + 𝒅𝟏𝒏 𝒙𝒏 = 𝒆𝟏 (𝟏)
𝒙𝟐 + 𝒅𝟐𝟑 𝒙𝟑 + ⋯ + 𝒅𝟐𝒏 𝒙𝒏 = 𝒆𝟐 (𝟐)
𝒙𝟑 + ⋯ + 𝒅𝟑𝒏 𝒙𝒏 = 𝒆𝟑 (𝟑)
.
.
𝒙𝒏−𝟏 + ⋯ + 𝒅𝒏−𝟏,𝒏 𝒙𝒏 = 𝒆𝒏−𝟏 (𝒏 − 𝟏)
𝒙𝒏 = 𝒆𝒏 (𝒏)
The Backward Pass:
This can be performed in two methods:
A. Analytical solution of equations (1) to (n) above:
From Eqn. (1): 𝒙𝒏 = 𝒆𝒏
From Eqn. (n-1): 𝒙𝒏−𝟏 = 𝒆𝒏−𝟏 − 𝒅𝒏−𝟏,𝒏 𝒙𝒏 = 𝒆𝒏−𝟏 − 𝒅𝒏−𝟏,𝒏 𝒆𝒏
From Eqn. (n-2): 𝒙𝒏−𝟐 = ⋯
….
….
In general:

Page 5 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan
𝒏

𝒙𝒊 = 𝒆𝒊 − ∑ 𝒅𝒊𝒋 𝒙𝒋 , 𝒊 = 𝒏 − 𝟏, 𝒏 − 𝟐, … . , 𝟏(𝒐𝒏𝒆 𝒗𝒂𝒍𝒖𝒆), 𝒋 = 𝒊 + 𝟏, … , 𝒏

𝒋=𝒊+𝟏

From the above equation:

 For 𝒊 = 𝒏 − 𝟏, → 𝒋 = 𝒏
𝒏

𝒙𝒏−𝟏 = 𝒆𝒏−𝟏 − ∑ 𝒅𝒏−𝟏,𝒋 𝒙𝒋 = 𝒆𝒏−𝟏 − 𝒅𝒏−𝟏,𝒏 𝒙𝒏

𝒋=𝒏

 For 𝒊 = 𝒏 − 𝟐, → 𝒋 = 𝒏 − 𝟏, … , 𝒏
𝒏

𝒙𝒏−𝟐 = 𝒆𝒏−𝟐 − ∑ 𝒅𝒏−𝟐,𝒋 𝒙𝒋 = 𝒆𝒏−𝟐 − 𝒅𝒏−𝟐,𝒏−𝟏 𝒙𝒏−𝟏 − 𝒅𝒏−𝟐,𝒏 𝒙𝒏

𝒋=𝒏−𝟏

B. Performing a set permissible operations on the matrix resulting from

the forward pass to get finally the identity matrix:
′
𝟏 𝒅𝟏𝟐 𝒅𝟏𝟑 𝒅𝟏𝟒 . . 𝒅𝟏𝒏 𝒆𝟏 𝟏 𝟎 𝟎 𝟎 . . 𝟎 𝒆𝟏
𝟎 𝟏 𝒅𝟐𝟑 𝒅𝟐𝟒 . . 𝒅𝟐𝒏 𝟐𝒆 𝟎 𝟏 𝟎 𝟎 . . 𝟎 𝒆′𝟐
|𝒆𝟑 |
𝟎 𝟎 𝟏 𝒅𝟑𝟒 . . 𝒅𝟑𝒏 𝟎 𝟎 𝟏 𝟎 . . 𝟎 𝒆′𝟑
. −→
. . . . . . . | . . . . . . . .
. |
. . . . . . . . . . . . . . .
[𝟎 𝟎 𝟎 𝟎 . . 𝟏 𝒆𝒏 ] [𝟎 𝟎 𝟎 𝟎 . . 𝟏 𝒆′𝒏 ]

Example:
Use the Gauss elimination method to solve the system of equations:
𝟐𝒙𝟏 + 𝟑𝒙𝟐 − 𝟐𝒙𝟑 − 𝒙𝟒 = −𝟐
𝟐𝒙𝟏 + 𝟓𝒙𝟐 − 𝟑𝒙𝟑 + 𝒙𝟒 = 𝟕
−𝟐𝒙𝟏 + 𝒙𝟐 + 𝟑𝒙𝟑 − 𝟐𝒙𝟒 = 𝟏
−𝟐𝟓 + 𝟐𝒙𝟐 − 𝒙𝟑 + 𝟑𝒙𝟒 = 𝟖
Solution
The Forward Pass:
Augmented matrix of the above system:

Page 6 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

𝑪𝑳𝟏 𝑪𝑳𝟐 𝑪𝑳𝟑 𝑪𝑳𝟒 𝑪𝟓

𝑹𝟏 𝟐 𝟑 −𝟐 −𝟏 . −𝟐
𝑹𝟐 𝟐 𝟓 −𝟑 𝟏 . 𝟕
[ ]
𝑹𝟑 −𝟐 𝟏 𝟑 −𝟐 . 𝟏
𝑹𝟒 −𝟓 𝟐 −𝟏 𝟑 . 𝟖
The Forward Pass:  Upper triangular matrix
Take the first row as the pivot row, and perform the following operations:
𝑹: 𝑹𝒐𝒘 𝒐𝒇 𝒕𝒉𝒆 𝒐𝒓𝒊𝒈𝒊𝒏𝒂𝒍 𝒎𝒂𝒕𝒓𝒊𝒙
𝑹′ : 𝑹𝒐𝒘 𝒐𝒇 𝒕𝒉𝒆 𝒐𝒓𝒆𝒔𝒖𝒍𝒕𝒊𝒏𝒈 𝒎𝒂𝒕𝒓𝒊𝒙
↓ 𝑷𝒊𝒗𝒐𝒕 𝒆𝒍𝒆𝒎𝒆𝒏𝒕
𝑷𝒊𝒗𝒐𝒕 𝒓𝒐𝒘 → 𝟐 𝟑 −𝟐 −𝟏 . −𝟐
. 𝟐 𝟓 −𝟑 𝟏 . 𝟕
[ ]
. −𝟐 𝟏 𝟑 −𝟐 . 𝟏
. −𝟓 𝟐 −𝟏 𝟑 . 𝟖
 Pivot element  1
 All elements below the pivot element are zeros
Step 1: Pivot row: 𝑹𝟏 , Pivot element: 𝒂𝟏𝟏 = 𝟐, Normalize 𝑹𝟏 , zero elements below
the pivot elements:
𝟐 𝟑 −𝟐 −𝟏 . −𝟐 𝑹′𝟏 = 𝑹𝟏 /𝒂𝟏𝟏 𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏
′ ′
𝟐 𝟓 −𝟑 𝟏 . 𝟕 𝑹𝟐 = 𝑹𝟐 − 𝒂𝟐𝟏 𝑹𝟏 𝟎 𝟐 −𝟏 𝟐 . 𝟗
[ ] ′ ′ → [𝟎 ]
−𝟐 𝟏 𝟑 −𝟐 . 𝟏 𝑹𝟑 = 𝑹𝟑 − 𝒂𝟑𝟏 𝑹𝟏 𝟒 𝟏 −𝟑 . −𝟏
−𝟓 𝟐 −𝟏 𝟑 . 𝟖 𝑹′𝟒 = 𝑹𝟒 − 𝒂𝟒𝟏 𝑹′𝟏 𝟎 𝟏𝟗/𝟐 −𝟔 𝟏/𝟐 . 𝟑

Step 2: Pivot row: 𝑹𝟐 , Pivot element: 𝒂𝟐𝟐 = 𝟐 Normalize 𝑹𝟐 , zero elements below
the pivot elements:
𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏 𝑹′𝟏 = 𝑹𝟏 𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏
𝟎 𝟐 −𝟏 𝟐 . 𝟗 𝑹′𝟐 = 𝑹𝟐 /𝒂𝟐𝟐 𝟎 𝟏 −𝟏/𝟐 𝟏 . 𝟗/𝟐
[ ] ′ ′ →[ ]
𝟎 𝟒 𝟏 −𝟑 . −𝟏 𝑹𝟑 = 𝑹𝟑 − 𝒂𝟑𝟐 𝑹𝟐 𝟎 𝟎 𝟑 −𝟕 . −𝟏𝟗
𝟎 𝟏𝟗/𝟐 −𝟔 𝟏/𝟐 . 𝟑 𝑹′𝟒 = 𝑹𝟒 − 𝒂𝟒𝟐 𝑹′𝟐 𝟎 𝟎 −𝟓/𝟒 −𝟗 . −𝟏𝟓𝟗/𝟒

Step 3: Pivot row: 𝑹𝟑 , Pivot element: 𝒂𝟑𝟑 = 𝟑, Normalize 𝑹𝟑 , zero elements below
the pivot elements:
𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏 𝑹′𝟏 = 𝑹𝟏 𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏
′
𝟎 𝟏 −𝟏/𝟐 𝟏 . 𝟗/𝟐 𝑹 𝟐 = 𝑹𝟐 𝟎 𝟏 −𝟏/𝟐 𝟏 . 𝟗/𝟐
[ ] ′ →[ ]
𝟎 𝟎 𝟑 −𝟕 . −𝟏𝟗 𝑹𝟑 = 𝑹𝟑 /𝒂𝟑𝟑 𝟎 𝟎 𝟏 −𝟕/𝟑 . −𝟏𝟗/𝟑
𝟎 𝟎 −𝟓/𝟒 −𝟗 . −𝟏𝟓𝟗/𝟒 𝑹′𝟒 = 𝑹𝟒 − 𝒂𝟒𝟑 𝑹′𝟑 𝟎 𝟎 𝟎 −𝟏𝟒𝟑/𝟏𝟐 . −𝟓𝟕𝟐/𝟏𝟐

Page 7 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

Step 4: Pivot row: 𝑹𝟒 , Pivot element: 𝒂𝟒𝟒 = −𝟏𝟒𝟑/𝟏𝟐, Normalize 𝑹𝟒 , zero elements
below the pivot elements:

𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏 𝑹′𝟏 = 𝑹𝟏 𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏

𝟎 𝟏 −𝟏/𝟐 𝟏 . 𝟗/𝟐 𝑹′𝟐 = 𝑹𝟐 𝟎 𝟏 −𝟏/𝟐 𝟏 . 𝟗/𝟐
[ ] →[ ]
𝟎 𝟎 𝟏 −𝟕/𝟑 . −𝟏𝟗/𝟑 𝑹′𝟑 = 𝑹𝟑 𝟎 𝟎 𝟏 −𝟕/𝟑 . −𝟏𝟗/𝟑
𝟎 𝟎 𝟎 −𝟏𝟒𝟑/𝟏𝟐 . −𝟓𝟕𝟐/𝟏𝟐 𝑹′𝟒 = 𝑹𝟒 /𝒂𝟒𝟒 𝟎 𝟎 𝟎 𝟏 . 𝟒

The resulting matrix is upper-triangular matrix.

All elements below the main diagonal are zeros.
The Backward Pass:
Method 1:
Starting from the last equation in the system:
𝒙𝟒 = 𝟒
𝟕 𝟏𝟗 𝟕 𝟏𝟗
𝒙𝟑 − 𝒙𝟒 = − → 𝒙𝟑 = × 𝟒 − =𝟑
𝟑 𝟑 𝟑 𝟑
𝟏 𝟗 𝟏 𝟗
𝒙𝟐 − 𝒙𝟑 + 𝒙𝟒 = → 𝒙𝟐 = × 𝟑 − 𝟒 + = 𝟐
𝟐 𝟐 𝟐 𝟐
𝟑 𝟏 𝟑 𝟏
𝒙𝟏 + 𝒙𝟐 − 𝒙𝟑 − 𝒙𝟒 = −𝟏 → 𝒙𝟏 = − × 𝟐 + 𝟑 + × 𝟒 − 𝟏 = 𝟏
𝟐 𝟐 𝟐 𝟐
Check the solution: substitute for 𝒙𝟏 , … 𝒙𝟒 in the given system of equations:
𝑬𝒒𝒏. 𝟏: 𝟐×𝟏+𝟑×𝟐−𝟐×𝟑−𝟒= 𝟐 𝑶𝑲
𝑬𝒒𝒏. 𝟐: 𝟐×𝟏+𝟓×𝟐−𝟑×𝟑+𝟒= 𝟕 𝑶𝑲
𝑬𝒒𝒏. 𝟑: −𝟐×𝟏+𝟐+𝟑×𝟑−𝟐×𝟒=𝟏 𝑶𝑲
𝑬𝒒𝒏. 𝟒: −𝟓×𝟏+𝟐×𝟐−𝟑+𝟑×𝟒=𝟖 𝑶𝑲

Method 2
Reduce the upper-triangular matrix resulting from the “Forward Pass” to identity
matrix
Step 1
′
𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏 𝑹𝟏 = 𝑹𝟏 − 𝒂𝟏𝟒 𝑹𝟒 𝟏 𝟑/𝟐 −𝟏 𝟎 . 𝟏
′
𝟎 𝟏 −𝟏/𝟐 𝟏 . 𝟗/𝟐 𝑹𝟐 = 𝑹𝟐 − 𝒂𝟐𝟒 𝑹𝟒 𝟎 𝟏 −𝟏/𝟐 𝟎 . 𝟏/𝟐
[ ] ′ →[ ]
𝟎 𝟎 𝟏 −𝟕/𝟑 . −𝟏𝟗/𝟑 𝑹𝟑 = 𝑹𝟑 − 𝒂𝟑𝟒 𝑹𝟒 𝟎 𝟎 𝟏 𝟎 . 𝟑
𝟎 𝟎 𝟎 𝟏 . 𝟒 𝑹′𝟒 = 𝑹𝟒 𝟎 𝟎 𝟎 𝟏 . 𝟒

Step 2

Page 8 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan
′
𝟏 𝟑/𝟐 −𝟏 𝟎 . 𝟏 𝑹𝟏 = 𝑹𝟏 − 𝒂𝟏𝟑 𝑹𝟑 𝟏 𝟑/𝟐 𝟎 𝟎 . 𝟒
′
𝟎 𝟏 −𝟏/𝟐 𝟎 . 𝟏/𝟐 𝑹𝟐 = 𝑹𝟐 − 𝒂𝟐𝟑 𝑹𝟑 𝟎 𝟏 𝟎 𝟎 . 𝟐
[ ] ′ →[ ]
𝟎 𝟎 𝟏 𝟎 . 𝟑 𝑹𝟑 = 𝑹𝟑 𝟎 𝟎 𝟏 𝟎 . 𝟑
𝟎 𝟎 𝟎 𝟏 . 𝟒 𝑹′𝟒 = 𝑹𝟒 𝟎 𝟎 𝟎 𝟏 . 𝟒

Step 3
′
𝟏 𝟑/𝟐 𝟎 𝟎 . 𝟒 𝑹𝟏 = 𝑹𝟏 − 𝒂𝟏𝟐 𝑹𝟐 𝟏 𝟎 𝟎 𝟎 . 𝟏
𝟎 𝟏 𝟎 𝟎 . 𝟐 𝑹′𝟐 = 𝑹𝟐 𝟎 𝟏 𝟎 𝟎 . 𝟐
[ ] →[ ]
𝟎 𝟎 𝟏 𝟎 . 𝟑 𝑹′𝟑 = 𝑹𝟑 𝟎 𝟎 𝟏 𝟎 . 𝟑
𝟎 𝟎 𝟎 𝟏 . 𝟒 ′ 𝟎 𝟎 𝟎 𝟏 . 𝟒
𝑹𝟒 = 𝑹𝟒

→ 𝒙𝟏 = 𝟏, 𝒙𝟐 = 𝟐, 𝒙𝟑 = 𝟑, 𝒙𝟒 = 𝟒
Notes on the Gauss Elimination Method
1. If a pivot element (𝒂𝒊𝒊 ) is zero: Re-arrange equations to avoid division by
zero.

Example:
𝟐 𝟐 𝟑 . 𝟏𝟎 𝟐 𝟐 𝟑 . 𝟏𝟎
[𝟏 𝟎 𝟐 . 𝟓 ] → [𝟐 𝟑 𝟐 . 𝟕 ] (interchange rows 2 and 3)
𝟐 𝟑 𝟐 . 𝟕 𝟏 𝟎 𝟐 . 𝟓
2. If coefficients (𝒂𝒊𝒋 ) change by orders of magnitude  resulting in large
round-off errors: Re-arrange equations so that the largest coefficients are
placed on the main diagonal of the coefficient matrix.

Example:
𝟏 𝟏𝟎𝟎 𝟎. 𝟏 . 𝟏𝟎𝟎 𝟓 𝟎. 𝟎𝟏 𝟐 . 𝟐𝟎
[𝟓 𝟎. 𝟎𝟏 𝟐 . 𝟐𝟎 ] → [𝟏 𝟏𝟎𝟎 𝟎. 𝟏 . 𝟏𝟎𝟎] (interchange rows 1 and 2)
𝟐 𝟏 𝟑 . 𝟏𝟎 𝟐 𝟏 𝟑 . 𝟏𝟎

Page 9 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

Gauss - Jordan Elimination Method

 Need only one pass to get the solution
 Operations are performed above and below the pivot row.

Example:
Use the Gauss-Jordan elimination method to solve the system of equations:
𝟐𝒙𝟏 + 𝟑𝒙𝟐 − 𝟐𝒙𝟑 − 𝒙𝟒 = −𝟐
𝟐𝒙𝟏 + 𝟓𝒙𝟐 − 𝟑𝒙𝟑 + 𝒙𝟒 = 𝟕
−𝟐𝒙𝟏 + 𝒙𝟐 + 𝟑𝒙𝟑 − 𝟐𝒙𝟒 = 𝟏
−𝟐𝟓 + 𝟐𝒙𝟐 − 𝒙𝟑 + 𝟑𝒙𝟒 = 𝟖
Solution:
Augmented matrix of the above system:
𝟐 𝟑 −𝟐 −𝟏 . −𝟐
𝟐 𝟓 −𝟑 𝟏 . 𝟕
[ ]
−𝟐 𝟏 𝟑 −𝟐 . 𝟏
−𝟓 𝟐 −𝟏 𝟑 . 𝟖
Step 1: Pivot row: R1, Pivot element: 𝒂𝟏𝟏 = 𝟐, Normalize Row 1, zero elements
below the pivot elements:

𝟐 𝟑 −𝟐 −𝟏 . −𝟐 𝑹′𝟏 = 𝑹𝟏 /𝒂𝟏𝟏 𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏

′ ′
𝟐 𝟓 −𝟑 𝟏 . 𝟕 𝑹 = 𝑹 𝟐 − 𝒂 𝑹
𝟐𝟏 𝟏 𝟎 𝟐 −𝟏 𝟐 . 𝟗
[ ] 𝟐 → [ ]
−𝟐 𝟏 𝟑 −𝟐 . 𝟏 𝑹′𝟑 = 𝑹𝟑 − 𝒂𝟑𝟏 𝑹′𝟏 𝟎 𝟒 𝟏 −𝟑 . −𝟏
−𝟓 𝟐 −𝟏 𝟑 ′
. 𝟖 𝑹𝟒 = 𝑹𝟒 − 𝒂𝟒𝟏 𝑹𝟏 ′ 𝟎 𝟏𝟗/𝟐 −𝟔 𝟏/𝟐 . 𝟑

Step 2: Pivot row: R2, Pivot element: 𝒂𝟐𝟐 = 𝟐, Normalize Row 2, zero elements
above and below the pivot elements:
′ ′
𝟏 𝟑/𝟐 −𝟏 −𝟏/𝟐 . −𝟏 𝑹𝟏 = 𝑹𝟏 − 𝒂𝟏𝟐 𝑹𝟐 𝟏 𝟎 −𝟏/𝟒 −𝟐 . −𝟑𝟏/𝟒
𝟎 𝟐 −𝟏 𝟐 . 𝟗 𝑹′𝟐 = 𝑹𝟐 /𝒂𝟐𝟐 𝟎 𝟏 −𝟏/𝟐 𝟏 . 𝟗/𝟐
[ ] ′ ′ →[ ]
𝟎 𝟒 𝟏 −𝟑 . −𝟏 𝑹𝟑 = 𝑹𝟑 − 𝒂𝟑𝟐 𝑹𝟐 𝟎 𝟎 𝟑 −𝟕 . −𝟏𝟗
𝟎 𝟏𝟗/𝟐 −𝟔 𝟏/𝟐 . 𝟑 𝑹′𝟒 = 𝑹𝟒 − 𝒂𝟒𝟐 𝑹′𝟐 𝟎 𝟎 −𝟓/𝟒 −𝟗 . −𝟏𝟓𝟗/𝟒

Step 3: Pivot row: R3, Pivot element: 𝒂𝟑𝟑 = 𝟑, Normalize Row 3, zero elements
above and below the pivot elements:
′ ′
𝟏 𝟎 −𝟏/𝟒 −𝟐 . −𝟑𝟏/𝟒 𝑹𝟏 = 𝑹𝟏 − 𝒂𝟏𝟑 𝑹𝟑 𝟏 𝟎 𝟎 −𝟑𝟏/𝟏𝟐 . −𝟏𝟏𝟐/𝟏𝟐
′ ′
𝟎 𝟏 −𝟏/𝟐 𝟏 . 𝟗/𝟐 𝑹 = 𝑹 − 𝒂𝟐𝟑 𝑹𝟑 𝟎 𝟏 𝟎 −𝟏/𝟔 . 𝟒/𝟑
[ ] 𝟐 ′ 𝟐 →[ ]
𝟎 𝟎 𝟑 −𝟕 . −𝟏𝟗 𝑹𝟑 = 𝑹𝟑 /𝒂𝟑𝟑 𝟎 𝟎 𝟏 −𝟕/𝟑 . −𝟏𝟗/𝟑
𝟎 𝟎 −𝟓/𝟒 −𝟗 . −𝟏𝟓𝟗/𝟒 𝑹𝟒 = 𝑹𝟒 − 𝒂𝟒𝟑 𝑹𝟑
′ ′ 𝟎 𝟎 𝟎 −𝟏𝟒𝟑/𝟏𝟐 . −𝟓𝟕𝟐/𝟏𝟐

Page 10 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

Step 4: Pivot row: R4, Pivot element: 𝒂𝟒𝟒 = −𝟏𝟒𝟑/𝟏𝟐, Normalize Row 4, zero
elements above the pivot elements:
′ ′
𝟏 𝟎 𝟎 −𝟑𝟏/𝟏𝟐 . −𝟏𝟏𝟐/𝟏𝟐 𝑹𝟏 = 𝑹𝟏 − 𝒂𝟏𝟒 𝑹𝟒 𝟏 𝟎 𝟎 𝟎 . 𝟏
′ ′
𝟎 𝟏 𝟎 −𝟏/𝟔 . 𝟒/𝟑 𝑹 = 𝑹𝟐 − 𝒂𝟐𝟒 𝑹𝟒 𝟎 𝟏 𝟎 𝟎 . 𝟐
[ ] 𝟐′ ′ →[ ]
𝟎 𝟎 𝟏 −𝟕/𝟑 . −𝟏𝟗/𝟑 𝑹𝟑 = 𝑹𝟑 − 𝒂𝟑𝟒 𝑹𝟒 𝟎 𝟎 𝟏 𝟎 . 𝟑
𝟎 𝟎 𝟎 −𝟏𝟒𝟑/𝟏𝟐 . −𝟓𝟕𝟐/𝟏𝟐 𝑹′𝟒 = 𝑹𝟒 /𝒂𝟒𝟒 𝟎 𝟎 𝟎 𝟏 . 𝟒
→ 𝒙𝟏 = 𝟏, 𝒙𝟐 = 𝟐, 𝒙𝟑 = 𝟑, 𝒙𝟒 = 𝟒
----------------------------------------------------------------------------------------------
Iteration Methods
 Jacobi method
1. Solve each equation for one unknown
2. Assume a solution (initial guess) 𝒙𝒊 , 𝒊 = 𝟏, 𝟐, … , 𝒏
3. Use the assumed solution to find an updated one, 𝒙′𝒊
4. Calculate the error
5. If the error is less than a specified tolerance: Stop, Solution is 𝒙′𝒊
6. Else, Use the updated solution in (3) to find a new solution:
𝒙𝒊 ← 𝒙′𝒊
Repeat the procedure until convergence is achieved.

Illustration:
For the system of equations:
𝒂𝟏𝟏 𝒙𝟏 + 𝒂𝟏𝟐 𝒙𝟐 + 𝒂𝟏𝟑 𝒙𝟑 + ⋯ + 𝒂𝟏𝒏 𝒙𝒏 = 𝑪𝟏
𝒂𝟐𝟏 𝒙𝟏 + 𝒂𝟐𝟐 𝒙𝟐 + 𝒂𝟐𝟑 𝒙𝟑 + ⋯ + 𝒂𝟐𝒏 𝒙𝒏 = 𝑪𝟐
𝒂𝟑𝟏 𝒙𝟏 + 𝒂𝟑𝟐 𝒙𝟐 + 𝒂𝟑𝟑 𝒙𝟑 + ⋯ + 𝒂𝟑𝒏 𝒙𝒏 = 𝑪𝟑
.
.
𝒂𝒏𝟏 𝒙𝟏 + 𝒂𝒏𝟐 𝒙𝟐 + 𝒂𝒏𝟑 𝒙𝟑 + ⋯ + 𝒂𝒏𝒏 𝒙𝒏 = 𝑪𝒏

Page 11 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

1. Solve each equation for one unknown:

𝒙′𝟏 = 𝑪𝟏 /𝒂𝟏𝟏 − [𝒂𝟏𝟐 𝒙𝟐 + 𝒂𝟏𝟑 𝒙𝟑 + ⋯ + 𝒂𝟏𝒏 𝒙𝒏 ]/𝒂𝟏𝟏

𝒙′𝟐 = 𝑪𝟐 /𝒂𝟐𝟐 − [𝒂𝟐𝟏 𝒙𝟏 + 𝒂𝟐𝟑 𝒙𝟑 + ⋯ + 𝒂𝟐𝒏 𝒙𝒏 ]/𝒂𝟐𝟐
𝒙′𝟑 = 𝑪𝟑 /𝒂𝟑𝟑 − [𝒂𝟑𝟏 𝒙𝟏 + 𝒂𝟑𝟐 𝒙𝟑 + ⋯ + 𝒂𝟑𝒏 𝒙𝒏 ]/𝒂𝟑𝟑
.
.
𝒙′𝒏 = 𝑪𝒏 /𝒂𝒏𝒏 − [𝒂𝒏𝟏 𝒙𝟏 + 𝒂𝒏𝟐 𝒙𝟑 + ⋯ + 𝒂𝒏−𝟏,𝒏 𝒙𝒏−𝟏 ]/𝒂𝒏𝒏
2. Assume a solution (initial guess) 𝒙𝒊 , 𝒊 = 𝟏, 𝟐, … , 𝒏
𝒙𝟏 =. . , 𝒙𝟐 =. . , 𝒙𝟑 =. . , … … … … 𝒙𝒏 =..
3. Use the assumed solution to find an updated one, 𝒙′𝒊 :
𝒙′𝟏 =. . , 𝒙′𝟐 =. . , 𝒙′𝟑 =. . , … … … … 𝒙′𝒏 =..
4. Calculate the error:
𝒙′𝒊 − 𝒙𝒊
𝑬𝒓𝒓𝒐𝒓 = | | , 𝒊 = 𝟏, … , 𝒏
𝒙′𝒊
5. If the error is less than a specified tolerance: Stop, Solution is 𝒙′𝒊
6. Else, use the updated solution in (3) as the initial guess, to find a new
solution:
𝒙𝒊 = 𝒙′𝒊 , 𝒊 = 𝟏, 𝟐, … . , 𝒏
Repeat the procedure until convergence is achieved.

Example
Use Jacobi method to solve the following system of equations to an accuracy of 1%.
𝟑𝒙𝟏 + 𝒙𝟐 − 𝟐𝒙𝟑 = 𝟗
−𝒙𝟏 + 𝟒𝒙𝟐 − 𝟑𝒙𝟑 = −𝟖
𝒙𝟏 − 𝒙𝟐 + 𝟒𝒙𝟑 = 𝟏
Solution:
Step 1:
𝒙′𝟏 = (𝟗 − 𝒙𝟐 + 𝟐𝒙𝟑 )/𝟑
𝒙′𝟐 = (−𝟖 + 𝒙𝟏 + 𝟑𝒙𝟑 )/𝟒
𝒙′𝟑 = (𝟏 − 𝒙𝟏 + 𝒙𝟐 )/𝟒
Page 12 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

Step 2:
Assume 𝒙𝟏 = 𝒙𝟐 = 𝒙𝟑 = 𝟏
Step 3:
𝒙′𝟏 = (𝟗 − 𝟏 + 𝟐 × 𝟏)/𝟑 = 𝟏𝟎/𝟑
𝒙′𝟐 = (−𝟖 + 𝟏 + 𝟑 × 𝟏)/𝟒 = −𝟏
𝒙′𝟑 = (𝟏 − 𝟏 + 𝟏)/𝟒 = 𝟏/𝟒
Step 4:
Calculate the error:
𝒙′𝟏 − 𝒙𝟏 𝟏𝟎/𝟑 − 𝟏
𝒙𝟏 : 𝑬𝒓𝒓𝒐𝒓 = | | = | | × 𝟏𝟎𝟎% = 𝟕𝟎%
𝒙′𝟏 𝟏𝟎/𝟑
𝒙′𝟐 − 𝒙𝟐 −𝟏 − 𝟏
𝒙𝟐 : 𝑬𝒓𝒓𝒐𝒓 = | | = | | × 𝟏𝟎𝟎% = 𝟐𝟎𝟎%
𝒙′𝟐 −𝟏
𝒙′𝟑 − 𝒙𝟑 𝟏/𝟒 − 𝟏
𝒙𝟑 : 𝑬𝒓𝒓𝒐𝒓 = | ′ |=| | × 𝟏𝟎𝟎% = 𝟑𝟎𝟎%
𝒙𝟑 𝟏/𝟒

Step 5:
Error > tolerance 
Step 6:
𝟏𝟎
𝑺𝒆𝒕 𝒙𝟏 = , 𝒙 = −𝟏, 𝒙𝟑 = 𝟏/𝟒
𝟑 𝟐
𝟐
(𝟗 + 𝟏 + 𝟒)
𝒙′𝟏 = = 𝟑. 𝟓
𝟑
𝟏𝟎 𝟑
(−𝟖 + 𝟑 + 𝟒)
𝒙′𝟐 = = −𝟎. 𝟗𝟕𝟗𝟏𝟔𝟔
𝟒
𝟏𝟎
(𝟏 − 𝟑 − 𝟏)
𝒙′𝟑 = = −𝟓/𝟔
𝟒

Page 13 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

Calculate the error:

𝒙′𝟏 − 𝒙𝟏 𝟑. 𝟓 − 𝟏𝟎/𝟑
𝒙𝟏 : 𝑬𝒓𝒓𝒐𝒓 = | | = | | × 𝟏𝟎𝟎% = 𝟒. 𝟕𝟔%
𝒙′𝟏 𝟑. 𝟓
𝒙′𝟐 − 𝒙𝟐 −𝟎. 𝟗𝟕𝟗 + 𝟏
𝒙𝟐 : 𝑬𝒓𝒓𝒐𝒓 = | | = | | × 𝟏𝟎𝟎% = 𝟐. 𝟏𝟑%
𝒙′𝟐 −𝟎. 𝟗𝟕𝟗
𝒙′𝟑 − 𝒙𝟑 −𝟓/𝟔 − 𝟏/𝟒
𝒙𝟑 : 𝑬𝒓𝒓𝒐𝒓 = | | = | | × 𝟏𝟎𝟎% = 𝟏𝟑𝟎%
𝒙′𝟑 −𝟓/𝟔

𝑺𝒆𝒕 𝒙𝟏 = 𝟑. 𝟓, 𝒙𝟐 = −𝟎. 𝟗𝟕𝟗𝟏𝟔𝟔, 𝒙𝟑 = −𝟓/𝟔

Repeat the procedure till error for all x falls below the specified tolerance.

 Gauss-Seidel Method
An improvement on the Jacobi method, where each new computed x
value is immediately used in the next equation to determine another x
value.
Example
Use the Gauss-Seidel method to obtain the solution of the following system of equations

Page 14 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

Page 15 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

HW Problem Set 3 (Linear Algebraic Equations)

Page 16 of 17
 College of Engineering, Department of Mechanical Engineering
Lecture Notes 0402307 Numerical Analysis Khalid Ramadan

Page 17 of 17