Scribd
Upload
12 views6 pages

Numerical Linear Algebra

The document discusses solving a least squares problem using different methods. It compares the normal equation method, QR decomposition using classical Gram-Schmidt and MATLAB's qr function, and SVD. It analyzes the condition numbers, number of correct digits, residuals, and testing errors of each method.

Uploaded by

Adebayo Emmanuel
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
12 views6 pages

Numerical Linear Algebra

The document discusses solving a least squares problem using different methods. It compares the normal equation method, QR decomposition using classical Gram-Schmidt and MATLAB's qr function, and SVD. It analyzes the condition numbers, number of correct digits, residuals, and testing errors of each method.

Uploaded by

Adebayo Emmanuel
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Comp.

Project 1 -Solving least squares problem


Adebayo
MATH 612
Week when the assignment was assigned: Week 5
October 2, 2023

1 Task 1
The dimension of matrix A is 50 × 12, number of rows is 50 and number of columns is 12.
The rank of matrix A is: 12 (full rank)
The Condition number of A is: 1.171777 × 108

2 Task 2
The degree of the interpolating polynomial is: 11 since the number of columns is 12.

3 Task 3
3.1 Mathematical Formulation and correct number of Digits

Method A(Normal Equation)


Mathematical Formulation
The normal equation is given by:
AT Aa = AT b (1)
Since  
∥δx∥ ∥δA∥
≤ Cond (A)
∥x + δx∥ ∥A∥
The possible relative error in the solution vector norm is Cond (A) × ϵmach
Hence, Cond(A) × ϵmach should give us the number of significant digits, m that are at least
correct in our solution by finding out the largest value of m for which Cond(A) × ϵmach is
less than 0.5 × 10−m
Correct digits
Cond(AT A) × ϵmach ≤ 0.5 × 10−m

1
 
0.5
m ≤ log T
Cond (A A) × εmach
m≤0
Hence, there is 0 correct digit(rounding to the nearest power of 10)

Algorithm 1 MATLAB Implementation of Normal Solution


Require: A (design matrix), b (target values)
Ensure: a = (AT A)−1 AT b (solution vector)
1: Compute the normal equation: AT Aa = AT b
2: Solve the linear system AT Aa = AT b for a
return a

Method B(QR Decomposition using Classical Gram


Mathematical Formulation
Solution of Aa = b by QR factorization method
If A = QR, then we have QRa = b
Ra = Q∗ b (2)

Algorithm 2 Computing the Classical Gram-Schmidt method for QR


Require: A ∈ Rm×n
Ensure: Q (matrix with orthonormal column) R(upper triangular matrix)
1: Q ← 0 ∈ Rm×n
2: R ← 0 ∈ Rn×n
3: R1,1 ← ∥A1 ∥ (A1 is the first column of A)
4: Q1 ← A1 /R1,1
5: for i ← (2 → n) do
6: vi = Ai
7: for j ← (1 → i − 1) do
8: Rj,i = ⟨vi , Qj ⟩
i−1
P
9: proj = Rji Qj
j=1
10: end for
11: ui = vi − proj
12: Ri,i = ∥ui ∥
13: Qi = Rui,ii
14: end for
return Q and R

If we let y = Q∗ b, then we solve for a from the equation

Ra = y (3)
Correct digits
Cond(R) × ϵmach ≤ 0.5 × 10−m

2
 
0.5
m ≤ log
Cond (R) × ϵmach
m≤8
Hence, there are 8 correct digit

Algorithm 3 MATLAB Implementation QR


Require: A (design matrix), b (target values)
Ensure: a = (AT A)−1 AT b (solution vector)
Compute the QR decomposition using Algorithm 2 : A = QR
Compute QT b
Solve the linear system Ra = QT b for a
return a

Method C(QR Decomposition using MATLAB qr function)


Mathematical Formulation Same as method 2
Correct digits
Cond(R) × ϵmach ≤ 0.5 × 10−m  
0.5
m ≤ log
Cond (R) × ϵmach
m≤7
Hence, there are 7 correct digit

Algorithm 4 MATLAB Implementation QR


Require: A (design matrix), b (target values)
Ensure: a = (AT A)−1 AT b (solution vector)
Compute the QR decomposition using MATLAB function qr: A = QR
Compute QT b
Solve the linear system Ra = QT b for a
return a

Method D(Using MATLAB SVD function)


Mathematical Formulation
T
Given Am×n = U m×n Σn×n (V n×n ) , A† = V Σ† U T
Where  
† 1 1
Σ = diag ,··· σ1 ≥ · · · ≥ σn > 0
σ1 σn
Hence,
a = A† b (4)

Correct digits
Cond(R) × ϵmach ≤ 0.5 × 10−m !
0.5
m ≤ log σmax
σmin
× ϵmach

3
Algorithm 5 MATLAB Implementation SVD
Require: A (design matrix), b (target values)
Ensure: a = A† b (solution vector)
Compute the SVD using MATLAB function svd(A,0): A = U ΣV T
Compute PseudoInverse of A : A† = V Σ† U T
Solve the linear system a = A† b for a
return a

m≤7
Hence, there are 7 correct digit

3.2 Plotting of Figures

Figure 1: Coefficients of the polynomial from each method

4
Figure 2: Interpolating polynomial and the original function

5
3.3 Evaluating and Tabulating the Residuum

∥Aa − b∥2 ∥Aa − b∥∞


Normal Equation 6.583886253457947 × 104 4.352268855341413 × 104
Classical QR 4.140817226320195 × 105 2.624790079464150 × 105
QR 6.108566281416130 × 104 4.042965082309044 × 104
SVD 6.108566252870014 × 104 4.042965064462445 × 104

Table 1: Residuum of the training fit

Comment: The method that would be best use in order to obtain accurate fit is the svd
pseudo-inverse method because it has the smallest 2 norm and ∞-norm

3.4 Exploring Testing Error

Figure 3: Exploring testing error of different methods

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy