Chapter 2

Solutions of Equations in One Variable

The Bisection Method

The Bisection Method

Suppose 𝑓 continuous on [𝑎, 𝑏], and 𝑓(𝑎), 𝑓(𝑏) opposite signs
By the IVT, there exists an 𝑥 in (𝑎, 𝑏) with 𝑓(𝑥) = 0
Divide the interval [𝑎, 𝑏] by computing the midpoint

𝑝 = (𝑎 + 𝑏)/2

If 𝑓(𝑝) has same sign as 𝑓(𝑎), consider new interval [𝑝, 𝑏]

If 𝑓(𝑝) has same sign as 𝑓(𝑏), consider new interval [𝑎, 𝑝]
Repeat until interval small enough to approximate 𝑥 well
The Bisection Method – Implementation

MATLAB Implementation
function p = bisection(f, a, b, tol)
% Solve f(p) = 0 using the bisection method.

while 1
p = (a+b) / 2;
if p-a < tol, break; end
if f(a)*f(p) > 0
a = p;
else
b = p;
end
end
Bisection Method

Termination Criteria
Many ways to decide when to stop:

|𝑝𝑁 − 𝑝𝑁−1 | < 𝜀

|𝑝𝑁 − 𝑝𝑁−1 |
<𝜀
|𝑝𝑁 |
|𝑓(𝑝𝑁 )| < 𝜀

None is perfect, use a combination in real software

Convergence

Theorem
Suppose that 𝑓 ∈ 𝐶[𝑎, 𝑏] and 𝑓(𝑎) ⋅ 𝑓(𝑏) < 0. The Bisection
method generates a sequence {𝑝𝑛 }∞
𝑛=1 approximating a zero 𝑝 of 𝑓
with
𝑏−𝑎
|𝑝𝑛 − 𝑝| ≤ 𝑛 , when 𝑛 ≥ 1.
2

Convergence Rate
The sequence {𝑝𝑛 }∞
𝑛=1 converges to 𝑝 with rate of
convergence 𝑂(1/2𝑛 ):

1
𝑝𝑛 = 𝑝 + 𝑂 ( ).
2𝑛
Fixed Points

Fixed Points and Root-Finding

A number 𝑝 is a fixed point for a given function 𝑔 if 𝑔(𝑝) = 𝑝
Given a root-finding problem 𝑓(𝑝) = 0, there are many 𝑔 with
fixed points at 𝑝:
𝑔(𝑥) = 𝑥 − 𝑓(𝑥)
𝑔(𝑥) = 𝑥 + 3𝑓(𝑥)
…
If 𝑔 has fixed point at 𝑝, then 𝑓(𝑥) = 𝑥 − 𝑔(𝑥) has a zero at 𝑝
Existence and Uniqueness of Fixed Points

Theorem
a. If 𝑔 ∈ 𝐶[𝑎, 𝑏] and 𝑔(𝑥) ∈ [𝑎, 𝑏] for all 𝑥 ∈ [𝑎, 𝑏], then 𝑔 has a
fixed point in [𝑎, 𝑏]
b. If, in addition, 𝑔′ (𝑥) exists on (𝑎, 𝑏) and a positive constant
𝑘 < 1 exists with

|𝑔′ (𝑥)| ≤ 𝑘, for all 𝑥 ∈ (𝑎, 𝑏),

then the fixed point in [𝑎, 𝑏] is unique.

Fixed-Point Iteration

Fixed-Point Iteration
For initial 𝑝0 , generate sequence {𝑝𝑛 }∞
𝑛=0 by 𝑝𝑛 = 𝑔(𝑝𝑛−1 ).
If the sequence converges to 𝑝, then

𝑝 = lim 𝑝𝑛 = lim 𝑔(𝑝𝑛−1 ) = 𝑔 ( lim 𝑝𝑛−1 ) = 𝑔(𝑝).

𝑛→∞ 𝑛→∞ 𝑛→∞

MATLAB Implementation
function p = fixedpoint(g, p0, tol)
% Solve g(p) = p using fixed-point iteration.

while 1
p = g(p0);
if abs(p-p0) < tol, break; end
p0 = p;
end
Convergence of Fixed-Point Iteration
Theorem (Fixed-Point Theorem)
Let 𝑔 ∈ 𝐶[𝑎, 𝑏] be such that 𝑔(𝑥) ∈ [𝑎, 𝑏], for all 𝑥 in [𝑎, 𝑏].
Suppose, in addition, that 𝑔′ exists on (𝑎, 𝑏) and that a constant
0 < 𝑘 < 1 exists with

|𝑔′ (𝑥)| ≤ 𝑘, for all 𝑥 ∈ (𝑎, 𝑏).

Then, for any number 𝑝0 in [𝑎, 𝑏], the sequence defined by

𝑝𝑛 = 𝑔(𝑝𝑛−1 ) converges to the unique fixed point 𝑝 in [𝑎, 𝑏].

Corollary
If 𝑔 satisfies the hypotheses above, then bounds for the error are
given by
|𝑝𝑛 − 𝑝| ≤ 𝑘𝑛 max{𝑝0 − 𝑎, 𝑏 − 𝑝0 }
𝑘𝑛
|𝑝𝑛 − 𝑝| ≤ |𝑝 − 𝑝0 |
1−𝑘 1
Newton’s Method

Taylor Polynomial Derivation

Suppose 𝑓 ∈ 𝐶 2 [𝑎, 𝑏] and 𝑝0 ∈ [𝑎, 𝑏] approximates solution 𝑝 of
𝑓(𝑥) = 0 with 𝑓 ′ (𝑝0 ) ≠ 0. Expand 𝑓(𝑥) about 𝑝0 :

(𝑝 − 𝑝0 )2 ″
𝑓(𝑝) = 𝑓(𝑝0 ) + (𝑝 − 𝑝0 )𝑓 ′ (𝑝0 ) + 𝑓 (𝜉(𝑝))
2
Set 𝑓(𝑝) = 0, assume (𝑝 − 𝑝0 )2 negligible:

𝑓(𝑝0 )
𝑝 ≈ 𝑝1 = 𝑝 0 −
𝑓 ′ (𝑝0 )

This gives the sequence {𝑝𝑛 }∞

𝑛=0 :

𝑓(𝑝𝑛−1 )
𝑝𝑛 = 𝑝𝑛−1 −
𝑓 ′ (𝑝𝑛−1 )
Newton’s Method

MATLAB Implementation
function p = newton(f, df, p0, tol)
% Solve f(p) = 0 using Newton's method.

while 1
p = p0 - f(p0)/df(p0);
if abs(p-p0) < tol, break; end
p0 = p;
end
Newton’s Method – Convergence

Fixed Point Formulation

Newton’s method is fixed point iteration 𝑝𝑛 = 𝑔(𝑝𝑛−1 ) with

𝑓(𝑥)
𝑔(𝑥) = 𝑥 −
𝑓 ′ (𝑥)

Theorem
Let 𝑓 ∈ 𝐶 2 [𝑎, 𝑏]. If 𝑝 ∈ [𝑎, 𝑏] is such that 𝑓(𝑝) = 0 and 𝑓 ′ (𝑝) ≠ 0,
then there exists a 𝛿 > 0 such that Newton’s method generates a
sequence {𝑝𝑛 }∞ 𝑛=1 converging to 𝑝 for any initial approximation
𝑝0 ∈ [𝑝 − 𝛿, 𝑝 + 𝛿].
Variations without Derivatives

The Secant Method

Replace the derivative in Newton’s method by

𝑓(𝑝𝑛−2 ) − 𝑓(𝑝𝑛−1 )
𝑓 ′ (𝑝𝑛−1 ) ≈
𝑝𝑛−2 − 𝑝𝑛−1
to get
𝑓(𝑝𝑛−1 )(𝑝𝑛−1 − 𝑝𝑛−2 )
𝑝𝑛 = 𝑝𝑛−1 −
𝑓(𝑝𝑛−1 ) − 𝑓(𝑝𝑛−2 )

The Method of False Position (Regula Falsi)

Like the Secant method, but with a test to ensure the root is
bracketed between iterations.
Order of Convergence

Definition
Suppose {𝑝𝑛 }∞ 𝑛=0 is a sequence that converges to 𝑝, with 𝑝𝑛 ≠ 𝑝
for all 𝑛. If positive constants 𝜆 and 𝛼 exist with

|𝑝𝑛+1 − 𝑝|
lim = 𝜆,
𝑛→∞ |𝑝𝑛 − 𝑝|𝛼

then {𝑝𝑛 }∞𝑛=0 converges to 𝑝 of order 𝛼, with asymptotic error

constant 𝜆.
An iterative technique 𝑝𝑛 = 𝑔(𝑝𝑛−1 ) is said to be of order 𝛼 if the
sequence {𝑝𝑛 }∞𝑛=0 converges to the solution 𝑝 = 𝑔(𝑝) of order 𝛼.

Special cases
If 𝛼 = 1 (and 𝜆 < 1), the sequence is linearly convergent
If 𝛼 = 2, the sequence is quadratically convergent
Fixed Point Convergence

Theorem
Let 𝑔 ∈ 𝐶[𝑎, 𝑏] be such that 𝑔(𝑥) ∈ [𝑎, 𝑏], for all 𝑥 ∈ [𝑎, 𝑏].
Suppose 𝑔′ is continuous on (𝑎, 𝑏) and that 0 < 𝑘 < 1 exists with
|𝑔′ (𝑥)| ≤ 𝑘 for all 𝑥 ∈ (𝑎, 𝑏). If 𝑔′ (𝑝) ≠ 0, then for any number 𝑝0
in [𝑎, 𝑏], the sequence 𝑝𝑛 = 𝑔(𝑝𝑛−1 ) converges only linearly to the
unique fixed point 𝑝 in [𝑎, 𝑏].

Theorem
Let 𝑝 be solution of 𝑥 = 𝑔(𝑥). Suppose 𝑔′ (𝑝) = 0 and 𝑔″
continuous with |𝑔″ (𝑥)| < 𝑀 on open interval 𝐼 containing 𝑝.
Then there exists 𝛿 > 0 s.t. for 𝑝0 ∈ [𝑝 − 𝛿, 𝑝 + 𝛿], the sequence
defined by 𝑝𝑛 = 𝑔(𝑝𝑛−1 ) converges at least quadratically to 𝑝, and

𝑀
|𝑝𝑛+1 − 𝑝| < |𝑝 − 𝑝|2 .
2 𝑛
Newton’s Method as Fixed-Point Problem

Derivation
Seek 𝑔 of the form

𝑔(𝑥) = 𝑥 − 𝜙(𝑥)𝑓(𝑥).

Find differentiable 𝜙 giving 𝑔′ (𝑝) = 0 when 𝑓(𝑝) = 0:

𝑔′ (𝑥) = 1 − 𝜙′ (𝑥)𝑓(𝑥) − 𝑓 ′ (𝑥)𝜙(𝑥)

𝑔′ (𝑝) = 1 − 𝜙′ (𝑝) ⋅ 0 − 𝑓 ′ (𝑝)𝜙(𝑝)

and 𝑔′ (𝑝) = 0 if and only if 𝜙(𝑝) = 1/𝑓 ′ (𝑝). This gives Newton’s
method
𝑓(𝑝 )
𝑝𝑛 = 𝑔(𝑝𝑛−1 ) = 𝑝𝑛−1 − ′ 𝑛−1
𝑓 (𝑝𝑛−1 )
Multiplicity of Zeros

Definition
A solution 𝑝 of 𝑓(𝑥) = 0 is a zero of multiplicity 𝑚 of 𝑓 if for 𝑥 ≠ 𝑝,
we can write 𝑓(𝑥) = (𝑥 − 𝑝)𝑚 𝑞(𝑥), where lim𝑥→𝑝 𝑞(𝑥) ≠ 0.

Theorem
𝑓 ∈ 𝐶 1 [𝑎, 𝑏] has a simple zero at 𝑝 in (𝑎, 𝑏) if and only if 𝑓(𝑝) = 0,
but 𝑓 ′ (𝑝) ≠ 0.

Theorem
The function 𝑓 ∈ 𝐶 𝑚 [𝑎, 𝑏] has a zero of multiplicity 𝑚 at point 𝑝
in (𝑎, 𝑏) if and only if

0 = 𝑓(𝑝) = 𝑓 ′ (𝑝) = 𝑓 ″ (𝑝) = ⋯ = 𝑓 (𝑚−1) (𝑝), but 𝑓 (𝑚) (𝑝) ≠ 0.

Variants for Multiple Roots

Newton’s Method for Multiple Roots

Define 𝜇(𝑥) = 𝑓(𝑥)/𝑓 ′ (𝑥). If 𝑝 is a zero of 𝑓 of multiplicity 𝑚 and
𝑓(𝑥) = (𝑥 − 𝑝)𝑚 𝑞(𝑥), then

𝑞(𝑥)
𝜇(𝑥) = (𝑥 − 𝑝)
𝑚𝑞(𝑥) + (𝑥 − 𝑝)𝑞 ′ (𝑥)

also has a zero at 𝑝. But 𝑞(𝑝) ≠ 0, so

𝑞(𝑝) 1
′
= ≠ 0,
𝑚𝑞(𝑝) + (𝑝 − 𝑝)𝑞 (𝑝) 𝑚
and 𝑝 is a simple zero of 𝜇. Newton’s method can then be applied
to 𝜇 to give

𝑓(𝑥)𝑓 ′ (𝑥)
𝑔(𝑥) = 𝑥 −
[𝑓 ′ (𝑥)]2 − 𝑓(𝑥)𝑓 ″ (𝑥)
Aitken’s Δ2 Method

Accelerating linearly convergent sequences

Suppose {𝑝𝑛 }∞
𝑛=0 linearly convergent with limit 𝑝
Assume that
𝑝𝑛+1 − 𝑝 𝑝 −𝑝
≈ 𝑛+2
𝑝𝑛 − 𝑝 𝑝𝑛+1 − 𝑝
Solving for 𝑝 gives
2
𝑝𝑛+2 𝑝𝑛 − 𝑝𝑛+1 (𝑝𝑛+1 − 𝑝𝑛 )2
𝑝≈ = ⋯ = 𝑝𝑛 −
𝑝𝑛+2 − 2𝑝𝑛+1 + 𝑝𝑛 𝑝𝑛+2 − 2𝑝𝑛+1 + 𝑝𝑛

Use this for new more rapidly converging sequence {𝑝𝑛 𝑛=0 :
̂ }∞

(𝑝𝑛+1 − 𝑝𝑛 )2
𝑝𝑛̂ = 𝑝𝑛 −
𝑝𝑛+2 − 2𝑝𝑛+1 + 𝑝𝑛
Delta Notation

Definition
For a given sequence {𝑝𝑛 }∞
𝑛=0 , the forward difference Δ𝑝𝑛 is
defined by
Δ𝑝𝑛 = 𝑝𝑛+1 − 𝑝𝑛 , for 𝑛 ≥ 0
Higher powers of the operator Δ are defined recursively by

Δ𝑘 𝑝𝑛 = Δ(Δ𝑘−1 𝑝𝑛 ), for 𝑘 ≥ 2

Aitken’s Δ2 method using delta notation

Since Δ2 𝑝𝑛 = 𝑝𝑛+2 − 2𝑝𝑛+1 + 𝑝𝑛 , we can write

(Δ𝑝𝑛 )2
𝑝𝑛̂ = 𝑝𝑛 − , for 𝑛 ≥ 0
Δ2 𝑝𝑛
Convergence of Aitken’s Δ2 Method

Theorem
Suppose that {𝑝𝑛 }∞
𝑛=0 converges linearly to 𝑝 and that

𝑝𝑛+1 − 𝑝
lim <1
𝑛→∞ 𝑝𝑛 − 𝑝

Then {𝑝𝑛 𝑛=0 converges to 𝑝 faster than {𝑝𝑛 }𝑛=0 in the sense that
̂ }∞ ∞

𝑝𝑛̂ − 𝑝
lim =0
𝑛→∞ 𝑝𝑛 − 𝑝
Steffensen’s Method

Accelerating fixed-point iteration

Aitken’s Δ2 method for fixed-point iteration gives

𝑝0 , 𝑝1 = 𝑔(𝑝0 ), 𝑝2 = 𝑔(𝑝1 ), 𝑝0̂ = {Δ2 }(𝑝0 ),

𝑝3 = 𝑔(𝑝2 ), 𝑝1̂ = {Δ2 }(𝑝1 ), …

Steffensen’s method assumes 𝑝0̂ is better than 𝑝2 :

(0) (0) (0) (0) (0) (1) (0)
𝑝0 , 𝑝1 = 𝑔(𝑝0 ), 𝑝2 = 𝑔(𝑝1 ), 𝑝0 = {Δ2 }(𝑝0 ),
(1) (1)
𝑝1 = 𝑔(𝑝0 ), …

Theorem
Suppose 𝑥 = 𝑔(𝑥) has solution 𝑝 with 𝑔′ (𝑝) ≠ 1. If exists 𝛿 > 0 s.t.
𝑔 ∈ 𝐶 3 [𝑝 − 𝛿, 𝑝 + 𝛿], then Steffensen’s method gives quadratic
convergence for 𝑝0 ∈ [𝑝 − 𝛿, 𝑝 + 𝛿].
Steffensen’s Method

MATLAB Implementation
function p = steffensen(g, p0, tol)
% Solve g(p) = p using Steffensen's method.

while 1
p1 = g(p0);
p2 = g(p1);
p = p0 - (p1-p0)^2 / (p2-2*p1+p0);
if abs(p-p0) < tol, break; end
p0 = p;
end
Zeros of Polynomials
Polynomial
A polynomial of degree 𝑛 has the form 𝑃 (𝑥) = 𝑎𝑛 𝑥𝑛 + 𝑎𝑛−1 𝑥𝑛−1 +
⋯ + 𝑎1 𝑥 + 𝑎0 with coefficients 𝑎𝑖 and 𝑎𝑛 ≠ 0.

Theorem (Fundamental Theorem of Algebra)

If 𝑃 (𝑥) polynomial of degree 𝑛 ≥ 1, with real or complex
coefficients, 𝑃 (𝑥) = 0 has at least one root.

Corollary
Exists unique 𝑥1 , … , 𝑥𝑘 and 𝑚1 , … , 𝑚𝑘 , with ∑𝑖=1 𝑚𝑖 = 𝑛 and
𝑘

𝑃 (𝑥) = 𝑎𝑛 (𝑥 − 𝑥1 )𝑚1 (𝑥 − 𝑥2 )𝑚2 ⋯ (𝑥 − 𝑥𝑘 )𝑚𝑘 .

Corollary
𝑃 (𝑥), 𝑄(𝑥) polynomials of degree at most 𝑛. If 𝑃 (𝑥𝑖 ) = 𝑄(𝑥𝑖 ) for
𝑖 = 1, 2, … , 𝑘, with 𝑘 > 𝑛, then 𝑃 (𝑥) = 𝑄(𝑥).
Horner’s Method

Theorem (Horner’s Method)

Let 𝑃 (𝑥) = 𝑎𝑛 𝑥𝑛 + 𝑎𝑛−1 𝑥𝑛−1 + ⋯ + 𝑎1 𝑥 + 𝑎0 . If 𝑏𝑛 = 𝑎𝑛 and

𝑏𝑘 = 𝑎𝑘 + 𝑏𝑘+1 𝑥0 , for 𝑘 = 𝑛 − 1, 𝑛 − 2, … , 1, 0,

then 𝑏0 = 𝑃 (𝑥0 ). Moreover, if

𝑄(𝑥) = 𝑏𝑛 𝑥𝑛−1 + 𝑏𝑛−1 𝑥𝑛−2 + ⋯ + 𝑏2 𝑥 + 𝑏1 ,

then 𝑃 (𝑥) = (𝑥 − 𝑥0 )𝑄(𝑥) + 𝑏0 .

Computing Derivatives
Differentiation gives

𝑃 ′ (𝑥) = 𝑄(𝑥) + (𝑥 − 𝑥0 )𝑄′ (𝑥) and 𝑃 ′ (𝑥0 ) = 𝑄(𝑥0 ).

Horner’s Method

MATLAB Implementation
function [y, z] = horner(a, x0)
% Evaluate polynomial:
% P(x) = a(1)x^n + a(2)x^(n-1) + ... + a(n)x + a(n+1)
% and its derivative at x0 using Horner's method.
% Outputs: y = P(x0), z = P'(x0).

n = length(a)-1;
y = a(1);
z = a(1);
for j = 2:n
y = x0*y + a(j);
z = x0*z + y;
end
y = x0*y + a(n+1);
Deflation

Deflation
Compute approximate root 𝑥1̂ using Newton. Then

𝑃 (𝑥) ≈ (𝑥 − 𝑥1̂ )𝑄1 (𝑥).

Apply recursively on 𝑄1 (𝑥) until the quadratic factor 𝑄𝑛−2 (𝑥)

can be solved directly.
Improve accuracy with Newton’s method on original 𝑃 (𝑥).
Müller’s Method

Müller’s Method
Similar to the Secant method, but parabola instead of line
Fit quadratic polynomial 𝑃 (𝑥) = 𝑎(𝑥 − 𝑝2 )2 + 𝑏(𝑥 − 𝑝2 ) + 𝑐
that passes through (𝑝0 , 𝑓(𝑝0 )),(𝑝1 , 𝑓(𝑝1 )),(𝑝2 , 𝑓(𝑝2 )).
Solve 𝑃 (𝑥) = 0 for 𝑝3 , choose root closest to 𝑝2 :

2𝑐
𝑝3 = 𝑝 2 − √ .
𝑏 + sgn(𝑏) 𝑏2 − 4𝑎𝑐
Repeat until convergence
Relatively insensitive to initial 𝑝0 , 𝑝1 , 𝑝2 , but e.g.
𝑓(𝑝𝑖 ) = 𝑓(𝑝𝑖+1 ) = 𝑓(𝑝𝑖+2 ) ≠ 0 gives problems