Aims

In this lecture, we will . . .

I Introduce the Fixed-Point Method
I Introduce the Newton’s Method
Fixed-Point Method

The basic idea of this method which is also called successive approximation
method or function iteration, is to rearrange the original equation

f (x) = 0, (1)

into an equivalent expression of the form

x = g(x). (2)

Any solution of (2) is called a fixed-point for the iteration function g(x) and hence
a root of (1).
Definition 1
(Fixed-Point of a Function)
A fixed-point of a function g(x) is a real number α such that α = g(α).
x2 − 4x + 8
For example, x = 2 is a fixed-point of the function g(x) = because
2
g(2) = 2. •
The fixed-point method essentially solves two functions simultaneously; y = x and
y = g(x). The point of intersection of these two functions is the solution to
x = g(x), and thus to f (x) = 0, see Figure 1.

y=x
y=g(x)

0
x5 x4 x3 x2 x1 x

Figure: Graphical Solution of Fixed-Point Method.

Definition 2
(Fixed-Point Method)
The iteration defined in the following

xn+1 = g(xn ); n = 0, 1, 2, . . . , (3)

is called the fixed-point method or the fixed-point iteration. •

The value of the initial approximation x0 is chosen arbitrarily and the hope is that
the sequence {xn }∞ n=0 converges to a number α which will automatically satisfies
(1). Moreover, since (1) is a rearrangement of (2), α is guaranteed to be a zero of
f (x). In general, there are many different ways of rearranging of (2) in (1) form.
However, only some of these are likely to give rise to successful iterations but
sometime we don’t have successful iterations. To describe such behaviour, we
discuss the following example.
Example 0.1
Consider the nonlinear equation x3 = 2x + 1 which has a root in the interval
[1.5, 2.0] using fixed-point method with x0 = 1.5, take three different
rearrangements for the equation.
Solution. Let us consider the three possible rearrangement of the given equation
as follows:
(x3n − 1)
(i) xn+1 = g1 (xn ) = ; n = 0, 1, 2, . . . ,
2
1
(ii) xn+1 = g2 (xn ) = ; n = 0, 1, 2, . . . ,
(x2n − 2)
s
(2xn + 1)
(iii) xn+1 = g3 (xn ) = ; n = 0, 1, 2, . . . ,
xn

then the numerical results for the corresponding iterations, starting with the
initial approximation x0 = 1.5 with accuracy 5 × 10−2 , are given in Table 1.
 Table: Solution of x3 = 2x + 1 by fixed-point method

n xn xn+1 = g1 (xn ) xn+1 = g2 (xn ) xn+1

p = g3 (xn )
= (x3n − 1)/2 = 1/(x2n − 2) = (2xn + 1)/xn
00 x0 1.500000 1.500000 1.500000
01 x1 1.187500 4.000000 1.632993
02 x2 0.337280 0.071429 1.616284
03 x3 -0.480816 -0.501279 1.618001
04 x4 -0.555579 -0.571847 1.618037
05 x5 -0.585745 -0.597731 1.618034

We note that the first two considered sequences diverge and the last one
converges. This example asks the need for a mathematical analysis of the method.
The following theorem gives sufficient conditions for the convergence of the
fixed-point iteration. •
Theorem 3
(Fixed-Point Theorem)
If g is continuously differentiable on the interval [a, b] and g(x) ∈ [a, b] for all
x ∈ [a, b], then
(a) g has at-least one fixed-point in the given interval [a, b].
Moreover, if the derivative g 0 (x) of the function g(x) exists on an interval [a, b]
which contains the starting value x0 , with

k ≡ max |g 0 (x)| < 1; for all x ∈ [a, b]. (4)

a≤x≤b

Then
(b) The sequence (3) will converge to the attractive (unique) fixed-point α in
[a, b].
(c) The iteration (3) will converge to α for any initial approximation.
(d) We have the error estimate

kn
|α − xn | ≤ |x1 − x0 |, for all n ≥ 1. (5)
1−k
(e) The limit holds:
α − xn+1
lim = g 0 (α). (6)
n→∞ α − xn
Now we come back to our previous Example 0.1 and discuss that why the first two
rearrangements we considered, do not converge but on the other hand, last
sequence has a fixed-point and converges.
Since, we observe that f (1.5)f (2) < 0, then the solution we seek is in the interval
[1.5, 2].
x3 − 1
(i) For g1 (x) = , we have g10 (x) = x2 , which is greater than unity
3
throughout the interval [1.5, 2]. So by Fixed-Point Theorem 3 this iteration
will fail to converge.
1 −2x
(ii) For g2 (x) = 2 , we have g20 (x) = 2 , and |g20 (1.5)| > 1, so from
x −2 (x − 2)2
Fixed-Point Theorem 3 this iteration will fail to converge.
r
2x + 1 √
(iii) For g3 (x) = , we have g30 (x) = x−3/2 /2 2x + 1 < 1, for all x in the
x
given interval [1.5, 2]. Also, g3 is decreasing function of x, and
g3 (1.5) = 1.63299 and g3 (2) = 1.58114 both lie in the interval [1.5, 2]. Thus
g3 (x) ∈ [1.5, 2], for all x ∈ [1.5, 2], so from Fixed-Point Theorem 3 the
iteration will converge, see Figure 2. •
 2

1.9

1.8
y = [(2x + 1)/x]1/2
1.7
α
1.6

y 1.5

1.4

1.3

1.2
y = x
1.1

1
1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2
x
p
Figure: Graphical Solution of x = (2x + 1)/x.
Note1
From (5) Note that the rate of convergence of the fixed-point method depends on
kn
the factor ; the smaller the value of k, then faster the convergence. The
(1 − k)
convergence may be very slow if the value of k is very close to 1. •
Note2
Assume that g(x) and g 0 (x) are continuous functions of x for some open interval I,
with the fixed-point α contained in this interval. Moreover assume that

|g 0 (α)| < 1, for α ∈ I,

then, there exists an interval [a, b], around the solution α for which all the
conditions of Theorem 3 are satisfied. But if

|g 0 (α)| > 1, for α ∈ I,

then the sequence (3) will not converge to α. In this case α is called a repulsive
fixed-point. If
|g 0 (α)| = 0, for α ∈ I,
then the sequence (3) converges very fast to the root α while if

|g 0 (α)| = 1, for α ∈ I,

then the convergence the sequence (3) is not guaranteed and if the convergence
happened, it would be very slow. Thus to get the faster convergence, the value of
|g 0 (α)| should be equal to zero or very close to zero. •
Example 0.2
2 − e x + x2
Find an interval [a, b] on which fixed-point problem x = will
3
converges. Estimate the number of iterations n within accuracy 10−5 .
2 − ex + x2
Solution. Since x = can be written as
3

f (x) = ex − x2 + 3x − 2 = 0,

and we observe that f (0)f (1) = (−1)(e1 ) < 0, then the solution we seek is in the
interval [0, 1].
2 − ex + x2 2x − ex
For g(x) = , we have g 0 (x) = < 1, for all x in the given
3 3
interval [0, 1]. Also, g is decreasing function of x and g(0) = 0.3333 and
3−e
g(1) = = 0.0939 both lie in the interval [0, 1]. Thus g(x) ∈ [0, 1], for all
3
x ∈ [0, 1], so from Fixed-Point Theorem 3 the g(x) has a unique fixed-point in
[0, 1]. Taking x0 = 0.5, we have

2 − ex0 + x20
x1 = g(x0 ) = = 0.2004.
3
Also, we have

k1 = |g 0 (0)| = 0.3333 and k2 = |g 0 (1)| = 0.2394,

which give k = max{k1 , k2 } = 0.3333. Thus the error estimate (5) within the
accuracy 10−5 is

(0.3333)n
|α − xn | ≤ 10−5 , gives (0.2996) ≤ 10−5 ,
1 − 0.3333
and by solving this inequality, we obtain n ≥ 9.7507. So we need ten
approximations to get the desired accuracy for the given problem. •
Example 0.3
Show that the function g(x) = 3−x on the interval [0, 1] has at least one
fixed-point but it is not unique.
Solution. Given x = g(x) = 3−x , and it can be written as

x − 3−x = f (x) = 0.

So f (0)(1) = (−1)(2/3) < 0, so f (x) has a root in the interval [0, 1], see Figure 3.
Note that g is decreasing function of x and g(0) = 1 and g(1) = 0.3333 both lie in
the interval [0, 1]. Thus g(x) ∈ [0, 1], for all x ∈ [0, 1], so from Fixed-Point
Theorem 3 the function g(x) has at least one fixed-point in [0, 1]. Since the
derivative of the function g(x) is

g 0 (x) = −3−x ln 3,

which is less than zero on [0, 1], therefore, the function g is decreasing on [0, 1]. But
g 0 (0) = − ln 3 = −1.0986, so

|g 0 (x)| > 1 on (0, 1).

Thus from Fixed-Point Theorem 3 the function g(x) has no unique fixed-point in
[0, 1]. •
 1

0.9
y = 3−x
0.8
y=x

0.7

0.6
y
0.5

0.4

0.3

0.2

0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x

Figure: Graphical Solution of x = 3−x .

Example 0.4
√
Show that the function g(x) = 2x − 1 on the interval [0, 1] that satisfies none of
the hypothesis of Theorem 3 √
but still has a unique fixed-point on [0, 1].
Solution. Since x = g(x) = 2x − 1, it gives

x2 − 2x + 1 = (x − 1)2 = f (x) = 0.

Then x = α = 1 ∈ [0, 1] is the root of the nonlinear equation f (x) = 0 and the
fixed-point of the function g(x) as g(1) = 1. But notice that the function g(x) is
not continuous on the interval [0, 1] and the derivative of the function g(x)

1
g 0 (x) = √ ,
2x − 1

does not exist on the interval (0, 1). So all the conditions of Fixed-Point
Theorem 3 fail. •
Example 0.5
Show that the fixed point form of the equation x = N 1/3 can be written as
x = N x−2 and the associated iterative scheme

xn+1 = N x−2
n , n ≥ 0,

will not successful (diverge) in finding the approximation of cubic root of the
positive number N .
Solution. Given x = N 1/3 and it can be written as
N
x3 − N = 0 or x= = N x−2 .
x2
It gives the iterative scheme

xn+1 = N x−2
n = g(xn ), n ≥ 0.

From this, we have

g(x) = N x−2 and g 0 (x) = −2N x−3 .

Since α = x = N 1/3 , therefore

g 0 (α) = −2N α−3 and g 0 (N 1/3 ) = −2N (N 1/3 )−3 = −2N N −1 = −2.

Thus
|g 0 (N 1/3 )| = | − 2| = 2 > 1,
which shows the divergence. •
Example 0.6
One of the possible rearrangement of the nonlinear equation ex = x + 2, which has
root in [1, 2] is

xn+1 = g(xn ) = ln(xn + 2); n = 0, 1, . . . .

(a) Show that g(x) has a unique fixed-point in [1, 2].

(b) Use fixed-point iteration formula (3) to compute approximation x3 , using
x0 = 1.5.
(c) Compute an error estimate |α − x3 | for your approximation.
(d) Determine the number of iterations needed to achieve an approximation with
accuracy 10−2 to the solution of g(x) = ln(x + 2) lying in the interval [1, 2] by
using the fixed-point iteration method.
Solution. Since, we observe that f (1)f (2) < 0, then the solution we seek is in the
interval [1, 2].
(a) For g(x) = ln(x + 2), we have g 0 (x) = 1/(x + 2) < 1, for all x in the given
interval [1, 2]. Also, g is increasing function of x, and
g(1) = ln(3) = 1.0986123 and g(2) = ln(4) = 1.3862944 both lie in the interval
[1, 2]. Thus g(x) ∈ [1, 2], for all x ∈ [1, 2], so from fixed-point theorem the g(x)
has a unique fixed-point, see Figure 4.
(b) using the given initial approximation x0 = 1.5, we have the other
approximations as
x1 = g(x0 ) = 1.252763, x2 = g(x1 ) = 1.179505, x3 = g(x2 ) = 1.156725.
(c) Since a = 1 and b = 2, then the value of k can be found as follows
k1 = |g 0 (1)| = |1/3| = 0.333 and k2 = |g 0 (2)| = |1/4| = 0.25,
which give k = max{k1 , k2 } = 0.333. Thus using the error formula (5), we
have
(0.333)3
|α − x3 | ≤ |1.252763 − 1.5| = 0.013687.
1 − 0.333
(d) From the error bound formula (5), we have
kn
|x1 − x0 | ≤ 10−2 .
1−k
By using above parts (b) and (c), we have
(0.333)n
|1.252763 − 1.5| ≤ 10−2 .
1 − 0.333
Solving this inequality, we obtain
n ln(0.333) ≤ ln(0.02698), gives, n ≥ 3.28539.
So we need four approximations to get the desired accuracy for the given
problem. •
 3.5 2

1.9
3
1.8
2.5
1.7
2 y = ex − x − 2 1.6
y

y
1.5 1.5

1.4 y=x y = ln(x + 2)

1
1.3
0.5
α 1.2 α
0
1.1

−0.5 1
1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2
x x

Figure: Graphical Solution of ex = x + 2 Graphical solution of x = ln(x + 2).

MATLAB command for the above given rearrangement x = g(x) of
f (x) = x3 − 2x − 1 by using the initial approximation x0 = 1.5, can be written as
follows:
f unction y = f n(x)
y = log(x + 2);
>> x0 = 1.5; tol = 0.01; sol = f ixpt(0 f n0 , x0, tol);

Program 2.2
MATLAB m-file for the Fixed-Point Method
function sol=fixpt(fn,x0,tol)
old= x0+1; while abs(x0-old) > tol; old=x0;
x0 = f eval(f n, old); end; sol=x0;
Procedure
(Fixed-Point Method))

1. Choose an initial approximation x0 such that x0 ∈ [a, b].

2. Choose a convergence parameter  > 0.
3. Compute new approximation xnew by using the iterative formula (3).
4. Check, if |xnew − x0 | <  then xnew is the desire approximate root; otherwise
set x0 = xnew and go to step 3.
Newton’s Method
This is one of the most popular and powerful iterative method for finding roots of
the nonlinear equation. It is also known as the method of tangents because after
estimated the actual root, the zero of the tangent to the function at that point is
determined. The Newton’s method consists geometrically of expanding the
tangent line at a current point xi until it crosses zero, then setting the next guess
xi+1 to the abscissa of that zero crossing, see Figure 5. This method is also called
the Newton-Raphson method.

y=f(x)

0
x x x1 x
3 2

Figure: Graphical Solution of Newton’s Method.

There are many description of the Newton’s method. We shall derive the method
from the familiar Taylor’s series expansion of a function in the neighborhood of a
point. Let f ∈ C 2 [a, b] and let xn be the nth approximation to the root α such
that f 0 (xn ) 6= 0 and |α − xn | is small. Consider the first Taylor polynomial for
f (x) expanded about xn , so we have

(x − xn )2 00
f (x) = f (xn ) + (x − xn )f 0 (xn ) + f (η(x)), (7)
2
where η(x) lies between x and xn . Since f (α) = 0, then (7), with x = α, gives

(α − xn )2 00
f (α) = 0 = f (xn ) + (α − xn )f 0 (xn ) + f (η(α)).
2
Since |α − xn | is small, then we neglect the term involving (α − xn )2 and so

0 ≈ f (xn ) + (α − xn )f 0 (xn ).

Solving for α, we get

f (xn )
α ≈ xn − , (8)
f 0 (xn )
which should be better approximation to α than is xn . We call this approximation
as xn+1 , then we get

f (xn )
xn+1 = xn − , f 0 (xn ) 6= 0, for all n ≥ 0. (9)
f 0 (xn )

The iterative method (9) is called the Newton’s method.

Example 0.7
Use Newton’s method to find the approximation x3 to the root of

cos x − x = 0,

where x0 = π/4. Solution.

Let f (x) = cos x − x = 0, and use Using the Newton’s iterative formula (9), we get

f (xn )
xn+1 = xn −
f 0 (xn )

to find the iterations, where x0 = π/4. Thus we get:

Table: Solution of cos x − x = 0 by Newton’s method

n xn f (xn )
1 0.7853981635 -0.078291381
2 0.7395361337 -0.000754873
3 0.7390851781 -0.000000074

Therefore, x3 = 0.7390851781.
Example 0.8
Use the Newton’s method to find the root of x3 = 2x + 1 that is located in the
interval [1.5, 2.0] accurate to 10−2 , take an initial approximation x0 = 1.5.
Solution. Given f (x) = x3 − 2x − 1 and so f 0 (x) = 3x2 − 2. Now evaluating f (x)
and f 0 (x) at the give approximation x0 = 1.5, gives

x0 = 1.5, f (1.5) = −0.625, f 0 (1.5) = 4.750.

Using the Newton’s iterative formula (9), we get

f (x0 ) (−0.625)
x1 = x0 − = 1.5 − = 1.631579.
f 0 (x0 ) 4.75

Now evaluating f (x) and f 0 (x) at the new approximation x1 , gives

x1 = 1.631579, f (1.631579) = 0.0801869, f 0 (1.631579) = 5.9861501.

Using the iterative formula (9) again to get other new approximation. The
successive iterates were shown in the Table 3.

Table: Solution of x3 = 2x + 1 by Newton’s method

n xn f (xn ) f 0 (xn ) Error = x − xn

00 1.500000 -0.625000 4.750000 0.1180339
01 1.631579 0.0801869 5.9861501 -0.0135451
02 1.618184 0.000878 5.855558 -0.0001501
03 1.618034 0.00000007 5.854102 -0.0000001
Just after the third iterations the required root is approximated to be
x3 = 1.618034 and the functional value is reduced to 7.0 × 10−8 . Since the exact
solution is 1.6180339, so the actual error is 1 × 10−7 . We see that the convergence
is quite faster than the methods considered previously. •

To get the above results using MATLAB command, firstly the function
x3 − 2x − 1 and its derivative 3x2 − 2 were saved in m-files called fn.m and dfn.m,
respectively written as follows:
f unction y = f n(x) f unction dy = df n(x)
y = x.ˆ 3 − 2 ∗ x − 1; dy = 3 ∗ x.ˆ 2 − 2;
after which we do the following:
>> x0 = 1.5; tol = 0.01; sol = newton(0 f n0 ,0 df n0 , x0, tol);
Example 0.9
If the difference of two numbers x and y is 6 and the square root of their product
is 4, then use Newton’s method to approximate, to within 10−4 , the largest value
of the number x and the corresponding number y using initial approximation
x0 = 7.5.
Solution. Given
√
x − y = 6 and xy = 4.
Solving the above equations for x, we have

x(x − 6) = 16 or x2 − 6x − 16 = f (x) = 0.

Applying Newton’s iterative formula (9) to find the approximation of this

equation, we have
x2 − 6xn − 16
xn+1 = xn − n .
2xn − 6
Finding the approximation to within 10−4 using the initial approximation
x0 = 7.5, we get
x2 − 6x0 − 16
x1 = x0 − 0 = 8.0278,
2x0 − 6
and continue in the same manner, we get the approximations within accuracy
10−4 as follows
x2 = 8.0001, x3 = 8.0000, x4 = 8.0000.
Thus the largest value of number x is 8 and its corresponding y value is 2. •
Example 0.10
The graphs of y = 2 sin x and y = ln(x) + k touch each other in the neighborhood
of point x = 8. Find the value of the constant k and the coordinates of point of
contact, use x0 = 8.
Solution. Since we know that the graphs will touch each other if the values of
derivatives at their point of contact is same. So for

y = 2 sin x, gives, y 0 = 2 cos x,

and
1
y = ln(x) + k, gives, y0 = .
x
Thus
1
2 cos x = , gives, x cos x − 0.5 = 0,
x
and from this we have the function and its derivative as follows

f (x) = x cos x − 0.5 and f 0 (x) = cos x − x sin x.

Using Newton’s iterative formula (9), we get

xn cos xn − 0.5
xn+1 = xn − ,
cos xn − xn sin xn
and for finding the approximations, starting x0 = 8, we obtain, x1 = 7.7936 and
x2 = 7.7897. Taking x = 7.79, we have y = 2 sin 7.79 = 1.996. Therefore, the point
of contact is (7.79, 1.996). To find the value of k, we solve the equation,
1.996 = ln(7.79) + k, and it gives, k = −0.0568, the required value of k. •
Example 0.11
Develop the iterative formula

x2n − b
xn+1 = , n ≥ 0,
2xn − a

for the approximate roots of the quadratic equation x2 − ax + b = 0 using the

Newton’s method. Then use the formula to find the third approximation of the
positive root of the equation x2 − 3x = 4, starting with x0 = 3.5.
Solution. Given
f (x) = x2 − ax + b,
therefore, we have (see Figure 6),

f (xn ) = x2n − axn + b and f 0 (xn ) = 2xn − a.

Using these functions values in the Newton’s iterative formula (9), we have

x2n − axn + b x2 − b
xn+1 = xn − = n , n ≥ 0.
2xn − a 2xn − a

Finding the first three approximations of the positive root of x2 − 3x = 4 using

the initial approximation x0 = 3.5 and a = 3, b = −4, we use the above formula by
taking n = 0, 1, 2 as follows
 x20 − b x21 − b x22 − b
x1 = = 4.0625, x2 = = 4.0008, x3 = = 4.0000,
2x0 − a 2x1 − a 2x2 − a
are the possible three approximations. Note that the positive root of
x2 − 3x − 4 = 0 is 4, so we have

|4 − x3 | = |4 − 4| = 0.0000,

the possible absolute error. •

3 4.5

2 y = (x2 + 4)/(2x − 3)
1
α
4
α
0
y

y
−1 y = x2 − 3x − 4
3.5 y=x
−2

−3

−4 3
3 3.5 4 4.5 3 3.5 4 4.5
x x

Figure: Graphical Solution of x2 − 3x = 4 and x = (x2 + 4)/(2x − 3).

Example 0.12
Develop an iterative procedure for evaluating the reciprocal of a positive number
N by using Newton’s method. Use the developed formula to find third
approximation to the reciprocal of 3, taking an initial approximation x0 = 0.4.
Compute absolute error.
Solution. Consider x = 1/N . This problem can be easily solved by noting that
we seek to find a root to the nonlinear equation

1/x − N = 0,

where N > 0 is the number whose reciprocal is to be found. Therefore, if

f (x) = 0, then x = 1/N is the exact root. Let

f (x) = 1/x − N and f 0 (x) = −1/x2 .

Hence, assuming an initial estimate to the root, say, x = x0 and by using iterative
formula (9), we get

(1/x0 − N )
x1 = x0 − = x0 + (1/x0 − N )x20 = x0 + x0 − N x20 = x0 (2 − N x0 ).
(−1/x20 )

In general, we have

xn+1 = xn (2 − N xn ), n = 0, 1, . . . , (10)
We have to find the approximation of the reciprocal of number N = 3. Given the
initial gauss of say x0 = 0.4, then by using the iterative formula (10), we get
x1 = 0.3200, x2 = 0.3328, x3 = 0.3333.

2 1

1.5
0.8 y=x
y = 1/x − 3
0.6
1 α
0.4
0.5
0.2
α
y

y
0 0

−0.2
−0.5 y = x(2 − 3x)
−0.4
−1
−0.6
−1.5
−0.8

−2 −1
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x x

Figure: Graphical Solution of 1/x = 3 and x = x(2 − 3x).

After just three iterations the estimated value compares rather favorably with the
exact value of 1/3 ≈ 0.3333, (see Figure 7). Thus the absolute error is
1
|E| = − x3 = |0.3333 − 0.3333| = 0.0000.
3
We can calculate the other reciprocal of the number in the same way by using the
general iterative formula (10). •
Procedure
(Newton’s Method)

1. Find the initial approximation x0 for the root by sketching the graph of the
function.
2. Evaluate function f (x) and the derivative f 0 (x) at initial approximation.
Check: if f (x0 ) = 0 then x0 is the desire approximation to a root. But if
f 0 (x0 ) = 0, then go back to step 1 to choose new approximation.
3. Establish Tolerance ( > 0) value for the function.
4. Compute new approximation for the root by using the iterative formula (9).
5. Check Tolerance. If |f (xn )| ≤ , for n ≥ 0, then end; otherwise, go back to
step 4, and repeat the process.
Summary

In this lecture, we ...

I Introduced the Fixed-Point Method
I Introduced the Newton’s Method