MATH1070 2 Error and Computer Arithmetic PDF

> 2.
Error and Computer Arithmetic
Numerical analysis is concerned with how to solve a problem

numerically, i.e., how to develop a sequence of numerical calculations
to get a satisfactory answer.
Part of this process is the consideration of the errors that arise in these
calculations, from the errors in the arithmetic operations or from other
sources.
2. Error and Computer Arithmetic
Math 1070
> 2. Error and Computer Arithmetic
Computers use binary arithmetic, representing each number as a

binary number: a finite sum of integer powers of 2.
Some numbers can be represented exactly, but others, such as
1
1
1
10 , 100 , 1000 , . . ., cannot.
For example,
2.125 = 21 + 23
has an exact representation in binary (base 2), but
3.1 21 + 20 + 24 + 25 + 28 +
does not.
And, of course, there are the transcendental numbers like that have
no finite representation in either decimal or binary number system.
Math 1070
> 2. Error and Computer Arithmetic
Computers use 2 formats for numbers.

Fixed-point numbers are used to store integers.
Typically, each number is stored in a computer word of 32 binary
digits (bits) with values of 0 and 1. at most 232 different
numbers can be stored.
If we allow for negative numbers, we can represent integers in the
range 231 x 231 1, since there are 232 such numbers.
Since 231 2.1 109 , the range for fixed-point numbers is too
limited for scientific computing. =
always get an integer answer.
the numbers that we can store are equally spaced.
very limited range of numbers.
Therefore they are used mostly for indices and counters.

An alternative to fixed-point, floating-point numbers
approximate real numbers.
the numbers that we can store are NOT equally spaced.
wide range of variably-spaced numbers that can be representeed
exactly.
Math 1070
> 2. Error and Computer Arithmetic > 2.1 Floating-point numbers
Numbers must be stores and used for arithmetic operations. Storing:

1
integer format
floating-point format
Definition (decimal Floating-point representation)

Let consider x 6= 0 written in decimal system.
Then it can be written uniquely as
x = x 10e
(4.1)
where
= +1 or 1 is the sign
e is an integer, the exponent
1 x < 10, the significand or mantissa
Example (
2
124.62 = (1.2462
| {z }) 10
= +1, the exponent e = 2, the significand x = 1.2462

Math 1070
The decimal floating-point representation of x R is given in (4.1),

with limitations on the
1
number of digits in mantissa x
size of e
Example
Suppose we limit
1
number of digits in x to 4
99 e 99
We say that a computer with such a representation has a four-digit decimal

floating point arithmetic.
This implies that we cannot store accurately more than the first four digits of
a number; and even the fourth digit may be changed by rounding.
What is the next smallest number bigger than 1? What is the next smallest
number bigger than 100? What are the errors and relative errors?
What is the smallest positive number?
Math 1070
Definition (Floating-point representation of a binary number x)

Let consider x written in binary format. Analogous to (4.1)
x = x 2e
(4.2)
where
= +1 or 1 is the sign
e is an integer, the exponent
x is a binary fraction satisfying
(1)2 x < (10)2
(in decimal:1 x < 2)
For example, if
x = (11011.0111)2
then = +1, e = 4 = (100)2 and x = (1.10110111)2
Math 1070
The floating-point representation of a binary number x is given by (4.2) with

a restriction on
1 number of digits in x: the precision of the binary floating-point
representation of x
2 size of e
The IEEE floating-point arithmetic standard is the format for floating point
numbers used in almost all computers.
the IEEE single precision floating-point representation of x has
a precision of 24 binary digits,
and the exponent e is limited by 126 e 127:
x = (1.a1 a2 . . . a23 ) 2e
where, in binary
(1111110)2 e (1111111)2
the IEEE double precision floating-point representation of x has
a precision of 53 binary digits,
x = (1.a1 a2 . . . a52 ) 2e
Math 1070
the IEEE single precision floating-point representation of x

has a precision of 24 binary digits,
x = (1.a1 a2 . . . a23 ) 2e
stored on 4 bytes (32 bits)
b1
|{z}
b2 b 3 b 9
| {z }
E=e+127
b b b
|10 11{z 32}
x
the IEEE double precision floating-point representation of x

has a precision of 53 binary digits,
x = (1.a1 a2 . . . a52 ) 2e
stored on 8 bytes (64 bits)
b1
|{z}
b b b
|2 3 {z 12}
E=e+1023
b13 b14 b64

|
{z
}
x
Math 1070
> 2. Error and Computer Arithmetic > 2.1.1 Accuracy of floating-point representation
Epsilon machine
How accurate can a number be stored in the floating point representation?
How can this be measured?
(1) Machine epsilon
Machine epsilon
For any format, the machine epsilon is the difference between 1 and the next
larger number that can be stored in that format.
In single precision IEEE, the next larger binary number is
1.0000000000000000000000 |{z}
1
a23
(1 + 224 cannot be stored exactly)

Then the machine epsilon in single precision IEEE format is
223 = 1.19 107

i.e., we can store approximately 7 decimal digits of a number x in decimal format.
Math 1070
Then the machine epsilon in double precision IEEE format is
252 = 2.22 1016

IEEE double-precision format can be used to store approximately 16
decimal digits of a number x in decimal format.
MATLAB: machine epsilon is available as the constant eps.
Math 1070
Another way to measure the accuracy of floating-point format:

(2) look for the largest integer M such that any integer x such that
0 x M can be stored and represented exactly in floating point form.
If n is the number of binary digits in the significand x, all integers less or
equal to

(1.11 . . . 1)2 2n1 = 1 + 1 21 + 1 22 + . . . + 1 2(n1) 2n1

1 21n
2n1 = 2n 1
=
1 12
can be represented exactly.
In IEEE single precision format
M = 224 = 16777216
and all 7-digit decimal integers will store exactly.
In IEEE double precision format
M = 253 = 9.0 1015
and all 15-digit decimal integers and most 16 digit ones will store exactly.
Math 1070
> 2. Error and Computer Arithmetic > 2.1.2 Rounding and chopping
Let say that a number x has a significand

x = 1.a1 a2 . . . an1 an an+1
but the floating-point representation may contain only n binary digits.
Then x must be shortened when stored.
Definition
We denote the machine floating-point version of x by f l(x).
1
truncate or chop x to n binary digits, ignoring the remaining

digits
round x to n binary digits, based on the size of the part of x
following digit n:
(a) if digit n + 1 is 0, chop x to n digits
(b) if digit n + 1 is 1, chop x to n digits and add 1 to the last digit of
the result
Math 1070
It can be shown that

f l(x) = x (1 + )
(4.3)
where is a small number depending of x.

(a) If chopping is used
2n+1 0
(4.4)
2n 2n
(4.5)
(b) If rounding is used
Characteristics of chopping:
1 the worst possible error is twice as large as when rounding is used
2 the sign of the error x f l(x) is the same as the sign of x
The worst of the two: no possibility of cancellation of errors.
Characteristics of rounding:
1 the worst possible error is only half as large as when chopping is used
2 More important: the error x f l(x) is negative for only half the
cases, which leads to better error propagation behavior
Math 1070
For single precision IEEE floating-point rounding arithmetic (there are

n = 24 digits in the significand):
1 chopping (rounding towards zero):
223 0
(4.6)
224 224
(4.7)
standard rounding:
For double precision IEEE floating-point rounding arithmetic:

1 chopping:
252 0
(4.8)
253 253
(4.9)
rounding:
Math 1070
> 2. Error and Computer Arithmetic >Conseq. for programming of floating-point arithm.
Numbers that have finite decimal expressions may have infinite binary
expansions. For example
(0.1)10 = (0.000110011001100110011 . . .)2
Hence (0.1)10 cannot be represented exactly in binary floating-point
arithmetic.
Possible problems:
Run into infinite loops
pay attention to the language used:
Fortran and C have both single and double precision, specify
double precision constants correctly
MATLAB does all computations in double precision
Math 1070
> 2. Error and Computer Arithmetic > 2.2 Errors: Definitions, Sources and Examples
Definition
The error in a computed quantity is defined as
Error(xA ) = xT - xA
where xT =true value, xA =approximate value. This is called also
absolute error.
The relative error Rel(xA ) is a measure off error related to the
size of the true value
error = xT xA
Rel(xA ) =
xT
true value
For example, for the approximation
22
=
7
we have xT = = 3.14159265 . . . and xA = 22
7 = 3.1428571

22
22
=
= 0.00126
Error
7
7

22
22
7
Rel
=
= 0.00042
7
Math 1070
The notion of relative error is a more intrinsic error measure.

1
The exact distance between 2 cities: x1T = 100km and the

measured distance is x1A = 99km

Error x1T = x1T x1A = 1km

Error x1T
1
Rel xT =
= 0.01 = 1%
x1T
The exact distance between 2 cities: x2T = 2km and the measured
distance is x2A = 1km

Error x2T = x2T x2A = 1km

Error x2T
2
Rel xT =
= 0.5 = 50%
x2T
Math 1070
Definition (significant digits)

The number of significant digits in xA is number of its leading digits
that are correct relative to the corresponding digits in the true value xT
More precisely, if xA , xT are written in decimal form; compute the error
xT = a1 a2 .a3
|xT xA | = 0
0 .0
am am+1 am+2
0 bm+1 bm+2
If the error is 5 units in the (m + 1)th digit of xT , counting

rightward from the first nonzero digit, then we say that xA has, at
least, m significant digits of accuracy relative to xT .
Example
1
xA = 0.222, xT =
2
9
xT = 0. 2 2 2 2 2 2
3
|xT xA | = 0. 0 0 0 2 2 2
Math 1070
Example
1
xA = 23.496, xT = 23.494:
xT = 2 3. 4 9 6
4
|xT xA | = 0 0. 0 0 1 9 9
xA = 0.02138, xT = 0.02144:
xT = 0. 0 2 1 4 4
|xT xA | = 0. 0 0 0 0 6
xA =
22
7
= 3.1428571 . . . , xT = = 3.14159265 . . . :
xT = 3. 1 4 1 5 9 2 6 5
3
|xT xA | = 0. 0 0 1 2 6 4 4 8
Math 1070
> 2. Error and Computer Arithmetic > 2.2.1 Sources of error
Errors in a scientific-mathematical-computational problem:

1 Original errors
(E1)
(E2)
(E3)
(E4)
(E5)
2
Modeling errors
Blunders and mistakes
Physical measurement errors
Machine representation and arithmetic errors
Mathematical approximation errors
Consequences of errors
(F1) Loss-of-significance errors
(F2) Noise in function evaluation
(F3) Underflow and overflow erros
Math 1070
(E1) Modeling errors: the mathematical equations are used to

represent physical reality - mathematical model.
Malthusian growth model (it can be accurate for some stages of
growth of a population, with unlimited resources)
N (t) = N0 ekt ,
N0 , k 0
where N (t) = population at time t. For large t the model

overestimates the actual population:
accurately models the growth of US population for
1790 t 1860, with k = 0.02975, N0 = 3, 929, 000 e1790k
but considerably overestimates the actual population for 1870
Math 1070
(E2) Blunders and mistakes: mostly programming errors

test by using cases where you know the solution
break into small subprograms that can be tested separately
(E2) Physical measurement errors. For example, the speed of light in

vacuum
c = (2.997925 + ) 1010 cm/sec,
|| 0.000003
Due to the error in data, the calculations will contain the effect
of this observational error. Numerical analysis cannot remove the
error in the data, but it can
look at its propagated effect in a calculation and
suggest the best form for a calculation that will minimize the
propagated effect of errors in data
Math 1070
(E4) Machine representation and arithmetics errors. For example

errors from rounding and chopping.
they are inevitable when using floating-point arithmetic
they form the main source of errors with some problems (solving
systems of linear equations). We will look at the effect of
rounding errors for some summation procedures
(E4) Mathematical approximation errors: major forms of error that we

will look at.
For example, when evaluating the integral
Z 1
2
I=
ex dx,
0
2
since there is no antiderivative for ex , I cannot be evaluated

explicitly. Instead, we approximate it with a quantity that can be
computed.
Math 1070
Using the Taylor approximation

2
ex 1 x2 +
x4 x6 x8
+
2!
3!
4!
we can easily evaluate

Z 1
x4 x6 x8
2
I
1x +
+
dx
2!
3!
4!
0
with the truncation error being evaluated by (3.10)
Math 1070
> 2. Error and Computer Arithmetic > 2.2.2 Loss-of-significance errors
Loss of significant digits
Example
Consider the evaluation of
f (x) = x[ x + 1 x]
for an increasing sequence of values of x.

x0 Computed f (x) True f(x)
1
0.414210
0.414214
10
1.54340
1.54347
100
4.99000
4.98756
1000
15.8000
15.8074
10, 000
50.0000
49.9988
100, 000 100.000
158.113
Table: results of using a 6-digit decimal calculator
As x increases, there are fewer digits of accuracy in the computed value f (x)
Math 1070
What happens?
For x = 100:
where
100 = 10.0000
| {z },
exact
101 = 10.04999
| {z }
rounded
101 is correctly rounded to 6 significant digits of accuracy.
x + 1 x = 101 100 = 0.0499000
while the true value should be 0.0498756.

The calculation
error. Three digits of
has a loss-of-significance
accuracy in x + 1 = 101 were
canceled
by subtraction of the
corresponding digits in x = 100.

The loss of accuracy was a by-product of
the form of f (x) and
the finite precision 6-digit decimal arithmetic being used
Math 1070
For this particular f , there is a simple way to reformulate it and avoid

the loss-of-significance error:
f (x) =
x+1 x
which on a 6 digit decimal calculator will imply

f (100) = 4.98756
the correct answer to six digits.
Math 1070
Example
Consider the evaluation of
1 cos(x)
f (x) =
x2
for a sequence approaching 0.
x0
0.1
0.01
0.001
0.0001
0.00001
Computed f (x)
0.4995834700
0.4999960000
0.5000000000
0.5000000000
0.0
True f(x)
0.4995834722
0.4999958333
0.4999999583
0.4999999996
0.5000000000
Table: results of using a 10-digit decimal calculator
Math 1070
Look at the calculation when x = 0.01:

cos(0.01) = 0.9999500004
(= 0.999950000416665)
has nine significant digits of accuracy, being off in the tenth digit by
two units. Next
1 cos(0.01) = 0.0000499996
(= 4.999958333495869e 05)
which has only five significant digits, with four digits being lost in the
subtraction.
To avoid the loss of significant digits, due to the subtraction of nearly
equal quantities, we use the Taylor approximation (3.7) for cos(x)
about x = 0:
cos(x) = 1
R6 (x) =
x2 x4 x6
+
+ R6 (x)
2! 4!
6!
x8
cos(),
8!
unknown number between 0 and x.

Math 1070
Hence

x2 x4 x6
1
+
+ R6 (x)
f (x) = 2 1 1
x
2!
4!
6!
1
x2 x4 x6
=
+
cos()
2!
4!
6!
8!
giving f (0) = 12 . For |x| 0.1

6
6
x

cos() 10 = 2.5 1011
8!

8!
Therefore, with this accuracy
f (x)
1
x2 x4
+ ,
2!
4!
6!
|x| < 0.1
Math 1070
Remark
When two nearly equal quantities are subtracted, leading significant
digits will be lost.
In the previous two examples, this was easy to recognize and we found
ways to avoid the loss of significance.
More often, the loss of significance is subtle and difficult to detect, as
in calculating sums (for example, in approximating a function f (x) by
a Taylor polynomial). If the value of the sum is relatively small
compared to the terms being summed, then there are probably some
significant digits of accuracy being lost in the summation process.
Math 1070
Example
Consider using the Taylor series approximation for ex to evaluate e5 :
(5) (5)2
(5)3
(5)4
e5 = 1 +
+
+
+
+ ...
1!
2!
3!
4!
Degree
Term
Sum
Degree
0
1.000
1.000
13
1
-5.000
-4.000
14
2
12.50
8.500
15
3
-20.83
-12.33
16
4
26.04
13.71
17
5
-26.04
-12.33
18
6
21.70
9.370
19
7
-15.50
-6.130
20
8
9.688
3.558
21
9
-5.382
-1.824
22
10
2.691
0.8670
23
11
-1.223 -0.3560
24
12
0.5097 0.1537
25
Table. Calculation of e5 = 0.006738 using
Term
Sum
-0.1960
-0.04230
0.7001E-1
0.02771
-0.2334E-1 0.004370
0.7293E-2
0.01166
-0.2145E-2 0.009518
0.5958E-3
0.01011
-0.1568E-3 0.009957
0.3920E-4
0.009996
-0.9333E-5 0.009987
0.2121E-5
0.009989
-0.4611 E-6 0.009989
0.9607 E-7 0.009989
-0.1921 E-7 0.009989
four-digit decimal arithmetic
Math 1070
There are loss-of-significance errors in the calculation of the sum.

To avoid the loss of significance is simple in this case:
e5 =
1
1
=
5
e
series for e5
and form e5 with a series not involving cancellation of positive and

negative terms.
Math 1070
> 2. Error and Computer Arithmetic > 2.2.3 Noise in function evaluation
Consider evaluating a continuous function f for all x [a, b]. The

graph is a continuous curve.
When evaluating f on a computer using floating-point arithmetic (with
rounding or chopping), the errors from arithmetic operations cause the
graph to cease being a continuous curve.
Let look at
f (x) = (x 1)3 = 1 + x(3 + x(3 + x))
on [0, 2].
Math 1070
Figure: f (x) = x3 3x2 + 3x 1
Math 1070
Figure: Detailed graph of f (x) = x3 3x2 + 3x 1 near x = 1
Here is a plot of the computed values of f (x) for 81 evenly spaced

values of x [0.99998, 1.00002].
A rootfinding program might consider f (x) t have a very large number
of solutions near 1 based on the many sign changes!!!
Math 1070
> 2. Error and Computer Arithmetic > 2.2.4 Underflow and overflow errors
From the definition of floating-point number, there are upper and

lower limits for the magnitudes of the numbers that can be expressed
in a floating-point form.
Attempts to create numbers
that are too small underflow errors: the default option is to
set the number to zero and proceed
that are too large overflow errors: generally fatal errors on
most computers. With the IEEE floating-point format, overflow
errors can be carried along as having a value of or N aN ,
depending on the context. Usually, an overflow error is an
indication of a more significant problem or error in the program.
Math 1070
Example (underflow errors)

Consider evaluating f (x) = x10 for x near 0.
With the IEEE single precision arithmetic, the smallest nonzero
positive number expressible in normalized floating point arithmetic is
m = 2126 = 1.18 1038

So f (x) is set to zero if
x10 < m

|x| < 10 m = 1.61 104
0.000161 < x < 0.000161
Math 1070
Sometimes is possible to eliminate the overflow error by just

reformulating the expression being evaluated.
Example (overflow errors)
p
Consider evaluating z = x2 + y 2 .
If x or y is very large, then x2 + y 2 might create an overflow error,
even though z might be within the floating-point range of the machine.
q
|x| 1 + xy 2 , 0 |y| |x|

r
2
z=
|y| 1 + xy , 0 |x| |y|
In both cases, the argument of 1 + w2 has |w| 1, which will not

cause any overflow error (except when z is too large to be expressed in
the floating-point format used).
Math 1070
> 2. Error and Computer Arithmetic > 2.3 Propagation of Error
When doing calculations with numbers that contain an error, the result
will be effected by these errors.
If xA , yA denote numbers used in a calculation, corresponding to
xT , yT , we wish to bound the propagated error
E = (xT yT ) (xA yA )
where denotes: + , , , .
The first technique used to bound E is interval arithmetic.
Suppose we know bounds on xT xA and yT yA . Using these bound
and xA yA , we look for an interval guaranteed to contain xT yT .
Math 1070
Example
Let xA = 3.14 and yA = 2.651 be correctly rounded from xT and yT ,

to the number of digits shown. Then
|xA xT | 0.005,
|yA yT | 0.0005
3.135 xT 3.145
2.6505 yT 2.6515
(4.10)
For addition ( : +00 )

xA + yA = 5.791
(4.11)
The true value, from (4.10)

5.7855 = 3.135+2.6505 xT +yT 3.145+2.6515 = 5.7965
(4.12)
To bound E, subtract (4.11) from (4.12) to get

0.0055 (xT + yT ) (xA + yA ) 0.0055
Math 1070
Example
For division ( : 00 )
xA
3.14
=
= 1.184459
yA
2.651
(4.13)
The true value, from (4.10),

3.135
xT
3.145
2.6515
yT
2.6505
dividing and rounding to 7 digits:
1.182350
xT
1.186569
yT
The bound on E
0.002109
xT
1.184459 0.002110
yT
Math 1070
Propagated error in multiplication
The relative error in xA yA compared to xT yT is

Rel(xA yA ) =
xT yT xA yA
xT yT
If xT = xA + , yT = yA + , then
xT yT xA yA
xT yT (xT )(yT )
=
xT yT
xT yT
xT + yT

=
=
+
xT yT
xT
yT
xT yT
= Rel(xA ) + Rel(yA ) Rel(xA )Rel(yA )
Rel(xA yA ) =
If Rel(xA ) 1 and Rel(yA ) 1, then

Rel(xA yA ) Rel(xA ) + Rel(yA )
Math 1070
Propagated error in division
The relative error in
xA
yA
compared to xyTT is

xT
xA
y
= T xT
Rel
yA
yT
xA
yA
If xT = xA + , yT = yA + , then

Rel
xA
yA

=
xT
yT
xA
yA
xT
yT
xT (yT ) (xT )yT

xT yA xA yT
=
xT yA
xT (yT )

xT + yT
x y
= T T
xT (yT )
1 xT
Rel(xA ) Rel(yA )
1 Rel(yA )
If Rel(yA ) 1, then

Rel
xA
yA

Rel(xA ) Rel(yA )
Math 1070
Propagated error in addition and subtraction

(xT yT ) (xA yA ) = (xT xA ) (yT yA ) =
Error(xA yA ) = Error(xA ) Error(yA )
Misleading: we can have much larger Rel(xA yA ) for small values of
Rel(xA ) and Rel(yA ) (this source of error is connected closely to
loss-of-significance errors).
Example
xA = 13
xT =
yT = 168, yA = 12.961
1
Rel(xA ) = 0, Rel(yA ) = 0.0000371
Error(xA yA ) = 0.0004814
Rel(xA yA ) = 0.0125
xT = xA = 3.1416
yT = 22
7 , yA = 3.1429
x x =
7.35 106 Rel(xA ) = 2.34 106 ,
T
A
2
(yT yA ) = 4.29 105 Rel(yA ) = 1.36 105
(xT yT )(xA yA ) = 0.0012645 (.0013) = 3.55 105
Rel(xA yA ) = 0.28
Math 1070
Total calculation error
With floating-point arithmetic on a computer, xA yA involves an

additional rounding or chopping error, as in (4.13). Hence the total
error in computing xA
yA (the complete operation in computer,
involving the propagated error plus the rounding or chopping error):
yA )
xT yT xA
yA = (xT yT xA yA ) +(xA yA xA
{z
}
{z
} |
|
propagated error
error in computing xA yA
When using IEEE arithmetic with the basic arithmetic operations

(4.3)
xA
yA = f l(xA yA ) = (1 + )(xA yA )
where as in (4.4)-(4.5), we get
xA yA xA
yA = (xA yA )
xA yA xA
yA
=
xA yA
Hence the process of rounding or chopping introduces a relatively small
new error into xA
yA as compared with xA yA .
Math 1070
> 2. Error and Computer Arithmetic > 2.3.1 Propagated error in function evaluation
Evaluate f (x) C 1 [a, b] at the approximate value xA instead of xT .

Using the mean value theorem
f (xT )f (xA ) = f 0 (c)(xT xA ), c between xT and xA
f 0 (xT )(xT xA ) f 0 (xA )(xT xA )
(4.14)
and
Rel(f (xA ))
f 0 (xT )
f 0 (xT )
(xT xA ) =
xT Rel(xA )
f (xT )
f (xT )
(4.15)
Math 1070
Example: ideal gas law

P V = nRT
with R a constant for all gases:
R = 8.3143 + ,
|| 0.0012
Let evaluate T assuming P = V = n = 1 : T =

i.e., evaluate f (x) = x1 at x = R. Let
xT = R,
xA = 8.3143,
For the error

E=
1
R,
|xT xA | 0.0012
1
1
R 8.3143
we have
(4.14)
|E| = |f (xT ) f (xA )| |f (xA )||xT xA |
1
x2A
0.0012 = 0.000144
Hence , the uncertainty in R relatively small error in the computed

1
1
=
R
8.3143
Math 1070
Example: evaluate bx , b > 0
Since f 0 (x) = (ln b)bx , by (4.15)

bxT bxA (ln b)bxT (xT xA )
Rel(bxA )
(ln b)xT
Rel(xA )
| {z }
K=conditioning number
If the conditioning number K 1 is large in size

Rel(bxA ) Rel(xA )
For example, if
Rel(xA ) = 107 ,
then
K = 104
Rel(bxA ) = 103
independent of how bxA is actually computed.
Math 1070
> 2. Error and Computer Arithmetic > 2.4 Summation
Let denote the sum

S = a1 + a2 + + an =
n
X
aj ,
j=1
where each aj is a floating-point number. It takes (n 1) additions, each of

which will probably involve a rounding or chopping error:
(4.3)
S2 = f l(a1 + a2 ) = (a1 + a2 )(1 + 2 )

(4.3)
S3 = f l(a3 + S2 ) = (a3 + S2 )(1 + 3 )

(4.3)
S4 = f l(a4 + S3 ) = (a4 + S3 )(1 + 4 )

..
.
(4.3)
Sn = f l(an + Sn1 ) = (an + Sn1 )(1 + n )

with Sn the computed version of S.
Each j satisfies the bounds (4.6)- (4.9), assuming the IEEE arithmetic is
used.
S Sn a1 (2 +. . .+n )a2 (2 +. . .+n )a3 (3 +. . .+n )
(4.16)
a4 (4 + . . . + n ) an n
Math 1070
Trying to minimize the error S SN :

1
before summing, arrange the terms a1 , a2 , . . . , an so they are

increasing in size
|a1 | |a2 | |a3 | |an |
then summate
Then the terms in (4.16) with the largest numbers of j s are

multiplied by the smaller values among aj s.
Example
1
j
decimal fraction
round it to 4 significant digits := aj
Use a decimal machine with 4 significant digits.

SL = adding S from smallest to largest,
LS = adding S from largest to smallest.
Math 1070
n
10
25
50
100
200
500
1000
True
2.929
3.816
4.499
5.187
5.878
6.793
7.486
SL
2.929
3.813
4.491
5.170
5.841
6.692
7.284
Error
0.001
0.003
0.008
0.017
0.037
0.101
0.202
LS
2.927
3.806
4.479
5.142
5.786
6.569
7.069
Error
0.002
0.010
0.020
0.045
0.092
0.224
0.417
Table: Calculating S on a machine using chopping

n
10
25
50
100
200
500
1000
True
2.929
3.816
4.499
5.187
5.878
6.793
7.486
SL
2.929
3.816
4.500
5.187
5.878
6.794
7.486
Error
0
0
-0.001
0
0
0.001
0
LS
2.929
3.817
4.498
5.187
5.876
6.783
7.449
Error
0
-0.001
0.001
0
0.002
0.010
0.037
Table: Calculating S on a machine using rounding

Math 1070
> 2. Error and Computer Arithmetic > 2.4.1 Rounding versus chopping
A more important difference in the errors in the previous Tables:

between rounding and chopping.
Rounding a far smaller error in the calculated sum than does chopping.
Let go back to the first term in (4.16):
T a1 (1 + 2 + . . . + n )
(4.17)
(1) Assuming that the previous example used rounding with four-digit
decimal machine, we know that all j satisfy
0.005 j 0.005
(4.18)
Treating rounding errors as being random, then the positive and negative
values of the j s will tend to cancel and the sum T will be nearly zero.
Moreover, from probability theory:
|T | 1.49 0.0005 n|a1 |
hence T is proportional to n and is small until n becomes quite large.

Similarly for the total error in (4.16).
Math 1070
(2) For chopping with the four-digit decimal machine, (4.18) is

replaced by
0.001 j 0
and all errors have one sign.
Again, the errors will be random in this interval, with the average
0.0005 and (4.17) will likely be
a1 (n 1) (0.0005)
hence T is proportional to n, which increases more rapidly than n.

Thus the error (4.17) and (4.16) will grow more rapidly when chopping
is used rather than rounding.
Math 1070
Example (difference between rounding and chopping on summation)

Consider
S=
n
X
1
j=1
in single precision accuracy of six decimal digits.

Errors occur both in calculation of
n
10
50
100
500
1000
True
2.92896825
4.49920534
5.18737752
6.79282343
7.48547086
1
j
and in the summation process.
Rounding error
-1.76E-7
7.00E-7
-4.12E-7
-1.32E-6
8.88E-8
Chopping Error
3.01E-7
3.56E-6
6.26E-6
3.59E-5
7.35E-5
Table: Calculation of S: rounding versus chopping

The true values were calculated using double precision arithmetic to
evaluate S; all sums were performed from the smallest to the largest.
Math 1070
> 2. Error and Computer Arithmetic > 2.4.2 A loop error
Suppose we wish to calculate:

x = a + jh,
(4.19)
for j = 0, 1, 2, . . . , n, for given h > 0, which is used later to evaluate a

function f (x).
Question
Should we compute x using (4.19) or by using
x=x+h
(4.20)
in the loop, having initially set x = a before beginning the loop?

Remark: These are mathematically equivalent ways to compute x, but
they are usually not computationally equivalent.
The difficulty arises when h does not have a finite binary expansion
that can be stored in the given floating-point significand; for example
h = 0.1
Math 1070
The computation (4.19)

x = a + jh
will involve two arithmetic operations, hence only two chopping or
rounding errors, for each value of x.
In contrast, the repeated use of (4.20)
x=x+h
will involve a succession of j additions, for the x of (4.19). As x
increases in size, the use of (4.20) involves a larger number of rounding
or chopping errors, leading to a different quantity than in (4.19).
Usually (4.19) is the preferred way to evaluate x
Example
Compute x in the two ways, with a = 0, h = 0.1 and verify with the
true value computed using a double precision computation.
Math 1070
j
10
20
30
40
50
60
70
80
90
100
x
1
2
3
4
5
6
7
8
9
10
Error using (4.19)

1.49E-8
2.98E-8
4.47E-8
5.96E-8
7.45E-8
8.94E-8
1.04E-7
1.19E-7
1.34E-7
1.49E-7
Error using (4.20)

-1.04 E-7
-2.09E-7
7.60 E-7
1.73 E-6
2.46 E-6
3.43 E-6
4.40 E-6
5.36 E-6
2.04 E-6
-1.76 E-6
Table: Evaluation of x = j h, h = 0.1
Math 1070
> 2. Error and Computer Arithmetic > 2.4.3 Calculation of inner products
Definition (inner product)

A sum of the form
S = a1 b1 + a2 b2 + + an bn =
n
X
aj bj
(4.21)
j=1
is called a dot product or inner product.

(1) If we calculate S in single precision:
a single precision rounding error for each multiplication and
each addition
2n 1 single precision rounding errors to calculate S.
The consequences of these errors can be analyzed as for (4.16)
and derive an optimal strategy of calculating (4.21).
Math 1070
> 2. Error and Computer Arithmetic > 2.4.3 Calculation of inner products
(2) Using double precision:

1
2
3
4
convert each aj and bj to double precision by extending their

significands with zeros
multiply in double precision
sum in double precision
round to single precision to obtain the calculated value of S
For machines with IEEE arithmetic, this is a simple and rapid

procedure to obtain more accurate inner products in single
precision.
There is no increase in storage space for the arrays
A = [a1 , a2 , . . . , an ] and B.
The accuracy is improved since there is only one single precision
rounding error, regardless of the size n.
Math 1070

MATH1070 2 Error and Computer Arithmetic PDF

Uploaded by

Copyright:

Available Formats

MATH1070 2 Error and Computer Arithmetic PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MATH1070 2 Error and Computer Arithmetic PDF

Uploaded by

Copyright:

Available Formats

> 2.

Error and Computer Arithmetic

Numerical analysis is concerned with how to solve a problem

2. Error and Computer Arithmetic

> 2. Error and Computer Arithmetic

Computers use binary arithmetic, representing each number as a

2. Error and Computer Arithmetic

> 2. Error and Computer Arithmetic

Computers use 2 formats for numbers.

Therefore they are used mostly for indices and counters.

> 2. Error and Computer Arithmetic > 2.1 Floating-point numbers

Numbers must be stores and used for arithmetic operations. Storing:

Definition (decimal Floating-point representation)

= +1, the exponent e = 2, the significand x = 1.2462

> 2. Error and Computer Arithmetic > 2.1 Floating-point numbers

The decimal floating-point representation of x R is given in (4.1),

number of digits in mantissa x

We say that a computer with such a representation has a four-digit decimal

> 2. Error and Computer Arithmetic > 2.1 Floating-point numbers

Definition (Floating-point representation of a binary number x)

(in decimal:1 x < 2)

> 2. Error and Computer Arithmetic > 2.1 Floating-point numbers

The floating-point representation of a binary number x is given by (4.2) with

> 2. Error and Computer Arithmetic > 2.1 Floating-point numbers

the IEEE single precision floating-point representation of x

the IEEE double precision floating-point representation of x

2. Error and Computer Arithmetic

b13 b14 b64

(1 + 224 cannot be stored exactly)

223 = 1.19 107

Then the machine epsilon in double precision IEEE format is

252 = 2.22 1016

2. Error and Computer Arithmetic

Another way to measure the accuracy of floating-point format:

Let say that a number x has a significand

truncate or chop x to n binary digits, ignoring the remaining

2. Error and Computer Arithmetic

It can be shown that

where  is a small number depending of x.

(b) If rounding is used

For single precision IEEE floating-point rounding arithmetic (there are

For double precision IEEE floating-point rounding arithmetic:

2. Error and Computer Arithmetic

2. Error and Computer Arithmetic

2. Error and Computer Arithmetic

The notion of relative error is a more intrinsic error measure.

The exact distance between 2 cities: x1T = 100km and the

2. Error and Computer Arithmetic

Definition (significant digits)

If the error is 5 units in the (m + 1)th digit of xT , counting

2. Error and Computer Arithmetic

2. Error and Computer Arithmetic

> 2. Error and Computer Arithmetic > 2.2.1 Sources of error

Errors in a scientific-mathematical-computational problem:

2. Error and Computer Arithmetic

> 2. Error and Computer Arithmetic > 2.2.1 Sources of error

(E1) Modeling errors: the mathematical equations are used to

where N (t) = population at time t. For large t the model

2. Error and Computer Arithmetic

where is a small number depending of x.

If Rel(xA ) 1 and Rel(yA ) 1, then