Solving A Quadratic Equation On A Computer 1966 Forsythe
Solving A Quadratic Equation On A Computer 1966 Forsythe
Solving A Quadratic Equation On A Computer 1966 Forsythe
gram. It was expected that the weather would soon be forecast routinely
by computer, and so it was. It was even hinted that computers might
make it possible to predict the future �our�e of hurricanes after a variety
of human interventions and thus lead to the eventual control of the weather.
This, disappointingly, has not come to pass.
Many intellectual steps are involved in a project like weather forecasting
by computer. In the first place, a reasonable model of the weather must
be reduced to systems of equations, both algebraic and differential. The
actual solution of such systems of equations is completely beyond the
powers of any computer because, among other things, this involves know
ing the wind at each of an infinite number of points of space.
Consequently, the second stage of numerical weather prediction is to
replace the infinite number of points of space by a finite number of points
arranged in a number of square-mesh layers spaced one above another,
made of squares perhaps one hundred miles on a side. Instead of trying
to describe and predict the wind at each point of space, one describes and
predicts it at the corners of the meshes. The equations that describe the
exact flow of air and moisture are replaced by much simpler equations,
relating these quantities at neighboring ·points of the mesh. A great deal
of mathematical analysis and experimental computation is needed before
we can discover simple equations for the mesh which reasonably well
simulate the actual equations for continuous space.
At the end of the second stage just described, we have a set of equatio.ns
to solve. Each equation deals with unknown quantities that are real num
bers. Recall that real numbers may be thought of as infinite decimal
expansions like
Floating-Point Numbers
We shall first describe a simplified computer number system, the
system of so-called floating-point numbers, and then show some of its
behavior with a simple mathematical computation.
The usual number system of a computer reduces the infinite number of
decimal places of real numbers to a fixed finite number. We first consider
decimal numbers with a sign ( + or -) and one nonzero digit to the left
of the decimal point, and exactly seven zero or nonzero numbers to the
right of the decimal point. Examples of such numbers are -7.3456780,
+1.0000000, +3.3333333, and -9.9808989. We say that such numbers
have 8 s£gnificant d£g£ts. (The choice of 8 significant base-10 digits is
purely for illustration; other systems also are used.) One can represent
approxim�tely 200,000,000 different numbers in this way, but they all
lie between -10 and -1 or between +1 and +10. To enable computers
to hold much bigger and smaller numbers, we add a sign and two more
decimal digits to serve as an exporien� of 10. In one commonly used system,
the exponent is allowed to range from -50 to +49. Thus, the number
-87 i- is represented by
-8.7666667 X 10+01.
In this system, which is much like so-called sc£entific notation, the repre
sentable numbers all have 8 significant decimal digits. They range from
and from
Computer Arithmetic
Besides holding floating-point numbers, every scientific computer must
be able to perform on them the elementary arithmetic operations of addi
tion and subtraction, multiplication and division. Let us consider addition
first. Sometimes the exact sum of two floating-point numbers is itself a
floating-point number; for example,
from the computer is that it will replace the product by zero and give the
program a signal that underflow has occurred.
The problems encountered in computer subtraction are analogous to
those in addition, and in this sense division is analogous to multiplication.
We assume that our computer operations of addition, subtraction,
multiplication, and division, in the absence of overflow or underflow,
will yield as an answer the floating-point number closest to the exact real
answer. (In case of a tie, we permit either choice.) In fact many actual
computer systems achieve this accuracy or come close to it.
6x 2 + Sx - 4 = 0. (1)
ax2 +bx+ c = 0
-b + -Yb 2 - 4ac
X1 (3)
2a
and
-b - Vb 2 - 4ac
X2 = ·(4)
2e
- -5 + 11 6 1
= -, X2
-s-VW
12 12 12
�
SOLVING A QUADRATIC EQUATION ON A COMPUTER 145
Example 2
Before examining the computer solution, we note that the true solutions,
rounded to eleven significant decimals, are
X1 = 99999. 999990
and
X2 = 0.000010000000001.
Moreover, it can be shown that x 1 and x 2 are w_ell determined by the data
in that small changes of the coefficients (1, -10 5, 1) would cause only
slight changes in x1 and x2•
Now let us apply the algorithm and see what are the computed values
of x1 and x2. We have
a = 1.0000000 X 10 +00,
b - -1.0000000 X 1o + os,
C = 1.0000000 X 10+00.
z = -b � 1.0000000 X 10+05,
y = b 2 = LOOOOOOO X 10+ 10,
w = 4a = 4.0000000 X 10+00,
V = WC = 4.0000000 X 10+00,
u = y - V = 1.0000000 X 10+ 10,
t = Vu = 1.0000000 X 10+0 s,
s = z + t = 2.0000000 X 10+05,
r = 2a = 2.0000000 X 10 +00,
x1 = s/r = 1.0000000 X 10 + •
05
z = - b = 1.0000000 X 1 o+ 05,
y - b 2 = 1.0000000 X 10+ 10,
w 4a = 4.000_0000 X 10+00,
V = WC = 4.0000000 X 10+ 00,
u = y - v = 1.0000000 X 10+10,
t = Vu = 1.0000000 X 1o +05,
s' = z - t =· 0,
r - 2a = 2.0000000 X 10 + 0 �
x2 = s'/r = 0.
helps cope with Example 3 will break down with Example 4. Certainly
our algorithm will not work.
Does the reader feel that equations-with such a large root will not occur
in practical computations? Let him be assured that they do. The final,
physically important result of a.computation is almost certain to lie safely
inside the range of floating-point numbers. InteMnediate results, however,
often have magnitudes smaller than 10- 50 or larger than 10 4 9•
Recently several computing experts agreed that one of the most serious
difficulties with many current computer systems is that they automatically
replace an underflowed answer by zero, without any warning message.
In such a system, 1o- 3 o X 1o- 3 o X 10 3 1 X 10 30 would be computed as 0
(because underflow would occur with the first multiplication step),
whereas 10- 30 X 10 3 1 X 10- 3 0 X 10 30 would be correctly computed as 10.
z = -b = 4.0000000,
y = b 2 = 16.000000,
w = 4a = 4.0000000,
V = WC = 16.000000,
U = y - V = 0,
t=Vu=O,
s - z + t = z - t = 4.0000000,
r - 2a = 2.0000000,
x1 - x2 = s/r = 2.0000000.
error for almost any equation whose coefficients are floating-point num
bers. Such performance can be achieved, and it is essential to have such
algorithms in the computer library.- Then, when a quadratic equation
occurs in the midst of a complex and imperfectly understood computation,
we can be sure that the quadratic equation solver can be relied upon to do
its part automatically and well, so that we can concentrate attention on the
rest of the computation.
We want a quadratic equation solver that will accept any floating-point
numbers a, b, c, and compute close decimal approximations to any of the
roots x1, x2 that lie safely within the range of floating-point numbers. Any
computed root should have an error in the last decimal place not exceed
ing, say, 10 units. If either x1 or x2 underflows, there should be a message
about what happened.
Such an algorithm has been devised by William Kahan of the Univer
sity of Toronto. The most difficult matter to take care of is the possibility
of overflow or underflow. It will not be possible to describe the complete
algorithm, but we can give some of the more accessible ideas.
First, we discuss the steps taken to overcome the great inaccuracy in
root x2, as computed in Example 2. Iri step (vii)' of Example 2, we sub
tracted two equal numbers, z and t, to get s' = 0. The true value of t
was not quite equal to z, because in step (v) the true u was not quite equ'al
to y. But t and z, like u and y, could not be distinguished with only 8
decimal digits at our disposal.
An easy cure for the difficulty is to use another method of computing
x2 which does not involve the subtraction of nearly equal numbers.
If a, b, c are any real numbers, and if a � 0 and c � 0, then the quad
ratic equation
ax 2+bx+c = 0
2c
X1
-b (5)
v'b2 - 4ac
and
2c
X2
-b + v'b2 - 4ac (6)
C (�) 2
+ b (�) + a - 0.
150 GEORGE E. FORSYTHE
z = -b = 4.0000000,
y = b 2 = 16.0000000 0000000,
w = 4a = 4.0000000 0000000,
V = WC = 15.9999996 0000000,
Note that x1 is in error by only 0.72 of a unit in the last decimal place.
We shall not discuss scaling and dealing with possible overflow· and
underflow. The details are many and technical, and they depend inti
mately on particular features of the computer on which they are carried
out. They are extremely important to actual computing but less generally
interesting than the ideas just presented. One of the obvious features
involves testing whether any or all of a, b, or c are zero.
Conclusion
We have described some of the pitfalls of applying the· quadratic
formula blindly with an automatic digital computer. We have given
SOLVING A QUADRATIC EQUATION ON A COMPUTER 151
sound cures for two of the pitf�lls and indicated what other work has been
done to create a first-class algorithm for solving a quadratic equation.
The quadratic equation is one of .the simplest mathematical entities
and is solved almost everywhere in applied mathematics. Its actual use
on a computer might be expected to be one of the best understood of
computer algorithms. Indeed it is not, and some more complex compu
tations were studied first. The fact that the obvious algorithm is so subject
to rounding error is not very widely known by computer users or writers
of elementary textbooks on computing methods or, certainly, most writers
of mathematics textbooks. Of course, it is familiar to specialists in numer
ical analysis. Thus, even in this elementary prol:5lem, we are working at
the frontiers of common computing knowledge.
The majority of practical computations are understood far less well
than the quadratic equation. A very great deal of difficult research and
development remains to be done before computers will be used as wisely
and well as they can be. Almost certainly various parts of the computations
for weather forecasting contain pitfalls like those of the quadratic equation,
and ignorance of these pitfalls is introducing computational errors that
are interfering with progress in weather forecasting. The same can be said
about most other nontrivial fields of scientific computation.
The moral of the story is that users of computers for mathematical
problems require some knowledge of numerical mathematics. It is not
sufficient to learn some programming language and then simply translate
formulas from a textbook of pure mathematics into the language of a
computer. The formulas and algorithms to be found in most mathematics
texts were devised for the exact · arithmetic of the real number system.
In order to understand the behavior of the formulas when they are used
with the approximate arithmetic of computers, the would-be scientific
computer must consult people and writings specifically concerned with
machine computation.
A further moral is that teachers of arithmetic and algebra should
consider the presentation of their subject in the light of its practical
application. The quadratic formula is very important for understanding
quadratic equations and should be taught. But why hide the fact that it
will not stand up to practical application on computers? In days· gone
by, when few quadratic equations were actually solved, it hardly mat
tered. But today, when quadratic equations are solved wholesale inside
computers, textbook writers should also present material that is not only
theoretically correct and easily taught but also easily transferable to
practical use. This is a real challenge. In the last analysis, the challenge
will be met by persons who have rich backgrounds in both mathem<1tical
theory and practical application. This is perhaps the most convincing
reason for maintaining the mathematical sciences as a unity.
152 GEORGE E. FORSYTHE
BIBLIOGRAPHY
L. F. Richardson, Weather Prediction by Numerical Process (1922); Reprinted by
Dover Publications, New York, 1965._
I. A. Stegun and M. Abramowitz, "Pitfalls in Computation," SIAM Journal, 4,
1956, 207-219.
R. W. Hamming, Numerical Methods for Scientists and Engineers, McGraw-Hill, New
York, 1962.
E. L. Stiefel, An Introduction to Numerical Mathematics, Academic Press, New York,.
1963.
BIOGRAPHICAL NOTE
George E. Forsythe heads the Computer Science Department at Stanford Uni
versity. When he founded it in 1961, it was one of the first such departments in a
United States university. From then until 1965, he also directed Stanford's
Computation Center. Forsythe served as president of the Association for Com
puting Machinery. His interest in computing began early, when, as a seventh
grader, he tried using a hand-cranked desk calculator to find the decimal expan
sion of 10,000/7699. He wanted to see how the digits repeated. In World War II,
he encountered more important computation problems as an Army Air Force
meteorology officer. He first formally entered the then-nascent field of computer
sciences while associated with the Institute for Numerical Analysis of the National
Bureau of Standards, Los Angeles.