The Newton's Method Formula
The Newton's Method Formula
The Newton's Method Formula
questions in mathematics involve the solution of equations or systems of equations. Unfortunately, many of them cannot be solved by algebraic manipulation. We therefore need to find a way to solve the equations numerically, in the hope of making accuracy as much as possible. There is a simple technique of elementary calculus known as Newtons method. Newtons method is a method for iteratively approximating the root of a function f(x) =0 using the derivative (under appropriate hypothesis). In other words, we want to find a value x* such that f(x*) =0, and we proceed it by giving an educated guess and then refining our guess over and over again. If the function is quite simple, we may be able to do some algebra to find a root x exactly. For instance, if f(x) is a quadratic polynomial, we can use the quadratic formula. If f(x) is a degree 3 or 4 polynomial, there are messier formulas which work as well. But if f(x) is a higher degree polynomial or an even more complicated function, there is no analog to the quadratic formula, i.e. there is no systematic process to algebraically determine the roots exactly. We have to approximate, and Newton's method is just one way of doing this. We will first start our discussion on Newton's method over the real number line, i.e. when x is always real and so is f(x). Later then we will see that the situation becomes much more interesting when we enlarge our scope, letting x and f(x) be complex numbers. For simplicity, we will use the function f(z) =z3-1 in this case. Of course, we all know that z=1 is the only real root, and there are also two other complex roots z= -1/2 + (31/2/2) i = -0.5 + 0.866 i and z = -1/2 - (31/2/2) i = -0.5 - 0.866 i. (1) , which can be calculated by formula to the k-th root of a complex number, and so we do not really need Newton's method in this example. However, since many physical science equation has no real solution, we introduce complex numbers and are much more interested in how a given complex number z converging to the root of the equation, for example, f(z)=z3-1 under the sequence given by Newtons method. We will later see that even in this simple case we already get pictures which are quite complicated enough than on the real number line.
Then x*= x1 - f(x0 )/ f '(x0 ) In general, this is not the case. However, the tangent line approximates f(x) for x near x0, and so we hope that x1 is not "too far off" from a root of f(x), in particular hopefully closer than x0. Repeating the process with x1 instead of x0, we will get the next approximation x2, and so on. Thus each xn is determined by f(xn-1) and the slope of the tangent line at xn, which is f '(xn-1). The appropriate formula is xn = xn-1 - f(xn-1) / f '(xn-1) (2) Demonstration of Newtons method:
If we choose x0=0, we are in trouble, since the tangent is horizontal, f '(x0) =0 and formula (*) crashes. Furthermore, if we choose a different x0 which eventually "lands" on xn=0, we have the same problem. The graph at right shows this in red for x0=-1.434, which crashes in 2 steps in this manner. There is an infinite sequence of "bad" negative points like this. Note that if we choose an initial guess very close to such a bad point, xn will eventually become closebut-not-equal-to 0, and then xn+1 will get blasted way out to the right and from there eventually converge to x=1. The closer we began to a bad point, the more steps it will take to get "reasonably close" to x=1. An example of this is shown in blue.
The picture shows that one choice (blue) converges to the real root, but another (green) converges to one of the other complex roots. But if we choose precisely z0 = 0.539 + 0.417 i (determined with some effort!), then z1= 0 .539 - 0.417 i, and z2=z0! So we get into an infinite cycle back and forth between these two values and do not converge to any root. This is a different sort of "bad point" than the ones we found on the negative real axis, when f '(x)=0. Incidentally, the grey shading in the graph represents |f(z)|. The dark squares are near the roots, when |f(z)| is 0, and the squares are progressively lighter as |f(z)| gets bigger, and hence f(z) gets "further from 0". So the darken square is the complex root of the equation f(z)=0. Recall that the complex-valued polynomial of degree n has n roots. From the graph above, we might question which root is being converged if we guess a z0 somewhere between two roots. The question leads to an interesting phenomenon called Newton basin, which is fractal. Using the same function f(z)=z3- 1 to give a fractal, since the polynomial has the three toots of unity as its roots. The three roots are 1, -0.5+0.866i, and -0.5-0.866i.
We see the three white small circles are the three roots of the f(z)=0. To answer to which root is converge, notice that if we pick z0 in the pink region, the sequence defined by Newtons method would converge to 1, likewise, z0 would converge to -0.5+0.866i if it is in blue and to -0.5-0.866i if in light blue. (See reference)
Reference: http://aleph0.clarku.edu/~djoyce/newton/examples.html