Newtons Method

Newton's method is an example of how differentiation is used to find zeros of functions and solve equations numerically. Newton's method or Newton-Raphson method is a procedure used to generate successive approximations to the zero of function f as follows:

x_n+1 = x_n- f(xn) / f '(xn), for n = 0,1,2,3,...

To understand how and why Newton's method works, let's first concentrate on the derivative of a function. One of the first things you learned about the derivative was that it could be used to find the slope, and thus the equation, of the tangent to a curve at a specified point. And one of the really nice things about the tangent to a curve is that it can, under certain circumstances, be used to approximate the curve.

The idea of the method is as follows: one starts with an initial guess which is reasonably close to the true root, then the function is approximated by its tangent line and one computes the x-intercept of this tangent line. This x-intercept will typically be a better approximation to the function's root than the original guess, and the method can be iterated.

Suppose f = [a, b] --> R is a differentiable function defined on the interval [a, b] with values in the real numbers R. The formula for converging on the root can be easily derived. Suppose we have some current approximation xn. Then we can derive the formula for a better approximation, x_n+1 by referring to the diagram on the right. We know from the definition of the derivative at a given point that it is the slope of a tangent at that point. That is

Here, f' denotes the derivative of the function f. Then by simple algebra we can derive

Newton's method, also called the Newton-Raphson method, is a numerical root-finding algorithm: a method for finding where a function obtains the value zero, or in other words, solving the equation f(x)=0 Most root-finding algorithms used in practice are variations of Newton's method.

Suppose the function f(x) has a root at x= r . The idea behind Newton's method is that, if f(x) is a smooth function, its graph can be approximated around a point x=x₀ by its tangent at x₀ . If the approximation is good enough, the point where the tangent crosses the x -axis must lie close to r .

This suggests that if we choose the point x₀ to lie somewhere close to r, the point where the tangent crosses the x - axis we may call that point x₁ will lie even closer to r . Now suppose that x₁ really does lie closer to r than x₀ . Then we can determine the tangent of at x=x₁ to obtain a second point x₂ that lies closer still. More generally, given x_k , we can obtain a better approximation x_{k+ 1} .