7.2 Solving a Nonlinear Equation

This section describes some numerical methods for finding real roots of the one-dimensional nonlinear function $ f(x)$, i.e., solving the nonlinear equation $ f(x) = 0$. The multidimensional case is treated in Section 7.3.

Our aim is to construct a sequence $ \{x_i\}$ of real numbers or real vectors that converges to the root we search for. $ n$-point iterative methods are used, i.e., $ x_{i+1}$ is computed as an $ n$-point iteration function depending upon the $ n$ previous values:

$\displaystyle x_{i+1} = f_i(x_i,x_{i-1},\dots,x_{i-n+1}) \quad\textrm{ for }i=n-1,n,\dots.
$

One-point iterative methods with the same function $ f_i = f$ for all $ i$ are used most often. In commonly used methods, the sequence is known to converge if the initial approximation is close enough to the root.


7.2.1 Termination of Iterative Methods

For any iterative method, one has to specify termination criteria. Natural ones are:


7.2.2 Newton-Raphson Method for One-dimensional Problems

The Newton-Raphson method (or just Newton method) is one of the most popular methods for root finding. Let $ x_i$ be $ i$-th approximation of the root. The function $ f$ in the vicinity of the root is approximated by the tangent to the curve $ y=f(x)$ at $ (x_i,f(x_i))$; the intersection of the tangent with the $ x$-axis defines the next approximation, $ x_{i+1}$ (see Fig. 7.1).

Figure: Principle of Newton-Raphson Method, 28072 XEGnum01.xpl
\includegraphics[width=1.0\defepswidth]{XEGnum01.ps}

The formula describing this procedure is

$\displaystyle x_{i+1} = x_i - \frac{f(x_i)}{f'(x_i)}.$ (7.1)

This formula can be also derived as follows: start with the Taylor series of $ f$ centered in $ x_i$:

$\displaystyle f(x_i+h) = f(x_i) + h f'(x_i) + O(h^2)
$

and take $ h = x_{i+1} - x_i$; then

$\displaystyle f(x_{i+1}) = f(x) + h f'(x_i) + O(h^2).
$

Now we neglect the term $ O(h^2)$ and setting $ f(x_{i+1}) = 0$ implies the formula (7.1).

REMARK 7.1   The Newton-Raphson method converges quadratically in case of distinct roots. Hence, it is very efficient for functions that have continuous and nonzero derivative in some neighbourhood of a root, having a good initial guess.

REMARK 7.2   If $ f'(x)$ is not known (or its computation is time-consuming too much), it can be approximated by differences as shown, e.g., in (Rektorys; 1995); see Section 7.6.1 for details. However, the convergence rate is lower in this case.