Some numerical methods for finding a local minimum of a given one-dimensional function will be described
in this section.
The same methods can be used for finding a maximum as well because
the function has its (local, global) maximum at
if and only if
has its (local, global) minimum at
.
A minimum (or maximum) is bracketed by three points , if
and
is less (or greater in case
of maximum) than both
and
. If this condition holds and the function
is continuous
in the interval
, then
has a minimum for some
,
.
A very simple iterative minimum bracketing procedure follows:
Start from any initial point. Moving a step in the downhill direction, one gets a new point in each iteration. From the third iteration on, store the last three points. New one in each iteration substitutes the oldest one in the triplet. Take larger and larger steps (the new step being computed by simple multiplying the previous one by a constant factor or obtained as a result of a parabolic interpolation applied on a previous triplet of points, see Section 7.4.3) until the downhill trend stops and one gets the last point, where the function value is greater than in the previous one.
An analogous procedure can be used for maximum bracketing; alternatively, one can use the minimum bracketing
procedure for .
The following quantlet can be used for bracketing of a minimum of a scalar function:
|
fname has to be a string with a name of the function whose minimum should be bracketed. If one has some idea about the location of the minimum, the input parameters a0 and b0 can be set to start the bracketing from a0 in the direction to b0 or vice versa, depending on the downhill direction. Otherwise one can use the simplest form
{a,b,c,fa,fb,fc} = nmbracket(fname)starting the bracketing with the defaults values
An example
XEGnum05.xpl
gives the following brackets for function
:
Contents of bracket.a [1,] 2.618 Contents of bracket.b [1,] 5.2361 Contents of bracket.c [1,] 9.4721 Contents of bracket.fa [1,] -6.7955 Contents of bracket.fb [1,] -23.743 Contents of bracket.fc [1,] 1.0622
Fig. 7.3 shows a graph of and the bracketing points shown as stars.
Let us denote the three points and the respective function values
. We are looking for
point
minimizing (or maximizing)
. Providing the parabola can be described as
![]() |
![]() |
![]() |
|
![]() |
![]() |
![]() |
|
![]() |
![]() |
![]() |
Can parabolic interpolation fail? The parabolic interpolation fails if the denumerator of the formula for
is equal to zero. This happens if and only if the three given points
are linear dependent,
i.e., they are on the same line. In this case, the information for parabolic interpolation is insufficient.
One can use the following quantlet to find the extremum of a parabola determined by three points.
|
The input parameters include the abscissas a,b and c as well as the respective function values
fa, fb and fc.
Using
nmparabint
, one can execute the inverse parabolic interpolation for more parabolas at the same time.
In this case all the input parameters will be vectors whose
-th components refer to the
-th parabola.
nmparabint
returns INF if the three points are linear dependent (i.e., on the same line).
An example
XEGnum06.xpl
uses the parabolic interpolation through the points
,
and
to find
the parabola extremum at
:
Contents of res [1,] 0.5
Fig. 7.4 shows the parabola
from which the input points (represented by stars) were taken.
The minimum found by
nmparabint.xpl
is represented by a triangle.
For golden section search, one has to bracket the minimum by some triplet first
(see Section 7.4.1 for details on minimum bracketing).
Now, let us choose a new point
in the bracketing interval, for example between
and
(
),
and evaluate
.
The new point
substitutes one of the original points in bracketing triplet:
if
, then
is the new triplet (with
less then both
and
);
if
, then
is the new triplet (with
).
The minimum function value we have found until now is at the middle point of the new triplet in both cases.
Continue this procedure until the size of the bracketing interval (i.e., the distance of its outer points) is
small enough.
It can be shown
divides the interval in the ratio
or, vice versa,
, with
.
This
is the so called golden section.
The following quantlet implements the golden section search method described above:
|
If used in the very basic form
min = nmgolden(fname),fname being a string with a name of function to be minimized,
min = nmgolden(fname,a,b,c)if a bracketing triplet
An example
XEGnum07.xpl
uses a golden section search with a function
given
a bracketing triplet
and finds its minimum
:
Contents of res.xmin [1,] 4.7124 Contents of res.fmin [1,] -1
Fig. 7.5 depicts a graph of with bracketing triplet represented by stars and the minimum shown
as a triangle.
Before starting the procedure, one should bracket the minimum (see section 7.4.1 for more details).
In each iteration, Brent's method reduces the bracketing interval and updates six stored points:
outer points of bracketing interval , three points (
,
)
with the three least values found so far and the most recently evaluated point
.
The method combines parabolic interpolation through
(as long as the process is convergent and does not leave the
bracketing interval) with golden section search.
The following quantlet implements Brent's method for minimization of a scalar function:
|
It is used the same way as
nmgolden
described in Section 7.4.6.
An example
XEGnum07b.xpl
uses the Brent's method for a function
given
a bracketing triplet
and finds its minimum
:
Contents of res.xmin [1,] 4.7124 Contents of res.fmin [1,] -1
Fig. 7.6 depicts a graph of with bracketing triplet represented by stars and the minimum shown
as a triangle.
|
finds a minimum of a given scalar function using Brent's method with first derivative.
Its usage is very similar to
nmgolden
; see Section 7.4.6 for description of the input parameters
a,b,c and xtol. In the simplest form
min = nmbrentder(fname)without the derivative of fname the quantlet calls
min = nmbrentder(fname,h)
If a function computing the first derivative of fname is available, one can input its name as a string fder:
min = nmbrentder(fname,fder),as in example
Contents of res.xmin [1,] -1.5708 Contents of res.fmin [1,] -1