4. Regression Methods
Jörg Aßmus
28 July 2004
Simply speaking, a regression problem is a way to determine
a function
describing a functional relation
between a
-dimensional variable
and an output variable
where
is a random error term.
There are two different approaches of fitting the function
. In the first
approach, we define
by a finite-dimensional parameter
:
where it is sufficient to estimate the parameter
. In this case, we are
able to approximate the function at each point
using only the
parameter estimate
of
by
This is called a parametric model. A very useful example is the linear
regression
On the other hand a model is called nonparametric if the dimension
of
the parameter
is infinite. This means that we do not know
anything about the function
. In this case we have to use the data set for
the calculation of the estimated function
at each point
. We investigate this question in
Smoothing Methods (6).
Both models can be estimated with
XploRe
.
A simple way of choosing the appropriate method is given here:
- If we know the functional form of
, we use a parametric
model:
is linear,
is one-dimensional,
We use simple linear regression. (
linreg
,
gls
)
is linear,
is
-dimensional,
We use multiple linear regression.
(
linreg
,
gls
,
linregbs
,
linregfs
,
linregstep
)
is nonlinear, e.g.
We use nonlinear regression.
- If we don't know the functional form of
, then we use a
nonparametric model. Nonparametric methods are introduced in
Smoothing Methods (6).
The nonlinear methods are more general than the linear methods. This means that
we can use the nonlinear regression for estimating linear models, but this
cannot be recommended in general. In the same way, the nonparametric models are
more general than the parametric ones.
In the following sections, the libraries
stats
containing the quantlets for the regression and
graphic
containing the plot quantlets
will be used. We should load them before we continue:
library("graphic") ; reads the library graphic
library("stats") ; reads the library stats