As in the case of density estimation, confidence intervals and bands can be based on the asymptotic normal distribution of the regression estimator. We will restrict ourselves to the Nadaraya-Watson case in order to show the essential concepts. In the latter part of this section we address the related topic of specification tests, which test the hypothesis of a parametric against the alternative nonparametric regression function.
Now that you have become familiar with nonparametric regression, you may want to know: How close is the smoothed curve to the true curve? Recall that we asked the same question when we introduced the method of kernel density estimation. There, we made use of (pointwise) confidence intervals and (global) confidence bands. But to construct this measure, we first had to derive the (asymptotic) sampling distribution.
The following theorem establishes the asymptotic distribution of the Nadaraya-Watson kernel estimator for one-dimensional predictor variables.
The asymptotic bias is proportional to the second moment of the
kernel and a measure of local curvature of
. This measure of local
curvature is
not a function of
alone but also of the marginal density. At maxima or
minima, the bias is a multiple of
alone; at inflection points it is
just a multiple of
only.
![]() |
We now use this result to define confidence intervals. Suppose that the bias
is of negligible magnitude compared to the variance, e.g. if the
bandwidth is sufficiently small. Then we can
compute approximate confidence intervals with the following
formula:
![]() |
(4.55) |
As we have seen in the density case, uniform confidence bands for
need rather restrictive assumptions. The derivation of
uniform confidence bands is again based on Bickel & Rosenblatt (1973).
![]() | |||
![]() |
In practice, the data
are transformed to the interval
, then the confidence bands are computed and rescaled to
the original scale of
.
The following comprehensive example covers local polynomial kernel regression as well as optimal smoothing parameter selection and confidence bands.
In the past, one of the most important exchange rates was that of Deutsche Mark (DM) to US Dollar (USD). The data that we consider here are from Olsen & Associates, Zürich. They contains the following numbers of quotes during the period Oct 1 1992 and Sept 30 1993. The data have been transformed as described in Bossaerts et al. (1996).
![]() |
We present now the regression smoothing approach with local linear estimation of the conditional mean (mean function) and the conditional variance (variance function) of the FX returns
![]() |
![]() |
![]() |
|
![]() |
![]() |
![]() |
![]() |
The estimated functions are plotted together with approximate 95%
confidence bands, which can be obtained from the asymptotic normal
distribution
of the local polynomial estimator.
The cross-validation optimal bandwidth is used for the local
linear estimation of the mean function in Figure 4.16.
As indicated by
the 95% confidence bands, the estimation is not very robust at the
boundaries. Therefore, Figure 4.16 covers a truncated range.
Analogously, the variance estimate is shown in Figure 4.17,
using the cross-validation optimal bandwidth
.
The basic results are the mean reversion and the ``smiling'' shape of the conditional variance.
Conditional heteroscedasticity appears to be very distinct.
For DM/USD a ``reverted leverage effect''
can be observed, meaning that the conditional
variance is higher for positive lagged returns than for negative
ones of the
same size. But note that
the difference is still within the 95% confidence bands.
In this book we will treat the topic of testing not as a topic of its own, being aware that this would be an enormous task. Instead, we concentrate on cases where regression estimators have a direct application in specification testing. We will only concentrate on methodology and skip any discussion about efficiency.
As this is the first section where we deal with testing, let us start with some brief, but general considerations about non- and semiparametric testing. Firstly, you should free your mind of the facts that you know about testing in the parametric world. No parameter is estimated so far, consequently it cannot be the target of interest to test for significance or linear restrictions of the parameters. Looking at our nonparametric estimates typical questions that may arise are:
Let us now turn to the fundamentals of nonparametric testing. Indeed, the appropriateness of a parametric model may be judged by comparing the parametric fit with a nonparametric estimator. This can be done in various ways, e.g. you may use a (weighted) squared deviation between the two models. A simple (but in many situations inefficient) approach would be to use critical values from the asymptotic distribution of this statistic. Better results are usually obtained by approximating the distribution of the test statistics using a resampling method.
Before introducing a specific test statistic we have
to specify the hypothesis and the alternative
.
To make it easy let us
start with a nonparametric regression
.
Our first null
hypothesis is that
has no impact on
.
If we assume
(otherwise take
),
then we may be interested to test
![]() |
![]() |
![]() |
It is clear that under both test statistics
and
must
converge to zero, whereas under
the condition
also lets the statistic increase to infinity.
Note that under
our estimate
does not have any bias (cf. Theorem 4.3)
that could matter in the squared
deviation
. Actually,
with the same assumptions we needed for the kernel estimator
, we find that under the null hypothesis
and
converge to a
distribution with
![]() |
![]() |
![]() |
(4.59) |
Let us now consider the more general null hypothesis. Suppose
we are interested in a specific parametric model given
by
and
is
a (parametric) function, known up to the parameter
.
This means
![]() |
![]() |
![]() |
A consistent estimator
for
is usually easy to obtain (by least squares,
maximum likelihood, or as a moment estimator, for example).
The analog to statistic (4.58) is then obtained
by using the deviation from
, i.e.
The remaining question of the example is: How to find the critical value for ``large''? The typical approach in parametric statistics is to obtain the critical value from the asymptotic distribution. This is principally possible in our nonparametric problem as well:
![]() | |||
![]() |
As in the parametric case, we have to estimate the
variance expression in the normal distribution.
However, with an appropriate estimate for
this is no obstacle. The main practical problem here is
the very slow convergence of
towards the normal distribution.
For that reason, approximations of the critical values corresponding to the finite sample distribution are used. The most popular way to approximate this finite sample distribution is via a resampling scheme: simulate the distribution of your test statistic under the hypothesis (i.e. ``resample'') and determine the critical values based on that simulated distribution. This method is called Monte Carlo method or bootstrap, depending on how the distribution of the test statistic can be simulated. Depending on the context, different resampling procedures have to be applied. Later on, for each particular case we will introduce not only the test statistic but also an appropriate resampling method.
For our current testing problem the possibly most popular resampling
method is the so-called wild bootstrap introduced by Wu (1986).
One of its advantages is
that it allows for a heterogeneous variance in the residuals.
Härdle & Mammen (1993) introduced wild bootstrap into the context of nonparametric
hypothesis testing as considered here. The principal idea is to resample
from the residuals
,
, that we got
under the null hypothesis. Each bootstrap residual
is drawn from a distribution that coincides
with the distribution of
up to the first
three moments. The testing procedure then consists of the
following steps:
![]() |
One famous method which fulfills the
conditions in step (c) is the
so-called golden cut method.
Here we draw
from the two-point
distribution with probability mass at
Let us mention that besides the type of test statistics
that we introduced here, other distance measures are plausible.
However, all test statistics can be considered as
estimates of one of the following expressions:
The question ``Which is the best test statistic?'' has no simple
answer. An optimal test should keep the nominal significance level
under the hypothesis and provide the highest power under the
alternative. However, in practice it turns out that the behavior of
a specific test may depend on the model, the error distribution,
the design density and the weight function. This leads to
an increasing number of proposals for the considered testing problem.
We refer to the bibliographic notes for additional references
to (4.63)-(4.66) and for further test
approaches.