This chapter examines the use of flexible methods to approximate an unknown density function, and techniques appropriate for visualization of densities in up to four dimensions. The statistical analysis of data is a multilayered endeavor. Data must be carefully examined and cleaned to avoid spurious findings. A preliminary examination of data by graphical means is useful for this purpose. Graphical exploration of data was popularized by [56] in his book on exploratory data analysis (EDA). Modern data mining packages also include an array of graphical tools such as the histogram, which is the simplest example of a density estimator. Exploring data is particularly challenging when the sample size is massive or if the number of variables exceeds a handful. In either situation, the use of nonparametric density estimation can aid in the fundamental goal of understanding the important features hidden in the data. In the following sections, the algorithms and theory of nonparametric density estimation will be described, as well as descriptions of the visualization of multivariate data and density estimates. For simplicity, the discussion will assume the data and functions are continuous. Extensions to discrete and mixed data are straightforward.
Statistical modeling of data has two general purposes: (1)
understanding the shape and features of data through the density
function,
, and (2) prediction
of
through the joint density function,
. When the
experimental setting is well-known, parametric models may be
formulated. For example, if the data are multivariate normal,
, then the features of the density may be
extracted from the maximum likelihood estimates of the parameters
and
. In particular, such data have one
feature, which is a single mode located at
. The shape of the
data cloud is elliptical, and the eigenvalues and eigenvectors of the
covariance matrix,
, indicate the orientation of the data and
the spread in those directions. If the experimental setting is not
well-known, or if the data do not appear to follow a parsimonious
parametric form, then nonparametric density
estimation is indicated. The
major features of the density may be found by counting and locating
the sample modes. The shape of the density cannot easily be
determined algebraically, but visualization methodology can assist in
this task. Similar remarks apply in the regression setting.
When should parametric methods be used and when should nonparametric
methods be used? A parametric model enjoys the advantages of
well-known properties and parameters which may be interpreted.
However, using parametric methods to explore data makes little sense.
The features and shape of a normal fit will always be the same no
matter how far from normal the data may be. Nonparametric approaches
can fit an almost limitless number of density functional forms.
However, at the model, parametric methods are always more
statistically accurate than the corresponding nonparametric estimates.
This statement can be made more precise by noting that parametric
estimators tend to have lower variance, but are susceptible to
substantial bias when the wrong parametric form is invoked.
Nonparametric methods are not unbiased, but the bias asymptotically
vanishes for any continuous target function. Nonparametric algorithms
generally have greater variance than a parametric algorithm.
Construction of optimal nonparametric estimates requires a data-based
approach in order to balance the variance and the bias, and the
resulting mean squared error generally converges at a rate slower than
the parametric rate of . In summary, nonparametric
approaches are always appropriate for exploratory purposes, and should
be used if the data do not follow a simple parametric form.