The object of nonmetric MDS, as well as of metric MDS, is to find the
coordinates of the points in
-dimensional space, so that there is a
good agreement between the observed proximities and the inter-point
distances. The development of nonmetric MDS was motivated by two
main weaknesses in the metric MDS (Fahrmeir and Hamerle; 1984, Page 679):
- the definition of an explicit functional connection
between dissimilarities and distances in order to derive
distances out of given dissimilarities, and
- the restriction to Euclidean geometry in order to determine the
object configurations.
The idea of a nonmetric MDS is to demand a less rigid
relationship between the dissimilarities and the distances.
Suppose that an unknown monotonic increasing function
,
 |
(15.19) |
is used to generate a set of distances
as a function of
given dissimilarities
.
Here
has the property that
if
, then
.
The scaling is based on the rank order of the dissimilarities.
Nonmetric MDS is therefore ordinal in character.
The most common approach used to determine the elements
and to obtain the
coordinates of the objects
given only
rank order information
is an iterative process commonly referred to as the Shepard-Kruskal algorithm.
In a first step, called the initial phase, we calculate Euclidean
distances
from an arbitrarily chosen initial configuration
in dimension
, provided that all objects have
different coordinates. One might use metric MDS
to obtain these initial coordinates.
The second step or nonmetric phase determines disparities
from
the distances
by constructing a monotone regression
relationship between the
's and
's, under the requirement that if
, then
.
This is
called the weak monotonicity requirement. To obtain the
disparities
, a useful
approximation method is the pool-adjacent violators (PAV) algorithm
(see Figure 15.6).
Let
 |
(15.20) |
be the rank order of dissimilarities of the
pairs of objects.
This corresponds to the points in Figure 15.5.
The PAV algorithm is described as follows:
``beginning with the lowest ranked value of
, the
adjacent
values are compared for each
to determine if they are monotonically related to the
's.
Whenever a block of consecutive values of
are encountered that violate the required monotonicity property the
values are averaged together with the most recent non-violator
value to obtain an estimator. Eventually this value is assigned
to all points in the particular block''.
In a third step, called the metric phase, the spatial
configuration of
is altered to obtain
. From
the
new distances
can be obtained
which are more closely related to the disparities
from step two.
EXAMPLE 15.3
Consider a small example with 4 objects based on the car marks data set.
Table 15.3:
Dissimilarities
for car marks.
|
j |
1 |
2 |
3 |
4 |
i |
|
Mercedes |
Jaguar |
Ferrari |
VW |
1 |
Mercedes |
- |
|
|
|
2 |
Jaguar |
3 |
- |
|
|
3 |
Ferrari |
2 |
1 |
- |
|
4 |
VW |
5 |
4 |
6 |
- |
|
Our aim is to find a representation with

via MDS.
Suppose that we choose as an initial configuration of

the
coordinates given in Table
15.4.
Table 15.4:
Initial coordinates for MDS.
i |
|
 |
 |
1 |
Mercedes |
3 |
2 |
2 |
Jaguar |
2 |
7 |
3 |
Ferrari |
1 |
3 |
4 |
VW |
10 |
4 |
|
The corresponding distances

are calculated in Table
15.5
Table 15.5:
Ranks and distances.
 |
 |
 |
 |
1,2 |
5.1 |
3 |
3 |
1,3 |
2.2 |
1 |
2 |
1,4 |
7.3 |
4 |
5 |
2,3 |
4.1 |
2 |
1 |
2,4 |
8.5 |
5 |
4 |
3,4 |
9.1 |
6 |
6 |
|
Figure 15.7:
Initial configuration of the MDS of the car data.
MVAnmdscar1.xpl
|
Figure 15.8:
Scatterplot of dissimilarities against distances.
MVAnmdscar2.xpl
|
A plot of the dissimilarities of Table 15.5
against the distance yields Figure 15.8.
This relation is not satisfactory since the ranking of the
did not result in a monotone relation of the corresponding distances
.
We apply therefore the PAV algorithm.
The first violator of monotonicity is the second point
. Therefore we
average the distances
and
to obtain the disparities
Applying the same procedure to

and

we obtain

. The plot of

versus
the disparities

represents a monotone regression
relationship.
In the initial configuration (Figure 15.7),
the third point (Ferrari)
could be moved so that the distance to object 2 (Jaguar) is reduced.
This procedure however also alters the distance between objects 3 and 4.
Care should be given when establishing a monotone relation
between
and
.
In order to assess how well the derived configuration fits the given
dissimilarities Kruskal suggests a measure called STRESS1 that is given by
 |
(15.21) |
An alternative stress measure is given by
 |
(15.22) |
where
denotes the average distance.
EXAMPLE 15.4
The Table
15.6
presents the STRESS calculations for the car example.
The average distance is
. The corresponding
STRESS measures are:
Table 15.6:
STRESS calculations for car marks example.
 |
 |
 |
 |
 |
 |
 |
(2,3) |
1 |
4.1 |
3.15 |
0.9 |
16.8 |
3.8 |
(1,3) |
2 |
2.2 |
3.15 |
0.9 |
4.8 |
14.8 |
(1,2) |
3 |
5.1 |
5.1 |
0 |
26.0 |
0.9 |
(2,4) |
4 |
8.5 |
7.9 |
0.4 |
72.3 |
6.0 |
(1,4) |
5 |
7.3 |
7.9 |
0.4 |
53.3 |
1.6 |
(3,4) |
6 |
9.1 |
9.1 |
0 |
82.8 |
9.3 |
 |
|
36.3 |
|
2.6 |
256.0 |
36.4 |
|
The goal is to find a point configuration that balances the effects STRESS
and non monotonicity. This is achieved by an iterative procedure. More
precisely, one defines a new position of object
relative to object
by
 |
(15.23) |
Here
denotes the step width of the iteration.
By (15.23) the configuration of object
is improved
relative to object
. In order to obtain an overall improvement
relative to all remaining points one uses:
 |
(15.24) |
The choice of step width
is crucial. Kruskal proposes a starting
value of
. The iteration is continued
by a numerical approximation procedure,
such as steepest descent or the Newton-Raphson procedure.
In a fourth step, the evaluation phase, the STRESS measure is used
to evaluate whether or not its change as a result of the last iteration is
sufficiently small that the procedure is terminated.
At this stage the optimal fit has been obtained for a given dimension.
Hence, the whole procedure needs to be carried out for several
dimensions.
EXAMPLE 15.5
Let us compute the new point configuration for

(Ferrari).
The initial coordinates from Table
15.4 are
Applying (
15.24) yields (for

):
Similarly we obtain

.
To find the appropriate number of dimensions,
, a plot of the minimum STRESS value as a function of the
dimensionality is made. One possible criterion in selecting the
appropriate dimensionality is to look for an elbow in the plot.
A rule of thumb that can be used
to decide if a STRESS value is sufficiently
small or not is provided by Kruskal:
 |
(15.25) |
Summary

-
Nonmetric MDS is only based on the rank order of dissimilarities.

-
The object of nonmetric MDS is to create a spatial representation of
the objects with low dimensionality.

-
A practical algorithm is given as:
- Choose an initial configuration.
- Find
from the configuration.
- Fit
, the disparities, by the PAV algorithm.
- Find a new configuration
by using the steepest descent.
- Go to 2.