Up to now strategies have been presented for factor analysis
that have concentrated
on the estimation of loadings and communalities and on their interpretations.
This was a logical step since the factors were
considered to be normalized random sources of information and were
explicitely addressed as
nonspecific (common factors). The estimated values of the factors, called
the factor scores,
may also be useful in the interpretation as well as in the
diagnostic analysis. To be more precise,
the factor scores are estimates of the
unobserved random vectors
,
, for each individual
,
. Johnson and Wichern (1998) describe three methods
which in practice yield very similar results. Here, we present the regression
method which has the advantage of being the simplest technique and is easy
to implement.
The idea is to consider the joint distribution of and
, and then
to proceed with the regression analysis presented in Chapter 5.
Under the factor model (10.4), the joint covariance matrix of
and
is:
Assuming joint normality, the conditional distribution of
is multinormal, see Theorem 5.1, with
The same rule can be followed when using instead of
. Then (10.18) remains valid when
standardized variables, i.e.,
,
are considered if
.
In this case the factors are given by
If the factors are rotated by the orthogonal matrix , the factor
scores have to be rotated accordingly, that is
No one method outperforms another in the practical implementation of factor analysis. However, by applying the tâtonnement process, the factor analysis view of the data can be stabilized. This motivates the following procedure.
Factor analysis and principal component analysis use the same set of
mathematical tools (spectral decomposition, projections, ). One
could conclude, on first sight, that they share the same view and
strategy and therefore yield very similar results. This is not true.
There are substantial differences between these two data analysis
techniques that we would like to describe here.
The biggest difference between PCA and factor analysis comes from the
model philosophy. Factor analysis imposes a strict structure of a fixed number
of common (latent) factors whereas the PCA determines factors in
decreasing order of importance. The most important factor in PCA is the one
that maximizes the projected variance. The most important factor in
factor analysis is the one that (after rotation) gives the maximal
interpretation. Often this is different from the direction of the
first principal component.
From an implementation point of view, the PCA is based on a well-defined, unique algorithm (spectral decomposition), whereas fitting a factor analysis model involves a variety of numerical procedures. The non-uniqueness of the factor analysis procedure opens the door for subjective interpretation and yields therefore a spectrum of results. This data analysis philosophy makes factor analysis difficult especially if the model specification involves cross-validation and a data-driven selection of the number of factors.