10. Factor Analysis

A frequently applied paradigm in analyzing data from multivariate observations is to model the relevant information (represented in a multivariate variable $X$) as coming from a limited number of latent factors. In a survey on household consumption, for example, the consumption levels, $X$, of $p$ different goods during one month could be observed. The variations and covariations of the $p$ components of $X$ throughout the survey might in fact be explained by two or three main social behavior factors of the household. For instance, a basic desire of comfort or the willingness to achieve a certain social level or other social latent concepts might explain most of the consumption behavior. These unobserved factors are much more interesting to the social scientist than the observed quantitative measures ($X$) themselves, because they give a better understanding of the behavior of households. As shown in the examples below, the same kind of factor analysis is of interest in many fields such as psychology, marketing, economics, politic sciences, etc.

How can we provide a statistical model addressing these issues and how can we interpret the obtained model? This is the aim of factor analysis. As in Chapter 8 and Chapter 9, the driving statistical theme of this chapter is to reduce the dimension of the observed data. The perspective used, however, is different: we assume that there is a model (it will be called the ``Factor Model'') stating that most of the covariances between the $p$ elements of $X$ can be explained by a limited number of latent factors. Section 10.1 defines the basic concepts and notations of the orthogonal factor model, stressing the non-uniqueness of the solutions. We show how to take advantage of this non-uniqueness to derive techniques which lead to easier interpretations. This will involve (geometric) rotations of the factors. Section 10.2 presents an empirical approach to factor analysis. Various estimation procedures are proposed and an optimal rotation procedure is defined. Many examples are used to illustrate the method.