8.4 Relations between Subspaces

The aim of this section is to present a duality relationship between the two approaches shown in Sections 8.2 and 8.3. Consider the eigenvector equations in $\mathbb{R}^n$

$$(\data{X}\data{X}^{\top})v_k=\mu _kv_k$$ (8.6)

for $k\le r$, where $r=\textrm{rank}(\data{X}\data{X}^{\top})=\textrm{rank}(\data{X})\le \min(p,n)$. Multiplying by $\data{X}^{\top}$, we have

$$\data{X}^{\top}(\data{X}\data{X}^{\top})v_k = \mu _k\data{X}^{\top}v_k$$ (8.7)

$$\textrm{or }\quad (\data{X}^{\top}\data{X})(\data{X}^{\top}v_k) = \mu _k(\data{X}^{\top}v_k)$$ (8.8)

so that each eigenvector $v_k$ of $\data{X}\data{X}^{\top}$ corresponds to an eigenvector $(\data{X}^{\top}v_k)$ of $\data{X}^{\top}\data{X}$ associated with the same eigenvalue $\mu _k$. This means that every non-zero eigenvalue of $\data{X}\data{X}^{\top}$ is an eigenvalue of $\data{X}^{\top}\data{X}$. The corresponding eigenvectors are related by

$$u_k=c_k\data{X}^{\top}v_k,$$

where $c_k$ is some constant.

Now consider the eigenvector equations in $\mathbb{R}^p$:

$$(\data{X}^{\top}\data{X})u_k=\lambda_ku_k$$ (8.9)

for $k\le r.$ Multiplying by $\data{X}$, we have

$$(\data{X}\data{X}^{\top})(\data{X}u_k)=\lambda _k(\data{X}u_k),$$ (8.10)

i.e., each eigenvector $u_k$ of $\data{X}^{\top}\data{X}$ corresponds to an eigenvector $\data{X}u_k$ of $\data{X}\data{X}^{\top}$ associated with the same eigenvalue $\lambda_k$. Therefore, every non-zero eigenvalue of $(\data{X}^{\top}\data{X})$ is an eigenvalue of $\data{X}\data{X}^{\top}$. The corresponding eigenvectors are related by

$$v_k=d_k\data{X}u_k,$$

where $d_k$ is some constant. Now, since $u^{\top}_ku_k=v^{\top}_kv_k=1$ we have $c_k=d_k=\frac{1 }{\sqrt {\lambda _k} }$. This lead to the following result:

THEOREM 8.4 (Duality Relations)   Let $r$ be the rank of $\data{X}$. For $k\le r$, the eigenvalues $\lambda_{k}$ of $\data{X}^{\top}\data{X}$ and $\data{X}\data{X}^{\top}$ are the same and the eigenvectors ($u_{k}$ and $v_{k}$, respectively) are related by

$$u_{k} = \frac{1}{\sqrt{\lambda_{k}}} \data{X}^{\top}v_{k}$$ (8.11)

$$v_{k} = \frac{1}{\sqrt{\lambda_{k}}} \data{X}u_{k}.$$ (8.12)

Note that the projection of the $p$ variables on the factorial axis $v_{k}$ is given by

$$w_{k} = \data{X}^{\top}v_{k} = \frac{1}{\sqrt{\lambda_{k}}} \data{X}^{\top}\data{X}u_{k} = \sqrt{\lambda_{k}}\ u_{k}.$$ (8.13)

Therefore, the eigenvectors $v_k$ do not have to be explicitly recomputed to get $w_{k}$.

Note that $u_k$ and $v_k$ provide the SVD of $\data{X}$ (see Theorem 2.2). Letting
$U=[u_1\; u_2\; \ldots \; u_r], \; V=[v_1\; v_2\; \ldots \; v_r]$ and $\Lambda=\mathop{\hbox{diag}}(\lambda_1, \ldots, \lambda_r)$ we have

$$\data{X} = V\; \Lambda^{1/2}\; U^{\top}$$

so that

$$x_{ij} = \sum_{k=1}^r \lambda_k^{1/2} \, v_{ik} \, u_{jk}.$$ (8.14)

In the following section this method is applied in analysing consumption behavior across different household types.

Summary

$\ast$

The matrices $\data{X}^{\top}\data{X}$ and $\data{X}\data{X}^{\top}$ have the same non-zero eigenvalues $\lambda_{1},\ldots,\lambda_{r}$, where $r=\mathop{\rm {rank}}(\data{X})$.

$\ast$

The eigenvectors of $\data{X}^{\top}\data{X}$ can be calculated from the eigenvectors of $\data{X}\data{X}^{\top}$ and vice versa:

$$u_{k}=\frac{1}{\sqrt{\lambda_{k}}}\data{X}^{\top}v_{k}\quad \textrm{and}\quad v_{k}=\frac{1}{\sqrt{\lambda_{k}}}\data{X}u_{k}.$$

$\ast$

The coordinates representing the variables (columns) of $\data{X}$ in a $q$-dimensional subspace can be easily calculated by $w_{k}=\sqrt{\lambda_{k}}u_{k}$.