5.4 Functional Principal Components

Principal Components Analysis yields dimension reduction in the multivariate framework. The idea is to find normalized weight vectors $ \mathbf{\gamma}_m\in \mathbb{R}^p,$ for which the linear transformations of a $ p$-dimensional random vector $ \mathbf{x}$, with $ \textrm{E}[\mathbf{x}]=0$:

$\displaystyle f_{m}=\mathbf{\gamma}_m^{\top}\mathbf{x}=\ \langle\mathbf{\gamma}_m,\mathbf{x}\rangle ,\ m=1,\ldots,p,$ (5.4)

have maximal variance subject to:

$\displaystyle \mathbf{\gamma}_l^{\top}\mathbf{\gamma}_m=\ \langle\mathbf{\gamma}_l,\mathbf{\gamma}_m\rangle \ =I(l=m)\ \textrm{for}\ l\leq m.$

Where $ I$ denotes the identificator function. The solution is the Jordan spectral decomposition of the covariance matrix, Härdle and Simar (2003).

In the Functional Principal Components Analysis (FPCA) the dimension reduction can be achieved via the same route, i.e. by finding orthonormal weight functions $ \gamma_1, \gamma_2, \ldots$, such that the variance of the linear transformation is maximal. In order to keep notation simple we assume $ \textrm{E}X(t)=0$. The weight functions satisfy:

$\displaystyle \vert\vert\gamma_m\vert\vert^2$ $\displaystyle =$ $\displaystyle \int \gamma_m(t)^2dt=1,$  
$\displaystyle \langle\gamma_l,\gamma_m\rangle$ $\displaystyle =$ $\displaystyle \int \gamma_l(t)\gamma_m(t)dt=0,\ l \neq m.$  

The linear transformation is:

$\displaystyle f_m=\ \langle\gamma_m,X\rangle \ =\int \gamma_m(t)X(t)dt,$

and the desired weight functions solve:
$\displaystyle \mathop{\rm arg\,max}\limits_{\langle\gamma_l,\gamma_m\rangle =I(l=m), l\leq m}\textrm{Var}\langle\gamma_m,X\rangle,$     (5.5)

or equivalently:

$\displaystyle \mathop{\rm arg\,max}\limits_{\langle\gamma_l,\gamma_m\rangle =I(l=m), l\leq m}\int\int\gamma_m(s)\textrm{Cov}(s,t)\gamma_m(t)dsdt.$

The solution is obtained by solving the Fredholm functional eigenequation

$\displaystyle \int \textrm{Cov}(s,t)\gamma(t)dt=\lambda \gamma(s).$ (5.6)

The eigenfunctions $ \gamma_1, \gamma_2, \ldots$ sorted with respect to the corresponding eigenvalues $ \lambda_1 \geq \lambda_2\geq \ldots$ solve the FPCA problem (5.5). The following link between eigenvalues and eigenfunctions holds:

$\displaystyle \lambda_m = \textrm{Var}(f_m) = \textrm{Var}\left[\int \gamma_m(t) X(t) dt\right]= \int \int \gamma_m(s) \textrm{Cov}(s,t) \gamma_m(t) ds dt.$

In the sampling problem, the unknown covariance function $ \textrm{Cov}(s,t)$ needs to be replaced by the sample covariance function $ \widehat{\textrm{Cov}}(s,t)$. Dauxois, Pousse, and Romain (1982) show that the eigenfunctions and eigenvalues are consistent estimators for $ \lambda_m$ and $ \gamma_m$ and derive some asymptotic results for these estimators.


5.4.1 Basis Expansion

Suppose that the weight function $ \gamma$ has expansion

$\displaystyle \gamma = \sum\limits_{l=1}^L \mathbf{b}_l \Theta_l(t) =\mathbf{\Theta}^{\top}\mathbf{b}.$

Using this notation we can rewrite the left hand side of eigenequation (5.6):
$\displaystyle \int\textrm{Cov}(s,t)\gamma(t)dt$ $\displaystyle =$ $\displaystyle \int\mathbf{\Theta}(s)^{\top}\textrm{Cov}(\mathbf{C})\mathbf{\Theta}(t)\mathbf{\Theta}(t)^{\top}\mathbf{b}dt$  
$\displaystyle ~$ $\displaystyle =$ $\displaystyle \mathbf{\Theta}^{\top}\textrm{Cov}(\mathbf{C})\mathbf{W}\mathbf{b},$  

so that:

$\displaystyle \textrm{Cov}(\mathbf{C})\mathbf{W}\mathbf{b}=\lambda\mathbf{b}.$

The functional scalar product $ \langle\gamma_l,\gamma_k\rangle $ corresponds to $ \mathbf{b}_l^{\top}\mathbf{W}\mathbf{b}_k$ in the truncated basis framework, in the sense that if two functions $ \gamma_l$ and $ \gamma_k$ are orthogonal, the corresponding coefficient vectors $ \mathbf{b}_l,\mathbf{b}_k$ satisfy $ \mathbf{b}_l^{\top}\mathbf{W}\mathbf{b}_k=0$. Matrix $ \mathbf{W}$ is symmetric by definition. Thus, defining $ \mathbf{u}=\mathbf{W}^{1/2}\mathbf{b}$, one needs to solve finally a symmetric eigenvalue problem:

$\displaystyle \mathbf{W}^{1/2}\textrm{Cov}(\mathbf{C})\mathbf{W}^{1/2}\mathbf{u}=\lambda\mathbf{u},$

and to compute the inverse transformation $ \mathbf{b}=\textbf{W}^{-1/2}\mathbf{u}$. For the orthonormal functional basis (i.e. also for the Fourier basis) $ \mathbf{W}=\mathbf{I}$, i.e. the problem of FPCA is reduced to the multivariate PCA performed on the matrix $ \mathbf{C}$.

Using the FPCA method on the IV-strings for 1M and 2M maturities we obtain the eigenfunctions plotted in Figure 5.4. It can be seen, that the eigenfunctions are too rough. Intuitively, this roughness is caused by the flexibility of the functional basis. In the next section we present a way of incorporating the smoothing directly into the PCA problem.

Figure 5.4: Weight functions for $ 1M$ and $ 2M$ maturity groups. Blue solid lines, $ \hat{\gamma}_1^{1M}$ and $ \hat{\gamma}_1^{2M}$, are the first eigenfunctions, green finely dashed lines, $ \hat{\gamma}_2^{1M}$ and $ \hat{\gamma}_2^{2M}$, are the second eigenfunctions, and cyan dashed lines, $ \hat{\gamma}_3^{1M}$ and $ \hat{\gamma}_3^{2M}$, are the third eigenfunctions.