5.5 Smoothed Principal Components Analysis

As we can see in Figure 5.4, the resulting eigenfunctions are often very rough. Smoothing them could result in a more natural interpretation of the obtained weight functions. Here we apply a popular approach known as roughness penalty. The downside of this technique is that we loose orthogonality in the $ L^2$ sense.

Assume that the underlying eigenfunctions of the covariance operator have a continuous and square-integrable second derivative. Let $ \mathcal{D}\gamma=\gamma'(t)$ be the first derivative operator and define the roughness penalty by $ \Psi(\gamma)=\vert\vert\mathcal{D}^2 \gamma\vert\vert^2.$ Moreover, suppose that $ \gamma_m$ has square-integrable derivatives up to degree four and that the second and the third derivatives satisfy one of the following conditions:

  1. $ \mathcal{D}^2\gamma$, $ \mathcal{D}^3\gamma$ are zero at the ends of the interval $ J$,
  2. the periodicity boundary conditions of $ \gamma$, $ \mathcal{D}\gamma$, $ \mathcal{D}^2\gamma$, and $ \mathcal{D}^3\gamma$ on $ J$.
Then we can rewrite the roughness penalty in the following way:
$\displaystyle \vert\vert\mathcal{D}^2 \gamma\vert\vert^2$ $\displaystyle =$ $\displaystyle \int \mathcal{D}^2\gamma(s)\mathcal{D}^2\gamma(s)ds$  
$\displaystyle ~$ $\displaystyle =$ $\displaystyle \mathcal{D}\gamma (u)\mathcal{D}^2\gamma (u) - \mathcal{D}\gamma(d)\mathcal{D}^2\gamma(d) - \int \mathcal{D}\gamma(s)\mathcal{D}^3\gamma(s)ds$ (5.7)
$\displaystyle ~$ $\displaystyle =$ $\displaystyle \gamma (u)\mathcal{D}^3\gamma (u) - \gamma(d)\mathcal{D}^3\gamma(d) - \int \gamma(s)\mathcal{D}^4\gamma(s)ds$ (5.8)
$\displaystyle ~$ $\displaystyle =$ $\displaystyle \langle\gamma,\mathcal{D}^4\gamma\rangle,$ (5.9)

where $ d$ and $ u$ are the boundaries of the interval $ J$ and the first two elements in (5.7) and (5.8) are both zero under any of the conditions mentioned above.

Given a eigenfunction $ \gamma$ with norm $ \vert\vert\gamma\vert\vert^2=1$, we can penalize the sample variance of the principal component by dividing it by $ 1 + \alpha\langle\gamma, \mathcal{D}^4\gamma\rangle $:

$\displaystyle PCAPV \stackrel{\mathrm{def}}{=}\frac{\int\int \gamma(s)\widehat{...
...t)\gamma(t)dsdt}{\int\gamma(t)(\mathcal{I} + \alpha \mathcal{D}^4)\gamma(t)dt},$     (5.10)

where $ \mathcal{I}$ denotes the identity operator. The maximum of the penalized sample variance (PCAPV) is an eigenfunction $ \gamma$ corresponding to the largest eigenvalue of the generalized eigenequation:

$\displaystyle \int \widehat{\mathop{\hbox{Cov}}} (s,t)\gamma(t)dt=\lambda(\mathcal{I}+\alpha \mathcal{D}^4) \gamma(s).$ (5.11)

As already mentioned above, the resulting weight functions (eigenfunctions) are no longer orthonormal in the $ L^2$ sense. Since the weight functions are used as smoothed estimators of principal components functions, we need to rescale them to satisfy $ \vert\vert\gamma_l\vert\vert^2=1$. The weight functions $ \gamma_l$ can be also interpreted as orthogonal in the modified scalar product of the Sobolev type

$\displaystyle (f,g)\stackrel{\mathrm{def}}{=}\langle f,g\rangle + \alpha\langle \mathcal{D}^2f,\mathcal{D}^2g\rangle .$

A more extended theoretical discussion can be found in Silverman (1991).


5.5.1 Basis Expansion

Define $ \mathbf{K}$ to be a matrix whose elements are $ \langle \mathcal{D}^2\Theta_j,\mathcal{D}^2\Theta_k\rangle$. Then the generalized eigenequation (5.11) can be transformed to:

$\displaystyle \mathbf{W}\mathop{\hbox{Cov}}(\mathbf{C})\mathbf{W} \mathbf{u}=\lambda(\mathbf{W}+\alpha\mathbf{K})\mathbf{u}.$ (5.12)

Using Cholesky factorization $ \mathbf{LL}^{\top}=\mathbf{W}+\alpha\mathbf{K}$ and defining $ \mathbf{S}=\mathbf{L}^{-1}$ we can rewrite (5.12) as:

$\displaystyle \{ \mathbf{S}\mathbf{W}\mathop{\hbox{Cov}}(\mathbf{C})\mathbf{W}\...
...{S}^{\top}\} (\mathbf{L}^{\top}\mathbf{u})=\lambda\mathbf{L}^{\top} \mathbf{u}.$

Applying Smoothed Functional PCA (SPCA) to the IV-strings, we get the smooth-eigenfunctions plotted in Figure 5.5. We use $ \alpha =10^{-7}$, the aim is to use a rather small degree of smoothing, in order to replace the high frequency fluctuations only. Some popular methods, like cross-validation, could be employed as well, Ramsay and Silverman (1997).

Figure 5.5: Smoothed weight functions with $ \alpha =10^{-7}$. Blue solid lines, $ \hat{\gamma}_1^{1M}$ and $ \hat{\gamma}_1^{2M}$, are the first eigenfunctions, green finely dashed lines, $ \hat{\gamma}_2^{1M}$ and $ \hat{\gamma}_2^{2M}$, are the second eigenfunctions, and cyan dashed lines, $ \hat{\gamma}_3^{1M}$ and $ \hat{\gamma}_3^{2M}$, are the third eigenfunctions.

The interpretation of the weight functions displayed in Figure 5.5 is as follows: The first weight function (solid blue) represents clearly the level of the volatility - weights are almost constant and positive. The second weight function (finely dashed green) changes sign near the at-the-money point, i.e. can be interpreted as the in-the-money/out-of-the-money identification factor or slope. The third (dashed cyan) weight function may play the part of the measure for a deep in-the-money or out-of-the-money factor or curvature. It can be seen that the weight functions for the 1M ( $ \widetilde{\gamma}_1^{1M}, \widetilde{\gamma}_2^{1M}, \widetilde{\gamma}_3^{1M}$) and 2M maturities ( $ \widetilde{\gamma}_1^{2M}, \widetilde{\gamma}_2^{2M}, \widetilde{\gamma}_3^{2M}$) have a similar structure. From a practical point of view it can be interesting to try to get common estimated eigenfunctions (factors in the further analysis) for both groups. In the next section, we introduce the estimation motivated by the Common Principal Component Model.