14.3 Stationary Long Memory Processes

A stationary process $ X$ has the long memory property, if for its autocorrelation function $ \rho(k)=\mathop{\hbox{Cov}}(X_i,X_{i+k})/\textrm{Var}(X_1)$ holds:

$\displaystyle \sum_{k=-\infty}^{\infty}{\rho (k)}$ $\displaystyle =$ $\displaystyle \infty \; .$ (14.1)

That is, the autocorrelations decay to zero so slowly that their sum does not converge, Beran (1994).

With respect to (14.1), note that the classical expression for the variance of the sample mean, $ \bar{X} \stackrel{\mathrm{def}}{=}n^{-1}\sum_{i=1}^{n}{X_i}$, for independent and identically distributed $ X_1,
\hdots, X_n$,

$\displaystyle \textrm{Var}(\bar{X}) = \frac{\sigma^2}{n} \textrm{ with } \sigma^2 = \textrm{Var}(X_i)$ (14.2)

is not valid anymore. If correlations are neither zero and nor so small to be negligible, the variance of $ \bar{X}$ is equal to
$\displaystyle \textrm{Var}(\bar{X})$ $\displaystyle =$ $\displaystyle \frac{\sigma^2}{n}\left(1+2\sum_{k=1}^{n-1}{\left(1-\frac{k}{n}\right)\rho(k)}\right).$ (14.3)

Thus, for long memory processes the variance of the sample mean converges to zero at a slower rate than $ n^{-1}$, Beran (1994). Note that long memory implies positive long range correlations. It is essential to understand that long range dependence is characterized by slowly decaying correlations, although nothing is said about the size of a particular correlation at lag $ k$. Due to the slow decay it is sometimes difficult to detect non zero but very small correlations by looking at the $ \pm
2/\sqrt{n}$-confidence band. Beran (1994) gives an example where the correct correlations are slowly decaying but within the $ \pm
2/\sqrt{n}$-band. So even if estimated correctly we would consider them as non significant.

Note that (14.1) holds in particular if the autocorrelation $ \rho(k)$ is approximately $ c\vert k\vert^{-\alpha}$ with a constant $ c$ and a parameter $ \alpha \in (0,1)$. If we know the autocorrelations we also know the spectral density $ f(\lambda)$, defined as

$\displaystyle f(\lambda)$ $\displaystyle =$ $\displaystyle \frac{\sigma^2}{2\pi}\sum_{k=-\infty}^{\infty}{\rho(k)e^{ik\lambda}}.$ (14.4)

The structure of the autocorrelation then implies, that the spectral density is approximately of the form $ c_f\vert k\vert^{\alpha-1}$ with a constant $ c_f$ as $ \lambda \rightarrow 0$. Thus the spectral density has a pole at 0.

To connect the long memory property with the Hurst coefficient, we introduce self similar processes. A stochastic process $ Y_t$ is called self similar with self similarity parameter $ H$, if for any positive stretching factor $ c$, the rescaled process $ c^{-H} Y_{ct}$ has the same distribution as the original process $ Y_t$. If the increments $ X_t = Y_t - Y_{t-1}$ are stationary, there autocorrelation function is given by

$\displaystyle \rho(k) = \frac{1}{2}\left(\vert k+1\vert^{2H} - 2\vert k\vert^{2H} + \vert k-1\vert^{2H}\right) \; ,$    

Beran (1994). From a Taylor expansion of $ \rho $ it follows

$\displaystyle \frac{\rho(k)}{H(2H-1) k^{2H-2}} \rightarrow 1 \textrm{ for } k \rightarrow \infty \; .$    

This means, that for $ H>0.5$, the autocorrelation function $ \rho(k)$ is approximately $ H(2H-1) k^{-\alpha}$ with $ \alpha = 2 - 2H \in (0,1)$ and thus $ X_t$ has the long memory property.


14.3.1 Fractional Brownian Motion and Noise

In this section, we introduce a particular self similar process with stationary increments, namely the fractional Brownian motion (FBM) and fractional Gaussian noise (FGN), Mandelbrot and van Ness (1968), Beran (1994).

DEFINITION 14.1   Let $ B_H(t)$ be a stochastic process with continuous sample paths and such that for any $ H \in (0,1)$ and $ \sigma^2$ a variance scaling parameter. Then $ B_H(t)$ is called fractional Brownian motion.

Essentially, this definition is the same as for standard Brownian motion besides that the covariance structure is different. For $ H=0.5$, definition 14.1 contains standard Brownian motion as a special case but in general ( $ H\not\neq 0.5$), increments $ B_H(t)-B_H(s)$ are not independent anymore. The stochastic process resulting by computing first differences of FBM is called FGN with parameter $ H$. The covariance at lag $ k$ of FGN follows from definition 14.1:
$\displaystyle \gamma (k)$ $\displaystyle =$ $\displaystyle \mathop{\hbox{Cov}}\left\{B_H(t)-B_H(t-1),B_H(t+k)-B_H(t+k-1)\right\}$  
  $\displaystyle =$ $\displaystyle \frac{\sigma^2}{2}\left(\vert k+1\vert^{2H} - 2\vert k\vert^{2H} + \vert k-1\vert^{2H}\right)$ (14.5)

For $ 0.5<H<1$ the process has long range dependence, and for $ 0<H<0.5$ the process has short range dependence.

Figures 14.2 and 14.3 show two simulated paths of $ N=1000$ observations of FGN with parameter $ H=0.8$ and $ H=0.2$ using an algorithm proposed by Davies and Harte (1987). For $ H=0.2$, the FBM path is much more jagged and the range of the $ y$-axis is about ten times smaller than for $ H=0.8$ which is due to the reverting behavior of the time series.

Figure 14.2: Simulated FGN with $ H=0.8$, $ N=1000$ and path of corresponding FBM.
\includegraphics[width=1.25\defpicwidth]{gFGN08.ps} \includegraphics[width=1.25\defpicwidth]{gFBM08.ps}

Figure: Simulated FGN with $ H=0.2$, $ N=1000$ and path of corresponding FBM. 28737 XFGSimFBM.xpl
\includegraphics[width=1.25\defpicwidth]{gFGN02.ps} \includegraphics[width=1.25\defpicwidth]{gFBM02.ps}

The estimated autocorrelation function (ACF) for the path simulated with $ H=0.8$ along with the $ \pm2/\sqrt{N}$-confidence band is shown in Figure 14.4. For comparison the ACF used to simulate the process given by (14.5) is superimposed (dashed line). The slow decay of correlations can be seen clearly.

Figure: Estimated and true ACF of FGN simulated with $ H=0.8$, $ N=1000$. 28741 XFGSimFBM.xpl
\includegraphics[width=1.5\defpicwidth]{gACF08.ps}

Applying R/S analysis we can retrieve the Hurst coefficient used to simulate the process. Figure 14.5 displays the estimated regression line and the data points used in the regression. We simulate the process with $ H=0.8$ and the R/S statistic yields $ \hat{H}=0.83$.

Finally, we mention that fractional Brownian motion is not the only stationary process revealing properties of systems with long memory. Fractional ARIMA processes are an alternative to FBM, Beran (1994). As well, there are non stationary processes with infinite second moments that can be used to model long range dependence, Samrodnitsky and Taqqu (1994).

Figure: Hurst regression and estimated Hurst coefficient ( $ \hat{H}=0.83$) of FBM simulated with $ H=0.8$, $ N=1000$. 28745 XFGSimFBMHurst.xpl
\includegraphics[width=1.25\defpicwidth]{gHurstPlot.ps}