12.2 Autoregressive Process

The linear autoregressive process of order $p$, (AR($p$)), is defined as

$$X_t = \nu + \alpha_1 X_{t-1} + \ldots + \alpha_p X_{t-p} + \varepsilon_t$$ (12.5)

Using the definition of the lag-operator $L$ (see Definition 10.13), (11.5) can also be written as

$$\alpha(L)X_t = \nu + \varepsilon_t,$$

with the lag-polynomial $\alpha(L) = 1- \alpha_1 L - \ldots - \alpha_p L^p.$ The process $X_t$ is stationary if all roots of the characteristic equation

$$\alpha(z) = 1-\alpha_1 z - \ldots - \alpha_p z^p = 0.$$ (12.6)

lie outside of the complex unit circle, that is, if for all $z$ with $\vert z\vert \le 1$ it holds that

$$\alpha(z) \ne 0.$$ (12.7)

In this case there is an inverted filter $\alpha^{-1}(L)$ for the linear filter $\alpha(L)$, such that the following holds,

$$\alpha(L) \alpha^{-1}(L)=1$$

and

$$\alpha^{-1}(L) = a_0 + a_1 L +a_2 L^2 + \ldots = \sum_{i=0}^{\infty}a_i L^i.$$

The process (11.5) can also be written under the condition (11.7)

$$X_t = \alpha^{-1}(1)\nu + \alpha^{-1}(L) \varepsilon_t$$

$$= \sum_{i=0}^{\infty}a_i \nu + \sum_{i=0}^{\infty}a_i L^i \varepsilon_t,$$

as a MA($\infty$) process.

A simple way to find and invert the autocovariance of an AR($p$) process with given parameters is to use the Yule-Walker equations. They are derived from the definition of an AR($p$) process in (11.5) by multiplying by $X_{t-\tau}$ and taking expectations.

$$\mathop{\text{\rm\sf E}}[X_t X_{t-\tau}] = \alpha_1 \mathop{\text... ...{t-1}X_{t-\tau}] +\ldots+ \alpha_p \mathop{\text{\rm\sf E}}[X_{t-p}X_{t-\tau}].$$ (12.8)

Since $\mathop{\text{\rm\sf E}}[X_t X_{t-\tau}]$ is the definition of the autocovariance function $\gamma_{\tau}$ for $\nu=0$, it can even be written simpler for $\tau=1,\ldots,p$

$$\begin{array}{*{3}{c@{\:+\:}}c@{\;=\;}c} \alpha_1 \gamma_0 & ... ..._{p-2} & \cdots & \alpha_p \gamma_{0} & \gamma_p \\ \end{array}$$ (12.9)

or by dividing by the variance $\gamma_0$

$$\rho = R \alpha$$ (12.10)

with $\rho = (\rho_1 \rho_2 \ldots \rho_p)^\top$, $\alpha = (\alpha_1 \alpha_2 \ldots \alpha_p)^\top$, and the $p \times p$-autocovariance matrix

$$R = \left( \begin{array}{*{3}{c}cc} 1 & \rho_1 & \cdots & \r... ... \\ \rho_{p-1} & \rho_{p-2} & \cdots & 1 \end{array}\right ).$$

The Yule-Walker equations are useful in determining the ACF for given parameters or, vice versa, in determining the estimated parameters for the given (empirical) autocorrelation.

Example 12.1 (AR(1))  
The AR(1) process from Example 10.1 with $\nu=0$ has the characteristic equation $1 - \alpha z = 0$. The explicit solution $z=1/\alpha$ and $\vert z\vert>1$ occurs exactly when $\vert\alpha\vert < 1$. The inverse filter of $\alpha(L) = 1- \alpha L$ is thus $\alpha^{-1}(L)=\sum_{i=0}^{\infty} \alpha^i L^i$ and the MA($\infty$) representation of the AR(1) process is

$$X_t = \sum_{i=0}^{\infty} \alpha^i \varepsilon_{t-i}.$$

The ACF of the AR(1) process is $\rho_{\tau} = \alpha^\tau$. For $\alpha>0$ all autocorrelations are positive, for $\alpha<0$ they alternate between positive and negative, see Figure 11.3.

Fig.: ACF of an AR(1) process with $\alpha =0.9$ (left) and $\alpha=-0.9$ (right). SFEacfar1.xpl

Example 12.2 (AR(2))  
The AR(2) process with $\nu=0$,

$$X_t = \alpha_1 X_{t-1} + \alpha_2 X_{t-2} + \varepsilon_t$$

is stationary when given the roots $z_1$ and $z_2$ of the quadratic equation

$$1-\alpha_1 z - \alpha_2 z^2 = 0,$$

it holds that $\vert z_1\vert>1$ and $\vert z_2\vert>1$. We obtain solutions as

$$z_{1,2} = -\frac{\alpha_1}{2\alpha_2} \pm \sqrt{\frac{\alpha_1^2}{4\alpha_2^2} + \frac{1}{\alpha_2}}$$

and $z_1 z_2 = -1/\alpha_2$. Due to $\vert z_1\vert>1$ and $\vert z_2\vert>1$ it holds that $\vert z_1 z_2\vert = 1/\vert\alpha_2\vert >1$ and

$$\vert\alpha_2\vert < 1.$$ (12.11)

From the Yule-Walker equations in the case of an AR(2) process

$$\rho_1 = \alpha_1 + \alpha_2 \rho_1$$ (12.12)

$$\rho_2 = \alpha_1 \rho_1 + \alpha_2$$ (12.13)

it follows that $\rho_1 = \alpha_1/(1-\alpha_2)$. The case $\rho_1 = \pm 1$ is excluded because a root would lie on the unit circle (at 1 or -1). Thus for a stationary AR(2) process it must hold that

$$\vert\rho_1\vert = \vert\alpha_1/(1-\alpha_2)\vert < 1,$$

from which, together with (11.11), we obtain the `stationarity triangle'

$$\alpha_1 + \alpha_2 < 1$$ (12.14)

$$\alpha_2 - \alpha_1 < 1$$ (12.15)

i.e., the region in which the AR(2) process is stationary.

The ACF of the AR(2) process is recursively given with (11.12), (11.13) and $\rho_\tau=\alpha_1 \rho_{\tau-1}+\alpha_2 \rho_{\tau-2}$ for $\tau>2$. Figure (11.4) displays typical patterns.

Fig.: ACF of a AR(2) process with $(\alpha_1=0.5,\alpha_2=0.4)$ (top left), $(\alpha_1=0.9,\alpha_2=-0.4)$ (top right), $(\alpha_1=-0.4,\alpha_2=0.5)$ (bottom left) and $(\alpha_1=-0.5,\alpha_2=-0.9)$ (bottom right). SFEacfar2.xpl