4.4 The Multinormal Distribution

The multinormal distribution with mean $\mu$ and covariance $\Sigma > 0$ has the density

$$f(x) = \vert 2 \pi \Sigma \vert^{-1/2} \exp \left \{ -\frac{1}{2} (x- \mu)^{\top} \Sigma^{-1}(x- \mu) \right \}.$$ (4.47)

We write $X \sim N_p (\mu, \Sigma)$.

How is this multinormal distribution with mean $\mu$ and covariance $\Sigma$ related to the multivariate standard normal $N_p(0,{\data{I}}_p)$? Through a linear transformation using the results of Section 4.3, as shown in the next theorem.

THEOREM 4.5   Let $X \sim N_p (\mu, \Sigma)$ and $Y = \Sigma ^{-1/2}(X-\mu )$ (Mahalanobis transformation). Then

$$Y\sim N_p(0,\data{I}_p),$$

i.e., the elements $Y_j\in \mathbb{R}$ are independent, one-dimensional $N(0,1)$ variables.

PROOF:
Note that $(X-\mu)^{\top} \Sigma^{-1}(X-\mu) = Y^{\top}Y$. Application of (4.45) gives ${\cal J} = \Sigma^{1/2}$, hence

$$f_Y(y) = (2 \pi)^{-p/2} \exp \left(-\frac{1}{2}y^{\top}y \right)$$ (4.48)

which is by (4.47) the pdf of a $N_p(0,\data{I}_p)$. ${\Box}$

Note that the above Mahalanobis transformation yields in fact a random variable $Y=(Y_1,\ldots ,Y_p)^{\top}$ composed of independent one-dimensional $Y_j\sim N_1(0,1)$ since

$$\begin{eqnarray*} f_Y(y) & = & \frac{1 }{(2\pi )^{p/2} }\exp\left (-\frac{1 }{2 ... ...frac{1 }{2}y^2_j\right )\\ & = & \prod ^p_{j=1}f_{Y_{j}}(y_j). \end{eqnarray*}$$

Here each $f_{Y_{j}}(y)$ is a standard normal density $\frac{1}{\sqrt{2\pi}} \exp \left(-\frac{y^2}{2} \right)$. From this it is clear that $E(Y)=0$ and $\Var(Y)= \data{I}_p$.

How can we create $N_p(\mu,\Sigma)$ variables on the basis of $N_p(0,\data{I}_p)$ variables? We use the inverse linear transformation

$$X = \Sigma^{1/2}Y + \mu.$$ (4.49)

Using (4.11) and (4.23) we can also check that $E(X)= \mu$ and $\Var(X) = \Sigma$. The following theorem is useful because it presents the distribution of a variable after it has been linearly transformed. The proof is left as an exercise.

THEOREM 4.6   Let $X \sim N_p (\mu, \Sigma)$ and $\data{A}(p\times p),\; c \in \mathbb{R}^p$, where $\data{A}$ is nonsingular.

Then $Y = \data{A} X +c$ is again a $p$-variate Normal, i.e.,

$$Y \sim N_p ( \data{A} \mu +c, \data{A} \Sigma \data{A}^{\top}).$$ (4.50)

Geometry of the $N_p(\mu,\Sigma)$ Distribution

From (4.47) we see that the density of the $N_p(\mu,\Sigma)$ distribution is constant on ellipsoids of the form

$$(x-\mu )^{\top} \Sigma^{-1}(x-\mu) = d^2.$$ (4.51)

EXAMPLE 4.14   Figure 4.3 shows the contour ellipses of a two-dimensional normal distribution. Note that these contour ellipses are the iso-distance curves (2.34) from the mean of this normal distribution corresponding to the metric $\Sigma^{-1}$.

Figure 4.3: Scatterplot of a normal sample and contour ellipses for $\mu=\left(3\atop 2\right)$ and $\Sigma=\left({1\atop -1.5}{-1.5\atop 4}\right)$. MVAcontnorm.xpl

According to Theorem 2.7 in Section 2.6 the half-lengths of the axes in the contour ellipsoid are $\sqrt{\frac{d^2}{\nu_{i}}}$ where $\nu_{i} = \frac{1}{\lambda_{i}}$ are the eigenvalues of $\Sigma^{-1}$ and $\lambda_{i}$ are the eigenvalues of $\Sigma$. The rectangle inscribing an ellipse has sides with length $2d\sigma_i$ and is thus naturally proportional to the standard deviations of $X_i\ (i=1,2)$.

The distribution of the quadratic form in (4.51) is given in the next theorem.

THEOREM 4.7   If $X \sim N_p (\mu, \Sigma)$, then the variable $U = (X-\mu)^{\top} \Sigma^{-1}(X-\mu)$ has a $\chi^2_p$ distribution.

THEOREM 4.8   The characteristic function (cf) of a multinormal $N_p(\mu,\Sigma)$ is given by

$$\varphi_X(t) = \exp(\textrm{\bf i}\ t^{\top} \mu -\frac{1}{2}t^{\top} \Sigma t).$$ (4.52)

We can check Theorem 4.8 by transforming the cf back:

$$\begin{eqnarray*} f(x) & = & \frac{1}{(2\pi)^p} \int \exp\left(-{\bf i}t^{\top}x... ...} \exp\left[-\frac{1}{2}\{(x-\mu)^{\top}\Sigma (x-\mu)\}\right] \end{eqnarray*}$$

since

$$\begin{eqnarray*} &&\int \frac{1}{\vert 2\pi\Sigma^{-1}\vert^{1/2}} \exp \left[-... ...top}\Sigma(t+{\bf i}\Sigma^{-1}(x-\mu))\}\right]\,dt \\ &&= 1. \end{eqnarray*}$$

Note that if $Y\sim N_p(0,\data{I}_p)$ (e.g., the Mahalanobis-transform), then

$$\begin{eqnarray*} \varphi_Y(t) & = & \exp\left(-\frac{1}{2}t^{\top} \data{I}_p t... ...\ & = & \varphi_{Y_1}(t_1)\cdot \ldots \cdot \varphi_{Y_p}(t_p) \end{eqnarray*}$$

which is consistent with (4.33).

Singular Normal Distribution

Suppose that we have $\mathop{\rm {rank}}(\Sigma ) = k < p$, where $p$ is the dimension of $X$. We define the (singular) density of $X$ with the aid of the $G$-Inverse $\Sigma^{-}$ of $\Sigma$,

$$f(x) = \frac{(2\pi)^{-k/2}} {(\lambda_1 \cdots \lambda_k)^{1... ...ft \{ -\frac{1}{2} (x-\mu)^{\top} \Sigma^{-} (x-\mu) \right \}$$ (4.53)

where

(1)

$x$ lies on the hyperplane $\data{N}^{\top} (x-\mu) = 0$ with $\data{N} (p \times (p-k)) : \data{N}^{\top} \Sigma = 0$ and $\data{N}^{\top}\data{N} = \data{I}_k$.

(2)

$\Sigma^{-}$ is the $G$-Inverse of $\Sigma$, and $\lambda_1, \ldots, \lambda_k$ are the nonzero eigenvalues of $\Sigma$.

What is the connection to a multinormal with $k$-dimensions? If

$$Y \sim N_k (0, \Lambda_1)\ \textrm{ and }\ \Lambda_1 = \mathop{\hbox{diag}}(\lambda_1, \ldots, \lambda_k),$$ (4.54)

then there exists an orthogonal matrix $\data{B} (p \times k)$ with $\data{B} ^{\top}\data{B} = \data{I}_k$ so that $X = \data{B} Y + \mu$ where $X$ has a singular pdf of the form (4.53).

Gaussian Copula

The second important copula that we want to present is the Gaussian or normal copula,

$$C_{\rho}(u, v) = \int_{- \infty}^{\Phi_1^{-1}(u)} \int_{- \infty}^{\Phi_2^{-1}(v)} f_\rho(x_1,x_2) d x_2 d x_1\; ,$$ (4.55)

see Embrechts et al. (1999). In (4.55), $f_\rho$ denotes the bivariate normal density function with correlation $\rho$ for $n=2$. The functions $\Phi_1$ and $\Phi_2$ in (4.55) refer to the corresponding one-dimensional standard normal cdfs of the margins.

In the case of vanishing correlation, $\rho = 0$, the Gaussian copula becomes

$$C_0(u, v) = \int_{- \infty}^{\Phi_1^{-1}(u)} f_{X_1}(x_1) d x_1 \; \int_{- \infty}^{\Phi_2^{-1}(v)} f_{X_2}(x_2) d x_2 \;$$

$$= u \, v$$

$$= \Pi (u,v)\; .$$

Summary

$\ast$

The pdf of a $p$-dimensional multinormal $X\sim N_{p}(\mu,\Sigma)$ is

$$f(x) = \vert 2 \pi \Sigma \vert^{-1/2} \exp \left \{ -\frac{1}{2} (x- \mu)^{\top} \Sigma^{-1}(x- \mu) \right \}.$$

The contour curves of a multinormal are ellipsoids with half-lengths proportional to $\sqrt{\lambda_{i}}$, where $\lambda_{i}$ denotes the eigenvalues of $\Sigma$ ($i=1,\dots,p$).

$\ast$

The Mahalanobis transformation transforms $X\sim N_{p}(\mu,\Sigma)$ to $Y=\Sigma^{-1/2}(X-\mu) \sim N_{p}(0,\data{I}_{p})$. Going the other direction, one can create a $X\sim N_{p}(\mu,\Sigma)$ from $Y \sim N_{p}(0,\data{I}_{p})$ via $X=\Sigma^{1/2}Y+\mu$.

$\ast$

If the covariance matrix $\Sigma$ is singular (i.e., $\mathop{\rm {rank}}(\Sigma) < p$), then it defines a singular normal distribution.

$\ast$

The density of a singular normal distribution is given by

$$\begin{eqnarray*}\frac{(2\pi)^{-k/2}} {(\lambda_1 \cdots \lambda_k)^{1/2}} \exp \left\{ -\frac{1}{2} (x-\mu)^{\top} \Sigma^{-} (x-\mu) \right\} . \end{eqnarray*}$$