7. Generalized Linear Models

Marlene Müller
28 July 2004

McCullagh and Nelder (1989) summarized many approaches to relax the distributional assumptions of the classical linear model under the common term Generalized Linear Models (GLM). A generalized linear model (GLM) is a regression model of the form

$$E {Y} = G(x^T\beta),$$

where $E {Y}$ denotes the expected value of the dependent variable $Y$, $x$ is a vector of explanatory variables, $\beta$ an unknown parameter vector and $G(\bullet)$ a known link function.

An essential feature of the GLM is that the expectation $\mu=EY$ is directly dependent on a function of the index $\eta=x^T\beta$. Additionally, one assumes that $\textrm{Var}(Y)=\sigma^2V(\mu)$. The function $G$ which relates $\mu$ and $\eta$ is called the link function. (Note that McCullagh and Nelder (1989) actually denote $G^{-1}$ as the link function.)

It is easy to see that GLM covers a range of widely used models, e.g.

• Linear regression (OLS)
  The model
  $$Y = x^T\beta + \varepsilon,\quad \varepsilon\sim N(0,\sigma^2)$$
  implies that
  $$E Y = x^T\beta, \quad \textrm{Var}(Y) = \sigma^2.$$
  Hence the classical linear model falls into the GLM framework with an identity link function and a variance function $V(\mu)=1$.

• Binary response models (Logit, Probit)
  The probability $P(Y=1)$ for Bernoulli distributed $Y$ is identical to the expectation $E {Y}$. Hence Logit or Probit models
  $$P(Y=1)= G(x^T\beta)=\mu,$$
  where $G(\bullet)$ is the Logistic or Gaussian distribution function, can be estimated within the GLM framework as well. The variance function is $V(\mu)=\mu(1-\mu)$ in this case.