7.1 Introduction

Generalized linear models (GLM) extend the concept of the well understood linear regression model. The linear model assumes that the conditional expectation of $Y$ (the dependent or response variable) is equal to a linear combination $\boldsymbol{X}^\top\boldsymbol{\beta}$, i.e.

$$E(Y\vert\boldsymbol{X}) = \boldsymbol{X}^\top\boldsymbol{\beta}.$$

This could be equivalently written as $Y = \boldsymbol{X}^\top\boldsymbol{\beta} +\varepsilon$. Unfortunately, the restriction to linearity cannot take into account a variety of practical situations. For example, a continuous distribution of the error $\varepsilon$ term implies that the response $Y$ must have a continuous distribution as well. Hence, the linear regression model may fail when dealing with binary $Y$ or with counts.

Example 1 (Bernoulli responses)  
Let us illustrate a binary response model (Bernoulli $Y$) using a sample on credit worthiness. For each individual in the sample we know if the granted loan has defaulted or not. The responses are coded as

$$Y=\left\{\begin{array}{ll} 1 & \quad \textrm{loan defaults},\\ [-1mm] 0 & \quad \textrm{otherwise}. \end{array}\right.$$

The term of interest is how credit worthiness depends on observable individual characteristics $\boldsymbol {X}$ (age, amount and duration of loan, employment, purpose of loan, etc.). Recall that for a Bernoulli variable $P(Y=1\vert\boldsymbol{X})=E(Y\vert\boldsymbol{X})$ holds. Hence, the default probability $P(Y=1\vert\boldsymbol{X})$ equals a regression of $Y$ on $\boldsymbol {X}$. A useful approach is the following logit model:

$$P(Y=1\vert\boldsymbol{X}=\boldsymbol{x})= \frac{1}% {1+\exp(-\boldsymbol{x}^\top \boldsymbol{\beta})}.$$

Here the function of interest $E(Y\vert\boldsymbol{X})$ is linked to a linear function of the explanatory variables by the logistic cumulative distribution function (cdf) $F(u)=1/(1+e^{-u})=e^u/(1+e^u)$.

The term generalized linear models (GLM) goes back to [29] and [27] who show that if the distribution of the dependent variable $Y$ is a member of the exponential family, then the class of models which connects the expectation of $Y$ to a linear combination of the variables $\boldsymbol{X}^\top\boldsymbol{\beta}$ can be treated in a unified way. In the following sections we denote the function which relates $\mu=E(Y\vert\boldsymbol{X})$ and $\eta=\boldsymbol{X}^\top \boldsymbol{\beta}$ by $\eta=G(\mu)$ or

$$E(Y\vert\boldsymbol{X}) = G^{-1}(\boldsymbol{X}^\top \boldsymbol{\beta}).$$

This function $G$ is called link function. For all considered distributions of $Y$ there exists at least one canonical link function and typically a set of frequently used link functions.