19. Neural Networks

A neural network is a non-linear system that converts a series of real input values $ x_1, \ldots, x_p$ over several intermediary steps to one or more terminal variables $ y_1, \ldots, y_q$. It represents a function $ \nu: \mathbb{R}^p \rightarrow
\mathbb{R}^q$:

$\displaystyle (y_1, \ldots, y_q)^\top = \nu(x_1, \ldots, x_p) ,$

Fig. 18.1: Neural feed forward network with a hidden layer
\includegraphics[width=1.2\defpicwidth]{neu1.ps}

that has a special form given by the network structure. This is graphically displayed in Figure 18.1 in the form of a directed graph whose nodes are grouped in various levels. In the input layer each node represents an input variable; in the output layer each node represents an output variable. In between there are one or more hidden layers, whose nodes are neither sources nor layers of the graph. The network in Figure 18.1 contains only one hidden layer. In additional it is a feed forward network, since it contains no edges that begin in a node and end in the same node or in a different node from the same or a previous layer.

Fig. 18.2: Neural feedback network with two hidden layers
\includegraphics[width=1.3\defpicwidth]{neu2.ps}

Figure 18.2 displays a feedback network, in that there is feedback among the nodes of the two hidden layers. In the following we will concentrate on the feed forward network.

Neural network are used in financial statistics to represent functions, which, for example, can represent the default probability of a credit, the forecast of an exchange rate or the volatility of a stock. Here the emphasis is on non-parametric applications, which in comparison to the local smoothing function discussed in Chapter 13 require an advanced modelling and can be quite involved to calculate. On the other hand it is still practical when numerous variables need to be considered in forecasts or quantifying risk, i.e., when the dimension $ p$ of the function arguments is large.

Since neural networks are still relatively unknown tools in statistics, in the first section we will give an elementary introduction in the structure of a neural network. It allows for the construction of complex functions using simple elements. In the second section we describe the popular numerical application for fitting neural networks to the data, before we conclude with various applications to financial problems and introduce the underlying assumptions.