2.4 Results

To judge the effectiveness of a Value-at-Risk model, it is common to use backtesting. A simple approach is to compare the predicted and empirical number of outliers, where the actual loss exceeds the VaR. We implement this test in a two risk factor model using real life time series, the FX rates USD/EUR and GBP/EUR, respectively their DEM counterparts before the introduction of the Euro. Our backtesting investigation is based on a time series ranging from 2 Jan. 1991 until 9 Mar. 2000 and simple linear portfolios $ i=1,\dots,4$:

$\displaystyle \textrm{Value}(a_i,t)[EUR] = a_{i,1} \times \textrm{USD}_t - a_{i,2} \times \textrm{GBP}_t \; .$ (2.17)

The Value-at-Risk is computed with confidence level $ 1-\alpha_i$ ( $ \alpha_1=0.1$, $ \alpha_2=0.05$, and $ \alpha_3=0.01$) based on a time series for the statistical estimators of length $ T=250$ business days. The actual next day value change of the portfolio is compared to the VaR estimate. If the portfolio loss exceeds the VaR estimate, an outlier has occurred. This procedure is repeated for each day in the time series.

The prediction error as the absolute difference of the relative number of outliers $ \hat \alpha$ to the predicted number $ \alpha$ is averaged over different portfolios and confidence levels. The average over the portfolios ( $ a_1=(-3,-2)$ $ a_2=(+3,-2)$ $ a_3=(-3,+2)$ $ a_4=(+3,+2)$) uses equal weights, while the average over the confidence levels $ i$ emphasizes the tails by a weighting scheme $ w_i$ ($ w_1=1$, $ w_2=5$, $ w_3=10$). Based on the result, an overall error and a relative ranking of the different methods is obtained (see Table 2.2).

As benchmark methods for Value-at-Risk we use the variance-covariance (vcv) method and historical simulation (his), for details see Deutsch and Eller (1999). The variance covariance method is an analytical method which uses a multivariate normal distribution. The historical simulation method not only includes the empirical copula, but also empirical marginal distributions. For the copula VaR methods, the margins are assumed to be normal, the only difference between the copula VaR's is due to different dependence structures (see Table 2.1). Mainly as a consequence of non-normal margins, the historical simulation has the best backtest results. However, even assuming normal margins, certain copulas (5, 12-14) give better backtest results than the traditional variance-covariance method.


Table: Relative number of backtest outliers $ \hat \alpha$ for the VaR with confidence $ 1-\alpha$, weighted average error $ \vert\hat\alpha-\alpha\vert$ and error ranking. 6870 XFGaccvar4.xpl
Copula as in Table 2.1
$ \alpha\!\!=$ $ a\!\!=$ his vcv 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 21
.10 $ a_1$ .103 .084 .111 .074 .100 .086 .080 .086 .129 .101 .128 .129 .249 .090 .087 .084 .073 .104 .080
.05 $ a_1$ .053 .045 .066 .037 .059 .041 .044 .040 .079 .062 .076 .079 .171 .052 .051 .046 .038 .061 .041
.01 $ a_1$ .015 .019 .027 .013 .027 .017 .020 .016 .032 .027 .033 .034 .075 .020 .022 .018 .015 .027 .018
.10 $ a_2$ .092 .078 .066 .064 .057 .076 .086 .062 .031 .049 .031 .031 .011 .086 .080 .092 .085 .065 .070
.05 $ a_2$ .052 .044 .045 .023 .033 .041 .049 .031 .012 .024 .012 .013 .003 .051 .046 .054 .049 .039 .032
.01 $ a_2$ .010 .011 .016 .002 .007 .008 .009 .006 .002 .002 .002 .002 .001 .015 .010 .018 .025 .011 .005
.10 $ a_3$ .099 .086 .126 .086 .064 .088 .096 .073 .032 .054 .033 .031 .016 .094 .086 .105 .133 .070 .086
.05 $ a_3$ .045 .048 .093 .047 .032 .052 .050 .040 .017 .026 .017 .016 .009 .049 .047 .058 .101 .034 .050
.01 $ a_3$ .009 .018 .069 .018 .012 .018 .016 .012 .007 .009 .006 .006 .002 .018 .015 .018 .073 .013 .020
.10 $ a_4$ .103 .090 .174 .147 .094 .095 .086 .103 .127 .094 .129 .127 .257 .085 .085 .085 .136 .088 .111
.05 $ a_4$ .052 .058 .139 .131 .056 .060 .058 .071 .084 .068 .084 .085 .228 .053 .054 .051 .114 .053 .098
.01 $ a_4$ .011 .020 .098 .108 .017 .025 .025 .035 .042 .056 .041 .042 .176 .016 .017 .016 .087 .015 .071
.10 Avg .014 .062 .145 .123 .085 .055 .052 .082 .193 .104 .194 .194 .478 .045 .061 .045 .110 .082 .075
.05 Avg .011 .021 .154 .124 .051 .030 .016 .060 .134 .080 .132 .136 .387 .006 .012 .017 .127 .041 .075
.01 Avg .007 .029 .169 .117 .028 .031 .032 .036 .065 .071 .065 .067 .249 .029 .025 .029 .160 .026 .083
Avg Avg .009 .028 .163 .120 .039 .032 .028 .047 .095 .076 .094 .096 .306 .022 .023 .026 .147 .034 .080
Rank 1 6 18 16 9 7 5 10 14 11 13 15 19 2 3 4 17 8 12