18.3 Robustifying Least Squares: the rLS filter


{filtX, KG, PreviousPs, clipInd} = 31370 rlsfil (y, mu, Sig, H, F, Q, R, bs, b)
calculates the rLS filter for a (multivariate) time series
{b, ctrl} = 31373 calibrLS (T, Sig, H, F, Q, R, e, N, eps, itmax, aus)
calibrates the rLS filter to a given efficiency loss in the ideal model.


18.3.1 Derivation

Instead of $ M\Delta y$ we use a huberized version of it, that is

$\displaystyle H_b(M\Delta y)=M\Delta y \min\{1, \frac{b}{\Vert M\Delta y\Vert}\},$ (18.4)

so that the correction is just as in the Kalman filter if it is not too large in absolute value, and if it is too large, the direction remains unchanged and it is projected to the ball with radius $ b$ in $ n$ dimensions.

This gives us

$\displaystyle x_{0\vert}$ $\displaystyle =$ $\displaystyle \mu,$ (18.5)
$\displaystyle x_{t\vert t-1}$ $\displaystyle =$ $\displaystyle F x_{t-1\vert t-1}$ (18.6)
$\displaystyle x_{t\vert t}$ $\displaystyle =$ $\displaystyle x_{t\vert t-1} + H_b(\hat M_t (y_t-Hx_{t\vert t-1})),$ (18.7)

where we have not yet specified how to choose $ M$ and $ b$, but (c.f. Ruckdeschel (1999)) We already note at this point that the rLS has preserved the crucial features from of the Kalman filter So in the implementation, we use $ \hat M$.


18.3.2 Calibration

As to the choice of $ b$ we propose an assurance criterion: How much efficiency in the ideal model relative to the optimal procedure ,i.e. the Kalman filter, am I ready to pay in order to get robustness under deviations from the ideal model? This efficiency loss is quantified as the relative degradation in MSE in the ideal model one obtains if one uses robust versions instead of the classical Kalman filter.

Of course the lower the clipping $ b$, the higher the relative efficiency loss, so due to this monoticity we may solve

$\displaystyle \mathop{\rm {{}E{}}}\nolimits \Vert\Delta x-H_b(\hat M \Delta y)\Vert^2 \stackrel{!}{=}(1+\delta) \mathop{\hbox{tr}}\Sigma_{t\vert t}$ (18.8)

in $ b$, uniquely for a given efficiency loss $ \delta$.

We note here that


18.3.3 Examples

All examples stem--at least for the state space model specification--from Petr Franek (see Härdle, Klinke, and Müller; 2000, Chapter 10). They have been slightly modified to better demonstrate the features of our methods.


18.3.3.1 Example 5

As first example we have taken Petr Franek's data kalman.dat together with his specifications for the system matrices and by manipulating observations $ 50$, $ 60$, $ 90$ entered three artificial outliers; the result is saved in kalmanao.dat .

  library("xplore")
  library("plot")
  library("kalman")

  serie = read("kalmanao.dat")
  y = serie[,2]
  mu = 10
  Sig = 0
  H = 1
  F = 1
  Q = 9
  R = 9
  T=dim(y)

  e=0.05
  N=100
  eps=0.01
  itmax=15
  aus=4
The rLS filter is then calibrated to an efficiency loss of $ 5\%$
  ergLS=calibrLS(T,Sig,H,F,Q,R,e,N,eps,itmax,aus)
  b=ergLS.b
Next we filter the data with the Kalman filter and the rLS filter.
  res = kfilter2(y,mu,Sig, H,F,Q,R)
  fx = res.filtX
  res= rlsfil(y,mu,Sig, H,F,Q,R,b)
  frx = res.filtX
The results are then displayed, the classical Kalman filter in red, the rLS filter in green and the observations from kalmanao.dat in blue. Additionally we set flags on the time instances where the rLS filter clips $ \hat M_t\Delta y_t$.
  origy= serie[,1]~serie[,2]
  origy= setmask(origy, "line", "blue", "thin")
  fx = serie[,1]~fx
  fx = setmask(fx, "line", "red", "thin")
  frx = serie[,1]~frx
  frx = setmask(frx, "line", "green", "thin")
  clip=serie[,1]~(res.clipInd)
  clip=paf(clip,clip[,2]==1)
  clip[,2]=0
  setmaskp(clip,4, 3, 4)  

  disp = createdisplay(1,1)
  show(disp,1,1, origy,fx,frx,clip)
  setgopt(disp,1,1, "title", "KalmanData1 + AO's in t=50,60,90")
  setgopt(disp,1,1, "xlabel", "t") 
  setgopt(disp,1,1, "ylabel", "y, rLS, Kalman")
31635 XAGrkalm04.xpl

The graphical result of this example is displayed in Figure 18.3.

Figure 18.3: Actual observations $ \hat=$ solid line, classical Kalman filter $ \hat=$ dashed and a bit lighter line, the rLS filter $ \hat=$ dotted line; clipped instances $ \hat=$ circles on top of the graphic.
\includegraphics[scale=0.6]{rlsbsp1}


18.3.3.2 Example 5

As a second example we took the state-space model underlying Petr Franek's Example 2.

  library("xplore")
  library("plot")
  library("kalman")

  mu = #(20,0)
  Sig = #(0,0)~#(0,0)
  H  = #(0.3,-0.3)~#(1,1)
  F  = #(1,0)~#(1,0)
  R  = #(0,0)~#(0,9)
  mid=#(0,0)
  Qid= #(9,0)~#(0,9)
According to this model we simulated $ 50$ states and observations from a $ 10\%$-AO-contamination using quantlets 31643 epscontnorm and 31646 kemitor2 . The contaminating distribution is $ {N}_2\left(\left(\!\!\begin{array}{c} 25\\ 30\end{array}\!\!\right),\left(\!\!\begin{array}{cc} 0.9& 0\\ 0&0.9 \end{array}\!\!\right)\right)$.
  T=50
  e=0.1
           
  mcont=#(25,30)
  Qcont=0.1.*Qid
  AOr=0.1
  randomize(0)
  ErrX = epscontnorm(T,0,mid,R,mcont,Qcont,0) 
  ErrY = epscontnorm(T,AOr,mid,Qid,mcont,Qcont,0)
  sim = kemitor2(T,mu,H,F,(ErrY.X),(ErrX.X))
  y=sim.Y        
  Xact=sim.X
The rLS filter is then calibrated to an efficiency loss of $ 10\%$.
  N=10000
  eps=0.01
  itmax=15
  aus=4

  ergLS=calibrLS(T,Sig,H,F,Qid,R,e,N,eps,itmax,aus)

  b=ergLS.b
The simulated data are filtered by the classical Kalman filter and the rLS filter
  res = kfilter2(y,mu,Sig, H,F,Qid,R)
  fx = res.filtX
  res= rlsfil(y,mu,Sig, H,F,Qid,R,b)
  frx = res.filtX
Next, the filtered values are prepared for graphical output: the classical Kalman filter is to be displayed in red, the rLS filter in green and the actual simulated states in blue.
  i=(1:T)
  Xact1 = i~(Xact[,1])
  Xact1 = setmask(Xact1, "line", "blue", "thin")
  fx1 = i~(fx[,1])
  fx1 = setmask(fx1, "line", "red", "thin")
  frx1= i~(frx[,1])
  frx1 = setmask(frx1, "line", "green", "thin")
Additionally we set green flags on the time instances where the rLS filter clips $ \hat M_t\Delta y_t$ on bottom of the graphics and red ones on top where an AO-outlier occurred.
  ym1=max(vec((Xact[,1])~(fx[,1])~(frx[,1]))) ;top of graphics
  ym2=min(vec((Xact[,1])~(fx[,1])~(frx[,1]))) ;bottom of graphics
  flagInd=i~(ErrY.Ind)
  flagInd=paf(flagInd,flagInd[,2]==1)
  flagInd[,2]=ym1*((ym1>0)*1.1+(ym1<0)*0.9)
  flagclip=i~(res.clipInd)
  flagclip=paf(flagclip,flagclip[,2]==1)
  flagclip[,2]= ym2*((ym2<0)*1.1+(ym2>0)*0.9)
  setmaskp(flagInd,4, 3, 4)  
  setmaskp(flagclip,2, 4, 4)
Finally, all the results of the 1-st coordinate are displayed in the left half of a graphic window.
  disp = createdisplay(1,2)
  show(disp,1,1,Xact1,fx1,frx1,flagInd,flagclip)
  setgopt(disp,1,1, "title", "simulated Model under AO 
                                                 -- 1st coord.")
  setgopt(disp,1,1, "xlabel", "t") 
  setgopt(disp,1,1, "ylabel", "x, rLS, Kalman")
31650 XAGrkalm05.xpl

Then the same is done for the second coordinate, replacing [,1] by [,2] on all instances, and the results are then plotted on the right half of the graphic-window. All the program may be seen in 31655 XAGrkalm05.xpl .

Figure 18.4: Simulated data according to model from Example 2: actual states $ \hat=$ solid line, the classical Kalman filter $ \hat=$ dashed / a bit lighter, the rLS filter $ \hat=$ dotted line; clipped instances are marked by circles on bottom of the graphic, the AO-instances by circles on top.
\includegraphics[scale=0.55]{rlsbsp2}


18.3.3.3 Example 6

To show that robustification does not go for free, we have taken Petr Franek's data set kalman2.dat as it was, together with his specification of the state space model. So the data stems from the ideal situation and the classical Kalman filter in this case should do better than the rLS.

  library("xplore")
  library("plot")
  library("kalman")

  serie = read("kalman2.dat")
  y = serie[,2]
  mu = #(0,0)
  Sig = #(0,0)~#(0,0)
  H = #(1,0)'
  F = #(0.5,1)~#(-0.3,0)
  R = #(1,0)~#(0,0)
  Q = 4

  e=0.05
  N=10000
  eps=0.01
  itmax=15
  aus=4
Here the rLS filter is calibrated to an efficiency loss of $ 5\%$.
  ergLS=calibrLS(T,Sig,H,F,Q,R,e,N,eps,itmax,aus)
  b=ergLS.b

  res= kfilter2(y,mu,Sig, H,F,Q,R)
  fx = res.filtX
  fy=(H*fx')'
  res= rlsfil(y,mu,Sig, H,F,Q,R,b)
  frx = res.filtX
  fry=(H*frx')'
As we do not have any information about the actual states in this example, we examine the filtered observations $ y_{t\vert t}$ generated by the Kalman filter and the rLS filter and compare them to the actual observations $ y_t$. The filtered observations are simply calculated as $ y_{t\vert t}=Hx_{t\vert t}$.

These filtered observations are then displayed, as usual.

  origy = serie[,1]~serie[,2]
  origy = setmask(origy, "line", "blue", "thin")

  fy = serie[,1]~fy
  fy = setmask(fy, "line", "red", "thin")
  fry = serie[,1]~fry
  fry = setmask(fry, "line", "green", "thin")
  flags = serie[,1]~(res.clipInd)
  flags=paf(flags,flags[,2]==1)
  setmaskp(flags,4, 3, 4)
  
  disp = createdisplay(1,1)
  show(disp,1,1,origy,fy,fry,flags)
  setgopt(disp,1,1, "title", "KalmanData2 in Observation Space")
  setgopt(disp,1,1, "xlabel", "t") 
  setgopt(disp,1,1, "ylabel", "y, y-rLS, y-Kalman")
31664 XAGrkalm06.xpl

The graphical result of this example is displayed in Figure 18.5.

Figure 18.5: actual observations $ \hat=$ solid line, the classical Kalman filter dashed and a bit lighter line, the rLS filter $ \hat=$ dotted line; the clipped instances are marked by circles.
\includegraphics[scale=0.6]{rlsbsp3}


18.3.4 Possible Extensions

Alternatively to using 31783 rlsbnorm and 31786 rlsbnorm1 although the assumption of normality cannot be exactly met, one could use

Simulation of a bundle of paths and then MC-Integration:

One can quite easily replace the routines 31789 rlsbnorm and 31792 rlsbnorm1 by MC-Routines calculating $ \mathop{\rm {{}E{}}}\nolimits \Vert\Delta x-H_b(\hat M \Delta y)\Vert^2$ ob basis of a large sample of $ (\Delta y_t, \Delta x_t)$, that is evolving according to recursions (18.2), simultaneously to $ x_{t\vert t}$.
Numerical Integration:

Even keeping the normality assumption, in order to speed up the calibration step, one could try and evaluate $ \mathop{\rm {{}E{}}}\nolimits \Vert\Delta x-H_b(\hat M \Delta y)\Vert^2$ for low dimension $ n$ by numerical integration routines instead of using MC-integration.