Next: 3.6 Estimation of Density
Up: 3. Markov Chain Monte
Previous: 3.4 The Gibbs Sampling
3.5 MCMC Sampling with Latent Variables
In designing MCMC simulations, it is sometimes helpful to modify the
target distribution by introducing latent variables or
auxiliary variables into the sampling. This idea was called
data augmentation by [58] in the context of missing data
problems.
Slice sampling, which we do not discuss in this chapter, is
a particular way of introducing auxiliary variables into the sampling,
for example see [20].
To fix notations, suppose that
denotes a vector of latent
variables and let the modified target distribution be
. If the latent variables are tactically
introduced, the conditional distribution of
(or sub
components of
given
may be easy to
derive. Then, a multiple-block M-H simulation is conducted with the
blocks
and
leading to the sample
where the draws on
, ignoring those on the latent data,
are from
, as required.
To demonstrate this technique in action, we return to the
probit regression example discussed in Sect. 3.3.2 to
show how a MCMC sampler can be developed with the help of latent
variables. The approach, introduced by [1], capitalizes
on the simplifications afforded by introducing latent or auxiliary
data into the sampling.
The model is rewritten as
This specification is equivalent to the probit regression model since
Now the Albert-Chib algorithm proceeds with the sampling of the full
conditional distributions
where both these distributions are tractable (i.e., requiring no M-H
steps). In particular, the distribution of
conditioned on the latent data becomes independent of the observed
data and has the same form as in the Gaussian linear regression model
with the response data given by and is multivariate normal
with mean
and variance matrix
. Next, the
distribution of the latent data conditioned on the data and the
parameters factor into a set of independent distributions with
each depending on the data through :
where the distribution
is the normal
distribution
truncated by the knowledge of
; if , then
and if , then
. Thus, one samples from
if and from
if , where
denotes
the
distribution truncated to the
region .
The results, based on MCMC draws beyond a burn-in of
a iterations, are reported in Fig. 3.4. The results
are close to those presented above, especially to the ones from the
tailored M-H chain.
Figure 3.6:
Caesarean data with Albert-Chib algorithm: Marginal posterior
densities (top panel) and autocorrelation plot (bottom
panel)
|
Next: 3.6 Estimation of Density
Up: 3. Markov Chain Monte
Previous: 3.4 The Gibbs Sampling