Improving the Data Augmentation Algorithm in the Two-Block Setup

Subhadip Pal, Kshitij Khare, James P. Hobert

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


The data augmentation (DA) approach to approximate sampling from an intractable probability density fX is based on the construction of a joint density, fX,Y , whose conditional densities, fX|Y and fY|X, can be straightforwardly sampled. However, many applications of the DA algorithm do not fall in this “single-block” setup. In these applications, X is partitioned into two components, X = (U,V), in such a way that it is easy to sample from fY|X, fU|V,Y , and fV|U,Y .We refer to this alternative version of DA, which is effectively a three-variable Gibbs sampler, as “two-block” DA. We develop two methods to improve the performance of the DA algorithm in the two-block setup. These methods are motivated by the Haar PX-DA algorithm, which has been developed in previous literature to improve the performance of the single-block DA algorithm. The Haar PX-DA algorithm, which adds a computationally inexpensive extra step in each iteration of the DA algorithm while preserving the stationary density, has been shown to be optimal among similar techniques. However, as we illustrate, the Haar PX-DA algorithm does not lead to the required stationary density fX in the two-block setup. Our methods incorporate suitable generalizations and modifications to this approach, and work in the two-block setup. A theoretical comparison of our methods to the two-block DAalgorithm, a much harder task than the single-block setup due to nonreversibility and structural complexities, is provided.We successfully apply our methods to applications of the two-block DA algorithm in Bayesian robit regression and Bayesian quantile regression. Supplementary materials for this article are available online.

Original languageEnglish
Pages (from-to)1114-1133
Number of pages20
JournalJournal of Computational and Graphical Statistics
Issue number4
Publication statusPublished - Oct 2 2015
Externally publishedYes


  • Data augmentation algorithm
  • Group action
  • Haar measure
  • Sandwich algorithm
  • Two-block DA algorithm

ASJC Scopus subject areas

  • Statistics and Probability
  • Discrete Mathematics and Combinatorics
  • Statistics, Probability and Uncertainty


Dive into the research topics of 'Improving the Data Augmentation Algorithm in the Two-Block Setup'. Together they form a unique fingerprint.

Cite this