Figure 2: MCMC runtimes for the All Scalar and All Blocked algorithms, for univariate normal, univariate gamma, and multivariate normal model structures. We have examined both the algorithmic and computational efficiency of MCMC algorithms, each of which fundamentally affect overall MCMC efficiency. We define the overall efficiency of Ψ simply as E (Ψ) = A (Ψ) /C (Ψ). We consider this to be a sensible measure of the overall efficiency of an MCMC algorithm, since E (Ψ) may be interpreted as the number of effective samples produced per unit of MCMC algorithm runtime, for the slowest mixing model parameter. If we can construct Ψ to maximize E (Ψ), then Ψ is the most time-efficient MCMC algorithm for generating effectively independent samples to approximate the true, joint posterior distribution of Θ. That being said, from our examination of algorithmic and computational efficiency, it is not immediately clear how to balance tradeoffs between A (Ψ) and C (Ψ) to maximize E (Ψ). We have generally considered the two boundary-case algorithms Ψ$_{scalar}$ and Ψ$_{block}$, but a huge number of intermediate algorithms exist. For Ψ ∈ Ψ$_{M}$ , we may gain useful insights regarding the factors affecting E (Ψ) in terms of the properties of each ψ$_{i}$ . C (Ψ) = ∑ k i $_{=1}$C ( ψ$_{i}$ ), and the values A (Ψ ,θ$_{i}$ ) which determine A (Ψ) each result from a single application of scalar or block sampling – although 13 2.4 Overall Efficiency 0 5 10 15 0 100 200 300 400 500 Model dimension (d) Runtime (seconds per 10,000 MCMC samples) Model Structure Nor mal Gamma MV Nor mal MCMC Algorithm All Scalar All Block ed