Using Autodiff to Estimate Posterior Moments, Marginals and Samples: Methods

cover
15 Apr 2024

This paper is available on arxiv under CC 4.0 license.

Authors:

(1) Sam Bowyer, Equal contribution, Department of Mathematics and sam.bowyer@bristol.ac.uk;

(2) Thomas Heap, Equal contribution, Department of Computer Science University of Bristol and thomas.heap@bristol.ac.uk;

(3) Laurence Aitchison, Department of Computer Science University of Bristol and laurence.aitchison@bristol.ac.uk.

Methods

Of course, the contributions of this paper are not in computing the unbiased marginal likelihood estimator, which previously has been used in learning general probabilistic models, but instead our major contribution is a novel approach to computing key quantities of interest in Bayesian computation by applying the source term trick to the massively parallel marginal likelihood estimator. In particular, in the following sections, we outline in turn how to compute posterior expectations, marginals and samples.

Figure 1: Results obtained in the MovieLens model. Columns a–c show the evidence lower bound, predictive log-likelihood and variance in the estimator of zm using the true MovieLens100K data. Column d shows the mean squared error in the estimator of zm when the data is sampled from the model and thus the true value of zm is known. The error bars in the top row represent the standard-deviation across different dataset splits.