A Journey from Posterior to Prior and the Aftermath
Kai W. Ng
Deaprtment of Statistics and Actuarial Science, University of Hong Kong
It all started with an integral equation, for which Tanner and Wong proposed to solve by successive substitution, called Data Augmentation Algorithm, in their 1987 invited paper in the Journal of American Statistical Association, with discussions by five prominent experts (two from Harvard University). The first author of the paper later gave further details and applications of the Algorithm in a volume of Springer Series in Statistics, Tools for Statistical Inference (3rd Printing of 1st Ed. in 1993 and 3rd Ed. in 1996). Because the sufficient conditions ensuring the convergence of the successive substitution were almost not verifiable in practice, I intended to get a plainer set of sufficient conditions for easier verification. Then certain thoughts struck me and changed my direction: solving the integral equation was equivalent to obtaining the prior pdf (probability density function) given the posterior pdf and the likelihood function. Going back to basics, I obtained the explicit solution in 1995 and coined it the Inverse Bayes Formula (IBF), because it went in opposite direction of the Bayes’ Formula. Under the positivity assumption for the integral equation, i.e. all functions are positive in the product space, the explicit solution has two equivalent forms, point-wise and function-wise. The point-wise form reveals the fundamental nature of the prior pdf as follows: With respect to repeated sampling from the data distribution conditional on a parameter value, the harmonic mean of the posterior density at that parameter value equals the prior density at the same parameter value. On the other hand, the function-wise formula facilitates easier numerical and sampling implementation and still applies to certain non-product supports. For general support spaces with haphazard positivity patterns, however, the inversion of Bayes’ formula (also IBF, but in a slightly different meaning) would be in the form of algorithms instead of nice and neat formulae, which had been briefly reported in the 1996 Sidney International Statistical Congress. I shall share with audience in the talk this unexpected journey of reversing the Bayesian inference and its interesting aftermath.
Date received: March 29, 2010
Copyright © 2010 by the author(s). The author(s) of this work and the organizers of the conference have granted their consent to include this abstract in Topology Atlas. Document # cbak-90.