Journal of Convex Analysis 29 (2022), No. 2, 623--647
Copyright Heldermann Verlag 2022
Entropic Convex Duality in the Determination of Data-Constrained Kernel-Based Bayes-Jaynes Priors
Richard Le Blanc
FMSS, Université de Sherbrooke, Sherbrooke, Québec, Canada
richard.le.blanc@usherbrooke.ca
Dense datasets produced by high-throughput technologies allow for the objective determination of Bayesian priors when understood as solutions to Fredholm's integral equation. In order to select an appropriate methodology, we conducted a comprehensive review of alternative derivations for the Bayesian prior and posterior update rules comprising: classical iterative alternating algorithms, information theory-motivated variational approaches, and contemporary convex geometry methods. All of these share the key concepts of Kullback-Leibler (KL) relative entropy and of variational minimization. Since the KL relative entropy is a Bregman divergence allowing only linear exploration of tangent hyperplanes of the convex entropy functional, the resulting algorithms -- reexpressed in terms of alternating Bregman projections -- are necessarily iterative in nature. It has been recently argued that the prior can often only be understood in the context of the model likelihood/kernel, but iterative algorithms obscure the prior dependency on the kernel. In contrast, Jaynes maximal entropy principle yields data-constrained structural priors which make very explicit their kernel dependency, with the modelled density entropic convex dual -- the convex analysis equivalent of coefficients from a Fourier decomposition -- providing the model kernel weight function. Jaynesian-Bayesian modelling of dense datasets null hypothesis statistical testing (NHST) p-value distributions in terms of the Bayes factor BF(p) allows for a natural extension of the NHST framework. We illustrate all concepts in terms of newly derived analytical expressions for the Fisher-Student-Snedecor's t and F noncentral hypersphere distributions which afford informative representations on compact 2D spaces.
Keywords: Fredholm equation, Kullback-Leibler relative entropy, Bregman divergence, Bregman projection, mirror update, Bayes-Jaynes prior.
MSC: 45B05, 94A17, 49N15, 62C10, 60E05.
[ Fulltext-pdf (1783 KB)] for subscribers only.