ICML 2020: Bayesian Neural Networks and the Cold Posterior.

In our ICML 2020 paper, we tackle the question: How Good is the Bayes Posterior in Deep Neural Networks Really? We cast doubt on the current understanding of Bayes posteriors in popular deep neural networks. We find that using cold posteriors improves the performance of BNNs but sharply deviate from the Bayesian paradigm. We put forward several hypotheses that could explain cold posteriors and evaluate the hypotheses through experiments.