In our NeurIPS paper, we leverage diversity stemming from models with different hyperparameters. This leads to SotA accuracy and more robust predictions. Hyper-deep ensembles expand on deep ensembles by integrating over a larger space of hyperparameters. Hyper-batch ensembles expand on efficient methods. Also check out the thread by Dustin Tran.