Uncertainty quantification is one of the most important challenges in machine learning, as it helps understand the models behavior and improves their performance in situations where the data is scarce. For instance, Large Language Models (LLMs) are prone to hallucinate, i.e., to generate text that is not coherent with the context in unseen situations. Self-driving cars fail to behave correctly in situations that are not covered by the training data. In these situations, it is essential to be able to detect when the model is uncertain about its prediction. The Bayesian approach to uncertainty quantification is a potential solution to this issue, but is often computationally expensive. At the same time, methods for reducing the cost of classical ensembling of neural networks have been developed. This project consists of developing a new method for Bayesian neural network ensembling that is computationally efficient and that can be used to quantify the uncertainty of the model under the Bayesian framework. The project includes prototyping a Bayesian neural network ensemble as in [1] and combining it with the optimizations made in [2]. Merging the methods could potentially make ensembling fast during both training and inference (whereas [2] only improves test-time inference). [1] [1810.05546] Uncertainty in Neural Networks: Approximately Bayesian Ensembling [2] [2002.06715] BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning