This research project aims to advance the trustworthiness and reliability of large-scale machine learning (ML) models, such as Large Language Models (LLMs) and vision-language models, through basic research in machine learning and the development of scalable and effective probabilistic methods for uncertainty quantification and reduction. In safety-critical domains, such as healthcare diagnostics or agentic systems, ML models must be able to make decisions under uncertainty and accurately capture and reflect their inherent uncertainties. Unfortunately, contemporary machine learning models are often overconfident in their predictions and lack the ability to "know when they don't know" [1]. Moreover, uncertainty quantification in contemporary machine learning models faces additional challenges, as these models often contain millions or billions of parameters or are pre-trained on proprietary data sets. This resource project accompanies the compute project NAISS 2025/5-586, which will provide the necessary compute resources to execute the research. Achieving the goals of the project (outlined in detail in the NAISS 2025/5-586 application), significant computational and storage resources will be needed. The research will be conducted by the PI, 1-2 PhD students, one postdoctoral researcher, and 1-2 research assistants.