AI-driven medical image analysis has advanced considerably but still faces challenges in diagnostic accuracy, disease localization, confidence calibration, generalization, visual understanding, and explainability. Most existing approaches rely primarily on a single data modality, while current vision-language models (VLMs) still have limitations in effectively integrating heterogeneous multimodal information and exploiting complementary or privileged information. In addition, many medical imaging models depend on fully supervised learning and require extensive pixel-level annotations, which are expensive and time-consuming to obtain. This project investigates computationally intensive multimodal learning approaches that can better integrate imaging and textual information while reducing dependence on detailed manual annotations. Main supervisor: Ida Häggström, Chalmers