NAISS
SUPR
NAISS Projects
SUPR
Multimodal and Generative learning with applications in biomedicine
Dnr:

NAISS 2025/5-662

Type:

NAISS Medium Compute

Principal Investigator:

Paolo Soda

Affiliation:

UmeƄ universitet

Start Date:

2025-12-01

End Date:

2026-12-01

Primary Classification:

20603: Medical Imaging

Secondary Classification:

10210: Artificial Intelligence

Allocation

Abstract

In the fourth industrial revolution we are witnessing a fast and widespread adoption of artificial intelligence (AI) in our life, healthcare included. Advancements in deep learning (DL) should make significant contributions in this area supporting diagnosis, prognosis and treatment decisions. Most of the DL models consider only unimodal data, neglecting information available in other modalities of patient digital phenotypes, e.g. clinical data, CT, MRI images, etc.. As interpreting medical findings is multimodal by its very nature, AI needs to be able to interpret different modalities together to progress towards higher informative clinical decision making. In this respect we plan to investigate multimodal deep learning, an area of great interest that studies how deep neural networks (DNNs) can learn shared representations between different modalities. Open issues are when to fuse the different modalities and how to embed in the training any process able to learn more powerful data representations. We plan also to search for an optimal MDL fusion architecture, robust to missing modalities or missing data, studying multimodal regularization to improve stability, algorithmic speed-up and to reduce overfitting. We would like to consider approaches mitigating training from scratch, even when datasets of reduced size are available as it happens in healthcare, such as the use of generative approaches, GANs included. Furthermore, Generative AI is another core of research, focusing on inter- and intra-modality generation, as well as on longitudinal generation. Key paradigms are GANs, already mentioned, and latent space manipulation techniques, such as latent diffusion models. This topic is also relevant with reference to (cyber)security issues, such as synthetic injection and removal of information in medical imaging, adversarial attacks and anonymization. We are aware that a key impediment to the use of DL-based systems in practice is their black-box nature that does not permit to directly explain the decisions taken: this is why we embed Explainable AI (XAI) in our investigation. This project aims to deploy such methodologies in different areas of healthcare, to prove how general they are and to provide experts domain useful insights into the data. In this respect, specific fields of experimentation concern: i) cancer research, where we are looking for quantitative signature from multi-omics data able to predict the prognosis and to select the right personalized therapy in non-small cell lung cancer, ii) neurodegenerative diseases, iii) wellbeing and time series data, iii) atherosclerosis research where, using data available within the VIPVIZA study.