NAISS
SUPR
NAISS Projects
SUPR
Continual Self-Supervised Learning with Diffusion Models and Flow Matching on Resource-Constrained Edge and IoTDevices
Dnr:

NAISS 2026/4-1792

Type:

NAISS Small

Principal Investigator:

Shuai Zhu

Affiliation:

Uppsala universitet

Start Date:

2026-10-05

End Date:

2027-11-01

Primary Classification:

10210: Artificial Intelligence

Allocation

Abstract

Building on our previous work on diffusion-based continual self-supervised learning, this follow-up project aims to develop more efficient and deployable adaptive learning methods for resource-constrained IoT devices. Our earlier research showed that generative models can support continual self-supervised learning by jointly providing representations from unlabeled data and synthetic replay for preserving previously acquired knowledge. This project extends that foundation toward more efficient generative learning and substantially smaller hardware platforms. The main research objective is to investigate flow matching as an alternative generative framework for continual self-supervised learning. Flow matching has the potential to reduce the computational cost of training and generation while retaining strong generative and representation-learning capabilities. We will study how it can be used for both representation learning and replay, and how these functions interact under class-incremental learning, domain shifts, and other forms of non-stationary data. A central focus will be resource efficiency. The project will investigate lightweight model architectures, compact generative models, efficient replay mechanisms, and implementation strategies that reduce memory use and computational demand. The aim is to enable not only inference, but also local training, adaptation, and synthetic data generation on constrained IoT hardware. This is essential for systems that must operate for long periods with limited connectivity and without continuous access to cloud-scale resources. The long-term goal is to develop learning systems that can adapt after deployment with little or no manually labeled data. Such systems should be able to maintain useful performance as data distributions change, sensing conditions evolve, and previously unseen classes appear. Combining self-supervised representation learning with continual learning offers a natural foundation for this setting, but practical deployment requires methods that are both robust and computationally efficient. The proposed methods will be evaluated on vision and sensor-based tasks under realistic continual-learning scenarios. Evaluation will consider representation quality, forgetting, adaptation to new data, generative quality, memory footprint, computational cost, and on-device feasibility. Particular attention will be given to the relationship between generative quality and continual representation learning, and to the extent to which model size and computational constraints affect long-term adaptation. The expected outcome is a more efficient framework for continual self-supervised learning that extends beyond diffusion-based methods and is suitable for highly constrained IoT platforms. The project will provide both scientific insight into the role of modern generative models in continual learning and practical methods for autonomous, long-term adaptation at the edge.