Generative models are becoming an increasingly important part of modern machine learning, with applications in image synthesis, scientific modeling, simulation, and data-driven engineering. At the same time, their training often requires substantial computational resources, and improvements in model quality are frequently achieved at the cost of significantly greater compute. This makes efficient use of training resources a central challenge. A key open question is whether training compute can be allocated more intelligently, so that limited computational budgets lead to greater learning progress.
This project addresses that question in the context of flow matching, a recent and influential framework for generative modeling based on learning continuous-time transformations between simple source distributions and complex data distributions. A distinctive feature of flow matching is that training signals are sampled along a continuous generative path, and existing work suggests that different regions of this path may not be equally informative for learning. This raises a fundamental problem: how should training effort be distributed across the generative process in order to improve efficiency, stability, and final model quality?
The goal of this project is to study information-aware compute allocation for generative model training through the concrete setting of stochastic interpolation paths in flow matching. The initial focus is on Brownian-bridge flow matching, where different regions of the path exhibit different learning characteristics, creating a nontrivial trade-off between informativeness and stability. The project will investigate how these differences evolve during training and whether adaptive, information-aware allocation strategies can exploit this structure to improve convergence and robustness under fixed training objectives.
The project combines empirical analysis with methodological development. It will characterize how training signals vary across the generative path, develop adaptive strategies for allocating training effort, and evaluate these strategies against standard fixed scheduling approaches on controlled benchmarks. Although the immediate scope is focused on a specific class of flow-matching models, the broader ambition is to contribute to a more general understanding of how training compute can be distributed effectively in modern generative learning.
If successful, the project is expected to have impact at two levels. First, it can provide practical methods for improving the efficiency and stability of flow-based generative model training under limited compute budgets. Second, it can contribute to a broader understanding of the relationship between information structure, optimization dynamics, and resource allocation in continuous-time generative modeling.
Main supervisor: Cristian R. Rojas
Affiliation: Department of Decision and Control Systems, KTH Royal Institute of Technology