NAISS
SUPR
NAISS Projects
SUPR
Revisit diffusion samplers: a unified framework and beyond
Dnr:

NAISS 2026/4-1647

Type:

NAISS Small

Principal Investigator:

Shuangshuang Chen

Affiliation:

Kungliga Tekniska högskolan

Start Date:

2026-09-25

End Date:

2027-10-01

Primary Classification:

10210: Artificial Intelligence

Webpage:

Allocation

Abstract

Sampling from, and estimating the normalizing constant of, a probability distribution known only up to proportionality is a fundamental problem in probabilistic inference, with applications spanning molecular biology, robotics, drug discovery, and cosmology. Classical approaches — Markov Chain Monte Carlo, Sequential Monte Carlo, and variational inference — trade off accuracy, computational cost, and asymptotic guarantees. Inspired by the success of diffusion- and score-based generative models, a rich family of diffusion samplers has recently emerged (DDS, PIS, DIS, MCD, CMCD, LDVI, GBS, and others), which formulate inference as learning controlled stochastic differential equations that bridge a tractable prior and the target distribution. Because these methods were developed independently, in different notations and with different motivating derivations, systematic comparison between them — and a precise understanding of how they relate — has remained difficult. This project develops and empirically validates a unified theoretical framework for diffusion samplers grounded in the forward-backward Radon–Nikodym derivative (FBRND) of the controlled forward and backward path measures. We derive three distinct but theoretically equivalent FBRND decompositions (FBRND-GBS, FBRND-DDS, FBRND-path), which weight terminal, running, and stochastic training costs differently, and relate them to a reference-process formulation (FBRND-Ref) that recovers the Schrödinger bridge perspective. This framework maps every existing diffusion sampler onto a (FBRND decomposition, controlled SDE process) pair, revealing that these two design choices — the training objective and the stochastic process — can be fully decoupled. This decoupling immediately expands the design space beyond the specific combinations proposed in prior work. The central empirical question this project addresses is whether objectives that are equivalent in continuous time remain equivalent once instantiated with realistic discrete-time integration, finite training budgets, and specific SDE parameterizations. Answering this requires training and evaluating a large combinatorial grid: multiple sampler algorithms, each crossed with several theoretically-equivalent-but-practically-distinct objective/evaluation variants (divergence, divergence-free, ratio-based), under systematic sweeps over diffusion-process hyperparameters (maximum diffusion rate, noise schedule, number of discretization steps, step size), across a diverse suite of benchmark targets spanning low-dimensional multimodal densities (Gaussian mixtures, Funnel), Bayesian posterior inference tasks (logistic regression on Sonar, Ionosphere, Credit, Cancer, Seeds), spatial statistics (log-Gaussian Cox process, dim 1600), dynamical systems (Brownian motion, Lorenz), and higher-dimensional generative tasks (NICE normalizing flows on MNIST/Fashion-MNIST). Each configuration additionally requires multiple random seeds to obtain statistically reliable estimates of sample quality (MMD, Sinkhorn distance) and log-normalizing-constant accuracy (ELBO/EUBO bounds). The outcome will be (i) a peer-reviewed paper establishing the FBRND framework and its practical implications for the design of diffusion-based samplers, and (ii) an open-source codebase and results archive that the community can use to compare and design new samplers within the unified design space, rather than treating each method as a bespoke algorithm. The computational resource requested below is needed specifically to run the large but individually inexpensive training jobs that this systematic empirical validation requires.