Event Details

MARS (Mathematics for AI in Real-world Systems) seminar series. Speaker: Christopher Nemeth (Lancaster). Title: Optimisation on the Space of Probability Measures

Abstract: Many modern sampling methods can instead be viewed as optimisation procedures over a space of probability measures endowed with an appropriate geometry. In this talk I will begin by developing this lens via Wasserstein gradient flows: the Fokker–Planck dynamics for the overdamped Langevin diffusion which arises as the Wasserstein-2 gradient flow of the Kullback–Leibler functional, and the unadjusted Langevin algorithm (ULA) is precisely a forward-flow time-discretisation of that flow. In parallel, particle variational methods such as Stein variational gradient descent (SVGD) can be read as explicit-Euler updates for the gradient flow of KL under a kernelised (Stein) Wasserstein metric. This unifying viewpoint clarifies why these algorithms decrease appropriate objective functionals and what structural assumptions (e.g., geodesic convexity) ensure their convergence.

The second part of the talk presents new work on how to practically implement these algorithms without the hassle of manually tuning the discretisation parameter. We introduce Fuse—a Functional Upper-bound Step-size Estimator—which yields adaptive, step-size-free discretisations of Wasserstein gradient flows. Fuse is a general approach which can be used practically to create tuning-free variants of algorithms such as ULA, SGLD, mean-field Langevin dynamics, SVGD, and variational gradient descent, to name a few. The resulting procedures retain the performance of optimally tuned baselines—provably up to logarithmic factors—under mild conditions such as geodesic convexity and locally bounded (stochastic) gradients.

I will cover the derivation of Fuse from functional inequalities on the Wasserstein space, non-asymptotic guarantees for both forward-flow and forward-Euler discretisations, and empirical results spanning target sampling and mean-field neural network training that match (or surpass) the best tuned alternatives—without manual learning-rate selection.