Across science, engineering, and finance, many of the most important risks come from low-likelihood, high-impact events. Estimating the probability of these events with brute-force Monte Carlo sampling—running a model repeatedly with randomly drawn inputs to estimate the probability of rare outcomes—can require an excessive volume of model iterations, especially when each sample comes from an expensive, physics-based model.
Guided diffusion models offer a new way to navigate towards rare events, but guidance introduces a critical challenge: If a model oversamples the tail of a distribution, how can we still estimate the true likelihood of those samples?
In a new paper, Towards accurate extreme event likelihoods from diffusion model climate emulators, we explore this problem in climate science by guiding a diffusion-based climate emulator toward tropical storms and computing odds ratios from guided and unguided probabilities. In this post we will illustrate a minimal working example using a tropical cyclone case.
There’s a ton more work to do: New methods deserve to be tried, performance optimizations are needed, and the impact could be unlocking radical cost efficiencies for next-generation risk analytics in science and economics.
