Decision-Making in Stochastic Environments Using Diffusion Models

Abstract

Sequential decision‐making under uncertainty is a difficult task in many real-world applications, and standard optimization methods often fail to capture complex stochastic dynamics, leading to suboptimal control. This thesis investigates the integration of diffusion-based probabilistic forecasting in Model Predictive Control (MPC) to enhance decision-making in partially observable, stochastic systems.

In this Thesis, we develop Diffusion‐Informed Model Predictive Control (D-I MPC), a unified framework that integrates powerful diffusion‐based probabilistic forecasting into MPC. Our approach generates an ensemble of future trajectories for the evolution of the system, using a diffusion model and then applies several MPC variants: deterministic MPC, stochastic MPC, multi-stage scenario tree-based MPC, and heuristic-augmented MPC.

We demonstrate the effectiveness of D-I MPC on an energy‐arbitrage task with a battery energy storage system in the New York day-ahead electricity market, where it consistently outperforms MPC implementations driven by classical forecasters and model-free reinforcement-learning baselines, and additionally it performs closely to idealized implementations that use perfect forecasts in their optimization processes.