ArXiv

Tamed Stochastic Gradient Hamiltonian Monte Carlo

2026-07-16 · 11:36 UTC ·Zhuoran Wang, Ying Zhang ·1 min read
Authors
Zhuoran Wang, Ying Zhang
Categories
math.OC, math.NA, stat.ML
arXiv
https://arxiv.org/abs/2607.14862v1
PDF
https://arxiv.org/pdf/2607.14862v1

Brief

The paper introduces tSGHMC to handle sampling and optimization problems with superlinearly growing stochastic gradients, proving a non-asymptotic W2 error bound with rate 1/4 under continuity-in-average and strong convexity and providing an expected excess risk upper bound. Experiments on newsvendor and CVaR tasks (synthetic and real data) show lower RMSE and excess risk versus tamed ULA.

Why it matters

Wang and Zhang (arXiv:2607.14862v1, published 2026-07-16) propose tamed stochastic gradient Hamiltonian Monte Carlo (tSGHMC) for sampling and stochastic optimization when stochastic gradients grow superlinearly.

Key details

  • Under a continuity-in-average condition and strong convexity, the paper proves a non-asymptotic Wasserstein-2 error bound for tSGHMC with convergence rate 1/4 and derives an upper bound on the associated expected excess risk.
  • Empirical tests on a newsvendor problem and Conditional Value-at-Risk (CVaR) minimization using synthetic and real datasets show tSGHMC attains lower root-mean-square error and lower expected excess risk than the tamed unadjusted stochastic Langevin algorithm (first-order counterpart).
Source evidence

Abstract

In this paper, we propose a novel tamed stochastic gradient Hamiltonian Monte Carlo (tSGHMC) algorithm for sampling and stochastic optimization problems with superlinearly growing stochastic gradients. Under a certain continuity in average condition and a strong convexity condition, we establish a non-asymptotic error bound in Wasserstein-2 distance for tSGHMC with the rate of convergence equal to $1/4$. Then, we derive an upper estimate for the associated expected excess risk, which provides a theoretical guarantee for the performance of tSGHMC. To illustrate the effectiveness of the proposed algorithm, we apply tSGHMC to practical examples, including a newsvendor problem and a Conditional Value-at-Risk minimization problem, using synthetic and real-world datasets. Numerical results support our theoretical findings. Furthermore, we compare tSGHMC with its first-order counterpart, namely, the tamed unadjusted stochastic Langevin algorithm. Simulation results demonstrate that tSGHMC achieves lower root mean square error and expected excess risk across a range of tasks.