Stochastic gradient Langevin dynamics (SGLD) is an optimization and sampling technique composed of characteristics from Stochastic gradient descent, a Robbins–Monro optimization algorithm, and Langevin dynamics, a mathematical extension of molecular dynamics models. Like stochastic gradient descent, SGLD is an iterative optimization algorithm which uses minibatching to create a stochastic gradient estimator, as used in SGD to optimize a differentiable objective function. Unlike traditional SGD, SGLD can be used for Bayesian learning as a sampling method. SGLD may be viewed as Langevin dynamics applied to posterior distributions, but the key difference is that the likelihood gradient terms are minibatched, like in SGD. SGLD, like Langevin dynamics, produces samples from a posterior distribution of parameters based on available data. First described by Welling and Teh in 2011, the method has applications in many contexts which require optimization, and is most notably applied in machine learning problems.
Formal definition Given some parameter vector θ {\displaystyle \theta } , its prior distribution p ( θ ) {\displaystyle p(\theta )} , and a set of data points X = { x i } i = 1 N {\displaystyle X=\{x_{i}\}_{i=1}^{N}} , Langevin dynamics samples from the posterior distribution p ( θ ∣ X ) ∝ p ( θ ) ∏ i = 1 N p ( x i ∣ θ ) {\displaystyle p(\theta \mid X)\propto p(\theta )\prod _{i=1}^{N}p(x_{i}\mid \theta )} by updating the chain:
Δ θ t = ε t 2 ( ∇ log p ( θ t ) + ∑ i = 1 N ∇ log p ( x t i ∣ θ t ) ) + η t {\displaystyle \Delta \theta _{t}={\frac {\varepsilon _{t}}{2}}\left(\nabla \log p(\theta _{t})+\sum _{i=1}^{N}\nabla \log p(x_{t_{i}}\mid \theta _{t})\right)+\eta _{t}}
Stochastic gradient Langevin dynamics uses a modified update procedure with minibatched likelihood terms:
Δ θ t = ε t 2 ( ∇ log p ( θ t ) + N n ∑ i = 1 n ∇ log p ( x t i ∣ θ t ) ) + η t {\displaystyle \Delta \theta _{t}={\frac {\varepsilon _{t}}{2}}\left(\nabla \log p(\theta _{t})+{\frac {N}{n}}\sum _{i=1}^{n}\nabla \log p(x_{t_{i}}\mid \theta _{t})\right)+\eta _{t}}
where n < N {\displaystyle n<N} is a positive integer, η t ∼ N ( 0 , ε t ) {\displaystyle \eta _{t}\sim {\mathcal {N}}(0,\varepsilon _{t})} is Gaussian noise, p ( x ∣ θ ) {\displaystyle p(x\mid \theta )} is the likelihood of the data given the parameter vector θ {\displaystyle \theta } , and our step sizes ε t {\displaystyle \varepsilon _{t}} satisfy the following conditions:
… excerpt ends here. Continue reading the full article.
