In the probability theory field of mathematics, Talagrand's concentration inequality is an isoperimetric-type inequality for product probability spaces. It was first proved by the French mathematician Michel Talagrand. The inequality is one of the manifestations of the concentration of measure phenomenon. Roughly, the product of the probability to be in some subset of a product space (e.g. to be in one of some collection of states described by a vector) multiplied by the probability to be outside of a neighbourhood of that subspace at least a distance t {\displaystyle t} away, is bounded from above by the exponential factor e − t 2 / 4 {\displaystyle e^{-t^{2}/4}} . It becomes rapidly more unlikely to be outside of a larger neighbourhood of a region in a product space, implying a highly concentrated probability density for states described by independent variables, generically. The inequality can be used to streamline optimisation protocols by sampling a limited subset of the full distribution and being able to bound the probability to find a value far from the average of the samples.
Statement The inequality states that if Ω = Ω 1 × Ω 2 × ⋯ × Ω n {\displaystyle \Omega =\Omega _{1}\times \Omega _{2}\times \cdots \times \Omega _{n}} is a product space endowed with a product probability measure and A {\displaystyle A}
is a subset in this space, then for any t ≥ 0 {\displaystyle t\geq 0}
Pr [ A ] ⋅ Pr [ A t c ] ≤ e − t 2 / 4 , {\displaystyle \Pr[A]\cdot \Pr \left[{A_{t}^{c}}\right]\leq e^{-t^{2}/4}\,,}
where A t c {\displaystyle {A_{t}^{c}}} is the complement of A t {\displaystyle A_{t}} where this is defined by
A t = { x ∈ Ω : ρ ( A , x ) ≤ t } {\displaystyle A_{t}=\{x\in \Omega \colon ~\rho (A,x)\leq t\}}
and where ρ {\displaystyle \rho } is Talagrand's convex distance defined as
ρ ( A , x ) = max α , ‖ α ‖ 2 ≤ 1 min y ∈ A ∑ i : x i ≠ y i α i {\displaystyle \rho (A,x)=\max _{\alpha ,\|\alpha \|_{2}\leq 1}\ \min _{y\in A}\ \sum _{i\colon ~x_{i}\neq y_{i}}\alpha _{i}}
where α ∈ R n {\displaystyle \alpha \in \mathbf {R} ^{n}} , x , y ∈ Ω {\displaystyle x,y\in \Omega } are n {\displaystyle n} -dimensional vectors with entries
α i , x i , y i {\displaystyle \alpha _{i},x_{i},y_{i}} respectively and ‖ ⋅ ‖ 2 {\displaystyle \|\cdot \|_{2}} is the ℓ 2 {\displaystyle \ell ^{2}} -norm. That is,
‖ α ‖ 2 = ( ∑ i α i 2 ) 1 / 2 {\displaystyle \|\alpha \|_{2}=\left(\sum _{i}\alpha _{i}^{2}\right)^{1/2}}
References