In probability theory, integral probability metrics are types of distance functions between probability distributions, defined by how well a class of functions can distinguish the two distributions. Many important statistical distances are integral probability metrics, including the Wasserstein-1 distance and the total variation distance. In addition to theoretical importance, integral probability metrics are widely used in areas of statistics and machine learning. The name "integral probability metric" was given by German statistician Alfred Müller; the distances had also previously been called "metrics with a ζ-structure."
Definition Integral probability metrics (IPMs) are distances on the space of distributions over a set X {\displaystyle {\mathcal {X}}} , defined by a class F {\displaystyle {\mathcal {F}}} of real-valued functions on X {\displaystyle {\mathcal {X}}} as
D F ( P , Q ) = sup f ∈ F | E X ∼ P f ( X ) − E Y ∼ Q f ( Y ) | = sup f ∈ F | P f − Q f | ; {\displaystyle D_{\mathcal {F}}(P,Q)=\sup _{f\in {\mathcal {F}}}{\big |}\mathbb {E} _{X\sim P}f(X)-\mathbb {E} _{Y\sim Q}f(Y){\big |}=\sup _{f\in {\mathcal {F}}}{\big |}Pf-Qf{\big |};}
here the notation P f refers to the expectation of f under the distribution P. The absolute value in the definition is unnecessary, and often omitted, for the usual case where for every f ∈ F {\displaystyle f\in {\mathcal {F}}} its negation − f {\displaystyle -f} is also in F {\displaystyle {\mathcal {F}}} . The functions f being optimized over are sometimes called "critic" functions; if a particular f ∗ ∈ F {\displaystyle f^{*}\in {\mathcal {F}}} achieves the supremum, it is often termed a "witness function" (it "witnesses" the difference in the distributions). These functions try to have large values for samples from P and small (likely negative) values for samples from Q; this can be thought of as a weaker version of classifers, and indeed IPMs can be interpreted as the optimal risk of a particular classifier. The choice of F {\displaystyle {\mathcal {F}}} determines the particular distance; more than one F {\displaystyle {\mathcal {F}}} can generate the same distance. For any choice of F {\displaystyle {\mathcal {F}}} , D F {\displaystyle D_{\mathcal {F}}} satisfies all the definitions of a metric except that we may have D F ( P , Q ) = 0 {\displaystyle D_{\mathcal {F}}(P,Q)=0} for some P ≠ Q; this is variously termed a "pseudometric" or a "semimetric" depending on the community. For instance, using the class F = { x ↦ 0 } {\displaystyle {\mathcal {F}}=\{x\mapsto 0\}} which only contains the zero function, D F ( P , Q ) {\displaystyle D_{\mathcal {F}}(P,Q)} is identically zero. D F {\displaystyle D_{\mathcal {F}}} is a metric if and only if F {\displaystyle {\mathcal {F}}} separates points on the space of probability distributions, i.e. for any P ≠ Q there is some f ∈ F {\displaystyle f\in {\mathcal {F}}} such that P f ≠ Q f {\displaystyle Pf\neq Qf} ; most, but not all, common particular cases satisfy this property.
Examples All of these examples are metrics except when noted otherwise.
… excerpt ends here. Continue reading the full article.
