In probability theory, a normalizing constant or normalizing factor is used to reduce any nonnegative function whose integral is finite to a probability density function. For example, a Gaussian function can be normalized into a probability density function, which gives the standard normal distribution. In Bayes' theorem, a normalizing constant is used to ensure that the sum of all possible hypotheses equals 1. Other uses of normalizing constants include making the value of a Legendre polynomial at 1 and in the orthogonality of orthonormal functions. A similar concept has been used in areas other than probability, such as for polynomials.
Definition In probability theory, a normalizing constant is a constant by which an everywhere non-negative function must be multiplied so the area under its graph is 1, e.g., to make it a probability density function or a probability mass function.
Examples If we start from the simple Gaussian function
p ( x ) = e − x 2 / 2 , x ∈ ( − ∞ , ∞ ) {\displaystyle p(x)=e^{-x^{2}/2},\quad x\in (-\infty ,\infty )}
we have the corresponding Gaussian integral
∫ − ∞ ∞ p ( x ) d x = ∫ − ∞ ∞ e − x 2 / 2 d x = 2 π , {\displaystyle \int _{-\infty }^{\infty }p(x)\,dx=\int _{-\infty }^{\infty }e^{-x^{2}/2}\,dx={\sqrt {2\pi \,}},}
Now if we use the latter's reciprocal value as a normalizing constant for the former, defining a function φ ( x ) {\displaystyle \varphi (x)} as
φ ( x ) = 1 2 π p ( x ) = 1 2 π e − x 2 / 2 {\displaystyle \varphi (x)={\frac {1}{\sqrt {2\pi \,}}}p(x)={\frac {1}{\sqrt {2\pi \,}}}e^{-x^{2}/2}}
so that its integral is unit
∫ − ∞ ∞ φ ( x ) d x = ∫ − ∞ ∞ 1 2 π e − x 2 / 2 d x = 1 {\displaystyle \int _{-\infty }^{\infty }\varphi (x)\,dx=\int _{-\infty }^{\infty }{\frac {1}{\sqrt {2\pi \,}}}e^{-x^{2}/2}\,dx=1}
then the function φ ( x ) {\displaystyle \varphi (x)} is a probability density function. This is the density of the standard normal distribution. (Standard, in this case, means the expected value is 0 and the variance is 1.) And constant 1 2 π {\textstyle {\frac {1}{\sqrt {2\pi }}}} is the normalizing constant of function p ( x ) {\displaystyle p(x)} . Similarly,
∑ n = 0 ∞ λ n n ! = e λ , {\displaystyle \sum _{n=0}^{\infty }{\frac {\lambda ^{n}}{n!}}=e^{\lambda },}
and consequently
f ( n ) = λ n e − λ n ! {\displaystyle f(n)={\frac {\lambda ^{n}e^{-\lambda }}{n!}}}
is a probability mass function on the set of all nonnegative integers. This is the probability mass function of the Poisson distribution with expected value λ. Note that if the probability density function is a function of various parameters, so too will be its normalizing constant. The parametrised normalizing constant for the Boltzmann distribution plays a central role in statistical mechanics. In that context, the normalizing constant is called the partition function.
… excerpt ends here. Continue reading the full article.
