A likelihood function (often simply called the likelihood) gives the relative merit of various statistical models for describing a data set. Often the models being compared are parameterized by a parameter, with the parameter often written as θ, or they are parameterized by multiple parameters given as the components of a vector. For a probability function (or probability density function) Pr[x | θ] that gives the probability (or probability density) of data x for a given model-specifying parameter θ, the likelihood is any function of θ equal to cPr[x | θ] for some positive value c. In maximum likelihood estimation, the model parameter(s) or argument that maximizes the likelihood function serves as a point estimate for the unknown parameter, while the Fisher information (often approximated by the likelihood's Hessian matrix at the maximum) gives an indication of the estimate's precision. In contrast, in Bayesian statistics, the estimate of interest is the converse of the likelihood, the so-called posterior probability of the parameter given the observed data, which is calculated via Bayes' rule.
Definition
The likelihood function, parameterized by a (possibly multivariate) parameter θ {\textstyle \theta } , is usually defined differently for discrete and continuous probability distributions (a more general definition is discussed below). Given a probability density or mass function
x ↦ f ( x ∣ θ ) , {\displaystyle x\mapsto f(x\mid \theta ),}
where x {\textstyle x} is a realization of the random variable X {\textstyle X} , the likelihood function is
θ ↦ f ( x ∣ θ ) , {\displaystyle \theta \mapsto f(x\mid \theta ),}
often written
L ( θ ∣ x ) . {\displaystyle {\mathcal {L}}(\theta \mid x).}
In other words, when f ( x ∣ θ ) {\textstyle f(x\mid \theta )} is viewed as a function of x {\textstyle x} with θ {\textstyle \theta } fixed, it is a probability density function, and when viewed as a function of θ {\textstyle \theta } with x {\textstyle x} fixed, it is a likelihood function. In the frequentist paradigm, the notation f ( x ∣ θ ) {\textstyle f(x\mid \theta )} is often avoided and instead f ( x ; θ ) {\textstyle f(x;\theta )} or f ( x , θ ) {\textstyle f(x,\theta )} are used to indicate that θ {\textstyle \theta } is regarded as a fixed unknown quantity rather than as a random variable being conditioned on. The likelihood function does not specify the probability that θ {\textstyle \theta } is the truth, given the observed sample X = x {\textstyle X=x} . Such an interpretation is a common error, with potentially disastrous consequences (see prosecutor's fallacy).
Discrete probability distribution Let X {\textstyle X} be a discrete random variable with probability mass function p {\textstyle p} depending on a parameter θ {\textstyle \theta } . Then the function
L ( θ ∣ x ) = p θ ( x ) = P θ ( X = x ) = Pr { X = x ∣ Θ = θ } , {\displaystyle {\mathcal {L}}(\theta \mid x)=p_{\theta }(x)=P_{\theta }(X=x)={\text{Pr}}\{X=x\mid \Theta =\theta \},}
considered as a function of θ {\textstyle \theta } , a possible value of the deterministic but unknown parameter Θ {\textstyle \Theta } , is the likelihood function, given the outcome x {\textstyle x} of the random variable X {\textstyle X} . Sometimes the probability of "the value x {\textstyle x} of X {\textstyle X} for the parameter value θ {\textstyle \theta } " is written as P(X = x | θ) or P(X = x; θ). The likelihood is the probability that a particular outcome x {\textstyle x} is observed when the true value of the parameter is θ {\textstyle \theta } , equivalent to the probability mass on x {\textstyle x} ; it is not a probability density over the parameter θ {\textstyle \theta } . The likelihood, L ( θ ∣ x ) {\textstyle {\mathcal {L}}(\theta \mid x)} , should not be confused with P ( θ ∣ x ) {\textstyle P(\theta \mid x)} , which is the posterior probability of θ {\textstyle \theta } given the data x {\textstyle x} .
Example
… excerpt ends here. Continue reading the full article.
