The posterior probability is a type of conditional probability that results from updating the prior probability with information summarized by the likelihood via an application of Bayes' rule. From an epistemological perspective, the posterior probability contains everything there is to know about an uncertain proposition (such as a scientific hypothesis, or parameter values), given prior knowledge and a mathematical model describing the observations available at a particular time. After the arrival of new information, the current posterior probability may serve as the prior in another round of Bayesian updating. In the context of Bayesian statistics, the posterior probability distribution usually describes the epistemic uncertainty about statistical parameters conditional on a collection of observed data. From a given posterior distribution, various point and interval estimates can be derived, such as the maximum a posteriori (MAP) or the highest posterior density interval (HPDI). But while conceptually simple, the posterior distribution is generally not tractable and therefore needs to be either analytically or numerically approximated.
Definition in the distributional case In Bayesian statistics, the posterior probability is the probability distribution of the parameters θ {\displaystyle \theta } given the evidence X {\displaystyle X} , and is denoted p ( θ | X ) {\displaystyle p(\theta |X)} . It contrasts with the likelihood function, which is the probability of the evidence given the parameters: p ( X | θ ) {\displaystyle p(X|\theta )} . The two are related as follows: Given a prior belief that a probability distribution function is p ( θ ) {\displaystyle p(\theta )} and that the observations x {\displaystyle x} have a likelihood p ( x | θ ) {\displaystyle p(x|\theta )} , then the posterior probability is defined as
p ( θ | x ) = p ( x | θ ) p ( x ) p ( θ ) {\displaystyle p(\theta |x)={\frac {p(x|\theta )}{p(x)}}p(\theta )} , where p ( x ) {\displaystyle p(x)} is the normalizing constant and is calculated as
p ( x ) = ∫ p ( x | θ ) p ( θ ) d θ {\displaystyle p(x)=\int p(x|\theta )p(\theta )d\theta }
for continuous θ {\displaystyle \theta } , or by summing p ( x | θ ) p ( θ ) {\displaystyle p(x|\theta )p(\theta )}
over all possible values of θ {\displaystyle \theta } for discrete θ {\displaystyle \theta } . The posterior probability is therefore proportional to the product Likelihood · Prior probability.
Example Suppose there is a school with 60% boys and 40% girls as students. The girls wear trousers or skirts in equal numbers; all boys wear trousers. An observer sees a (random) student from a distance; all the observer can see is that this student is wearing trousers. What is the probability this student is a girl? The correct answer can be computed using Bayes' theorem. The event G is that the student observed is a girl, and the event T is that the student observed is wearing trousers. To compute the posterior probability P ( G | T ) {\displaystyle P(G|T)} , we first need to know:
P ( G ) {\displaystyle P(G)} , or the probability that the student is a girl regardless of any other information. Since the observer sees a random student, meaning that all students have the same probability of being observed, and the percentage of girls among the students is 40%, this probability equals 0.4.
P ( B ) {\displaystyle P(B)} , or the probability that the student is not a girl (i.e. a boy) regardless of any other information (B is the complementary event to G). This is 60%, or 0.6.
P ( T | G ) {\displaystyle P(T|G)} , or the probability of the student wearing trousers given that the student is a girl. As they are as likely to wear skirts as trousers, this is 0.5.
P ( T | B ) {\displaystyle P(T|B)} , or the probability of the student wearing trousers given that the student is a boy. This is given as 1.
… excerpt ends here. Continue reading the full article.
