In probability theory and statistics, the marginal distribution of a subset of a collection of random variables is the probability distribution of the variables contained in the subset. It gives the probabilities of various values of the variables in the subset without reference to the values of the other variables. This contrasts with a conditional distribution, which gives the probabilities contingent upon the values of the other variables. Marginal variables are those variables in the subset of variables being retained. These concepts are "marginal" because they can be found by summing values in a table along rows or columns, and writing the sum in the margins of the table. The distribution of the marginal variables (the marginal distribution) is obtained by marginalizing (that is, focusing on the sums in the margin) over the distribution of the variables being discarded, and the discarded variables are said to have been marginalized out. The context here is that the theoretical studies being undertaken, or the data analysis being done, involves a wider set of random variables but that attention is being limited to a reduced number of those variables. In many applications, an analysis may start with a given collection of random variables, then first extend the set by defining new ones (such as the sum of the original random variables) and finally reduce the number by placing interest in the marginal distribution of a subset (such as the sum). Several different analyses may be done, each treating a different subset of variables as the marginal distribution.
Definition
Marginal probability mass function Given a known joint distribution of two discrete random variables, say, X and Y, the marginal distribution of either variable – X for example – is the probability distribution of X when the values of Y are not taken into consideration. This can be calculated by summing the joint probability distribution over all values of Y. Naturally, the converse is also true: the marginal distribution can be obtained for Y by summing over the separate values of X.
p X ( x i ) = ∑ j p ( x i , y j ) , and p Y ( y j ) = ∑ i p ( x i , y j ) {\displaystyle p_{X}(x_{i})=\sum _{j}p(x_{i},y_{j}),\quad {\text{and}}\quad p_{Y}(y_{j})=\sum _{i}p(x_{i},y_{j})}
A marginal probability can always be written as an expected value:
p X ( x ) = ∫ y p X ∣ Y ( x ∣ y ) p Y ( y ) d y = E Y [ p X ∣ Y ( x ∣ Y ) ] . {\displaystyle p_{X}(x)=\int _{y}p_{X\mid Y}(x\mid y)\,p_{Y}(y)\,\mathrm {d} y=\operatorname {E} _{Y}[p_{X\mid Y}(x\mid Y)]\;.}
Intuitively, the marginal probability of X is computed by examining the conditional probability of X given a particular value of Y, and then averaging this conditional probability over the distribution of all values of Y. This follows from the definition of expected value (after applying the law of the unconscious statistician)
E Y [ f ( Y ) ] = ∫ y f ( y ) p Y ( y ) d y . {\displaystyle \operatorname {E} _{Y}[f(Y)]=\int _{y}f(y)p_{Y}(y)\,\mathrm {d} y.}
Therefore, marginalization provides the rule for the transformation of the probability distribution of a random variable Y and another random variable X=g(Y):
p X ( x ) = ∫ y p X ∣ Y ( x ∣ y ) p Y ( y ) d y = ∫ y δ ( x − g ( y ) ) p Y ( y ) d y . {\displaystyle p_{X}(x)=\int _{y}p_{X\mid Y}(x\mid y)\,p_{Y}(y)\,\mathrm {d} y=\int _{y}\delta {\big (}x-g(y){\big )}\,p_{Y}(y)\,\mathrm {d} y.}
Marginal probability density function Given two continuous random variables X and Y whose joint distribution is known, then the marginal probability density function for X can be obtained by integrating the joint probability density, f, over Y, and vice versa. That is
… excerpt ends here. Continue reading the full article.
