In multivariate statistics, if ε {\displaystyle \varepsilon } is a vector of n {\displaystyle n} random variables, and Λ {\displaystyle \Lambda } is an n {\displaystyle n} -dimensional symmetric matrix, then the scalar quantity ε T Λ ε {\displaystyle \varepsilon ^{T}\Lambda \varepsilon } is known as a quadratic form in ε {\displaystyle \varepsilon } .
Expectation It can be shown that
E [ ε T Λ ε ] = tr [ Λ Σ ] + μ T Λ μ {\displaystyle \operatorname {E} \left[\varepsilon ^{T}\Lambda \varepsilon \right]=\operatorname {tr} \left[\Lambda \Sigma \right]+\mu ^{T}\Lambda \mu }
where μ {\displaystyle \mu } and Σ {\displaystyle \Sigma } are the expected value and variance-covariance matrix of ε {\displaystyle \varepsilon } , respectively, and tr denotes the trace of a matrix. This result only depends on the existence of μ {\displaystyle \mu } and Σ {\displaystyle \Sigma } ; in particular, normality of ε {\displaystyle \varepsilon } is not required. A book treatment of the topic of quadratic forms in random variables is that of Mathai and Provost.
Proof Since the quadratic form is a scalar quantity, ε T Λ ε = tr ( ε T Λ ε ) {\displaystyle \varepsilon ^{T}\Lambda \varepsilon =\operatorname {tr} (\varepsilon ^{T}\Lambda \varepsilon )} . Next, by the cyclic property of the trace operator,
E [ tr ( ε T Λ ε ) ] = E [ tr ( Λ ε ε T ) ] . {\displaystyle \operatorname {E} [\operatorname {tr} (\varepsilon ^{T}\Lambda \varepsilon )]=\operatorname {E} [\operatorname {tr} (\Lambda \varepsilon \varepsilon ^{T})].}
Since the trace operator is a linear combination of the components of the matrix, it therefore follows from the linearity of the expectation operator that
E [ tr ( Λ ε ε T ) ] = tr ( Λ E ( ε ε T ) ) . {\displaystyle \operatorname {E} [\operatorname {tr} (\Lambda \varepsilon \varepsilon ^{T})]=\operatorname {tr} (\Lambda \operatorname {E} (\varepsilon \varepsilon ^{T})).}
A standard property of variances then tells us that this is
tr ( Λ ( Σ + μ μ T ) ) . {\displaystyle \operatorname {tr} (\Lambda (\Sigma +\mu \mu ^{T})).}
Applying the cyclic property of the trace operator again, we get
tr ( Λ Σ ) + tr ( Λ μ μ T ) = tr ( Λ Σ ) + tr ( μ T Λ μ ) = tr ( Λ Σ ) + μ T Λ μ . {\displaystyle \operatorname {tr} (\Lambda \Sigma )+\operatorname {tr} (\Lambda \mu \mu ^{T})=\operatorname {tr} (\Lambda \Sigma )+\operatorname {tr} (\mu ^{T}\Lambda \mu )=\operatorname {tr} (\Lambda \Sigma )+\mu ^{T}\Lambda \mu .}
Variance in the Gaussian case In general, the variance of a quadratic form depends greatly on the distribution of ε {\displaystyle \varepsilon } . However, if ε {\displaystyle \varepsilon } does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that Λ {\displaystyle \Lambda } is a symmetric matrix. Then,
… excerpt ends here. Continue reading the full article.
