-dimensional symmetric matrix, then the scalar quantity
are the expected value and variance-covariance matrix of
, respectively, and tr denotes the trace of a matrix.
This result only depends on the existence of
A book treatment of the topic of quadratic forms in random variables is that of Mathai and Provost.
[2] Since the quadratic form is a scalar quantity,
ε = tr (
Next, by the cyclic property of the trace operator, Since the trace operator is a linear combination of the components of the matrix, it therefore follows from the linearity of the expectation operator that A standard property of variances then tells us that this is Applying the cyclic property of the trace operator again, we get In general, the variance of a quadratic form depends greatly on the distribution of
does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable.
Assume for the moment that
is a symmetric matrix.
Then, In fact, this can be generalized to find the covariance between two quadratic forms on the same
must both be symmetric): In addition, a quadratic form such as this follows a generalized chi-squared distribution.
The case for general
can be derived by noting that so is a quadratic form in the symmetric matrix
, so the mean and variance expressions are the same, provided
and an operator matrix
, then the residual sum of squares can be written as a quadratic form in
: For procedures where the matrix
is symmetric and idempotent, and the errors are Gaussian with covariance matrix
degrees of freedom and noncentrality parameter
, where may be found by matching the first two central moments of a noncentral chi-squared random variable to the expressions given in the first two sections.
with no bias, then the noncentrality
follows a central chi-squared distribution.