Normal-inverse-Wishart distribution

In probability theory and statistics, the normal-inverse-Wishart distribution (or Gaussian-inverse-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions.

It is the conjugate prior of a multivariate normal distribution with unknown mean and covariance matrix (the inverse of the precision matrix).

[1] Suppose has a multivariate normal distribution with mean

and covariance matrix

, where has an inverse Wishart distribution.

has a normal-inverse-Wishart distribution, denoted as The full version of the PDF is as follows:[2]

{\displaystyle f({\boldsymbol {\mu }},{\boldsymbol {\Sigma }}|{\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Psi }},\nu )={\frac {\lambda ^{D/2}|{\boldsymbol {\Psi }}|^{\nu /2}|{\boldsymbol {\Sigma }}|^{-{\frac {\nu +D+2}{2}}}}{(2\pi )^{D/2}2^{\frac {\nu D}{2}}\Gamma _{D}({\frac {\nu }{2}})}}{\text{exp}}\left\{-{\frac {1}{2}}Tr({\boldsymbol {\Psi \Sigma }}^{-1})-{\frac {\lambda }{2}}({\boldsymbol {\mu }}-{\boldsymbol {\mu }}_{0})^{T}{\boldsymbol {\Sigma }}^{-1}({\boldsymbol {\mu }}-{\boldsymbol {\mu }}_{0})\right\}}

is the multivariate gamma function and

is the Trace of the given matrix.

By construction, the marginal distribution over

is an inverse Wishart distribution, and the conditional distribution over

is a multivariate normal distribution.

The marginal distribution over

is a multivariate t-distribution.

Suppose the sampling density is a multivariate normal distribution where

With the mean and covariance matrix of the sampling distribution is unknown, we can place a Normal-Inverse-Wishart prior on the mean and covariance parameters jointly The resulting posterior distribution for the mean and covariance matrix will also be a Normal-Inverse-Wishart where

To sample from the joint posterior of

, one simply draws samples from

To draw from the posterior predictive of a new observation, draw

, given the already drawn values of

[3] Generation of random variates is straightforward: