This allows accounting for both any prior knowledge on the parameters to be determined as well as uncertainties in observations.
The utility is most commonly defined in terms of a measure of the accuracy of the information provided by the experiment (e.g., the Shannon information or the negative of the variance) but may also involve factors such as the financial cost of performing the experiment.
What will be the optimal experiment design depends on the particular utility criterion chosen.
If the model is linear, the prior probability density function (PDF) is homogeneous and observational errors are normally distributed, the theory simplifies to the classical optimal experimental design theory.
In numerous publications on Bayesian experimental design, it is (often implicitly) assumed that all posterior probabilities will be approximately normal.
This allows for the expected utility to be calculated using linear theory, averaging over the space of model parameters.
In many cases, the posterior distribution is not available in closed form and has to be approximated using numerical methods.
The most common approach is to use Markov chain Monte Carlo methods to generate samples from the posterior, which can then be used to approximate the expected utility.
Another approach is to use a variational Bayes approximation of the posterior, which can often be calculated in closed form.
This approach has the advantage of being computationally more efficient than Monte Carlo methods, but the disadvantage that the approximation might not be very accurate.
is the marginal probability density in observation space The expected utility of an experiment with design
[6] It is worth noting that the second term on the second equation line will not depend on the design
, so if the goal is to choose the design with the highest utility, the term need not be computed at all.
Several authors have considered numerical techniques for evaluating and optimizing this criterion.
An example of Bayesian design for linear dynamical model discrimination is given in Bania (2019).
The Kelly criterion also describes such a utility function for a gambler seeking to maximize profit, which is used in gambling and information theory; Kelly's situation is identical to the foregoing, with the side information, or "private wire" taking the place of the experiment.