The control variates method is a variance reduction technique used in Monte Carlo methods.
It exploits information about the errors in estimates of known quantities to reduce the error of an estimate of an unknown quantity.
[1] [2][3] Let the unknown parameter of interest be
such that the expected value of m is μ:
, i.e. m is an unbiased estimator for μ.
Suppose we calculate another statistic
Then is also an unbiased estimator for
The variance of the resulting estimator
is By differentiating the above expression with respect to
, it can be shown that choosing the optimal coefficient minimizes the variance of
(Note that this coefficient is the same as the coefficient obtained from a linear regression.)
With this choice, where is the correlation coefficient of
, the greater the variance reduction achieved.
are unknown, they can be estimated across the Monte Carlo replicates.
This is equivalent to solving a certain least squares system; therefore this technique is also known as regression sampling.
When the expectation of the control variable,
, is not known analytically, it is still possible to increase the precision in estimating
(for a given fixed simulation budget), provided that the two conditions are met: 1) evaluating
is significantly cheaper than computing
; 2) the magnitude of the correlation coefficient
[3] We would like to estimate using Monte Carlo integration.
Using a sample of size n denote the points in the sample as
Then the estimate is given by Now we introduce
as a control variate with a known expected value
and combine the two into a new estimate Using
realizations and an estimated optimal coefficient
we obtain the following results The variance was significantly reduced after using the control variates technique.