In other words, each expert has a subjeciive probability distribution over a given set of outcomes.
Documented applications of belief aggregation include: During COVID-19, the European Academy of Neurology developed an ad-hoc three-round voting method to aggregate expert opinions and reach a consensus.
Geometric aggregation is justified when the experts' beliefs are based on the same information, and multiplicative aggregation is justified when the experts' beliefs are based on private information.
In some settings, it is possible to pay the experts a certain sum of money, depending both on their expressed belief and on the realized outcome.
Careful design of the payment function (often called a "scoring rule") can lead to a truthful mechanism.
To develop truthful mechanisms, one must make assumptions about the experts' preferences over the set of accepted probability-distributions.
They define a new concept called level-strategyproofness (Level-SP), which is relevant when society's decision is based on the question of whether the probability of some event is above or below a given threshold.
Level-SP provably implies strategyproofness for a rich class of cdf-single-peaked preferences.