Bayesian statistics (/ˈbeɪziən/ BAY-zee-ən or /ˈbeɪʒən/ BAY-zhən)[1] is a theory in the field of statistics based on the Bayesian interpretation of probability, where probability expresses a degree of belief in an event.
Bayesian statistical methods use Bayes' theorem to compute and update probabilities after obtaining new data.
[3][4] For example, in Bayesian inference, Bayes' theorem can be used to estimate the parameters of a probability distribution or statistical model.
Since Bayesian statistics treats probability as a degree of belief, Bayes' theorem can directly assign a probability distribution that quantifies the belief to the parameter or set of parameters.
In several papers spanning from the late 18th to the early 19th centuries, Pierre-Simon Laplace developed the Bayesian interpretation of probability.
[5] Laplace used methods now considered Bayesian to solve a number of statistical problems.
Throughout much of the 20th century, Bayesian methods were viewed unfavorably by many statisticians due to philosophical and practical considerations.
Many of these methods required much computation, and most widely used approaches during that time were based on the frequentist interpretation.
However, with the advent of powerful computers and new algorithms like Markov chain Monte Carlo, Bayesian methods have gained increasing prominence in statistics in the 21st century.
[2][6] Bayes's theorem is used in Bayesian methods to update probabilities, which are degrees of belief, after obtaining new data.
Although Bayes's theorem is a fundamental result of probability theory, it has a specific interpretation in Bayesian statistics.
usually represents a proposition (such as the statement that a coin lands on heads fifty percent of the time) and
represents the evidence, or new data that is to be taken into account (such as the result of a series of coin flips).
is difficult to calculate as the calculation would involve sums or integrals that would be time-consuming to evaluate, so often only the product of the prior and likelihood is considered, since the evidence does not change in the same analysis.
The maximum a posteriori, which is the mode of the posterior and is often computed in Bayesian statistics using mathematical optimization methods, remains the same.
[2] The general set of statistical techniques can be divided into a number of activities, many of which have special Bayesian versions.
[8] In classical frequentist inference, model parameters and hypotheses are considered to be fixed.
For example, it would not make sense in frequentist inference to directly assign a probability to an event that can only happen once, such as the result of the next flip of a fair coin.
However, it would make sense to state that the proportion of heads approaches one-half as the number of coin flips increases.
For example, a coin can be represented as samples from a Bernoulli distribution, which models two possible outcomes.
Devising a good model for the data is central in Bayesian inference.
In most cases, models only approximate the true process, and may not take into account certain factors influencing the data.
[2] In Bayesian inference, probabilities can be assigned to model parameters.
Bayesian inference uses Bayes' theorem to update probabilities after more evidence is obtained or known.
[2][10] Furthermore, Bayesian methods allow for placing priors on entire models and calculating their posterior probabilities using Bayes' theorem.
These posterior probabilities are proportional to the product of the prior and the marginal likelihood, where the marginal likelihood is the integral of the sampling density over the prior distribution of the parameters.
In complex models, marginal likelihoods are generally computed numerically.
For conducting a Bayesian statistical analysis, best practices are discussed by van de Schoot et al.[15] For reporting the results of a Bayesian statistical analysis, Bayesian analysis reporting guidelines (BARG) are provided in an open-access article by John K.
[16] The Bayesian design of experiments includes a concept called 'influence of prior beliefs'.
The correct visualization, analysis, and interpretation of these distributions is key to properly answer the questions that motivate the inference process.