Log probability

Since the probabilities of independent events multiply, and logarithms convert multiplication to addition, log probabilities of independent events add.

Log probabilities are thus practical for computations, and have an intuitive interpretation in terms of information theory: the negative expected value of the log probabilities is the information entropy of an event.

Similarly, likelihoods are often transformed to the log scale, and the corresponding log-likelihood can be interpreted as the degree to which an event supports a statistical model.

The log probability is widely used in implementations of computations with probability, and is studied as a concept in its own right in some applications of information theory, such as natural language processing.

In that case the log probabilities in the following formulas would be inverted.

However, in many applications a multiplication of probabilities (giving the probability of all independent events occurring) is used more often than their addition (giving the probability of at least one of mutually exclusive events occurring).

Additionally, the cost of computing the addition can be avoided in some situations by simply using the highest probability as an approximation.

Since probabilities are non-negative this gives a lower bound.

, provided one takes advantage of the asymmetry in the addition formula.

This also produces the correct behavior if one of the operands is floating-point negative infinity, which corresponds to a probability of zero.

The above formula alone will incorrectly produce an indeterminate result in the case where both arguments are

For numerical reasons, one should use a function that computes