Markov kernel

In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that in the general theory of Markov processes plays the role that the transition matrix does in the theory of Markov processes with a finite state space.

A Markov kernel with source

with the following properties: In other words it associates to each point

Then a Markov kernel is fully determined by the probability it assigns to singletons

for the random walk are equivalent to the Markov kernel.

Again a Markov kernel is defined by the probability it assigns to singleton sets for each

define a (countable) stochastic matrix

i.e. We then define Again the transition probability, the stochastic matrix and the Markov kernel are equivalent reformulations.

a measurable function with respect to the product

i.e. the mapping defines a Markov kernel.

Moreover it encompasses other important examples such as the convolution kernels, in particular the Markov kernels defined by the heat equation.

The latter example includes the Gaussian kernel on

standard Lebesgue measure and Take

This example allows us to think of a Markov kernel as a generalised function with a (in general) random rather than certain value.

That is, it is a multivalued function where the values are not equally weighted.

with the standard sigma algebra of Borel sets.

is the number of element at the state

For the simple case of coin flips this models the different levels of a Galton board.

we consider a Markov kernel

a sharply defined point

the kernel assigns a "fuzzy" point in

which is only known with some level of uncertainty, much like actual physical measurements.

by the Chapman-Kolmogorov equation The composition is associative by the Monotone Convergence Theorem and the identity function considered as a Markov kernel (i.e. the delta measure

This composition defines the structure of a category on the measurable spaces with Markov kernels as morphisms, first defined by Lawvere,[4] the category of Markov kernels.

A composition of a probability space

defines a probability space

Then there exists a unique measure

-valued random variable on the measure space

is a version of the conditional expectation

It is called regular conditional distribution of