The data processing inequality is an information theoretic concept that states that the information content of a signal cannot be increased via a local physical operation.
This can be expressed concisely as 'post-processing cannot increase information'.
[1] Let three random variables form the Markov chain
, implying that the conditional distribution of
and is conditionally independent of
Specifically, we have such a Markov chain if the joint probability mass function can be written as In this setting, no processing of
, deterministic or random, can increase the information that
Using the mutual information, this can be written as : with the equality
also forms a Markov chain.
[2] One can apply the chain rule for mutual information to obtain two different decompositions of
are conditionally independent, given
, which means the conditional mutual information,
The data processing inequality then follows from the non-negativity of
This computer science article is a stub.
You can help Wikipedia by expanding it.