Transfer entropy is a non-parametric statistic measuring the amount of directed (time-asymmetric) transfer of information between two random processes.
[1][2][3] Transfer entropy from a process X to another process Y is the amount of uncertainty reduced in future values of Y by knowing the past values of X given past values of Y.
denote two random processes and the amount of information is measured using Shannon's entropy, the transfer entropy can be written as: where H(X) is Shannon's entropy of X.
The above definition of transfer entropy has been extended by other types of entropy measures such as Rényi entropy.
[3][4] Transfer entropy is conditional mutual information,[5][6] with the history of the influenced variable
in the condition: Transfer entropy reduces to Granger causality for vector auto-regressive processes.
[7] Hence, it is advantageous when the model assumption of Granger causality doesn't hold, for example, analysis of non-linear signals.
[8][9] However, it usually requires more samples for accurate estimation.
[10] The probabilities in the entropy formula can be estimated using different approaches (binning, nearest neighbors) or, in order to reduce complexity, using a non-uniform embedding.
[11] While it was originally defined for bivariate analysis, transfer entropy has been extended to multivariate forms, either conditioning on other potential source variables[12] or considering transfer from a collection of sources,[13] although these forms require more samples again.
Transfer entropy has been used for estimation of functional connectivity of neurons,[13][14][15] social influence in social networks[8] and statistical causality between armed conflict events.
[16] Transfer entropy is a finite version of the directed information which was defined in 1990 by James Massey[17] as
The directed information places an important role in characterizing the fundamental limits (channel capacity) of communication channels with or without feedback[18][19] and gambling with causal side information.