Interaction information

[4] Interaction information expresses the amount of information (redundancy or synergy) bound up in a set of variables, beyond that which is present in any subset of those variables.

These functions, their negativity and minima have a direct interpretation in algebraic topology.

[5] The conditional mutual information can be used to inductively define the interaction information for any finite number of variables as follows: where Some authors[6] define the interaction information differently, by swapping the two terms being subtracted in the preceding equation.

This has the effect of reversing the sign for an odd number of variables.

is the mutual information between variables

is the conditional mutual information between variables

The interaction information is symmetric, so it does not matter which variable is conditioned on.

This is easy to see when the interaction information is written in terms of entropy and joint entropy, as follows: In general, for the set of variables

, the interaction information can be written in the following form (compare with Kirkwood approximation): For three variables, the interaction information measures the influence of a variable

on the amount of information shared between

, the interaction information can be negative as well as positive.

Positive interaction information indicates that variable

inhibits (i.e., accounts for or explains some of) the correlation between

, whereas negative interaction information indicates that variable

Interaction information is bounded.

Therefore Positive interaction information seems much more natural than negative interaction information in the sense that such explanatory effects are typical of common-cause structures.

For example, clouds cause rain and also block the sun; therefore, the correlation between rain and darkness is partly accounted for by the presence of clouds,

The result is positive interaction information

A car's engine can fail to start due to either a dead battery or a blocked fuel pump.

Ordinarily, we assume that battery death and fuel pump blockage are independent events,

{\displaystyle I({\text{blocked fuel}};{\text{dead battery}})=0}

But knowing that the car fails to start, if an inspection shows the battery to be in good health, we can conclude that the fuel pump must be blocked.

{\displaystyle I({\text{blocked fuel}};{\text{dead battery}}\mid {\text{engine fails}})>0}

, and the result is negative interaction information.

The possible negativity of interaction information can be the source of some confusion.

[3] Many authors have taken zero interaction information as a sign that three or more random variables do not interact, but this interpretation is wrong.

[7] To see how difficult interpretation can be, consider a set of eight independent binary variables

's overlap each other (are redundant) on the three binary variables

, we would expect the interaction information

This is correct in the sense that but it remains difficult to interpret.