It prescribes that a function that maps a set of input values
should be chosen or learned so as to maximize the average Shannon mutual information between
As the InfoMax objective is difficult to compute exactly, a related notion uses two models giving two outputs
[2] Infomax, in its zero-noise limit, is related to the principle of redundancy reduction proposed for biological sensory processing by Horace Barlow in 1961,[3] and applied quantitatively to retinal processing by Atick and Redlich.
[4] (Becker and Hinton, 1992)[2] showed that the contrastive InfoMax objective allows a neural network to learn to identify surfaces in random dot stereograms (in one dimension).
Infomax-based ICA was described by (Bell and Sejnowski, 1995),[5] and (Nadal and Parga, 1995).