Information field theory

For example, the posterior expectation value of a field generated by a known Gaussian process and measured by a linear device with known Gaussian noise statistics is given by a generalized Wiener filter applied to the measured data.

IFT extends such known filter formula to situations with nonlinear physics, nonlinear devices, non-Gaussian field or noise statistics, dependence of the noise statistics on the field values, and partly unknown parameters of measurement.

For this it uses Feynman diagrams, renormalisation flow equations, and other methods from mathematical physics.

They describe the spatial variations of a quantity, like the air temperature, as a function of position.

[4] In IFT Bayes theorem is usually rewritten in the language of a statistical field theory,

the negative logarithm of the joint probability of data and signal and with the partition function being

However, physical fields have much more regularity than most elements of function spaces, as they are continuous and smooth at most of their locations.

Therefore, less general, but sufficiently flexible constructions can be used to handle the infinite number of degrees of freedom of a field.

This way, one deals with finite dimensional field spaces, over which probability densities are well definable.

In order for this description to be a proper field theory, it is further required that the pixel resolution

, however, all what is necessary for IFT to be consistent is that this determinant can be estimated for any finite resolution field representation with

A Gaussian probability distribution requires the specification of the field two point correlation function

If the noise follows a signal independent zero mean Gaussian statistics with covariance

denotes equality up to irrelevant constants, which, in this case, means expressions that are independent of

where equality between the right and left hand sides holds as both distributions are normalized,

, meaning that the data variance in a Fourier band has to exceed the expected noise level by a certain threshold before the signal reconstruction

Whenever the data variance exceeds this threshold slightly, the signal reconstruction jumps to a finite excitation level, similar to a first order phase transition in thermodynamic systems.

perception of the signal starts continuously as soon the data variance exceeds the noise level.

The critical filter, extensions thereof to non-linear measurements, and the inclusion of non-flat spectrum priors, permitted the application of IFT to real world signal inference problems, for which the signal covariance is usually unknown a priori.

The generalized Wiener filter, that emerges in free IFT, is in broad usage in signal processing.

From the Helmholtz free energy, any connected moment of the field can be calculated via

Situations where small expansion parameters exist that are needed for such a diagrammatic expansion to converge are given by nearly Gaussian signal fields, where the non-Gaussianity of the field statistics leads to small interaction coefficients

For example, the statistics of the Cosmic Microwave Background is nearly Gaussian, with small amounts of non-Gaussianities believed to be seeded during the inflationary epoch in the Early Universe.

Thus, the effective action approach of IFT is equivalent to the variational Bayesian methods, which also minimize the Kullback-Leibler divergence between approximate and exact posteriors.

Minimizing the Gibbs free energy provides approximatively the posterior mean field

itself, it can be pulled out of the path integral of the internal information energy construction,

The resulting expression can be calculated by commuting the mean field annihilator

By the usage of the field operator formalism the Gibbs free energy can be calculated, which permits the (approximate) inference of the posterior mean field via a numerical robust functional minimization.

The usage of path integrals for field inference was proposed by a number of authors, e.g. Edmund Bertschinger[9] or William Bialek and A.

[10] The connection of field theory and Bayesian reasoning was made explicit by Jörg Lemm.

Radio interferometric image of radio galaxies in the galaxy cluster Abell 2219. The images were constructed by data back-projection (top), the CLEAN algorithm (middle), and the RESOLVE algorithm (bottom). Negative and therefore not physical fluxes are displayed in white.
First three Feynman diagrams contributing to the posterior mean estimate of a field. A line expresses an information propagator, a dot at the end of a line to an information source, and a vertex to an interaction term. The first diagram encodes the Wiener filter, the second a non-linear correction, and the third an uncertainty correction to the Wiener filter.