Neural modeling fields

NMF is interpreted as a mathematical description of the mind's mechanisms, including concepts, emotions, instincts, imagination, thinking, and understanding.

This increase in similarity can be interpreted as satisfaction of an instinct for knowledge, and is felt as aesthetic emotions.

Suppose that signal X(n) is coming from sensory neurons n activated by object m, which is characterized by parameters Sm.

Various models compete for evidence in the bottom-up signals, while adapting their parameters for better match as described below.

Perception of minute features, or everyday objects, or cognition of complex abstract concepts is due to the same mechanism described below.

Learning is an essential part of perception and cognition, and in NMF theory it is driven by the dynamics that increase a similarity measure between the sets of models and signals, L({X},{M}).

Therefore, a partial similarity measure is constructed so that it treats each model as an alternative (a sum over concept-models) for each input neuron signal.

If learning is successful, it approximates probabilistic description and leads to near-optimal Bayesian decisions.

This is a well known problem, it is addressed by reducing similarity L using a "skeptic penalty function," (Penalty method) p(N,M) that grows with the number of models M, and this growth is steeper for a smaller amount of data N. For example, an asymptotically unbiased maximum likelihood estimation leads to multiplicative p(N,M) = exp(-Npar/2), where Npar is a total number of adaptive parameters in all models (this penalty function is known as Akaike information criterion, see (Perlovsky 2001) for further discussion and references).

The learning process consists of estimating model parameters S and associating signals with concepts by maximizing the similarity L. Note that all possible combinations of signals and models are accounted for in expression (2) for L. This can be seen by expanding a sum and multiplying all the terms resulting in MN items, a huge number.

This is the source of Combinatorial Complexity, which is solved in NMF by utilizing the idea of dynamic logic.

[7][8] An important aspect of dynamic logic is matching vagueness or fuzziness of similarity measures to the uncertainty of models.

Initially, parameter values are not known, and uncertainty of models is high; so is the fuzziness of the similarity measures.

Then the association variables f(m|n) are computed, Equation for f(m|n) looks like the Bayes formula for a posteriori probabilities; if l(n|m) in the result of learning become conditional likelihoods, f(m|n) become Bayesian probabilities for signal n originating from object m. The dynamic logic of the NMF is defined as follows: The following theorem has been proved (Perlovsky 2001): Theorem.

However, when the locations and orientations of patterns are not known, it is not clear which subset of the data points should be selected for fitting.

A standard approach for solving this kind of problem is multiple hypothesis testing (Singer et al. 1974).

Since all combinations of subsets and models are exhaustively searched, this method faces the problem of combinatorial complexity.

If one attempts to fit 4 models to all subsets of 10,000 data points, computation of complexity, MN ~ 106000.

An alternative computation by searching through the parameter space, yields lower complexity: each pattern is characterized by a 3-parameter parabolic shape.

Fitting 4x3=12 parameters to 100x100 grid by a brute-force testing would take about 1032 to 1040 operations, still a prohibitive computational complexity.

To apply NMF and dynamic logic to this problem one needs to develop parametric adaptive models of expected patterns.

1(a) true 'smile' and 'frown' patterns are shown without noise; (b) actual image available for recognition (signal is below noise, signal-to-noise ratio is between –2 dB and –0.7 dB); (c) an initial fuzzy model, a large fuzziness corresponds to uncertainty of knowledge; (d) through (m) show improved models at various iteration stages (total of 22 iterations).

Fig.1. Finding 'smile' and 'frown' patterns in noise, an example of dynamic logic operation: (a) true 'smile' and 'frown' patterns are shown without noise; (b) actual image available for recognition (signal is below noise, signal-to-noise ratio is between –2dB and –0.7dB); (c) an initial fuzzy blob-model, the fuzziness corresponds to uncertainty of knowledge; (d) through (m) show improved models at various iteration stages (total of 22 iterations). Between stages (d) and (e) the algorithm tried to fit the data with more than one model and decided, that it needs three blob-models to 'understand' the content of the data. There are several types of models: one uniform model describing noise (it is not shown) and a variable number of blob-models and parabolic models, which number, location, and curvature are estimated from the data. Until about stage (g) the algorithm 'thought' in terms of simple blob models, at (g) and beyond, the algorithm decided that it needs more complex parabolic models to describe the data. Iterations stopped at (m), when similarity L stopped increasing. This example is discussed in more details in (Linnehan et al. 2003).

Fig.2. Hierarchical NMF system. At each level of a hierarchy there are models, similarity measures, and actions (including adaptation, maximizing the knowledge instinct - similarity). High levels of partial similarity measures correspond to concepts recognized at a given level. Concept activations are output signals at this level and they become input signals to the next level, propagating knowledge up the hierarchy.