Competitive learning

Competitive learning is a form of unsupervised learning in artificial neural networks, in which nodes compete for the right to respond to a subset of the input data.

[1][2] A variant of Hebbian learning, competitive learning works by increasing the specialization of each node in the network.

It is well suited to finding clusters within data.

Models and algorithms based on the principle of competitive learning include vector quantization and self-organizing maps (Kohonen maps).

There are three basic elements to a competitive learning rule:[3][4] Accordingly, the individual neurons of the network learn to specialize on ensembles of similar patterns and in so doing become 'feature detectors' for different classes of input patterns.

The fact that competitive networks recode sets of correlated inputs to one of a few output neurons essentially removes the redundancy in representation which is an essential part of processing in biological sensory systems.

[5][6] Competitive Learning is usually implemented with Neural Networks that contain a hidden layer which is commonly known as “competitive layer”.

[7] Every competitive neuron is described by a vector of weights

and calculates the similarity measure between the input data

The winner neuron m sets its output

and all the other competitive neurons set their output

Usually, in order to measure similarity the inverse of the Euclidean distance is used:

Here is a simple competitive learning algorithm to find three clusters within some input data.

Let the weights that each node gives to its sensors be set randomly between 0.0 and 1.0.

When the net is shown an input, the node with the highest output is deemed the winner.

The input is classified as being within the cluster corresponding to that node.

The winner updates each of its weights, moving weight from the connections that gave it weaker signals to the connections that gave it stronger signals.