Echo state network

[9] An early clear formulation of the reservoir computing idea is due to K. Kirby, who disclosed this concept in a largely forgotten conference contribution.

[10] The first formulation of the reservoir computing idea known today stems from L. Schomaker,[11] who described how a desired target output could be obtained from an RNN by learning to combine signals from a randomly configured ensemble of spiking neural oscillators.

In addition to the solutions for errors with smallest squares, margin maximization criteria, so-called training support vector machines, are used to determine the output values.

[12] Other variants of echo state networks seek to change the formulation to better match common models of physical systems, such as those typically those defined by differential equations.

[15] The fixed RNN acts as a random, nonlinear medium whose dynamic response, the "echo", is used as a signal base.

[2] RNNs were rarely used in practice before the introduction of the ESN, because of the complexity involved in adjusting their connections (e.g., lack of autodifferentiation, susceptibility to vanishing/exploding gradients, etc.).

In early studies, ESNs were shown to perform well on time series prediction tasks from synthetic datasets.

[1][17] Today, many of the problems that made RNNs slow and error-prone have been addressed with the advent of autodifferentiation (deep learning) libraries, as well as more stable architectures such as long short-term memory and Gated recurrent unit; thus, the unique selling point of ESNs has been lost.