“Storing Infinite Numbers of Patterns in a Spin-Glass Model of Neural Networks”, Daniel J. Amit, Hanoch Gutfreund, H. Sompolinsky1985-09-30 (neural net; backlinks)⁠:

The Hopfield model for a neural network is studied in the limit when the number p of stored patterns increases with the size N of the network, as p = αN.

It is shown that, despite its spin-glass features, the model exhibits associative memory for α < αc, αc ≳ 0.14.

This is a result of the existence at low temperature of 2p dynamically stable degenerate states, each of which is almost fully correlated with one of the patterns. These states become ground states at α < 0.05.

The phase diagram of this rich spin-glass is described.

See Also:

• Large Associative Memory Problem in Neurobiology and Machine Learning

• Memorization Without Generalization in a Multilayered Neural Network

• Statistical mechanics of learning from examples

• The statistical mechanics of learning a rule

• Statistical Mechanics of Generalization

• Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior

• Why does deep and cheap learning work so well?

• Exhaustive Learning

• The Loss Surfaces of Multilayer Networks

• Learning through atypical “phase transitions” in overparameterized neural networks

• Efficient supervised learning in networks with binary synapses

• Heavy-tailed neuronal connectivity arises from Hebbian self–organization

• The Phase Transition In Human Cognition

• The Phase Transition In Human Cognition § Phase Transitions in Language Processing

• Observation of Phase Transitions in Spreading Activation Networks

• Hopfield Networks is All You Need

Backlinks: