Beyond catastrophic forgetting in associative networks with self-interactions
Maurizio Mattia*1, Gianni V. Vinci1, Andrea Galluzzi1
1 Natl. Center for Radiation Protection and Computational Physics,
Istituto Superiore di Sanità, 00161 Roma, Italy*Email: maurizio.mattia@iss.it
IntroductionSpin-glass
models of associative memories are a cornerstone between statistical physics
and theoretical neuroscience [1]. In these networks, stochastic spin-like units
interact through a synaptic matrix shaped by local Hebbian learning. In absence
of self-interactions, the free energy reveals catastrophic forgetting of all
stored patterns when their number exceeds a critical memory load [2]. MethodsWe modeled
associative memory using a recurrent neural network (RNN) of N units, each
representing the firing rate of a local neuron assembly. Unit activities
evolved deterministically, driven by a monotonically increasing gain function
of their synaptic inputs, which were computed as weighted sums of all other
units' activities. For stationary external inputs, the resulting dynamics
matched well-known analog associative memory models. All simulations were
performed without external input. ResultsHere, we bridge the gap with biology by
considering deterministic RNNs with graded units coupled via the same
Amari-Hopfield synaptic matrix [3,4], while retaining self-interactions. Contrary
to the assumption that self-couplings play a negligible role, we demonstrate
that they qualitatively reshape the energy landscape, confining the recurrent
dynamics to the subspace hosting the stored patterns. This allows for the
derivation of an exact overlap-dependent Lyapunov function, valid even for
networks with finite size. Moreover, self-interactions generate an auxiliary
internal field aligned with the target memory pattern, widening the repertoire
of accessible attractor states. Discussion Consequently,
pure recall states act as robust associative memories for any memory load,
beyond the critical threshold for catastrophic forgetting observed in
spin-glass models -all without requiring nonlocal learning prescriptions or
significant reshaping of the Hebbian synaptic matrix [5,6]. All the details about
this work can be found in the preprint [7]. Work partially funded by the NEXTGENERATIONEU and MUR (PNRRM4C2I1.3) project MNESYS (PE0000006-DD 1553 11.10.2022) and project EBRAINS-Italy (IR0000011- DD 10116.6.2022) to MM. - Amit, D. J. (1989). Modeling brain function (Cambridge University Press).
- https://doi.org/10.1103/PhysRevLett.55.1530
- https://doi.org/10.1109/T-C.1972.223477
- https://doi.org/10.1073/pnas.79.8.2554
- https://doi.org/10.1051/jphyslet:01985004608035900
- https://doi.org/10.1038/304158a0
- https://doi.org/10.48550/arXiv.2504.04560
