P148 ELiSe: Efficient Learning of Sequences in Structured Recurrent Networks
Laura Kriener¹⸴²
Ben von Hünerbein*¹
Kristin Völk³
Timo Gierlich¹
Federico Benitez¹
Walter Senn¹
Mihai A. Petrovici¹
¹ Department of Physiology, University of Bern, 3012 Bern, Switzerland.
² Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
³ Catlab Engineering GmbH, Grafrath, Germany
*Email: ben.vonhuenerbein@unibe.ch
Introduction
To learn complex action sequences, neural networks must maintain memories of past states. Typically, the required transients are produced by strong network recurrence. The biological plausibility of existing solutions for recurrent weight learning suffers from issues with locality (BPTT [1]), resource scaling (RTRL [2]), or parameter scales (FORCE [3]). To alleviate these, we introduce dendritic computation and a static structural scaffold to our recurrent networks. Leveraging this, our always-on local plasticity rule carves out strong attractors which generate the target activation sequences. We show that with few neurons, our model learns to reproduce complex non-Markovian sequences robustly despite external disturbances.
Methods
Our network contains two populations of structured neurons with somatic and dendritic compartments and leaky-integrator dynamics that integrate presynaptic inputs (Fig. 1a1). Output rates are computed as non-linear functions on the voltage. During development, a sparse scaffold of static somato-somatic connections with random delays is formed (Fig. 1a2,3). A teacher nudges output neurons towards a target pattern, and the somato-somatic scaffold transports this signal throughout the network. The dense, plastic, and randomly delayed somato-dendritic weights (Fig. 1a4)use these signals to adapt based on a local error-correcting learning rule [4].This gives rise to a robust dynamical attractor which generates the correct output pattern in the absence of a teacher.
Results
We demonstrate our model's ability to learn complex, non-Markovian sequences by exposing it repeatedly to a sample of Beethoven's "Für Elise" (Fig. 1b). We find that learning the recurrent weights is critical by showing that our model outperforms a same-size reservoir, both in its ability to learn and then sustain a pattern during replay (Fig. 1c). Next, we demonstrate robust learning across large ranges of the network parameter space. Further, despite severe temporary disruptions of the output population activity during pattern replay, the network is able to recover a correct replay of the learned pattern. Finally, we show that our network is able to extract the denoised signal from noisy target activities.
Discussion
Compared to other models of sequence learning in cortex, we suggest that ours is more resource-efficient, more biologically plausible, and, in general, more robust. It starts with only a sparse, random connection scaffold generating weak and unstructured activity. We show that this is enough for local plasticity to extract useful information in order to imprint strong attractor dynamics, in a manner that is robust to parameter variability and external disturbance. Unlike other approaches, learning in our networks is phaseless and is not switched off during validation and replay.
Figure 1. (a) Development and learning in ELiSe. (a1) Sparse somato-somatic scaffold based on p and q (a2) with interneuron driven inhibition (a3). Dense somato-dendritic synapses (green) adapted during learning (a4). (b) Learning in early, intermediate and final stages (teacher removal at red line). (c) Learning accuracy and stability during learning and replay compared to an equivalent reservoir.
Acknowledgements
We thank Richard Hahnloser and his lab for valuable feedback on learning in songbirds. We gratefully acknowledge funding from the European Union for the Human Brain Project (grant #945539) and Fenix Infrastructure resources (grant #800858), the Swiss National Science Foundation (grants #310030L\_156863 and #CRSII5\_180316) and the Manfred Stärk Foundation.
References
[1] Werbos,Paul J. "Backpropagation through time: what it does and how to do it" Proceedings of the IEEE 78.10 (1990): 1550-1560.
[2] Marschall,Owen,Kyunghyun Cho, and Cristina Savin. "A unified framework of online learning algorithms for training recurrent neural networks." Journal of machine learning research 21.135 (2020): 1-34.
[3]Sussillo,David, and Larry F. Abbott. "Generating coherent patterns of activity from chaotic neural networks" Neuron 63.4 (2009): 544-557.
[4] Urbanczik, Robert, and Walter Senn. "Learning by the dendritic prediction of somatic spiking" Neuron 81.3 (2014): 521-528.
