P253 Higher-Threshold Neurons Boost Information Encoding in Spiking Neural Networks
Farhad Razi*1, Fleur Zeldenrust1
1Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
*Email: farhad.razi@donders.ru.nl
Introduction The brain exhibits remarkable neural heterogeneity. Studies suggest this boosts performing sequential tasks [1], efficient coding [2], and working memory [3] in artificial neural networks. Specifically, heterogeneity in spike thresholds is shown to improve information encoding by reducing trial-to-trial variability in network responses [4]. However, the mechanisms behind this reduced variability remain unclear. We propose that spike threshold heterogeneity introduces variability in neuronal firing sensitivity, with higher-threshold neurons significantly contributing to enhanced information encoding and reduced variability. Our findings advance understanding of the brain's computational capacities. Methods A recurrent spiking network with leaky integrate-and-fire neurons was used (Fig. 1A). Heterogeneity was introduced by varying the width of uniform spike-threshold distributions. The distribution of firing rates was assessed. The dimensionality of network activity was quantified using the participation ratio. An input was applied to a subset of the network. A linear decoder was trained to decode the input from spiking responses of the stimulated subset and the whole network. Information encoding was quantified by comparing root mean square error (RMSE) between the decoded and original input. The decoder, trained on the original input, was used to decode a novel input from network responses, evaluating its generalization to unfamiliar inputs. Results Increasing spike threshold heterogeneity enhances network information encoding. Heterogeneity increases firing rate variability and participation ratio, indicating a higher dimensionality of network activity (Fig. 1B), consistent with previous studies [4]. Decoding performance improves with heterogeneity, particularly when using the whole network (Fig. 1C). This enhanced decoding performance is largely carried by neurons that have higher spike thresholds (Fig. 1D). Notably, decoders trained on heterogeneous networks show superior generalization performance on a novel input (Fig. 1E). These results support our hypothesis that heterogeneity yields more robust network-wide information encoding capacities via higher-threshold neurons. Discussion Our results highlight heterogeneity's crucial role in the brain's capacity in information encoding. However, heterogeneity may not always be beneficial. Improved encoding could potentially consume neural resources, possibly hindering certain task performances. Future work will investigate how heterogeneity impacts networks trained for specific prediction and decoding tasks to reveal the trade-off between information encoding and processing, identifying task-dependent optimal ranges for the neural heterogeneity. Our findings offer insights into the brain function and can guide the development of efficient, task-adaptive neuromorphic systems, potentially bridging the gap between biological and artificial neural networks.
Figure 1. A, Computational experimental design. B, Network characteristics and heterogeneity. C, RMSE between decoded and original input decreases with heterogeneity. D, Neurons with higher spike thresholds possess larger decoder weights, indicating their heightened role in encoding. E, Decoder generalization on a novel input improves with increasing heterogeneity. Acknowledgements This work was supported by Dutch Research Council, NWO Vidi grant VI.Vidi.213. 137. References [1]https://doi.org/10.1038/s41467-021-26022-3 [2]https://doi.org/10.1371/journal.pcbi.1008673 [3]https://doi.org/10.1523/JNEUROSCI.1641-13.2013 [4]https://doi.org/10.1073/pnas.2311885121
