P004 Synergistic high-order statistics in a neural network is related to task complexity and attractor characteristics
Ignacio Ampuero1, Javier Díaz1,Patricio Orio1,2,3 1Centro Interdisciplinario de Neurociencia de Valparaíso, Universidad de Valparaíso, Valparaíso, Chile. 2Instituto de Neurociencia, Facultad de Ciencias, Universidad de Valparaíso, Valparaíso, Chile 3Advanced Center for Electrical and Electronic Engineering AC3E, Valparaíso, Chile.
Email:patricio.orio@uv.cl Introduction: Understanding how collective functions emerge in the brain is a significant challenge in neuroscience, as emergent behaviors (or their disruptions) are believed to underlie consciousness, behavioral outputs, and brain disorders. Information theory provides tools that can be used to measure high-order interactions (HOIs): statistical structures that are present in a group of variables but not in pair-wise interactions. It is unknown how these measurable emergent behaviors can originate and be sustained, contributing to information processing. To this end, we study the self-emergence of HOIs in RNNs that undergo plasticity to learn to perform cognitive tasks of different complexity.Methods: We trained continuous-time RNNs to perform one of the following tasks: Go/NoGo, Negative patterning, Temporal Discrimination, Context-dependent Decision making. After network training, a long duration input consisting of either noise or a series of task inputs was applied to evaluate the dynamics of the hidden layer. HOIs were evaluated using the O-info and S-info metrics implemented in the JIDT toolbox (1) using the KSG estimator, at different orders of interaction taking all combinations from 3 to 11 nodes. The dimension of the trajectory was assessed by the amount of variance explained by the first 5 PCA components. Graph metrics were employed to characterize the weight matrix of the hidden layer.Results: Training causes the dynamics of hidden layer to show HOIs with high redundancy at higher orders of interaction and synergistic interactions measured at lower order (i.e. smaller groups). More synergy is observed after training with the compound, context-dependent task, while more redundancy is originated by the simpler Go/NoGo. The existence of synergistic interactions is also correlated with more complex dynamics as suggested by a trajectory of higher dimension. Finally, we tested different pruning procedures to obtain sparser weight matrices, without observing an effect on the HOIs measured. Discussion: Our results show that the type of task that a network is solving determines a different pattern of HOIs, suggesting that complex tasks induce the emergence of synergistic interactions. In the future, it will be of interest to study how the HOIs emerge in networks trained to solve multiple tasks, and how the HOIs relate to the resilience of the network to noisy or faulty conditions. In addition, more study cases will be explored to assess whether the synergistic nature of HOIs always correlates with trajectories of higher dimension.
Acknowledgements This work is funded by Fondecyt grant 1241469 (ANID, Chile). AC3E is funded by Basal grant AFB240002 (ANID, Chile) References (1)Joseph T. Lizier, "JIDT: An information-theoretic toolkit for studying the dynamics of complex systems", Frontiers in Robotics and AI 1:11, 2014; doi:10.3389/frobt.2014.00011 (pre-print: arXiv:1408.3270)
