P315 Towards brain scale simulations using NEST GPU
José Villamar*1,2, Gianmarco Tiddia3, Luca Sergi3,4, Pooja Babu1,5, Luca Pontisso6, Francesco Simula6, Alessandro Lonardo6, Elena Pastorelli6, Pier Stanislao Paolucci6, Bruno Golosio3,4, Johanna Senk1,7
1Institute for Advanced Simulation (IAS-6), Jülich Research Centre, Jülich, Germany
2RWTH Aachen University, Aachen, Germany
3Istituto Nazionale di Fisica Nucleare, Sezione di Cagliari, Monserrato, Italy
4Dipartimento di Fisica, Università di Cagliari, Monserrato, Italy
5Simulation and Data Laboratory Neuroscience, Jülich Supercomputing Centre, Jülich Research Centre, Jülich, Germany
6Istituto Nazionale di Fisica Nucleare, Sezione di Roma, Roma, Italy
7Sussex AI, School of Engineering and Informatics, University of Sussex, Brighton, United Kingdom
*Email:j.villamar@fz-juelich.de
Introduction
Efficient simulation of large-scale spiking neuronal networks is important for neuroscientific research, and both the simulation speed and the time it takes to instantiate the network in computer memory are key factors. NEST GPU is a GPU-based simulator under the NEST Initiative written in CUDA-C++ that demonstrates high simulation speeds with models of various network sizes on single-GPU and multi-GPU systems [1,2,3]. On the path toward models of the whole brain, neuroscientists show an increasing interest in studying networks that are larger by several orders of magnitude. Here, we show the performance of our simulation technology with a scalable network model across multiple network sizes approaching human cortex magnitudes.
Methods
For this, we propose a novel method to efficiently instantiate large networks on multiple GPUs in parallel. Our approach relies on the deterministic initial state of pseudo-random number generators (PRNGs). While requiring synchronization of network construction directives between MPI processes and a small memory overhead, this approach enables dynamical neuron creation and connection at runtime. The method is evaluated through a two-population recurrently connected network model designed for benchmarking an arbitrary number of GPUs while maintaining first-order network statistics across scales.
Results
The benchmarking model was tested during an exclusive reservation of the LEONARDO Booster cluster. While keeping constant the number of neurons and incoming synapses to each neuron per GPU, we performed several simulation runs exploiting in parallel from 400 to 12,000 (full system) GPUs. Each GPU device contained approximately 281 thousand neurons and 3.1 billion synapses. Our results show network construction times of less than a second using the full system and stable dynamics across scales. At full system scale, the network model was composed of approximately 3.37 billion neurons and 37.96 trillion synapses (~25% human cortex).
Discussion
To conclude, our novel approach enabled network model instantiation of magnitudes nearing human cortex scale while keeping fast construction times, on average of 0.5s across trials. The stability of dynamics and performance across scales obtained in our model is a proof of feasibility paving the way for biologically more plausible and detailed brain scale models.
Acknowledgements
ISCRA for awarding access to the LEONARDO supercomputer (EuroHPC Joint Undertaking) via theBRAINSTAIN - INFN Scientific Committee 5 project, hosted by CINECA (Italy); HiRSE_PS, Helmholtz Platform for Research Software Engineering - Preparatory Study (2022-01-01 - 2023-12-31), the Horizon Europe Grant 101147319, Joint lab SMHB; FAIR CUPI53C22001400006 Italian PNRR grant.
References
●
https://doi.org/10.3389/fncom.2021.627620
●
https://doi.org/10.3389/fninf.2022.883333
●
https://doi.org/10.3390/app13179598
