P014 Investigating the mechanisms underpinning behavioral resilience using an extended Multi-agent Reinforcement learning model
Chirayush Mohanty*,Priya Gole*, Sanket Houde, Aadya Umrao, Pragathi Priyadharsini Balasubramani
Translational Neuroscience and Technology Labs, IIT Kanpur, India
*co-first authors
Email: cmohanty21@iitk.ac.in
Introduction:Reinforcement learning models of choice behavior specifically focuses on expected reinforcement based learning and decision making, and to our knowledge, the models haven’t explored well the reward maximization strategy that is controlled by energy constraints and social constraints, and if subjective policy relates to someone’s ability to adapt well during difficult times. In Particular, we asked whether participant’s risk taking, resource (intake of food energy) influence on decisions, or social conformity bias, can explain their resilience levels.
Methods:We here for the first time performed a repeated experimental design, before and after lunch period, on school kids of age 13-15 years old (N=32, males = 21) followed by computational modeling to understand the effects of risk taking ability, food energy resource modulation, and conformity with partner’s choices, in our participants. The task tested the participant’s trade off in maximization of reward magnitude versus the frequency (loss/gains) as in Balasubramani et al., (2022). We also obtained information of participant’s personality through the Big 5 questionnaire, adapted for participant’s age. We built a Multi-agent reinforcement learning (MARL) model to investigate the relationship between the meta-parameters: exploration index, social conformity bias computed based on marginal value theorem, and resource level index, in explaining the choice dynamics.
Results:We found that the extent of reward magnitude maximization of choices correlated (Spearman r=0.37, p=0.035) with resilience, and the social conformity (r = -0.27, p = 0.12) was fairly related to resilience as well. Particularly the extent of choosing the option with frequent losses negatively related to openness and extraversion (p<0.001), while the extent of choosing min expected reward with max risk related to neuroticism (p=0.001). Our MARL model was fit to capture the reward maximization and social conformity behavior, and it provided a population exploration index of 0.85± 0.12 across blocks, and a social conformity or influential bias of 0.22±0.83 (0±0.82) in the competitive (cooperative) block, respectively.
Discussion:Our MARL model finds that increased resilience in our population may be explained by two distinct patterns and were block dependent: The social bias didn’t seem to matter for relating to resilience in the cooperation block, rather the higher exploration index related to resilience levels. Whereas in the competitive block, resilience was exhibited by those who conform to other’s values and explore less, or those who do not conform with others but explore more. Furthermore, the resilience levels were positively related to the social conformity bias measures, and interestingly, we find that increase of resource availability post lunch specifically increased the extent of social bias.
Figure 1.
Acknowledgements
We are thankful to the Kendriya Vidyalaya school at IIT Kanpur, Principal R.C. Pandey, and all supporting teachers for giving us the permission and assisting us to conduct this study.
References
1.Balasubramani, P.P*., Walke, A., Grennan, G., Purpura, S., Perley, A., Ramanathan, D., Coleman, T., & Mishra, J. (2022). Simultaneous gut-brain electrophysiology shows cognition and satiety specific coupling. Sensors, 22(23), 9242. https://doi.org/10.3390/s22239242 *corresponding
2.Balasubramani, P. P*., Diaz-Delgado, J., Grennan, G., Alim, F., Zafar-Khan, M., Maric, V., ... & Mishra, J*. (2022). Distinct neural activations correlate with maximization of reward magnitude versus frequency. Cerebral Cortex, 2022;, bhac482, https://doi.org/10.1093/cercor/bhac482 *corresponding
