1School of Psychology, University of Sheffield, Sheffield, UK 2The Neuroscience Institute, University of Sheffield, Sheffield, UK 3Insigneo Institute for in silico Medicine, University of Sheffield, Sheffield, UK
*Email: h.toutounji@sheffield.ac.uk
Introduction People often encounter decisions that may lead to gains or losses. As they learn the value of available choices, whether positive in the case of gains and rewards, or negative in the case of losses, people need also to balance between gathering information (exploration) and capitalising on their current knowledge (exploitation). Exploration itself can either be random or directed towards reducing uncertainty[1]. While psychiatric traits like impulsivity are known to influence exploration[2], there is no account on how this influence relates to the learning context such as gain or loss. This study investigates how impulsivity modulates different exploration strategies and decision performance in a context-dependent manner.
Methods Human participants (N = 115) completed a two-armed bandit task where in different rounds they can win or lose points. Each arm delivered or cost either a fixed or a variable (uncertain) number of points. Learning and exploration behaviour was modelled using reinforcement learning. Crucially, uncertainty in each trial was incorporated into the model using the Kalman filter for the learning process and a hybrid choice model with three components[1]: value-dependent random exploration, and uncertainty-dependent random and directed exploration. Impulsivity was measured using the UPPS-P Impulsive Behaviour Scale[3]. A general linear mixed model quantified the interaction between impulsivity, exploration strategies, and context.
Results Participants engaged in significantly more value-dependent random exploration and less uncertainty-dependent random exploration in the loss context compared to the win context. However, impulsive individuals showed the opposite trend, relying significantly more on uncertainty-dependent random exploration in the loss context. Impulsivity was also positively linked to task performance in loss contexts, suggesting that impulsive individuals adaptively leveraged random exploration to manage uncertainty. In other words, impulsive individuals engaged in more uncertainty-dependent random exploration, especially when facing losses, and benefited from this strategy.
Discussion
Our findings highlight the adaptive role of impulsivity in uncertain environments, particularly when leading to losses. Impulsive individuals appear to be more sensitive to total uncertainty, effectively using random exploration to improve performance. These results contrast with prior studies that emphasise the maladaptive nature of impulsivity, suggesting instead its potential benefits in high-stakes loss contexts. Our findings contradict prospect theory[4], showing more risk aversion to losses than gains. Further, this win-loss asymmetry is amplified in impulsive individuals, highlighting the importance of taking individual traits into account when developing theories of human learning and decision making.
Acknowledgements This work was funded by the University of Sheffield. References 1. doi: 10.1016/j.cognition.2017.12.014 2. doi: 10.1038/s41467-022-31918-9 3. doi: 10.3389/fpsyt.2019.00139
