P013 Targeted striatal activation and reward uncertainty promote exploration in mice
Jyotika Bahuguna1*, Julia Badyna2,3,Krista A. Bond4, Eric A. Yttri3*, Jonathan E. Rubin3,5*, Timothy D.
Verstynen1,3*
1 LNCA, Faculte de Psychologie, Universite de Strasbourg, Strasbourg, France
2Department of Biological Sciences, Carnegie Mellon University, Pittsburgh, PA, US
3Center for the Neural Basis of Cognition, Pittsburgh, PA, USA
4Psychiatry, Yale, New Haven, CT
5Department of Mathematics, University of Pittsburgh, Pittsburgh, PA, USA
*Email: jyotika.bahuguna@gmail.com , timothyv@andrew.cmu.edu , eyttri@andrew.cmu.edu , jonrubin@pitt.edu
Introduction
Decision policies, which moderate what choices are made and how fast they are executed, are influenced by contextual factors such as uncertainty about reward or sudden changes in action-outcome contingencies. To help resolve the mechanisms involved, we explored a critical neural substrate,namely dSPNs and iSPNs in the striatum, that are known to modulate both choice and vigour aspect of the decision making [1, 2]. We also explored, if the modulation of decision policies were aimed at optimizing reward rate.
Methods
We manipulated two forms of contextual uncertainty -- relative difference in reward probability between options (conflict), and unexpected changes in action-outcome contingencies (volatility)-- as D1-cre and A2A-cre mice underwent optogenetic stimulation of striatal direct pathway (dSPNs) or indirect pathway spiny projection neurons (iSPNs).The trial-by-trial behavioral outcomes (choice and decision times) were fit to a hierarchical drift diffusion model (DDM) [3], using a Bayesian delta rule model [4,5] as a trialwise regressor on DDM parameters. The values of DDM parameters obtained, in particular drift rate and boundary height, provided an estimate of the instantaneous decision policy on each trial.
Results
We found that during stable environmental periods unstimulated mice maintained a high drift rate and high boundary height, reflecting relatively exploitative decision strategies (Fig1B). When action-outcome mappings switched, both drift rate and boundary height quickly dropped, reflecting a shift to fast exploratory decision policies(Fig1B). These modulations in decision policies reflect a drive to maintain immediate reward rate (Fig1A). We see the same shift in decision policies as a result of increase in conflict, i.e as the reward probabilities become uncertain, the trajectories shift deeper into the exploration regime and this also reflects the drive to maintain reward rate(Fig1C). iSPN stimulation shifted animals into overall more exploratory states, with lower drift rates, but altered the response to change points such that boundary height increased, instead of decreasing (Fig1D). We characterized this regime as a slow exploration regime. dSPN stimulation did not seem to affect decision policies.
Discussion
These results suggest that reward and environmental uncertainty modulates the decision policy to be more exploratory and the modulation reflects the drive to maintain the reward rate. Morever, amplifying striatal indirect pathway activity fundamentally shifts how animals change decision policies in response to environmental feedback, promoting a slowing of the exploration strategies that are adopted.
Figure 1. Figure1 A) DDM manifolds showing how accuracy, reaction times and reward rate change with change in DDM parameters. B) Mice show exploitative policy at stable conditions but switch to exploration during contingency changes C) High conflict pushes the behavior towards exploration regime D) iSPN stimulation imposes a slow exploration policy on mice whereas dSPN stimulation does not have a significan
Acknowledgements
JB is supported by ANR-CPJ-2024DRI00039. TV, JBad, JBah, EAY and JER are partly supported by NIH awards R01DA053014 and R01DA059993 as part of the CRCNS program. JER is partly supported by NIH award R01NS125814, also part of the CRCNS program.
References
[1] Freeze, B. S., Kravitz, A. V., Hammack, N., Berke, J. D., & Kreitzer, A. C. (2013). https://doi.org/10.1523/JNEUROSCI.1278-13.2013
[2] Geddes, C. E., Li, H., & Jin, X. (2018). https://doi.org/10.1016/j.cell.2018.06.012
[3] Wiecki, T. V., Sofer, I., & Frank, M. J. (2013). https://doi.org/10.3389/fninf.2013.00014
[4] Nassar, M. R., Wilson, R. C., Heasly, B., & Gold, J. I. (2010). https://doi.org/10.1523/JNEUROSCI.0822-10.2010
[5] Vaghi, M. M., Luyckx, F., Sule, A., Fineberg, N. A., Robbins, T. W., & De Martino, B. (2017). https://doi.org/10.1016/j.neuron.2017.09.006
