Abstract
We present a Reinforcement Learning (RL)-based model of serotonin which tries to reconcile some of the diverse roles of the neuromodulator. The proposed model uses a novel formulation of utility function, which is a weighted sum of the traditional value function and the risk function. Serotonin is represented by the weightage, α, used in this combination. The model is applied to three different experimental paradigms: 1) bee foraging behavior, which involves decision making based on risk, 2) temporal reward prediction task, in which serotonin (α) controls the time-scale of reward prediction, and 3) reward/punishment prediction task, in which punishment prediction error depends on serotonin levels. The three diverse roles of serotonin – in time-scale of reward prediction, risk modeling, and punishment prediction – is explained within a single framework by the model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Schultz, W., Dayan, P., Montague, P.R.: A neural substrate of prediction and reward. Science 275(5306), 1593–1599 (1997)
Cools, R., Nakamura, K., Daw, N.D.: Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 36(1), 98–113 (2011)
Tops, M., et al.: Serotonin: modulator of a drive to withdraw. Brain Cogn. 71(3), 427–436 (2009)
Rogers, R.D.: The roles of dopamine and serotonin in decision making: evidence from pharmacological experiments in humans. Neuropsychopharmacology 36(1), 114–132 (2011)
Schultz, W.: Dopamine signals for reward value and risk: basic and recent data. Behav. Brain Funct. 6, 24 (2010)
Daw, N.D., Kakade, S., Dayan, P.: Opponent interactions between serotonin and dopamine. Neural Netw. 15(4-6), 603–616 (2002)
Doya, K.: Metalearning and neuromodulation. Neural Netw. 15(4-6), 495–506 (2002)
Tanaka, S.C., et al.: Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum. PLoS One 2(12), e1333 (2007)
Bell, D.E.: Risk, return and utility. Management Science 41, 23–30 (1995)
Montague, P.R., et al.: Bee foraging in uncertain environments using predictive hebbian learning. Nature 377(6551), 725–728 (1995)
Cools, R., Robinson, O.J., Sahakian, B.: Acute tryptophan depletion in healthy volunteers enhances punishment prediction but does not affect reward prediction. Neuropsychopharmacology 33(9), 2291–2299 (2008)
d’Acremont, M., et al.: Neural correlates of risk prediction error during reinforcement learning in humans. Neuroimage 47(4), 1929–1939 (2009)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. Adaptive Computations and Machine Learning (1998)
Kahneman, D., Tversky, A.: Prospect theory: an analysis of decision under risk. Econometrica 47, 263–292 (1979)
Robinson, O.J., Cools, R., Sahakian, B.J.: Tryptophan depletion disinhibits punishment but not reward prediction: implications for resilience. Psychopharmacology (Berl) 219(2), 599–605 (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pragathi Priyadharsini, B., Ravindran, B., Srinivasa Chakravarthy, V. (2012). Understanding the Role of Serotonin in Basal Ganglia through a Unified Model. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds) Artificial Neural Networks and Machine Learning – ICANN 2012. ICANN 2012. Lecture Notes in Computer Science, vol 7552. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33269-2_59
Download citation
DOI: https://doi.org/10.1007/978-3-642-33269-2_59
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33268-5
Online ISBN: 978-3-642-33269-2
eBook Packages: Computer ScienceComputer Science (R0)