Abstract
One of the most critical properties of a versatile intelligent agent is its capacity to adapt autonomously to any change in the environment without overly complexifying its cognitive architecture. In this paper, we propose that understanding the role of neuromodulation in the brain is of central interest for this purpose. More precisely, we propose that an accurate estimation of the nature of uncertainty present in the environment is performed by specific brain regions and broadcast throughout the cerebral network by neuromodulators, resulting in appropriate changes in cerebral functioning and learning modes. Better understanding the principles of these mechanisms in the brain might tremendously inspire the field of Artificial General Intelligence. The original contribution of this paper is to relate the four major neuromodulators to four fundamental dimensions of uncertainty.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychol. Rev. 111(4), 1036–1060 (2004). http://dx.doi.org/10.1037/0033-295x.111.4.1036
Aston-Jones, G., Rajkowski, J., Kubiak, P.: Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience 80(3), 697–715 (1997).http://dx.doi.org/10.1016/s0306-4522(97)00060–2
Aston-Jones, G., Cohen, J.D.: An integrative theory of Locus Coeruleus-Norepinephrine function: adaptive gain and optimal performance. Ann. Rev. Neurosci. 28(1), 403–450 (2005). http://dx.doi.org/10.1146/annurev.neuro.28.061604.135709
Aston-Jones, G., Rajkowski, J., Cohen, J.: Role of locus coeruleus in attention and behavioral flexibility. Biol. Psychiatry 46(9), 1309–1320 (1999). http://dx.doi.org/10.1016/s0006-3223(99)00140–7
Balasubramani, P.P., Chakravarthy, V.S., Ravindran, B., Moustafa, A.A.: An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning. Front. Comput. Neurosci. 8 (2014)
Berger-Tal, O., Nathan, J., Meron, E., Saltz, D.: The exploration-exploitation dilemma: a multidisciplinary framework. PLoS ONE 9(4), 1–8 (2014)
Berridge, K.C., Robinson, T.E.: What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Rev. 28(3), 309–369 (1998). http://dx.doi.org/10.1016/s0165-0173(98)00019–8
Bouret, S., Sara, S.J.: Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28(11), 574–582 (2005). http://dx.doi.org/10.1016/j.tins.2005.09.002
Calandreau, L., Trifilieff, P., Mons, N., Costes, L., Marien, M., Marighetto, A., Micheau, J., Jaffard, R., Desmedt, A.: Extracellular hippocampal acetylcholine level controls amygdala function and promotes adaptive conditioned emotional response. J. Neurosci. Official J. Soc. Neurosci. 26(52), 13556–13566 (2006)
Carrere, M., Alexandre, F.: A pavlovian model of the amygdala and its influence within the medial temporal lobe. Front. Syst. Neurosci. 9(41) (2015)
Cohen, J.D., McClure, S.M., Yu, A.J.: Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. Ser. B, Biol. Sci. 362(1481), 933–942 (2007). http://dx.doi.org/10.1098/rstb.2007.2098
Cools, R., Nakamura, K., Daw, N.D.: Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 36(1), 98–113 (2011). http://dx.doi.org/10.1038/npp.2010.121
Daw, N.D., Kakade, S., Dayan, P.: Opponent interactions between serotonin and dopamine. Neural Netw. 15(4–6), 603–616 (2002).http://dx.doi.org/10.1016/s0893-6080(02)00052–7
Daw, N.D., O’Doherty, J.P., Dayan, P., Seymour, B., Dolan, R.J.: Cortical substrates for exploratory decisions in humans. Nature 441(7095), 876–879 (2006). http://dx.doi.org/10.1038/nature04766
Dayan, P.: Twenty-five lessons from computational neuromodulation. Neuron 76(1), 240–256 (2012). http://dx.doi.org/10.1016/j.neuron.2012.09.027
Doya, K.: Metalearning and neuromodulation. Neural Netw. 15(4–6), 495–506 (2002). http://dx.doi.org/10.1016/s0893-6080(02)00044-8
Doya, K., Samejima, K., Katagiri, K., Kawato, M.: Multiple model-based reinforcement learning. Neural Comp. 14(6), 1347–1369 (2002). http://neco.mitpress.org/cgi/content/abstract/14/6/1347
Friston, K.: Functional integration and inference in the brain. Prog. Neurobiol. 68(2), 113–143 (2002). http://view.ncbi.nlm.nih.gov/pubmed/12450490
Grossberg, S.: Adaptive resonance theory: how a brain learns to consciously attend, learn, and recognize a changing world. Neural Netw. 37, 1–47 (2013). http://dx.doi.org/10.1016/j.neunet.2012.09.017
Haber, S., Fudge, J., McFarland, N.: Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20(6), 2369–2382 (2000)
Humphries, M.D., Khamassi, M., Gurney, K.: Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. Front. Neurosci. 6(9), 1–14 (2012)
Mannella, F., Gurney, K., Baldassarre, G.: The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis. Front. Behav. Neurosci. 7 (2013)
McClelland, J.L., McNaughton, B.L., O’Reilly, R.C.: Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102(3), 419–457 (1995)
McClure, S., Gilzenrat, M., Cohen, J.: An exploration-exploitation model based on norepinepherine and dopamine activity. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, pp. 867–874. MIT Press (2006). http://www.csbmb.princeton.edu/~smcclure/pdf/MGC_NIPS.pdf
Niv, Y.: Cost, benefit, tonic, phasic: what do response rates tell us about dopamine and motivation. Annals of the New York Academy of Sciences 1104(1), 357–376 (2007). http://dx.doi.org/10.1196/annals.1390.018
Pauli, W.M., Hazy, T.E., O’Reilly, R.C.: Expectancy, ambiguity, and behavioral flexibility: separable and complementary roles of the orbital frontal cortex and amygdala in processing reward expectancies. J. Cogn. Neurosci. 24(2), 351–366 (2011). http://dx.doi.org/10.1162/jocn_a_00155
Pauli, W.M., O’Reilly, R.C.: Attentional control of associative learning-a possible role of the central cholinergic system. Brain Res. 1202, 43–53 (2008)
Sara, S.J., Bouret, S.: Orienting and reorienting: the locus coeruleus mediates cognition through arousal. Neuron 76(1), 130–141 (2012). http://dx.doi.org/10.1016/j.neuron.2012.09.011
Schultz, W.: Predictive reward signal of dopamine neurons. J. Neurophysiol. 80(1), 1–27 (1998). http://jn.physiology.org/content/80/1/1
Silver, D., Yang, Q., Li, L.: Lifelong machine learning systems: beyond learning algorithms. In: AAAI Spring Symposium Series (2013)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Trappenberg, T.P.: Fundamentals of Computational Neuroscience, 2nd edn. Oxford University Press, Oxford (2009)
Yu, A.J., Dayan, P.: Uncertainty, neuromodulation, and attention. Neuron 46(4), 681–692 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Alexandre, F., Carrere, M. (2016). Modeling Neuromodulation as a Framework to Integrate Uncertainty in General Cognitive Architectures. In: Steunebrink, B., Wang, P., Goertzel, B. (eds) Artificial General Intelligence. AGI 2016. Lecture Notes in Computer Science(), vol 9782. Springer, Cham. https://doi.org/10.1007/978-3-319-41649-6_33
Download citation
DOI: https://doi.org/10.1007/978-3-319-41649-6_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-41648-9
Online ISBN: 978-3-319-41649-6
eBook Packages: Computer ScienceComputer Science (R0)