Abstract
Decision making in both animals and humans is influenced by the anticipation of reward and/or punishment. Little is known about how reward and punishment interact in the context of decision making. The Avoidance–Reward Conflict (ARC) Task is a new paradigm that varies the degree of reward and the probability of punishment in a single paradigm that can be used in both non-human primates (NHPs) and humans. This study examined the behavioral pattern in the ARC task in both NHPs and humans. Two adult male NHPs (macaca mulatta) and 20 healthy human volunteers (12 females) participated in the ARC task. NHPs and humans perform similarly on the ARC task. With a high probability of punishment (an aversive air puff to the eye), both NHPs and humans are more likely to forgo reward if it is small or medium magnitude than when it is large. Both NHPs and humans perform similarly on the same behavioral task suggesting the reliability of animal models in predicting human behavior.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adida M, Clark L, Pomietto P et al (2008) Lack of insight may predict impaired decision making in manic patients. Bipolar Disord 10:829–837. doi:10.1111/j.1399-5618.2008.00618.x
Amorapanth P, LeDoux JE, Nader K (2000) Different lateral amygdala outputs mediate reactions and actions elicited by a fear-arousing stimulus. Nat Neurosci 3:74–79
Apicella P, Ljungberg T, Scarnati E, Schultz W (1991a) Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res 85:491–500
Apicella P, Scarnati E, Schultz W (1991b) Tonically discharging neurons of monkey striatum respond to preparatory and rewarding stimuli. Exp Brain Res 84:672–675
Asaad WF, Eskandar EN (2008a) Achieving behavioral control with millisecond resolution in a high-level programming environment. J Neurosci Methods 173:235–240. doi:10.1016/j.jneumeth.2008.06.003
Asaad WF, Eskandar EN (2008b) A flexible software tool for temporally-precise behavioral control in Matlab. J Neurosci Methods 174:245–258. doi:10.1016/j.jneumeth.2008.07.014
Asaad WF, Eskandar EN (2011) Encoding of both positive and negative reward prediction errors by neurons of the primate lateral prefrontal cortex and caudate nucleus. J Neurosci 31:17772–17787. doi:10.1523/JNEUROSCI.3793-11.2011
Baeg EH, Kim YB, Jang J et al (2001) Fast spiking and regular spiking neural correlates of fear conditioning in the medial prefrontal cortex of the rat. Cereb Cortex 11:441–451
Balleine BW, O’Doherty JP (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35:48–69. doi:10.1038/npp.2009.131
Barberini CL, Morrison SE, Saez A et al (2012) Complexity and competition in appetitive and aversive neural circuits. Front Neurosci 6:170. doi:10.3389/fnins.2012.00170
Bechara A, Damasio AR, Damasio H, Anderson SW (1994) Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50:7–15
Boileau I, Assaad J-M, Pihl RO et al (2003) Alcohol promotes dopamine release in the human nucleus accumbens. Synapse 49:226–231. doi:10.1002/syn.10226
Botvinick M, Nystrom LE, Fissell K et al (1999) Conflict monitoring versus selection-for-action in anterior cingulate cortex. Nature 402:179–181. doi:10.1038/46035
Breiter HC, Aharon I, Kahneman D et al (2001) Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30:619–639
Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009) Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci USA 106:4894–4899. doi:10.1073/pnas.0811507106
Bush G, Vogt BA, Holmes J et al (2002) Dorsal anterior cingulate cortex: a role in reward-based decision making. Proc Natl Acad Sci USA 99:523–528. doi:10.1073/pnas.012470999
Carter CS, Braver TS, Barch DM et al (1998) Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280:747–749
Chandler RA, Wakeley J, Goodwin GM, Rogers RD (2009) Altered risk-aversion and risk-seeking behavior in bipolar disorder. Biol Psychiatry 66:840–846. doi:10.1016/j.biopsych.2009.05.011
Delgado MR, Nystrom LE, Fissell C et al (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84:3072–3077
Delgado MR, Jou RL, LeDoux JE, Phelps EA (2009) Avoiding negative outcomes: tracking the mechanisms of avoidance learning in humans during fear conditioning. Front Behav Neurosci 3:33. doi:10.3389/neuro.08.033.2009
Drevets WC, Gautier C, Price JC et al (2001) Amphetamine-induced dopamine release in human ventral striatum correlates with euphoria. Biol Psychiatry 49:81–96
Elliott R, Friston KJ, Dolan RJ (2000) Dissociable neural responses in human reward systems. J Neurosci 20:6159–6165
Figee M, Vink M, de Geus F et al (2011) Dysfunctional reward circuitry in obsessive–compulsive disorder. Biol Psychiatry 69:867–874. doi:10.1016/j.biopsych.2010.12.003
First MB, Spitzer RL, Gibbon M, Williams J (1995) Structured clinical interview for DSM-IV Axis I Disorders-Patient Edition (SCID-I/D, Version 2.0), Biometrics Research Department, New York
Gehring WJ, Willoughby AR (2002) The medial frontal cortex and the rapid processing of monetary gains and losses. Science 295:2279–2282. doi:10.1126/science.1066893
Gilmartin MR, McEchron MD (2005) Single neurons in the medial prefrontal cortex of the rat exhibit tonic and phasic coding during trace fear conditioning. Behav Neurosci 119:1496–1510. doi:10.1037/0735-7044.119.6.1496
Greenberg BDB, Malone DAD, Friehs GMG et al (2006) Three-year outcomes in deep brain stimulation for highly resistant obsessive–compulsive disorder. Neuropsychopharmacology 31:2384–2393. doi:10.1038/sj.npp.1301165
Gruber J (2011) A review and synthesis of positive emotion and reward disturbance in bipolar disorder. Clin Psychol Psychother 18:356–365. doi:10.1002/cpp.776
Guitart-Masip M, Fuentemilla L, Bach DR et al (2011) Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. J Neurosci 31:7867–7875. doi:10.1523/JNEUROSCI.6376-10.2011
Haber SN, Knutson B (2010) The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35:4–26. doi:10.1038/npp.2009.129
Haber SN, Kim K-S, Mailly P, Calzavara R (2006) Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J Neurosci 26:8368–8376. doi:10.1523/JNEUROSCI.0271-06.2006
Hayden BY, Heilbronner SR, Platt ML (2010) Ambiguity aversion in rhesus macaques. Front Neurosci. doi:10.3389/fnins.2010.00166
Hayden BY, Pearson JM, Platt ML (2011) Neuronal basis of sequential foraging decisions in a patchy environment. Nat Neurosci 14:933–939. doi:10.1038/nn.2856
Hikosaka O, Bromberg-Martin E, Hong S, Matsumoto M (2008) New insights on the subcortical representation of reward. Curr Opin Neurobiol 18:203–208. doi:10.1016/j.conb.2008.07.002
Holroyd CB, Coles MGH (2002) The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev 109:679–709
Holtzheimer PE, Kelley ME, Gross RE et al (2012) Subcallosal cingulate deep brain stimulation for treatment-resistant unipolar and bipolar depression. Arch Gen Psychiatry 69:150–158. doi:10.1001/archgenpsychiatry.2011.1456
Hoover WB, Vertes RP (2007) Anatomical analysis of afferent projections to the medial prefrontal cortex in the rat. Brain Struct Funct 212:149–179. doi:10.1007/s00429-007-0150-4
Irwin W, Davidson RJ, Lowe MJ et al (1996) Human amygdala activation detected with echo-planar functional magnetic resonance imaging. Neuroreport 7:1765–1769
Kalisch R, Korenfeld E, Stephan KE et al (2006) Context-dependent human extinction memory is mediated by a ventromedial prefrontal and hippocampal network. J Neurosci 26:9503–9511
Kerns JG, Cohen JD, MacDonald AW et al (2004) Anterior cingulate conflict monitoring and adjustments in control. Science 303:1023–1026. doi:10.1126/science.1089910
Kim H, Shimojo S, O’Doherty JP (2006) Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. PLoS Biol 4:e233. doi:10.1371/journal.pbio.0040233
Knutson B, Adams CM, Fong GW, Hommer D (2001) Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21:RC159
LaBar KS, Gatenby JC, Gore JC et al (1998) Human amygdala activation during conditioned fear acquisition and extinction: a mixed-trial fMRI study. Neuron 20:937–945
LeDoux J (2012a) Rethinking the emotional brain. Neuron 73:653–676. doi:10.1016/j.neuron.2012.02.004
LeDoux JE (2012b) Evolution of human emotion: a view through fear. Prog Brain Res 195:431–442. doi:10.1016/B978-0-444-53860-4.00021-0
Mataix-Cols D, Wooderson S, Lawrence N et al (2004) Distinct neural correlates of washing, checking, and hoarding symptom dimensions in obsessive–compulsive disorder. Arch Gen Psychiatry 61:564–576. doi:10.1001/archpsyc.61.6.564
Matsumoto M, Hikosaka O (2009) Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459:837–841. doi:10.1038/nature08028
Mian MK, Campos M, Sheth SA, Eskandar EN (2010) Deep brain stimulation for obsessive–compulsive disorder: past, present, and future. Neurosurg Focus 29:E10–E10. doi:10.3171/2010.4.FOCUS10107
Milad MR, Rauch SL (2012) Obsessive–compulsive disorder: beyond segregated cortico–striatal pathways. Trends Cogn Sci (Regul Ed) 16:43–51. doi:10.1016/j.tics.2011.11.003
Milad MR, Quirk GJ, Pitman RK et al (2007) A role for the human dorsal anterior cingulate cortex in fear expression. Biol Psychiatry 62:1191–1194. doi:10.1016/j.biopsych.2007.04.032
Milad MR, Pitman RK, Ellis CB et al (2009) Neurobiological basis of failure to recall extinction memory in posttraumatic stress disorder. Biol Psychiatry 66:1075–1082. doi:10.1016/j.biopsych.2009.06.026
Milad MR, Furtak SC, Greenberg JL et al (2013) Deficits in conditioned fear extinction in obsessive–compulsive disorder and neurobiological changes in the fear circuit. JAMA Psychiatry 70:608–618. doi:10.1001/jamapsychiatry.2013.914
Mirenowicz J, Schultz W (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72:1024–1027
Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449–451. doi:10.1038/379449a0
Mischel W, Grusec J (1967) Waiting for rewards and punishments: effects of time and probability on choice. J Pers Soc Psychol 5:24–31
Mischel W, Metzner R (1962) Preference for delayed reward as a function of age, intelligence, and length of delay interval. J Abnorm Soc Psychol 64:425–431
Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936–1947
O’Doherty J, Dayan P, Schultz J et al (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452–454. doi:10.1126/science.1094285
Paton JJ, Belova MA, Morrison SE, Salzman CD (2006) The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439:865–870. doi:10.1038/nature04490
Phelps EA, LeDoux JE (2005) Contributions of the amygdala to emotion processing: from animal models to human behavior. Neuron 48:175–187. doi:10.1016/j.neuron.2005.09.025
Phelps EA, Delgado MR, Nearing KI, LeDoux JE (2004) Extinction learning in humans: role of the amygdala and vmPFC. Neuron 43:897–905
Phillips ML, Marks IM, Senior C et al (2000) A differential neural response in obsessive–compulsive disorder patients with washing compared with checking symptoms to disgust. Psychol Med 30:1037–1050
Prescott MJ, Brown VJ, Flecknell PA et al (2010) Refinement of the use of food and fluid control as motivational tools for macaques used in behavioural neuroscience research: report of a Working Group of the NC3Rs. J Neurosci Methods 193:167–188. doi:10.1016/j.jneumeth.2010.09.003
Quirk GJ, Mueller D (2008) Neural mechanisms of extinction learning and retrieval. Neuropsychopharmacology 33:56–72. doi:10.1038/sj.npp.1301555
Rauch SLS, Jenike MAM, Alpert NMN et al (1994) Regional cerebral blood flow measured during symptom provocation in obsessive–compulsive disorder using oxygen 15-labeled carbon dioxide and positron emission tomography. Arch Gen Psychiatry 51:62–70. doi:10.1001/archpsyc.1994.03950010062008
Saxena S, Brody AL, Maidment KM et al (2004) Cerebral glucose metabolism in obsessive–compulsive hoarding. Am J Psychiatry 161:1038–1048
Schlund MW, Magee S, Hudgins CD (2011) Human avoidance and approach learning: evidence for overlapping neural systems and experiential avoidance modulation of avoidance neurocircuitry. Behav Brain Res 225:437–448. doi:10.1016/j.bbr.2011.07.054
Schultz WW (2000) Multiple reward signals in the brain. Nat Rev Neurosci 1:199–207. doi:10.1038/35044563
Schultz W (2006) Behavioral theories and the neurophysiology of reward. Annu Rev Psychol 57:87–115. doi:10.1146/annurev.psych.56.091103.070229
Schultz W, Apicella P, Scarnati E, Ljungberg T (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12:4595–4610
Sescousse G, Caldú X, Segura B, Dreher J-C (2013) Processing of primary and secondary rewards: a quantitative meta-analysis and review of human functional neuroimaging studies. Neurosci Biobehav Rev 37:681–696. doi:10.1016/j.neubiorev.2013.02.002
Sierra-Mercado D, Corcoran KA, Lebrón-Milad K, Quirk GJ (2006) Inactivation of the ventromedial prefrontal cortex reduces expression of conditioned fear and impairs subsequent recall of extinction. Eur J Neurosci 24:1751–1758. doi:10.1111/j.1460-9568.2006.05014.x
Sierra-Mercado D, Padilla-Coreano N, Quirk GJ (2011) Dissociable roles of prelimbic and infralimbic cortices, ventral hippocampus, and basolateral amygdala in the expression and extinction of conditioned fear. Neuropsychopharmacology 36:529–538. doi:10.1038/npp.2010.184
Small DM, Zatorre RJ, Dagher A et al (2001) Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 124:1720–1733
Sotres-Bayon F, Sierra-Mercado D, Pardilla-Delgado E, Quirk GJ (2012) Gating of fear in prelimbic cortex by hippocampal and amygdala inputs. Neuron 76:804–812. doi:10.1016/j.neuron.2012.09.028
Townsend J, Altshuler LL (2012) Emotion processing and regulation in bipolar disorder: a review. Bipolar Disord 14:326–339. doi:10.1111/j.1399-5618.2012.01021.x
Vidal-Gonzalez I, Vidal-Gonzalez B, Rauch SL, Quirk GJ (2006) Microstimulation reveals opposing influences of prelimbic and infralimbic cortex on the expression of conditioned fear. Learn Mem 13:728–733. doi:10.1101/lm.306106
Weiskrantz L (1956) Behavioral changes associated with ablation of the amygdaloid complex in monkeys. J Comp Physiol Psychol 49:381–391. doi:10.1037/h0088009
Yechiam E, Hayden EP, Bodkins M et al (2008) Decision making in bipolar disorder: a cognitive modeling approach. Psychiatry Res 161:142–152. doi:10.1016/j.psychres.2007.07.001
Zaghloul KA, Blanco JA, Weidemann CT et al (2009) Human substantia nigra neurons encode unexpected financial rewards. Science 323:1496–1499. doi:10.1126/science.1167342
Acknowledgments
This study was supported by Conte Project Grant (P50-MH086400) to ENE and DDD, a supplement for underrepresented minorities from DA026297 and a Young Investigator Grant from the Brain and Behavior Research Foundation (NARSAD) to DS-M, a Northeastern University Provost Award and Conte Project Summer Fellowship to EJM, and an HHMI fellowship to CAE. We thank the animal care staff of the Center for Comparative Medicine.
Conflict of interest
Dr. Deckersbach’s research has been funded by NIMH, NARSAD, TSA, IOCDF, Tufts University and the Depression and Bipolar Disorder Alternative Treatment Foundation. He has received honoraria, consultation fees and/or royalties from the MGH Psychiatry Academy, BrainCells Inc., Systems Research and Applications Corporation, Boston University, the Catalan Agency for Health Technology Assessment and Research, the National Association of Social Workers Massachusetts, the Massachusetts Medical Society, Tufts University, NIDA, NIMH, and Oxford University Press. He has also participated in research funded by NIH, NIA, AHRQ, Janssen Pharmaceuticals, The Forest Research Institute, Shire Development Inc., Medtronic, Cyberonics, Northstar, and Takeda. Dr. Dougherty has received honoraria from Reed Elsevier and Medtronic. He has also served as a consult to Medtronic and received grant/research support from Medtronic, Eli Lilly, Cyberonics, and Roche. All other authors report no financial interests or potential conflicts of interest.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Sierra-Mercado, D., Deckersbach, T., Arulpragasam, A.R. et al. Decision making in avoidance–reward conflict: a paradigm for non-human primates and humans. Brain Struct Funct 220, 2509–2517 (2015). https://doi.org/10.1007/s00429-014-0796-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00429-014-0796-7