Skip to main content
SpringerLink
Log in

Decision Making, Models

  • Living reference work entry
  • First Online:
Encyclopedia of Computational Neuroscience

  • 281 Accesses

Definition

Models of decision making attempt to describe, using stochastic differential equations which represent either neural activity or more abstract psychological variables, the dynamical process that produces a commitment to a single action/outcome as a result of incoming evidence that can be ambiguous as to the action it supports.

Detailed Description

Background

Decision making can be separated into four processes (Doya 2008):

  1. 1.

    Acquisition of sensory information to determine the state of the environment and the organism within it

  2. 2.

    Evaluation of potential actions (options) in terms of the cost and benefit to the organism given its belief about the current state

  3. 3.

    Selection of an action based on, ideally, an optimal trade-off between the costs and benefits

  4. 4.

    Use of the outcome of the action to update the costs and benefits associated with it

Models of the dynamics of decision making have focused on perceptual decisions with only two possible responses available. The term...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Balci F, Simen P, Niyogi R, Saxe A, Hughes JA, Holmes P, Cohen JD (2011) Acquisition of decision making criteria: reward rate ultimately beats accuracy. Atten Percept Psychophys 73:640–657

    Article  PubMed Central  PubMed  Google Scholar 

  • Barto AG (1994) Reinforcement learning control. Curr Opin Neurobiol 4:888–893

    Article  PubMed  CAS  Google Scholar 

  • Barto AG, Mahadevan S (2003) Recent advances in hierarchical reinforcement learning. Discrete Event Dyn Syst Theory Appl 13:343–379

    Google Scholar 

  • Beck JM, Ma WJ, Kiani R, Hanks T, Churchland AK, Roitman J, Shadlen MN, Latham PE, Pouget A (2008) Probabilistic population codes for Bayesian decision making. Neuron 60:1142–1152

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Bellman R (1957) Dynamic programming. Princeton University Press, Princeton

    Google Scholar 

  • Bogacz R, Brown E, Moehlis J, Holmes P, Cohen JD (2006) The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychol Rev 113:700–765

    Article  PubMed  Google Scholar 

  • Bogacz R, McClure SM, Li J, Cohen JD, Montague PR (2007a) Short-term memory traces for action bias in human reinforcement learning. Brain Res 1153:111–121

    Article  PubMed  CAS  Google Scholar 

  • Bogacz R, Usher M, Zhang J, McClelland JL (2007b) Extending a biologically inspired model of choice: multi-alternatives, nonlinearity and value-based multidimensional choice. Philos Trans R Soc Lond B Biol Sci 362:1655–1670

    Article  PubMed Central  PubMed  Google Scholar 

  • Botvinick MM (2012) Hierarchical reinforcement learning and decision making. Curr Opin Neurobiol 22:956–962

    Article  PubMed  CAS  Google Scholar 

  • Brunton BW, Botvinick MM, Brody CD (2013) Rats and humans can optimally accumulate evidence for decision-making. Science 340:95–98

    Article  PubMed  CAS  Google Scholar 

  • Cain N, Shea-Brown E (2012) Computational models of decision making: integration, stability, and noise. Curr Opin Neurobiol 22:1047–1053

    Article  PubMed  CAS  Google Scholar 

  • Churchland AK, Ditterich J (2012) New advances in understanding decisions among multiple alternatives. Curr Opin Neurobiol 22:920–926

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Churchland AK, Kiani R, Shadlen MN (2008) Decision-making with multiple alternatives. Nat Neurosci 11:693–702

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Cisek P, Puskas GA, El-Murr S (2009) Decisions in changing conditions: the urgency-gating model. J Neurosci 29:11560–11571

    Article  PubMed  CAS  Google Scholar 

  • Daw ND, Doya K (2006) The computational neurobiology of learning and reward. Curr Opin Neurobiol 16:199–204

    Article  PubMed  CAS  Google Scholar 

  • Dayan P, Daw ND (2008) Decision theory, reinforcement learning, and the brain. Cogn Affect Behav Neurosci 8:429–453

    Article  PubMed  Google Scholar 

  • Dayan P, Niv Y (2008) Reinforcement learning: the good, the bad and the ugly. Curr Opin Neurobiol 18:185–196

    Article  PubMed  CAS  Google Scholar 

  • Deco G, Rolls ET (2005) Attention, short-term memory, and action selection: a unifying theory. Prog Neurobiol 76:236–256

    Article  PubMed  Google Scholar 

  • Deneve S (2012) Making decisions with unknown sensory reliability. Front Neurosci 6:75

    Article  PubMed Central  PubMed  Google Scholar 

  • Ditterich J (2010) A comparison between mechanisms of multi-alternative perceptual decision making: ability to explain human behavior, predictions for neurophysiology, and relationship with decision theory. Front Neurosci 4:184

    Article  PubMed Central  PubMed  Google Scholar 

  • Doya K (2008) Modulators of decision making. Nat Neurosci 11:410–416

    Article  PubMed  CAS  Google Scholar 

  • Drugowitsch J, Moreno-Bote R, Churchland AK, Shadlen MN, Pouget A (2012) The cost of accumulating evidence in perceptual decision making. J Neurosci Off J Soc Neurosci 32:3612–3628

    Article  CAS  Google Scholar 

  • Furman M, Wang XJ (2008) Similarity effect and optimal control of multiple-choice decision making. Neuron 60:1153–1168

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Gillespie DT (1992) Markov processes: an introduction for physical scientists. Academic, San Diego

    Google Scholar 

  • Glimcher PW (2001) Making choices: the neurophysiology of visual-saccadic decision making. Trends Neurosci 24:654–659

    Article  PubMed  CAS  Google Scholar 

  • Glimcher PW (2003) The neurobiology of visual-saccadic decision making. Annu Rev Neurosci 26:133–179

    Article  PubMed  CAS  Google Scholar 

  • Gold JI, Shadlen MN (2001) Neural computations that underlie decisions about sensory stimuli. Trends Cogn Sci 5:10–16

    Article  PubMed  Google Scholar 

  • Gold JI, Shadlen MN (2007) The neural basis of decision making. Annu Rev Neurosci 30:535–574

    Article  PubMed  CAS  Google Scholar 

  • Hanks TD, Mazurek ME, Kiani R, Hopp E, Shadlen MN (2011) Elapsed decision time affects the weighting of prior probability in a perceptual decision task. J Neurosci Off J Soc Neurosci 31:6339–6352

    Article  CAS  Google Scholar 

  • Huk AC, Shadlen MN (2005) Neural activity in macaque parietal cortex reflects temporal integration of visual motion signals during perceptual decision making. J Neurosci Off J Soc Neurosci 25:10420–10436

    Article  CAS  Google Scholar 

  • Ito M, Doya K (2011) Multiple representations and algorithms for reinforcement learning in the cortico-basal ganglia circuit. Curr Opin Neurobiol 21:368–373

    Article  PubMed  CAS  Google Scholar 

  • Izhikevich EM (2007) Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 17:2443–2452

    Article  PubMed  Google Scholar 

  • Joel D, Niv Y, Ruppin E (2002) Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw Off J Int Neural Netw Soc 15:535–547

    Article  Google Scholar 

  • Johnson A, van der Meer MA, Redish AD (2007) Integrating hippocampus and striatum in decision-making. Curr Opin Neurobiol 17:692–697

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Lawler GF (2006) Introduction to stochastic processes. Chapman & Hall/CRC, Boca Raton

    Google Scholar 

  • Lee D, Seo H (2007) Mechanisms of reinforcement learning and decision making in the primate dorsolateral prefrontal cortex. Ann N Y Acad Sci 1104:108–122

    Article  PubMed  Google Scholar 

  • Ludwig CJ, Davies JR (2011) Estimating the growth of internal evidence guiding perceptual decisions. Cogn Psychol 63:61–92

    Article  PubMed  Google Scholar 

  • Machens CK, Romo R, Brody CD (2005) Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 307:1121–1124

    Article  PubMed  CAS  Google Scholar 

  • Miller P, Katz DB (2013) Accuracy and response-time distributions for decision-making: linear perfect integrators versus nonlinear attractor-based neural circuits. J Comput Neurosci 35:261–294

    Google Scholar 

  • Miller P, Wang XJ (2006) Discrimination of temporally separated stimuli by integral feedback control. Proc Natl Acad Sci U S A 103:201–206

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Newsome WT, Britten KH, Movshon JA (1989) Neuronal correlates of a perceptual decision. Nature 341:52–54

    Article  PubMed  CAS  Google Scholar 

  • Niwa M, Ditterich J (2008) Perceptual decisions between multiple directions of visual motion. J Neurosci Off J Soc Neurosci 28:4435–4445

    Article  CAS  Google Scholar 

  • Ratcliff R (1978) A theory of memory retrieval. Psychol Rev 85:59–108

    Article  Google Scholar 

  • Ratcliff R (2002) A diffusion model account of response time and accuracy in a brightness discrimination task: fitting real data and failing to fit fake but plausible data. Psychon Bull Rev 9:278–291

    Article  PubMed  Google Scholar 

  • Ratcliff R, Hasegawa YT, Hasegawa RP, Smith PL, Segraves MA (2007) Dual diffusion model for single-cell recording data from the superior colliculus in a brightness-discrimination task. J Neurophysiol 97:1756–1774

    Article  PubMed Central  PubMed  Google Scholar 

  • Ratcliff R, McKoon G (2008) The diffusion decision model: theory and data for two-choice decision tasks. Neural Comput 20:873–922

    Article  PubMed Central  PubMed  Google Scholar 

  • Ratcliff R, Smith PL (2004) A comparison of sequential sampling models for two-choice reaction time. Psychol Rev 111:333–367

    Article  PubMed Central  PubMed  Google Scholar 

  • Redish AD, Jensen S, Johnson A, Kurth-Nelson Z (2007) Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol Rev 114:784–805

    Article  PubMed  Google Scholar 

  • Romo R, Salinas E (2003) Flutter discrimination: neural codes, perception, memory and decision making. Nat Rev Neurosci 4:203–218

    Article  PubMed  CAS  Google Scholar 

  • Rorie AE, Gao J, McClelland JL, Newsome WT (2010) Integration of sensory and reward information during perceptual decision-making in lateral intraparietal cortex (LIP) of the macaque monkey. PLoS ONE 5:e9308

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Rüter J, Marcille N, Sprekeler H, Gerstner W, Herzog MH (2012) Paradoxical evidence integration in rapid decision processes. PLoS Comput Biol 8:e1002382

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Salinas E (2004) Fast remapping of sensory stimuli onto motor actions on the basis of contextual modulation. J Neurosci 24:1113–1118

    Article  PubMed  CAS  Google Scholar 

  • Seymour B, O'Doherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ, Friston KJ, Frackowiak RS (2004) Temporal difference models describe higher-order learning in humans. Nature 429:664–667

    Article  PubMed  CAS  Google Scholar 

  • Shadlen MN, Newsome WT (2001) Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86:1916–1936

    PubMed  CAS  Google Scholar 

  • Shankar S, Massoglia DP, Zhu D, Costello MG, Stanford TR, Salinas E (2011) Tracking the temporal evolution of a perceptual judgment using a compelled-response task. J Neurosci Off J Soc Neurosci 31:8406–8421

    Article  CAS  Google Scholar 

  • Shea-Brown E, Gilzenrat MS, Cohen JD (2008) Optimization of decision making in multilayer networks: the role of locus coeruleus. Neural Comput 20:2863–2894

    Article  PubMed  Google Scholar 

  • Simen P, Contreras D, Buck C, Hu P, Holmes P, Cohen JD (2009) Reward rate optimization in two-alternative decision making: empirical tests of theoretical predictions. J Exp Psychol Hum Percept Perform 35:1865–1897

    Article  PubMed Central  PubMed  Google Scholar 

  • Smith PL, Ratcliff R (2004) Psychology and neurobiology of simple decisions. Trends Neurosci 27:161–168

    Article  PubMed  CAS  Google Scholar 

  • Soltani A, Wang XJ (2006) A biophysically-based neural model of matching law behavior: melioration by stochastic synapses. J Neurosci 26:3731–3744

    Article  PubMed  CAS  Google Scholar 

  • Soltani A, Wang XJ (2008) From biophysics to cognition: reward-dependent adaptive choice behavior. Curr Opin Neurobiol 18:209–216

    Article  PubMed  CAS  Google Scholar 

  • Soltani A, Wang XJ (2010) Synaptic computation underlying probabilistic inference. Nat Neurosci 13:112–119

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Stanford TR, Shankar S, Massoglia DP, Costello MG, Salinas E (2010) Perceptual decision making in less than 30 milliseconds. Nat Neurosci 13:379–385

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Sugrue LP, Corrado GS, Newsome WT (2005) Choosing the greater of two goods: neural currencies for valuation and decision making. Nat Rev Neurosci 6:363–375

    Article  PubMed  CAS  Google Scholar 

  • Thura D, Beauregard-Racine J, Fradet CW, Cisek P (2012) Decision making by urgency gating: theory and experimental support. J Neurophysiol 108:2912–2930

    Article  PubMed  Google Scholar 

  • Usher M, McClelland JL (2001) The time course of perceptual choice: the leaky, competing accumulator model. Psychol Rev 108:550–592

    Article  PubMed  CAS  Google Scholar 

  • Wald A (1947) Sequential analysis. Wiley, New York

    Google Scholar 

  • Wald A, Wolfowitz J (1948) Optimum character of the sequential probability ratio test. Ann Math Stat 19:326–339

    Article  Google Scholar 

  • Wang XJ (2002) Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36:955–968

    Article  PubMed  CAS  Google Scholar 

  • Wang XJ (2008) Decision making in recurrent neuronal circuits. Neuron 60:215–234

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Wong KF, Wang XJ (2006) A recurrent network mechanism of time integration in perceptual decisions. J Neurosci Off J Soc Neurosci 26:1314–1328

    Article  CAS  Google Scholar 

  • Wyart V, de Gardelle V, Scholl J, Summerfield C (2012) Rhythmic fluctuations in evidence accumulation during decision making in the human brain. Neuron 76:847–858

    Article  PubMed  CAS  PubMed Central  Google Scholar 

  • Zhou X, Wong-Lin K, Philip H (2009) Time-varying perturbations can distinguish among integrate-to-threshold models for perceptual decision making in reaction time tasks. Neural Comput 21:2336–2362

    Article  PubMed Central  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Volen Center for Complex Systems and Department of Biology, Brandeis University, Waltham, MA, USA

    Dr. Paul Miller

Authors
  1. Dr. Paul Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul Miller .

Editor information

Editors and Affiliations

  1. Atlanta, Georgia, USA

    Dieter Jaeger

  2. Department of Biomedical Engineering, Florida International University, Miami, Florida, USA

    Ranu Jung

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this entry

Cite this entry

Miller, P. (2013). Decision Making, Models. In: Jaeger, D., Jung, R. (eds) Encyclopedia of Computational Neuroscience. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7320-6_312-3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-7320-6_312-3

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, New York, NY

  • Online ISBN: 978-1-4614-7320-6

  • eBook Packages: Springer Reference Biomedicine and Life SciencesReference Module Biomedical and Life Sciences

Publish with us

Policies and ethics

Search

Navigation