Skip to main content
SpringerLink
Log in

Testing Optimal Timing in Value-Linked Decision Making

  • Published:
Computational Brain & Behavior Aims and scope Submit manuscript

Abstract

Influential theories of the decision making process hold that a choice is made once the cumulative weight of noisily sampled information reaches a desired level. While these theories were originally motivated as optimal solutions to statistical problems, the extent to which people optimally spend time deliberating is less well explored. I conduct an experimental test of optimality in a setting where the speed of information processing reflects the difference in value between options. In this case, spending a long time without having arrived at a conclusion signals both that the problem is hard and that the options are similar in value, so the confidence level required to trigger a decision should decline over time. I find that a recently developed theory of the optimal time-varying threshold improves model fit by accurately predicting observed truncation of response time tails. Principles of optimality may thus help account for patterns of choice and response time that characterize the process of deliberation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. This body of work builds on related applications which do not explicitly tie the accumulation rate to value (e.g., Hawkins et al. 2014; Otter et al. 2008; Trueblood et al. 2014).

  2. For more on Bayesian formulations of diffusion models, see Bitzer et al. (2014) and Fard et al. (2017).

  3. While this is the most widely used optimality criterion across fields of study, it is not the only one that has been proposed (for discussion, see Bogacz et al. 2006; Pirrone et al. 2014; and Bhui 2019). Tajima et al. (2016) calculate numerical solutions for collapsing boundaries under alternative criteria such as reward rate maximization, though Pirrone et al. (2018b) fail to find empirical support for the resulting predictions.

  4. Although this setup bears some resemblance to the standard race model due to the specification of two independent accumulators, the two models should not be confused. The race model assumes that a response is triggered when either accumulator crosses a given threshold. The uncertain-difference DDM instead treats the accumulators as two sources of information which are used to inform the optimal balance between reward and time expenditure. Hence, the decision criterion can be defined based on any combination of accumulator values. The setup entails that the difference in accumulators is a sufficient statistic for solving the optimization problem in Expression (2), and thus the uncertain-difference model boils down to a version of the DDM. See also Bogacz et al. (2006) for further explication of the technical connections between various sequential sampling models.

  5. In the present data, an analysis in the Appendix reveals no effect of absolute value on response time, contrary to observations in Teodorescu et al. (2016) and Pirrone et al. (2018a).

  6. Distinctions between blocks will not be explored in the present analysis since the effects of experience, fatigue, and incentives are confounded.

  7. It must be noted that due to Caltech’s particular nature, all students have strong quantitative backgrounds and are familiar with the normal distribution.

  8. Relatively long response times were also observed in Lam and Kalaska (2014).

  9. Since the properties of the DDM depend only on the ratios between the drift rate, decision threshold, and accumulation noise, one parameter is routinely fixed at some arbitrary level. Typically, this is the noise parameter, but for consistency with the notation of Fudenberg et al. (2018), instead I fix the (conditional) drift rate and allow the accumulation noise to be a free parameter.

  10. In accordance with the experimental design, σ0 was fixed at 7.

  11. There were 10,000 replicates per level of value difference, of which there were approximately 40.

  12. A negative association can also be driven by between-trial parameter variability (e.g., Laming 1968; Ratcliff 1978).

References

  • Bather, J. A. (1962). Bayes procedures for deciding the sign of a normal mean. Mathematical Proceedings of the Cambridge Philosophical Society, 58(4), 599–620.

    Article  Google Scholar 

  • Bhui, R. (2019). A statistical test for the optimality of deliberative time allocation. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-018-1555-1.

  • Bitzer, S., Park, H., Blankenburg, F., & Kiebel, S. J. (2014). Perceptual decision making: drift-diffusion model is equivalent to a Bayesian model. Frontiers in Human Neuroscience, 8, 102.

  • Bode, S., Sewell, D. K., Lilburn, S., Forte, J. D., Smith, P. L., & Stahl, J. (2012). Predicting perceptual decision biases from early brain activity. Journal of Neuroscience, 32(36), 12488–12498.

    Article  PubMed  Google Scholar 

  • Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. (2006). The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychological Review, 113(4), 700–765.

    Article  PubMed  Google Scholar 

  • Bollimunta, A., & Ditterich, J. (2012). Local computation of decision-relevant net sensory evidence in parietal cortex. Cerebral Cortex, 22(4), 903–917.

    Article  PubMed  Google Scholar 

  • Bollimunta, A., Totten, D., & Ditterich, J. (2012). Neural dynamics of choice: single trial analysis of decision-related activity in parietal cortex. Journal of Neuroscience, 32(37), 12684–12701.

    Article  PubMed  Google Scholar 

  • Bowman, N. E., Kording, K. P., & Gottfried, J. A. (2012). Temporal integration of olfactory perceptual evidence in human orbitofrontal cortex. Neuron, 75(5), 916–927.

    Article  PubMed  PubMed Central  Google Scholar 

  • Britten, K. H., Shadlen, M. N., Newsome, W. T., & Movshon, J. A. (1992). The analysis of visual motion: a comparison of neuronal and psychophysical performance. Journal of Neuroscience, 12(12), 4745–4765.

    Article  PubMed  Google Scholar 

  • Brown, S., & Heathcote, A. (2003). QMLE: fast, robust, and efficient estimation of distribution functions based on quantiles. Behavior Research Methods, Instruments, & Computers, 35(4), 485–492.

    Article  Google Scholar 

  • Brown, J. W., Hanes, D. P., Schall, J. D., & Stuphorn, V. (2008). Relation of frontal eye field activity to saccade initiation during a countermanding task. Experimental Brain Research, 190(2), 135–151.

    Article  PubMed  PubMed Central  Google Scholar 

  • Churchland, A. K., Kiani, R., & Shadlen, M. N. (2008). Decision-making with multiple alternatives. Nature Neuroscience, 11(6), 693–702.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cisek, P., Puskas, G. A., & El-Murr, S. (2009). Decisions in changing conditions: the urgency-gating model. Journal of Neuroscience, 29(37), 11560–11571.

    Article  PubMed  Google Scholar 

  • Ding, L., & Gold, J. I. (2010). Caudate encodes multiple computations for perceptual decisions. Journal of Neuroscience, 30(47), 15747–15759.

    Article  PubMed  Google Scholar 

  • Ding, L., & Gold, J. I. (2012). Separate, causal roles of the caudate in saccadic choice and execution in a perceptual decision task. Neuron, 75(5), 865–874.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ditterich, J. (2006a). Evidence for time-variant decision making. European Journal of Neuroscience, 24(12), 3628–3641.

    Article  PubMed  Google Scholar 

  • Ditterich, J. (2006b). Stochastic models of decisions about motion direction: behavior and physiology. Neural Networks, 19(8), 981–1012.

    Article  PubMed  Google Scholar 

  • Drugowitsch, J., Moreno-Bote, R., Churchland, A. K., Shadlen, M. N., & Pouget, A. (2012). The cost of accumulating evidence in perceptual decision making. Journal of Neuroscience, 32(11), 3612–3628.

    Article  PubMed  Google Scholar 

  • Fard, P. R., Park, H., Warkentin, A., Kiebel, S. J., & Bitzer, S. (2017). A Bayesian reformulation of the extended drift-diffusion model in perceptual decision making. Frontiers in Computational Neuroscience, 11, 29.

  • Forstmann, B. U., Dutilh, G., Brown, S., Neumann, J., Von Cramon, D. Y., Ridderinkhof, K. R., & Wagenmakers, E.-J. (2008). Striatum and pre-SMA facilitate decision-making under time pressure. Proceedings of the National Academy of Sciences, 105(45), 17538–17542.

    Article  Google Scholar 

  • Forstmann, B. U., Anwander, A., Scha¨fer, A., Neumann, J., Brown, S., Wagenmakers, E. J., Bogacz, R., & Turner, R. (2010). Cortico-striatal connections predict control over speed and accuracy in perceptual decision making. Proceedings of the National Academy of Sciences, 107(36), 15916–15920.

    Article  Google Scholar 

  • Forstmann, B. U., Ratcliff, R., & Wagenmakers, E.-J. (2016). Sequential sampling models in cognitive neuroscience: advantages, applications, and extensions. Annual Review of Psychology, 67, 641–666.

    Article  PubMed  Google Scholar 

  • Fudenberg, D., Strack, P., & Strzalecki, T. (2018). Speed, accuracy, and the optimal timing of choices. American Economic Review, 108(12), 3651–3684.

    Article  Google Scholar 

  • Hanks, T. D., Mazurek, M. E., Kiani, R., Hopp, E., & Shadlen, M. N. (2011). Elapsed decision time affects the weighting of prior probability in a perceptual decision task. Journal of Neuroscience, 31(17), 6339–6352.

    Article  PubMed  Google Scholar 

  • Hawkins, G. E., Marley, A., Heathcote, A., Flynn, T. N., Louviere, J. J., & Brown, S. D. (2014). Integrating cognitive process and descriptive models of attitudes and preferences. Cognitive Science, 38(4), 701–735.

    Article  PubMed  Google Scholar 

  • Hawkins, G. E., Forstmann, B. U., Wagenmakers, E.-J., Ratcliff, R., & Brown, S. D. (2015). Revisiting the evidence for collapsing boundaries and urgency signals in perceptual decision-making. Journal of Neuroscience, 35(6), 2476–2484.

    Article  Google Scholar 

  • Heathcote, A., Brown, S., & Mewhort, D. J. (2002). Quantile maximum likelihood estimation of response time distributions. Psychonomic Bulletin & Review, 9(2), 394–401.

    Article  Google Scholar 

  • Heathcote, A., Brown, S., & Cousineau, D. (2004). QMPE: estimating Lognormal, Wald, and Weibull RT distributions with a parameter-dependent lower bound. Behavior Research Methods, Instruments, & Computers, 36(2), 277–290.

    Article  Google Scholar 

  • Khodadadi, A., & Townsend, J. T. (2015). On mimicry among sequential sampling models. Journal of Mathematical Psychology, 68, 37–48.

    Article  Google Scholar 

  • Krajbich, I., & Rangel, A. (2011). Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proceedings of the National Academy of Sciences, 108(33), 13852–13857.

    Article  Google Scholar 

  • Krajbich, I., Armel, C., & Rangel, A. (2010). Visual fixations and the computation and comparison of value in simple choice. Nature Neuroscience, 13(10), 1292–1298.

    Article  PubMed  Google Scholar 

  • Krajbich, I., Lu, D., Camerer, C., & Rangel, A. (2012). The attentional drift-diffusion model extends to simple purchasing decisions. Frontiers in Psychology, 3, 193.

  • Krajbich, I., Oud, B., & Fehr, E. (2014). Benefits of neuroeconomic modeling: new policy interventions and predictors of preference. American Economic Review, 104(5), 501–506.

    Article  Google Scholar 

  • Krajbich, I., Hare, T., Bartling, B., Morishima, Y., & Fehr, E. (2015). A common mechanism underlying food choice and social decisions. PLoS Computational Biology, 11(10), e1004371.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lam, E. and Kalaska, J. F. (2014). Choosing sides: the psychophysics of target choices using random dot kinematograms with mutually contradictory evidence. Unpublished manuscript.

  • Laming, D. R. J. (1968). Information theory of choice-reaction times. Cambridge: Academic Press.

    Google Scholar 

  • Milosavljevic, M., Malmaud, J., Huth, A., Koch, C., & Rangel, A. (2010). The drift diffusion model can account for the accuracy and reaction time of value-based choices under high and low time pressure. Judgment and Decision making, 5(6), 437–449.

    Google Scholar 

  • Moran, R. (2015). Optimal decision making in heterogeneous and biased environments. Psychonomic Bulletin & Review, 22(1), 38–53.

    Article  Google Scholar 

  • Mullen, K. M., Ardia, D., Gil, D. L., Windover, D., & Cline, J. (2011). DEoptim: an R package for global optimization by differential evolution. Journal of Statistical Software, 40(6), 1–26.

    Article  Google Scholar 

  • Newsome, W. T., Britten, K. H., & Movshon, J. A. (1989). Neuronal correlates of a perceptual decision. Nature, 341(6237), 52–54.

    Article  PubMed  Google Scholar 

  • Niwa, M., & Ditterich, J. (2008). Perceptual decisions between multiple directions of visual motion. Journal of Neuroscience, 28(17), 4435–4445.

    Article  PubMed  Google Scholar 

  • O’Connell, R. G., Dockree, P. M., & Kelly, S. P. (2012). A supramodal accumulationto-bound signal that determines perceptual decisions in humans. Nature Neuroscience, 15(12), 1729–1735.

    Article  Google Scholar 

  • Otter, T., Allenby, G. M., & van Zandt, T. (2008). An integrated model of discrete choice and response time. Journal of Marketing Research, 45(5), 593–607.

    Article  Google Scholar 

  • Oud, B., Krajbich, I., Miller, K., Cheong, J., Botvinick, M., & Fehr, E. (2016). Irrational time allocation in decision-making. Proceedings of the Royal Society B: Biological Sciences, 283(1822), 20151439.

    Article  PubMed  Google Scholar 

  • Palmer, J., Huk, A. C., & Shadlen, M. N. (2005). The effect of stimulus strength on the speed and accuracy of a perceptual decision. Journal of Vision, 5(5), 376–404.

    Article  PubMed  Google Scholar 

  • Pirrone, A., Stafford, T., & Marshall, J. A. (2014). When natural selection should optimize speed-accuracy trade-offs. Frontiers in Neuroscience, 8, 73.

  • Pirrone, A., Azab, H., Hayden, B. Y., Stafford, T., & Marshall, J. A. (2018a). Evidence for the speed–value trade-off: human and monkey decision making is magnitude sensitive. Decision, 5(2), 129–142.

    Article  PubMed  Google Scholar 

  • Pirrone, A., Wen, W., & Li, S. (2018b). Single-trial dynamics explain magnitude sensitive decision making. BMC Neuroscience, 19(54), 54.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ramakrishnan, A., & Murthy, A. (2013). Brain mechanisms controlling decision making and motor planning. Progress in Brain Research, 202, 321–345.

    Article  PubMed  Google Scholar 

  • Ramakrishnan, A., Sureshbabu, R., & Murthy, A. (2012). Understanding how the brain changes its mind: microstimulation in the macaque frontal eye field reveals how saccade plans are changed. Journal of Neuroscience, 32(13), 4457–4472.

    Article  PubMed  Google Scholar 

  • Rao, R. P. (2010). Decision making under uncertainty: a neural model based on partially observable markov decision processes. Frontiers in Computational Neuroscience, 4, 146.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(2), 59–108.

    Article  Google Scholar 

  • Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: theory and data for two-choice decision tasks. Neural Computation, 20(4), 873–922.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ratcliff, R., Philiastides, M. G., & Sajda, P. (2009). Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG. Proceedings of the National Academy of Sciences, 106(16), 6539–6544.

    Article  Google Scholar 

  • Ratcliff, R., Smith, P. L., Brown, S. D., & McKoon, G. (2016). Diffusion decision model: current issues and history. Trends in Cognitive Sciences, 20(4), 260–281.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rigoux, L., Stephan, K. E., Friston, K. J., & Daunizeau, J. (2014). Bayesian model selection for group studies—revisited. NeuroImage, 84, 971–985.

    Article  PubMed  Google Scholar 

  • Salinas, E., & Stanford, T. R. (2013). The countermanding task revisited: fast stimulus detection is a key determinant of psychophysical performance. Journal of Neuroscience, 33(13), 5668–5685.

    Article  PubMed  Google Scholar 

  • Sanders, A. and Ter Linden, W. (1967). Decision making during paced arrival of probabilistic information. Acta Psychologica, 27, 170–177.

  • Schall, J. D. (2003). Neural correlates of decision processes: neural and mental chronometry. Current Opinion in Neurobiology, 13(2), 182–186.

    Article  PubMed  Google Scholar 

  • Schurger, A., Sitt, J. D., & Dehaene, S. (2012). An accumulator model for spontaneous neural activity prior to self-initiated movement. Proceedings of the National Academy of Sciences, 109(42), E2904–E2913.

    Article  Google Scholar 

  • Smith, P. L., & McKenzie, C. R. (2011). Diffusive information accumulation by minimal recurrent neural models of decision making. Neural Computation, 23(8), 2000–2031.

    Article  PubMed  Google Scholar 

  • Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009). Bayesian model selection for group studies. NeuroImage, 46(4), 1004–1017.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stone, M. (1960). Models for choice-reaction time. Psychometrika, 25(3), 251–260.

    Article  Google Scholar 

  • Tajima, S., Drugowitsch, J., & Pouget, A. (2016). Optimal policy for value-based decisionmaking. Nature Communications, 7(12400), 1–12.

    Google Scholar 

  • Teodorescu, A. R., Moran, R., & Usher, M. (2016). Absolutely relative or relatively absolute: violations of value invariance in human decision making. Psychonomic Bulletin & Review, 23(1), 22–38.

    Article  Google Scholar 

  • Thura, D., & Cisek, P. (2014). Deliberation and commitment in the premotor and primary motor cortex during dynamic decision making. Neuron, 81(6), 1401–1416.

    Article  PubMed  Google Scholar 

  • Thura, D., Beauregard-Racine, J., Fradet, C.-W., & Cisek, P. (2012). Decision making by urgency gating: theory and experimental support. Journal of Neurophysiology, 108(11), 2912–2930.

    Article  PubMed  Google Scholar 

  • Trueblood, J. S., Brown, S. D., & Heathcote, A. (2014). The multiattribute linear ballistic accumulator model of context effects in multialternative choice. Psychological Review, 121(2), 179–205.

    Article  PubMed  Google Scholar 

  • Tuerlinckx, F., Maris, E., Ratcliff, R., & De Boeck, P. (2001). A comparison of four methods for simulating the diffusion process. Behavior Research Methods, Instruments, & Computers, 33(4), 443–456.

    Article  Google Scholar 

  • Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: the leaky, competing accumulator model. Psychological Review, 108(3), 550–592.

    Article  PubMed  Google Scholar 

  • Viviani, P. (1979a). Choice reaction times for temporal numerosity. Journal of Experimental Psychology: Human Perception and Performance, 5(1), 157–167.

    PubMed  Google Scholar 

  • Viviani, P. (1979b). A diffusion model for discrimination of temporal numerosity. Journal of Mathematical Psychology, 19(2), 108–136.

    Article  Google Scholar 

  • Viviani, P., & Terzuolo, C. (1972). On the modeling of the performances of the human brain in a two-choice task involving decoding and memorization of simple visual patterns. Kybernetik, 10(3), 121–137.

    Article  PubMed  Google Scholar 

  • Voskuilen, C., Ratcliff, R., & Smith, P. L. (2016). Comparing fixed and collapsing boundary versions of the diffusion model. Journal of Mathematical Psychology, 73, 59–79.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wald, A. (1947). Sequential analysis. New York: Wiley.

    Google Scholar 

  • Wang, X.-J. (2002). Probabilistic decision making by slow reverberation in cortical circuits. Neuron, 36(5), 955–968.

    Article  PubMed  Google Scholar 

  • Wasserman, L. (2000). Bayesian model selection and model averaging. Journal of Mathematical Psychology, 44(1), 92–107.

    Article  PubMed  Google Scholar 

  • Wong, K.-F., & Wang, X.-J. (2006). A recurrent network mechanism of time integration in perceptual decisions. Journal of Neuroscience, 26(4), 1314–1328.

    Article  PubMed  Google Scholar 

  • Zhang, S., Lee, M. D., Vandekerckhove, J., Maris, G., & Wagenmakers, E.-J. (2014). Time-varying boundaries for diffusion models of decision making and response time. Frontiers in Psychology, 5, 1364.

Download references

Acknowledgements

Thanks to Colin Camerer, Jaron Colas, Taisuke Imai, Ian Krajbich, and Tomasz Strzalecki for helpful comments and discussions. An earlier version of this paper was circulated under the title "Evidence on Optimally Collapsing Thresholds in Value-Linked Decision Making".

Funding

Funding from the Social Sciences and Humanities Research Council of Canada and the Harvard Mind Brain Behavior Interfaculty Initiative is gratefully acknowledged.

Author information

Authors and Affiliations

  1. Departments of Psychology and Economics & Center for Brain Science, Harvard University, 52 Oxford St, Cambridge, MA, 02138, USA

    Rahul Bhui

Authors
  1. Rahul Bhui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rahul Bhui.

Ethics declarations

The experiment was approved by the Caltech Committee for the Protection of Human Subjects.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic Supplementary Material

ESM 1

(PDF 1854 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhui, R. Testing Optimal Timing in Value-Linked Decision Making. Comput Brain Behav 2, 85–94 (2019). https://doi.org/10.1007/s42113-019-0025-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42113-019-0025-9

Keywords

Search

Navigation