Abstract
An agent exercises instrumental rationality to the degree that she adopts appropriate means to achieving her ends. Adopting appropriate means to achieving one’s ends can, in turn, involve overcoming one’s strongest desires, that is, it can involve exercising synchronic self-control. However, contra prominent approaches, I deny that synchronic self-control is possible. Specifically, I draw on computational models and empirical evidence from cognitive neuroscience to describe a naturalistic, multi-system model of the mind. On this model, synchronic self-control is impossible. Must we, then, give up on a meaningful conception of instrumental rationality? No. A multi-system view still permits something like synchronic self-control: an agent can control her very strong desires. Adopting a multi-system model of the mind thus places limitations on our conceptions of instrumental rationality, without requiring that we abandon the notion altogether.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
At present, the truism captures the principle that an agent need only believe that she is free to act in a certain way, as well as unsuccessful attempts at action. Although it is still vulnerable to additional counterexamples, it can, in principle, be revised to accommodate them. My goal is not to establish a version of the truism that precludes all possible counterexamples, however. My goal is to provide a plausible, i.e., empirically-informed principle of motivation, together with an account of synchronic self-control that does not result in a paradox.
Philosophical folk psychology should be distinguished from other senses of the term ‘folk psychology.’ It is related to, but is not the same as, what can be called ‘lay folk psychology,’ or everyday theories of behavior (regarding the relationship, see Section 3.1). Lay folk psychology is used in everyday explanations of human choice and action, for example, when I ask, “Why was Bob so angry?,” and someone responds, “Because Bob just got a speeding ticket.” This kind of folk psychology takes place in everyday conversations and so on. Similarly, philosophical folk psychology should be distinguished from what are sometimes called ‘mindreading,’ ‘theory-theory,’ and ‘mental simulation,’ respectively. These kinds of folk psychology largely have to do with the mind. Mindreading refers to the cognitive capacities used to explain and predict behavior. Theory-theory is a theory about those cognitive capacities. It suggests that the cognitive capacities of mindreading depend on the representation of a kind of theory known as ‘folk psychology.’ Mental simulation is a different theory about those cognitive capacities. It suggests that we explain and predict others’ behavior by simulating their mental states.
In reinforcement learning, this system is formally called the Pavlovian system (for key papers, see Dayan and Balleine 2002; Dayan et al. 2006; Rangel et al. 2008; Balleine and O’Doherty 2010; Huys et al. 2011; Guitart-Masip et al. 2012; Huys et al., 2012; Daw and O’Doherty 2014; Gęsiarz and Crockett 2015; see especially Dayan 2008 and Daw and O’Doherty 2014). However, the term ‘Pavlovian’ frequently leads confusion among researchers in other fields (for an interesting discussion of how psychologists and machine learning scientists characterize the Pavlovian system differently, see Rescorla’s “Pavlovian conditioning: It’s not what you think it is,” 1988). In most fields, as well as in everyday usage, the term ‘Pavlovian’ is usually associated with Pavlov’s original experiments with dogs, where Pavlov trained his dogs by repeatedly ringing a bell and then consistently feeding them afterwards. Famously, the dogs learned to associate the ringing of the bell with the delivery of food, and the dogs’ expectation of food delivery was measured by their salivation. In this context, the food was defined as the unconditioned stimulus, the salivation as the unconditioned response, and the bell as the conditioned stimulus. When the dogs salivated in response to the ringing of the bell, this was identified as the conditioned response. In both public understanding and psychology, then, attention is paid to Pavlov’s discovery of the bell as the conditioned stimulus and its association with the conditioned response, the salivating. By contrast, in reinforcement learning, it is really the relationship between the unconditioned stimulus (i.e., the food) and the unconditioned response (i.e., the salivating) that is of interest. One added complication is that Pavlovian conditioning, i.e., the learning type, is sometimes also characterized in terms of methods from reinforcement learning methods, i.e., in terms of model-free and model based methods (see especially Dayan and Berridge 2014; Pool et al. 2019; see also Raab and Hartley 2019). Thanks to an anonymous reviewer for pressing me on this last point.
The deliberative system is formally called the goal-directed or model-based system (e.g. see Dayan 2011).
Hence its alternative name, tree search.
Example from Peter Dayan (2011).
The habitual system is formally called the habit-based or model-free system (e.g. see Dayan 2011).
This formulation is owed to Crockett (2013).
The feedback signal works much like exclamations of ‘Hotter!’ and ‘Colder’ in the children’s game Hot-or-Cold. The Seeker moves around the room with the general goal of finding a hidden object. The Hider helps the Seeker by telling her whether she is getting closer or farther away. The Hider’s suggestions operate like an error signal by helping the Seeker refine her predictions, albeit without giving her detailed instructions about where to go (analogy from Montague 2006).
For instance, in addressing precisely this question, Daw, Gershman, Seymour, Dayan, & Dolan (2011, p. 1211) observe that that such a hybrid approach of interaction-before-arbitration “would appear to remain consistent with the lesion data suggesting that the systems can function in isolation (Coutureau and Killcross 2003; Yin et al., 2004, 2005), and with behavioral data demonstrating that distinct decision systems may have different properties and can be differentially engaged in different circumstances (Doeller and Burgess, 2008; Frank et al., 2007; Fu and Anderson, 2008). It also remains consistent with other fMRI studies (Doeller et al., 2008; Poldrack et al., 2001; Venkatraman et al., 2009) suggesting that overall activity in different brain systems associated with either system can modulate with time or circumstances, presumably in relation to the extent that either process is engaged.”
Elsewhere, Sripada characterizes the same objection in terms of personhood (2014, p. 22).
References
Arpaly, N., and T. Schroeder. 2014. In praise of desire. New York: Oxford University Press.
Balleine, B.W., and A. Dickinson. 1998. Goal-directed instrumental action: Contingency and incentive learning and their cortical substrates. Neuropharmacology 37 (4–5): 407–419.
Balleine, B.W., and A. Dickinson. 2000. The effect of lesions of the insular cortex on instrumental conditioning: Evidence for a role in incentive memory. Journal of Neuroscience 20 (23): 8954–8964.
Balleine, B.W., and J.P. O’Doherty. 2010. Human and rodent homologies in action control: Corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35: 48–69. https://doi.org/10.1038/npp.2009.131.
Belova, M.A., J.J. Paton, and C.D. Salzman. 2008. Moment-to-moment tracking of state value in the amygdala. The Journal of Neuroscience 28 (40): 10023–10030.
Bouton, M.E. 2006. Learning and behavior: A contemporary synthesis. USA: Sinauer.
Carruthers, P. 2011. The opacity of mind: An integrative theory of self-knowledge. Oxford: Oxford University Press.
Churchland, P.M. 1981. Eliminative materialism and propositional attitudes. The Journal of Philosophy 78 (2): 67–90.
Coutureau, E., and S. Killcross. 2003. Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats. Behavioural Brain Research 146 (1–2): 167–174.
Crockett, M.J. 2013. Models of morality. Trends in Cognitive Sciences 17 (8): 363–366.
Davidson, D. 1970. How is weakness of the will possible?. In Davidson, D. (1980), Essats on actions and events. Oxford: Clarendon Press, pp. 21-42.
Daw, N.D., and Peter Dayan, P. 2014. The algorithmic anatomy of model-based evaluation. Philosophical Transactions of the Royal Society B: Biological Sciences 369 (1655):20130478.
Daw, N.D., and J.P. O’Doherty. 2014. Multiple systems for value learning. In Neuroeconomics, 393–410. Academic Press.
Daw, N.D., Y. Niv, and P. Dayan. 2005. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience 8 (12): 1704–1711.
Daw, N.D., S.J. Gershman, B. Seymour, P. Dayan, and R.J. Dolan. 2011. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69 (6): 1204–1215.
Dayan, P. 2008. The role of value systems in decision making. In Strüngmann Forum reports. Better than conscious? Decision making, the human mind, and implications for institutions, ed. C. Engel and W. Singer, 51–70. MIT Press.
Dayan, P. 2011. Interactions Between Model-Free and Model-Based Reinforcement Learning,’ Seminar Series from the Machine Learning Research Group. University of Sheffield, Sheffield. Lecture recording. <http://ml.dcs.shef.ac.uk/>. Accessed August 2019.
Dayan, P., and B.W. Balleine. 2002. Reward, motivation, and reinforcement learning. Neuron 36 (2): 285–298.
Dayan, P., and K.C. Berridge. 2014. Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation. Cognitive, Affective, & Behavioral Neuroscience.
Dayan, P., and Q.J. Huys. 2008. Serotonin, inhibition, and negative mood. PLoS Computational Biology 4 (2): e4.
Dayan, P., and Y. Niv. 2008. Reinforcement learning: The good, the bad and the ugly. Current Opinion in Neurobiology 18 (2): 185–196.
Dayan, P., Y. Niv, B. Seymour, D. Daw, and N. 2006. The misbehavior of value and the discipline of the will. Neural Networks 19 (8): 1153–1160.
De Caro, M., and D. Macarthur. 2004. The nature of naturalism. In Naturalism in question, ed. M. De Caro and D. Macarthur, 1–20. Cambridge: Harvard University Press.
Deneve, S., and A. Pouget. 2004. Bayesian multisensory integration and cross-modal spatial links. Journal of Physiology Paris 98 (1–3): 249–258.
Dennett, D.C. 1978. Three kinds of intentional psychology. Perspectives in the philosophy of language: A concise anthology: 163–186.
Dickinson, A. 1985. Actions and habits: The development of a behavioural autonomy. Philosophical Transactions of the Royal Society B: Biological Sciences 308: 67–78.
Dickinson, A., and B. Balleine. 2002. The role of learning in the operation of the motivational systems. In Stevens’ handbook of experimental psychology, Learning, motivation, and emotion, ed. C.R. Gallistel, vol. 3, 497–533. New York: Wiley.
Dickinson, A., S. Squire, Z. Varga, and J.W. Smith. 1998. Omission learning after instrumental pre-training. The Quarterly Journal of Experimental Psychology 51: 271–286.
Dolan, R.J., and P. Dayan. 2013. Goals and habits in the brain. Neuron 80: 312–325. https://doi.org/10.1016/j.neuron.2013.09.007.
Flanagan, O. 2006. Varieties of naturalism. In Oxford handbook of religion and science, ed. P. Clayton and Z. Simpson, 430–452. Oxford: Oxford University Press.
Gershman, S.J., A.B. Markman, and A.R. Otto. 2014. Retrospective revaluation in sequential decision making: A tale of two systems. Journal of Experimental Psychology: General 143 (1): 182–194.
Gęsiarz, F., and Crockett, M. J. 2015. Goal-directed, habitual and Pavlovian prosocial behavior. Frontiers in behavioral neuroscience 9, 135.
Glimcher, P.W., and E. Fehr, eds. 2013. Neuroeconomics: Decision making and the brain. New York: Academic Press.
Guitart-Masip, M., Q.J. Huys, L. Fuentemilla, P. Dayan, E. Duzel, and R.J. Dolan. 2012. Go and no-go learning in reward and punishment: Interactions between affect and effect. Neuroimage 62 (1): 154–166.
Hampton, A.N., P. Bossaerts, and J.P. O’Doherty. 2006. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. The Journal of Neuroscience 26 (32): 8360–8367.
Huys, Q.J., R. Cools, M. Gölzer, E. Friedel, A. Heinz, R.J. Dolan, and P. Dayan. 2011. Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding. PLoS Computational Biology 7 (4): e1002028.
Huys, Q.J., N. Eshel, E. O'Nions, L. Sheridan, P. Dayan, and J.P. Roiser. 2012. Bonsai trees in your head: How the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Computational Biology 8 (3): e1002410.
Kennett, J., and M. Smith. 1996. Frog and toad lose control. Analysis 56 (2): 63–73.
Kennett, J., and M. Smith. 1997. Synchronic self-control is always non-actional. Analysis 57 (2): 123–131.
Keramati, M., A. Dezfouli, and P. Piray. 2011. Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Computational Biology 7 (5): e1002055.
Kiani, R., and M.N. Shadlen. 2009. Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324 (5928): 759–764.
Killcross, S., and Coutureau, E. 2003. Coordination of actions and habits in the medial prefrontal cortex of rats. Cerebral cortex 13(4), 400-408.
Laudan, L., A. Donovan, R. Laudan, P. Barker, H. Brown, J. Leplin, P. Thagard, and S. Wykstra. 1986. Scientific change: Philosophical models and historical research. Synthese 69 (2): 141–223.
Lee, S.W., S. Shimojo, and J.P. O’Doherty. 2014. Neural computations underlying arbitration between model-based and model-free learning. Neuron 81 (3): 687–699.
Lewis, D. 1972. Psychophysical and theoretical identifications. Australasian Journal of Philosophy 50 (3): 249–258.
Macintosh, N.J. 1983. Conditioning and associative learning. Oxford: Oxford University Press.
McCauley, R. 2012. About face: Philosophical naturalism, the heuristic identity theory, and recent findings about prosopagnosia. In New perspectives on type identity: The mental and the physical, ed. S. Gozzano and C.S. Hill, 186–206. Cambridge: Cambridge University Press.
Mele, A.R. 1987. Irrationality: An essay on Akrasia, self-deception, and self-control. New York: Oxford University Press.
Mele, Alfred. 1992. Irrationality: An essay on Akrasia, self-deception, and self-control. New York: Oxford Univ Press.
Mele, A.R. 1997. Underestimating self-control: Kennett and Smith on frog and toad. Analysis 57 (2): 119–123.
Mele, R. 1998. Synchronic self-control revisited: Frog and toad shape up. Analysis 58 (4): 305–310.
Mele, A.R. 2003. Motivation and agency. New York: Oxford University Press.
Mele, A.R. 2014. Self-control, motivational strength, and exposure therapy. Philosophical Studies 170 (2): 359–375.
Montague, R. 2006. Why choose this book? How we make decisions. New York: Dutton.
O'Doherty, J.P., P. Dayan, K. Friston, H. Critchley, and R.J. Dolan. 2003. Temporal difference models and reward-related learning in the human brain. Neuron 38 (2): 329–337.
Otto, A.R., A. Skatova, S. Madlon-Kay, and N.D. Daw. 2014. Cognitive control predicts use of model-based reinforcement learning. Journal of Cognitive Neuroscience 27 (2): 319–333.
Parkinson, J.A., M.C. Olmstead, L.H. Burns, T.W. Robbins, and B.J. Everitt. 1999. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity byd-amphetamine. The Journal of Neuroscience 19 (6): 2401–2411.
Parkinson, J.A., J.W. Dalley, R.N. Cardinal, A. Bamford, B. Fehnert, G. Lachenal, N. Rudarakanchana, K.M. Halkerston, T.W. Robbins, and B.J. Everitt. 2002. Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: Implications for mesoaccumbens dopamine function. Behavioural Brain Research 137 (1): 149–163.
Paton, J.J., M.A. Belova, S.E. Morrison, and C.D. Salzman. 2006. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439 (7078): 865–870.
Pezzulo, G., F. Rigoli, and F. Chersi. 2013. The mixed instrumental controller: Using value of information to combine habitual choice and mental simulation. Frontiers in Psychology 4: 92.
Pezzulo, G., F. Rigoli, and K. Friston. 2015. Active inference, homeostatic regulation and adaptive behavioural control. Progress in Neurobiology 134: 17–35.
Pool, E.R., W.M. Pauli, C.S. Kress, and J.P. O’Doherty. 2019. Behavioural evidence for parallel outcome-sensitive and outcome-insensitive Pavlovian learning systems in humans. Nature Human Behaviour 3 (3): 284–296.
Raab, H.A., and C.A. Hartley. 2019. More than two forms of Pavlovian prediction. Nature Human Behaviour 3 (3): 212–213.
Rangel, A., C. Camerer, and P.R. Montague. 2008. A framework for studying the neurobiology of value-based decision making. Nature Reviews Neuroscience 9 (7): 545–556.
Redish, D.A. 2013. The mind within the brain. Oxford: Oxford University Press.
Russell, S. J., and P. Norvig. 2009. Artificial intelligence: A modern approach. Prentice Hall.
Schroeder, T. 2004. Three faces of desire. New York: Oxford University Press.
Schultz, W., L. Tremblay, and J.R. Hollerman. 1998. Reward prediction in primate basal ganglia and frontal cortex. Neuropharmacology 37 (4–5): 421–429.
Schultz, W., Tremblay, L., and Hollerman, J.R. 2000. Reward processing in primate orbitofrontal cortex and basal ganglia. Cerebral cortex 10(3), 272-283.
Schwitzgebel, E. 2008. The unreliability of naive introspection. Philosophical Review 117 (2): 245–273.
Sripada, C.S. 2010. Philosophical questions about the nature of willpower. Philosophy Compass 5: 793–805.
Sripada, C.S. 2012. How is willpower possible? The puzzle of synchronic self-control and the divided mind. Nous 48 (1): 41–74.
Staffel, J. 2017. Should I pretend I’m perfect? Res Philosophica 94 (2): 301–324.
Tremblay, L. and Schultz, W., 1999. Relative reward preference in primate orbitofrontal cortex. Nature 398(6729), pp.704-708.
Tremblay, L., and W. Schultz. 2000a. Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. Journal of Neurophysiology 83 (4): 1864–1876.
Tremblay, L., and W. Schultz. 2000b. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. Journal of Neurophysiology 83 (4): 1877–1885.
Tricomi, E., B.W. Balleine, and J.P. O’Doherty. 2009. A specific role for posterior dorsolateral striatum in human habit learning. European Journal of Neuroscience 29 (11): 2225–2232.
Valentin, V.V., A. Dickinson, and J.P. O’Doherty. 2007. Determining the neural substrates of goal-directed learning in the human brain. Journal of Human Neuroscience 27: 4019–4026.
Yin, H.H., Knowlton, B.J., and Balleine, B.W. 2004. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. European Journal of Neuroscience 19 (1):181-189
Yin, H.H., Ostlund, S.B., Knowlton, B.J. and Balleine, B.W. 2005. The role of the dorsomedial striatum in instrumental conditioning. European Journal of Neuroscience 22(2), pp.513-523.
Acknowledgements
The author would like to thank Ivan Gonzalez Cabrera, Peter Clutton, Caitrin Donovan, Benjamin Henke, Linus Huang, Colin Klein, Kathryn Lindeman, Ross Pain, Elizabeth Schechter, Chandra Sripada, and Julia Staffel. Special thanks to Thor Grunbaum.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Haas, J. Is Synchronic Self-Control Possible?. Rev.Phil.Psych. 12, 397–424 (2021). https://doi.org/10.1007/s13164-020-00490-w
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13164-020-00490-w