Model of Cue Extraction from Distractors by Active Recall

Abstract

Cues are informative signals animals must use to make decisions in order to obtain rewards, usually after intervening temporal delays, typified in the cue-action-reward task. In behavioural experiments the cue is often clearly distinguished from other stimuli, by a salience such as brightness for example, however in the real world animals face the problem of recognizing real cues from among other environmental distracting stimuli. Furthermore once the cue is recognized it must cause the animal to make a certain action to obtain reward. Therefore the animal faces a compound chicken-and-egg problem to obtain reward. First it must recognize the cue and then it must learn that the cue must initiate a certain action. But how can the animal recognize the cue before it has learned the action to obtain reward, since in this initial learning stage the cue is only partially predictive of reward? Here we present a simple neural network model of how animals extract cues from background distractor stimulus, all presented with equal salience, based on successive testing of different stimulus-action allocations over several trials. A stimulus is selected and gated into working memory to drive an action and then reactivated at the end period to be reinforced if correct. If the stimulus is not reinforced over several trials it is suppressed and a different stimulus is selected. If the stimulus is a real cue but it drives the incorrect action, its cue-action allocation is suppressed. This mechanism is enhanced by the property of cue mutual exclusion in trials which also provides a simple model of bottom-up attention and pop-out. The model is based on the cortical and hippocampal projections to the dopamine system through the striatum including a model of salience gated working memory and a reinforcement and punishment system based on dopamine feedback balance. We illustrate the model by numerical simulations of a rat learning to navigate a T-maze and show how it deterministically discovers the correct cue-action allocations.