Work is presented aimed at understanding the function of the basal ganglia in reward-related learning. Behavioral, fMRI, and computational techniques are used to examine basal ganglia activation during the reinforcement of physical (i.e., motoric) and cognitive (i.e., non-motoric) actions. In a single experiment design, participants received positive and negative reinforcement for performing actions in one of four possible directions depending on a color cue stimulus. During different phases of the experiment, participants performed either hand movements, eye movements, or covert attention shifts. Behavioral and fMRI data collected during the task were used to test predictions from simulated Reinforcement Learning (RL) agents trained on the same sequences of stimulus, action, and outcome experienced by the human participants. Behavioral data showed that participants were able to learn the three types of action equally well and at similar rates, providing behavioral evidence that a common algorithm might be involved. Further, RL simulations fit the learning of the three types of action equally well, suggesting that RL might be that common algorithm. A deconvolution analysis of striatal fMRI BOLD data suggested that: (1) the striatum computed reward prediction errors for both physical and cognitive actions, and (2) this computation was localized to different regions of the striatum depending on the type of action that was being rewarded. The localization of these computations replicates prior findings implicating those regions in action-specific voluntary control, but extends them to include the fact that these regions compute action-specific reward-prediction errors. Together, these data suggest that distinct circuits linking neocortex with the basal ganglia are involved in RL-related computations for the actions controlled by those circuits. The results point to a theoretical framework in which the basal ganglia mediate the reinforcement of actions whose control is delegated to local neocortical regions. Implications for theories of learning, theories of cognitive control, and mapping RL to the basal ganglia are discussed. Finally, the possibility that phasic dopamine might mediate the prediction error signals is considered in view of some theoretical challenges such as its non-specificity, its asymmetric representation of negative reinforcement, and its presence during non-rewarding events.
| Identifer | oai:union.ndltd.org:PITT/oai:PITTETD:etd-04212010-142229 |
| Date | 21 June 2010 |
| Creators | Laurent, Patryk Alix |
| Contributors | Mark E. Wheeler, Marc A. Sommer, A. David Redish, Peter L. Strick, Julie A. Fiez, Erik D. Reichle |
| Publisher | University of Pittsburgh |
| Source Sets | University of Pittsburgh |
| Language | English |
| Detected Language | English |
| Type | text |
| Format | application/pdf |
| Source | http://etd.library.pitt.edu/ETD/available/etd-04212010-142229/ |
| Rights | restricted, I hereby certify that, if appropriate, I have obtained and attached hereto a written permission statement from the owner(s) of each third party copyrighted matter to be included in my thesis, dissertation, or project report, allowing distribution as specified below. I certify that the version I submitted is the same as that approved by my advisory committee. I hereby grant to University of Pittsburgh or its agents the non-exclusive license to archive and make accessible, under the conditions specified below, my thesis, dissertation, or project report in whole or in part in all forms of media, now or hereafter known. I retain all other ownership rights to the copyright of the thesis, dissertation or project report. I also retain the right to use in future works (such as articles or books) all or part of this thesis, dissertation, or project report. |
Page generated in 0.0017 seconds