An adage posits that we learn from our mistakes; however, this is not entirely true. According to reinforcement learning theory, we learn when the expectation of our actions differs from outcomes. Here, we examined whether expectancy driven learning lends a role in motor learning. Given the vast amount of overlapping anatomy and circuitry within the brain with respect to reward and motor processes, it is appropriate to examine both motor control and expectancy processes within a singular task. In the current study, participants performed a line drawing task via tablet under conditions of changing expectancies. Participants were provided feedback in a reinforcement-learning manner, as positive (✓) or negative (x) based off their performance. Modulation of expected outcomes were reflected by changes in amplitude of the human event-related potential (ERP), the reward positivity. The reward positivity is thought to reflect phasic dopamine release from the mesolimbic dopaminergic system to the basal ganglia and cingulate cortex. Due to the overlapping circuitry of reward and motor pathways, another human ERP, the bereitschatftspotential (BP), was examined. The BP is implicated in motor planning and execution; however, the late aspect of the BP shares similarity with the contingent negative variability (CNV). Current evidence demonstrates a relationship between expectancy and reward positivity amplitude in a motor learning context, as well as modulation of the BP under difficult task conditions. Behavioural data supports prior literature and may suggest a connection between sensory motor prediction errors working in concert with reward prediction errors. Further evidence supports a frontal-medial evaluation system for motor errors. Additionally, results support prior evidence of motor plans being formed upon target observation and held in memory until motor execution, rather than their formation before movement onset. / Graduate

The reward positivity is a component of the human event-related brain potential (ERP) elicited by feedback stimuli in trial-and-error learning and guessing tasks. A prominent theory holds that the reward positivity reflects a reward prediction error that is differentially sensitive to the valence of the outcomes, namely, larger for unexpected positive events relative to unexpected negative events (Holroyd & Coles, 2002). Although the theory has found substantial empirical support, most of these studies have utilized either monetary or performance feedback to test the hypothesis. However, in apparent contradiction to the theory, a recent study found that unexpected physical punishments (a shock to the finger) also elicit the reward positivity (Talmi, Atkinson, & El-Deredy, 2013). Accordingly, these investigators argued that this ERP component reflects a salience prediction error rather than a reward prediction error. To investigate this finding further, I adapted the task paradigm by Talmi and colleagues to a more standard guessing task often used to investigate the reward positivity. Participants navigated a virtual T-maze and received feedback on each trial under two conditions. In a reward condition the feedback indicated that they would either receive a monetary reward or not for their performance on that trial. In a punishment condition the feedback indicated that they would receive a small shock or not at the end of the trial. I found that the feedback stimuli elicited a typical reward positivity in the reward condition and an apparently delayed reward positivity in the punishment condition. Importantly, this signal was more positive to the stimuli that predicted the omission of a possible punishment relative to stimuli that predicted a forthcoming punishment, which is inconsistent with the salience hypothesis. / Graduate / 0633 / 0317 / heydari@uvic.ca

Optimal decision-making entails outcome evaluation, comparing received costs and benefits with predicted costs and benefits. The anterior cingulate cortex (ACC), a brain area with major connections to the appetitive and aversive motivation systems, may provide the neural substrate of this evaluation process. One way to measure the relative contribution of these systems on decision-making is to measure individual differences in risk-taking behaviors. For individuals who make risky choices, this evaluation step may be biased: some show a preference for immediate, short-term rewards (increased appetitive system), while devaluing the long-term consequences of their choices (decreased aversive system). However, most studies supporting this theory have utilized monetary loss as the punishment. Other punishments that represent the presence of an aversive outcome, such as delivery of a painful stimulus, may be processed in a separate brain area and thus, may have differing effects on decision-making. The current study aimed to answer two main questions. First, we asked: is the ACC engaged by both appetitive stimuli and aversive stimuli? To answer this question, we recorded the Feedback-Related Negativity (FRN) response, a component thought to represent activity of the ACC, during a passive reward and punishment prediction task. Results indicated that the FRN responded to whether the outcome was A) unexpected and B) delivered or withheld, but not to the valence of the outcome. Second, we asked: do individual differences in these two systems have a differential impact on decision-making? To answer this question, participants completed a gambling task where they had to choose between large and small bets based on a probability of winning while we recorded their FRN response. They also completed self-report questionnaires indicating their sensitivity to reward/punishment and risk-taking behaviors. Results indicated that increased sensitivity of the appetitive system and decreased sensitivity of the aversive system (measured by both self-report and ERPs) predicted risky choice on the self-report measure and less so on the behavioral measure. Taken together, these results complement those that suggest the ACC is involved in evaluating both costs and benefits and may be influenced by both appetitive and aversive motivational systems.

The study of semantics in the brain explores how the brain represents, processes, and learns the meaning of language. In this thesis we show both that semantic representations can be decoded from electroencephalography data, and that we can detect the emergence of semantic representations as participants learn an artificial language mapping. We collected electroencephalography data while participants performed a reinforcement learning task that simulates learning an artificial language, and then developed a machine learning semantic representation model to predict semantics as a word-to-symbol mapping was learned. Our results show that 1) we can detect a reward positivity when participants correctly identify a symbol's meaning; 2) the reward positivity diminishes for subsequent correct trials; 3) we can detect neural correlates of the semantic mapping as it is formed; and 4) the localization of the neural representations is heavily distributed. Our work shows that language learning can be monitored using EEG, and that the semantics of even newly-learned word mappings can be detected using EEG. / Graduate

The Reward Positivity (RewP) is a neurophysiological marker of reward sensitivity that has been found to be impacted in depression. However, there have been some mixed findings regarding the relationship between the RewP and depression, suggesting there are other factors which impact this relationship. The current study investigated how the demographic factors of sex, age, and socio-economic status might moderate the RewP-depression relationship, and examined if these effects generalize across three different inventories for symptoms of depression. 194 people were recruited by random digit dialing (55.2% male, mean age = 51.34 years, mean monthly income = $6625.95). They completed the SCID, HAM-D, and IDAS measures of depression, and an EEG session in which they did a random guessing task to elicit the RewP. We found that there was a trend-level interaction of a moderate effect size between symptoms of depression, age, and sex in predicting RewP amplitude. Further exploration of this interaction revealed that for females, there was an interactive effect between age and symptoms of depression, such that for younger females, increased symptoms of depression were associated with a blunted RewP, and lower symptoms of depression were associated with an enhanced RewP. These effects were specific to the SCID, but did not generalize to the HAM-D or IDAS. Moreover, there was no interactive effects between age and depression symptoms for males, nor did SES interact with depression and other demographic factors in predicting the RewP. This study provides evidence that demographic factors can impact the strength and nature of the relationships between the RewP and depression, and that future researchers might wish to over-sample younger females when investigating other moderating factors of the RewP in order to increase power.

Substantial research has been done on the electroencephalogram (EEG) neural signals generated by feedback within a simple choice task, and there is much evidence for the existence of a reward prediction error signal generated in the anterior cingulate cortex of the brain when the outcome of this type of choice does not match expectations. However, less research has been done to date on the neural responses to intermediate outcomes in a multi-step choice task. Here, I investigated the neural signals generated by a complex, non-deterministic task that involved multiple choices before final win/loss feedback in order to see if the observed signals correspond to predictions made by reinforcement learning theory. In Experiment One, I conducted an EEG experiment to record neural signals while participants performed a computerized task designed to elicit the reward positivity, an event-related brain potential (ERP) component thought to be a biological reward prediction error signal. EEG results revealed a difference in amplitude of the reward positivity ERP component between experimental conditions comparing unexpected to expected feedback, as well as an interaction between valence and expectancy of the feedback. Additionally, results of an ERP analysis of the amplitude of the P300 component also showed an interaction between valence and expectancy. In Experiment Two, I used machine learning to classify epoched EEG data from Experiment One into experimental conditions to determine if individual states within the task could be differentiated based solely on the EEG data. My results showed that individual states could be differentiated with above-chance accuracy. I conclude by discussing how these results fit with the predictions made by reinforcement learning theory about the type of task investigated herein, and implications of those findings on our understanding of learning and decision-making in humans. / Graduate

Like other animals, humans explore to learn about the world, and exploit what we have learned in order to maximize reward. The trade-off between exploration and exploitation is a widely-studied topic that cuts across multiple domains, including animal ecology, economics, and computer science. This work approaches the explore-exploit dilemma from the perspective of cognitive neuroscience. In particular, how are our decisions to explore or exploit represented computationally? And how is that representation implemented in the brain? Experiment 1 examined neural signals following outcomes in a risk-taking task. Explorations – defined as slower responses – were preceded by an enhancement of the P300, a component of the human event-related brain potential thought to reflect a phasic release of norepinephrine from locus coeruleus. Experiment 2 revealed that the same neural signal precedes feedback in a learning task called a two-armed bandit. There, a reinforcement learning model was used to classify responses as either exploitations or explorations; exploitations were driven by previous rewards, and explorations were not. Experiments 3 and 4 extended these results in three important ways. First, evidence is presented that the neural signal observed in Experiments 1 and 2 was driven not only by the upcoming decision, but also by the preceding decision (perhaps even more so). Second, Experiments 3 and 4 involved increasingly larger action spaces. Experiment 3 involved choosing from among either 4, 9, or 16 options. Experiment 4 involved searching for rewards in continuous two-dimensional map. In both experiments, the feedback-locked P300 was enhanced following exploration. Third, exploitation was the more common strategy in Experiments 1 and 2. Thus, it was unclear whether the exploration-related P300 enhancement observed there was due to exploration per se, to exploration rate, or to the fact that exploration was rare compared to exploitation. Experiment 3 partially address this by eliciting different rates of exploration; the exploration-related P300 effect correlated with rate of exploration. In Experiment 4, exploration was more common than exploitation (in contrast to Experiments 1–3); even so, exploration was followed by a P300 enhancement. Together, Experiments 1–4 suggest the presence of a general neural system related to exploration that operates across multiple task types (discrete to continuous), regardless of whether exploration or exploitation is the more common task strategy. The proposed purpose of this neural signal is to interrupt one mode of decision-making (exploration) in favour of another (exploitation). / Graduate

Why is it that only some people who use drugs actually become addicted? In fact, addiction depends on a complicated process involving a confluence of risk factors related to biology, cognition, behaviour, and personality. Notably, all addictive drugs act on a neural system for reinforcement learning called the midbrain dopamine system, which projects to and regulates the brain's system for cognitive control, called frontal cortex and basal ganglia. Further, the development and expression of the dopamine system is determined in part by genetic factors that vary across individuals such that dopamine related genes are partly responsible for addiction-proneness. Taken together, these observations suggest that the cognitive and behavioral impairments associated with substance abuse result from the impact of disrupted dopamine signals on frontal brain areas involved in cognitive control: By acting on the abnormal reinforcement learning system of the genetically vulnerable, addictive drugs hijack the control system to reinforce maladaptive drug-taking behaviors. The goal of this research was to investigate this hypothesis by conducting a series of experiments that assayed the integrity of the dopamine system and its neural targets involved in cognitive control and decision making in young adults using a combination of electrophysiological, behavioral, and genetic assays together with surveys of substance use and personality. First, this research demonstrated that substance dependent individuals produce an abnormal Reward-positivity, an electrophysiological measure of a cortical mechanism for dopamine-dependent reward processing and cognitive control, and behaved abnormally on a decision making task that is diagnostic of dopamine dysfunction. Second, several dopamine-related neural pathways underlying individual differences in substance dependence were identified and modeled, providing a theoretical framework for bridging the gap between genes and behavior in drug addiction. Third, the neural mechanisms that underlie individual differences in decision making function and dysfunction were identified, revealing possible risk factors in the decision making system. In sum, these results illustrate how future interventions might be individually tailored for specific genetic, cognitive and personality profiles. / Graduate