The primary motor cortex (M1) is an important site for learning new motor skills. While rewardis known to both enhance and accelerate motor learning, the mechanism by which reward exertsthese effects remains unclear. Previous studies in primates have demonstrated reward-relatedactivity in M1, however, it is not known whether reward is represented among different neuronalcell types in M1, or if the representations change over the course of reward-based associativelearning. We begin by reviewing advances in optogenetic methods that have enabled thedissection of cortical circuits underlying sensorimotor behaviours with a special focus on thefunctional roles of cell-type specific connections in governing sensorimotor informationprocessing and learning and memory. We then used in vivo, two-photon calcium imaging tocharacterize reward and reward-related responses in pyramidal neurons (PNs), PV-INs, SST-INsand VIP-INs while mice simultaneously performed a head-fixed auditory classical conditioningtask. We found that different cell types had distinct responses to the conditioned stimulus (CS)and to reward, and these responses underwent differential changes over the course of associativelearning. Notably, VIP-INs preferentially represented reward and their reward responsesincreased with learning, while PV-INs preferentially represented the CS, and their CS responsesincreased with learning. Lastly, to identify which brain regions might provide reward-relatedinput to VIP-INs, we performed cell-type specific monosynaptic rabies tracing and generatedcomparative brain-wide maps of input to VIP-INs, PV-INs, SST-INs and PNs in M1. Weidentified preferential input from the orbital frontal cortex (ORB) to VIP-INs compared to theother IN subtypes. These results suggest that ORB may convey reward-related input to VIP-INsand thereby disinhibit local MOP circuitry during reward-based learning. Together, these studiesprovide a potential mechanism for how reward modulates motor learning.

Background: Disrupted reward learning has been suggested to contribute to the etiology and maintenance of depression. If deficits in reward learning are core to depression, we would expect that improving reward learning would decrease depression symptoms across time. Whereas previous studies have shown that changing reward learning can be done in a single study session, effecting clinically meaningful change in learning requires this change to endure beyond task completion and transfer to real world environments. With a longitudinal design, we investigate the potential for repeated sessions of behavioral training to create change in reward learning and decrease depression symptoms across time. Methods: 929 online participants (497 depression-present; 432 depression-absent) recruited from Amazon’s Mechanical Turk platform completed a behavioral training paradigm and clinical selfreport measures for up to eight total study visits. Participants were randomly assigned to one of 12 arms of the behavioral training paradigm, in which they completed a probabilistic reward learning task interspersed with queries about a feature of the task environment (11 learning arms) or a control query (1 control arm). Learning queries trained participants on one of four computational-based learning targets known to affect reinforcement learning (probability, average or extreme outcome values, and value comparison processes). A reinforcement learning model previously shown to distinguish depression related differences in learning was fit to behavioral responses using hierarchical Bayesian estimation to provide estimates of reward sensitivity and learning rate for each participant on each visit. Reward sensitivity captured participants’ value dissociation between high versus low outcome values, while learning rate informed how much participants learned from previously experienced outcomes. Mixed linear models assessed relationships between model-agnostic task performance, computational model-derived reinforcement learning parameters, depression symptoms, and study progression. Results: Across time, learning queries increased individuals’ reward sensitivities in depression-absent participants (β = 0.036, p =< 0.001, 95% CI (0.022, 0.049)). In contrast, control queries did not change reward sensitivities in depression-absent participants across time ((β = 0.016, p = 0.303, 95% CI (-0.015, 0.048)). Learning rates were not affected across time for participants receiving learning queries (β = 0.001, p = 0.418, 95% CI (-0.002, 0.004)) or control queries (β = 0.002, p = 0.558, 95% CI (-0.005, 0.009). Of the learning queries, those targeting value comparison processes improved depression symptoms (β = -0.509, p = 0.015, 95% CI (-0.912, - 0.106)) and increased reward sensitivities across time (β = 0.052, p =< 0.001, 95% CI (0.030, 0.075)) in depression-present participants. Increased reward sensitivities related to decreased depression symptoms across time in these participants (β = -2.905, p = 0.002, 95% CI (-4.75, - 1.114)). Conclusions: Multiple sessions of targeted behavioral training improved reward learning for participants with a range of depression symptoms. Improved behavioral reward learning was associated with improved clinical symptoms with time, possibly because learning transferred to real world scenarios. These results support disrupted reward learning as a mechanism contributing to the etiology and maintenance of depression and suggest the potential of repeated behavioral training to target deficits in reward learning. / Master of Science / Disrupted reward learning has been suggested to be central to depression. Work investigating how changing reward learning affects clinical symptoms has the potential to clarify the role of reward learning in depression. Here, we address this question by investigating if multiple sessions of behavioral training changes reward learning and decreases depression symptoms across time. We recruited 929 online participants to complete up to eight study visits. On each study visit participants completed a depression questionnaire and one of 12 arms of a behavioral training paradigm, in which they completed a reward learning task interspersed with queries about the task. Queries trained participants on one of four learning targets known to affect reward learning (probability, average or extreme outcome values, and value comparison processes). We used reinforcement learning to quantify specific reward learning processes, including how much participants valued high vs. low outcomes (reward sensitivity) and how much participants learned from previously experienced outcomes (learning rates). Across study visits, we found that participants without depression symptoms that completed the targeted behavioral training increased reward sensitivities (β = 0.036, p =< 0.001, 95% CI (0.022, 0.049)). Of the queries, those targeting value comparison processes improved both depression symptoms (β = -0.509, p = 0.015, 95% CI (-0.912, -0.106)) and reward sensitivities (β = 0.052, p =< 0.001, 95% CI (0.030, 0.075)) across study visits for participants with depression symptoms. These results suggest that multiple sessions of behavioral training can increase reward learning across time for participants with and without depression symptoms. Further, these results support the role of disrupted reward learning in depression and suggest the potential for behavioral training to improve both reward learning and symptoms in depression.

No description available.

Humans tend to preferentially process information relevant to themselves. For instance, in experiments where participants learn to manipulate stimuli referenced to themselves or someone else, participants exhibit larger reward processing signals for themselves. Additionally, attention and perception are biased not only towards one’s self but those related to them. However, the aspect of processing information related to known-others has not been addressed in reward learning. Here, I sought to address this issue. Specifically, I recorded electroencephalographic (EEG) data from 15 undergraduate student participants who played a simple two-choice “bandit” gambling game where a photo presented before each gamble indicated whether it benefited either the participant, an individual they knew, or a stranger. EEG data from 64 electrodes on a standard 10-20 layout were analyzed for event-related potentials (ERPs) elicited by target photos and gambling outcomes. Post experiment, I examined the relationship between relatedness and the amplitude reward learning ERPs, namely the reward positivity and the P300, with one-way repeated measures analyses of variance. My results demonstrate that the amplitudes of reward learning ERPs are sensitive to the target of a gamble. A secondary goal of this research was to determine if these differences could be explained by attentional and perceptual responses to cues of who a given gamble was for. Indeed, stepwise linear regression analyses identified the P2, N2, and P3 indexed relevance to self as predictors of resultant reward signals. My findings provide further evidence that a reward learning system within the medial-frontal cortex is sensitive to others with varying self-relevance, which may be a function of biases in attention and perception. / Graduate

The role of chemokines as chemotactic cytokines and their functions in the immune system and related pathologies are well defined. Recently, strong evidence supporting the hypothesis that chemokines can act as modulators of neuronal activity and influence neurotransmission has been reported. The chemokine CXCL12 is constitutively expressed in adult brain and expression of CXCL12 and its cognate receptor CXCR4 have been reported in regions of rat brain that construct dopamine (DA) and glutamate (GLU) pathways such as ventral tegmental area (VTA), substantia nigra (SN), and nucleus accumbens (NAc). In the central nervous system (CNS), activation of CXCR4 on dopaminergic neurons and astrocytes initiate cascade of events that leads to DA and GLU release and influence synaptic transmission. In vivo, intracerebroventricular (ICV) CXCL12 has been shown to potentiate cocaine-induced locomotor activity. Based on these evidences, the studies, as outlined in this dissertation, aimed to expand understanding of how CXCL12/CXCR4 interaction can affect cocaine-induced behavior and reinforcement, with special focus on mechanisms involving GLU. We first evaluated involvement of CXCR4 activation by CXCL12 on cocaine-induced locomotor activity using a selective CXCR4 antagonist AMD3100. Results demonstrated that AMD3100 (5, 10 mg/kg, IP) pretreatment dose-dependently attenuated cocaine-induced locomotor activity without affecting the baseline activity. Thereafter, effects of AMD3100 on cocaine’s reinforcing efficacy were tested using a biased conditioned place preference (CPP) paradigm. In all CPP experiments, saline pretreated controls established a significant preference for the cocaine-paired context following four pairings with cocaine (10 mg/kg, IP). Rats pretreated with 2.5 and 5 mg/kg AMD3100 prior to each pairing session showed significantly lower preference for the cocaine-paired side, whereas rats pretreated with 1 mg/kg AMD3100 showed similar preference for the cocaine-paired side as the saline controls when tested in the absence of the drug. Rats pretreated with AMD3100 (5 mg/kg, IP) just once prior to testing showed significantly lower preference for the cocaine-paired side. These results demonstrate that CXCR4 antagonism reduces development and expression of cocaine-induced CPP. Intravenous cocaine self-administration (SA) was performed to examine the effects of AMD3100 on the acquisition of cocaine-taking behavior and reinstatement to cocaine-seeking. Acquisition of cocaine SA was studied using three doses of cocaine (0.375, 0.5, 0.75 mg/kg/infusion) on a fixed-ratio 1 (FR-1) schedule of reinforcement. Two doses of AMD3100 (5, 10 mg/kg, IP) were tested. In all SA experiments, saline pretreated controls readily acquired cocaine self-administration. The lower and higher AMD3100 decreased the number of reinforcers earned during the two hour sessions compared to saline controls when tested against acquisition of 0.375 and 0.5 mg/kg/infusion cocaine. However, when tested against the 0.75 mg/kg/infusion cocaine, only 10 mg/kg of AMD3100 resulted in reduction in responding, but not the lower dose. When a dose response curve was plotted using all doses of cocaine and AMD3100 tested, the effects of AMD3100 were represented by a significant downward shift of the dose response curve. Effects of AMD3100 on cocaine-seeking were evaluated using the reinstatement model, in which rats that were extinguished from self-administration of 0.5 mg/kg/infusion cocaine underwent reinstatement testing with cue or cue + drug presentation. A compound cue (light and tone stimulus), that was used during the acquisition training, and submaximal dose of cocaine (5mg/kg, IP) were used. In both reinstatement conditions, rats pretreated with saline reinstated following the presentation of cue or cue + drug prime. Rats pretreated with AMD3100, 30 minutes prior to reinstatement session, showed significantly lower number of lever presses indicating drug-seeking was not as robust compared to the saline pretreated controls. Following the observations that cocaine-induced behaviors may be partially mediated by CXCL12/CXCR4 interaction, neurochemical changes were examined to elucidate the underlying mechanisms. CXCR4 immunoreactivity in prefrontal cortex (PFC) following withdrawal from 7 days of repeated cocaine (15 mg/kg, IP) was evaluated. Although positive CXCR4 immunoreactivity was observed in PFC, there were no significant differences in the intensity of CXCR4 expression compared to the saline treated controls at acute (2 hours, 2 and 10 days) or protracted (30 days) withdrawal time points. In contrast, CXCL12 protein levels in PFC and NAc were negatively influenced after protracted withdrawal from 7 days of repeated cocaine (15 mg/kg, IP) administration. Following assessment of regional expression of CXCR4, cellular expression was evaluated using triple labeling immunohistochemistry which revealed the positive CXCR4-immunoreactivity on cells staining positively for vesicular glutamate transporter 1 (vGlut1) and glial fibrillary acidic protein (GFAP) showing that the glutamatergic neurons and astrocytes in the PFC express CXCR4. To assess the effects of CXCL12 on GLU transmission, microdialysis of NAc was performed. Following unilateral injection of 50 ng CXCL12 into the lateral ventricle, increase in extracellular GLU was observed. In a follow up study, AMD3100 pretreatment (10 mg/kg, IP) attenuated cocaine-induced increase in extracellular GLU in the NAc. Since the glutamate transporter subtype-1 (GLT-1) is a major regulator of extracellular GLU and upregulation of GLT-1 expression and function has been shown to attenuate reinstatement to cocaine-seeking, its expression was evaluated using immunoblot analysis of the NAc and PFC of rats that self-administered cocaine. The results revealed that AMD3100 (10 mg/k, IP; daily injection 30 min prior to SA session) pretreatment upregulated GLT-1 levels in the NAc but not PFC. In summary, results of the present study show that CXCL12/CXCR4 interaction may modulate cocaine-induced behavioral effects including reinforcement and reinstatement to cocaine-seeking. Neurochemical assessments revealed the presence of CXCR4-expressing glutamatergic neurons in the PFC and AMD3100-induced up-regulation of GLT-1. Most importantly, we provided direct evidence of CXCL12/CXCR4 mediated GLU transmission in NAc. Together, these results expand our understanding of chemokine’s role as neuromodulators and identify CXCR4 as a novel target for development of new pharmacotherapies for the treatment of cocaine addiction. / Pharmacology

Real world decision-making requires simultaneously determining what we are observing in our environment (perceptual decision-making; PDM) and what the stimuli and actions are worth (reward-based decision-making; RDM). There is evidence of a bi-directional relationship between reward and perceptual information in guiding choice, with some studies suggesting that individuals optimally combine the two. Uncertainty in both reward expectations and perception have been shown to alter choice behavior, however few studies have manipulated both variables simultaneously. Given the distinct theoretical and computational foundations of PDM and RDM, it has also been assumed that the underlying behavioral and neural substrates of perceptual and reward-based choice are separable. However, there is evidence that task structure and subjective value/uncertainty more generally contribute to activity in large-scale networks of the brain, rather than domain specific features (perceptual salience/reward). Variability in task structures and methods of manipulating and modeling sensory and reward uncertainty, make it hard to draw definitive conclusions across these perspectives with currently available data. The current study used behavioral and fMRI techniques to investigate the neurobehavioral substrates of decision-making under simultaneous perceptual and reward uncertainty in a sample of healthy adult volunteers. The primary objectives of this project were to test: a) how simultaneous manipulations in sensory and reward uncertainty influence choice, b) whether task structure alters the influence of sensory and reward information on choice behavior, and c) whether activity in underlying neural substrates reflect domain-specific or domain-general processes. Results showed that choices were best predicted by a combined model of perceptual salience and reward, with an overall bias towards perceptual salience information. Choice percentage was not impacted by task structure, however choices were better predicted by individual features (salience and reward) when they were manipulated stably, than dynamically. Activity in the brain showed greater overlap between dynamic task conditions when compared to both salience and reward conditions. There was also greater overlap between stable task conditions when compared to reward but not salience conditions. Preliminary evidence suggests that activity in decision-relevant regions of the brain varied by uncertainty and value rather than salience and reward per se. / Doctor of Philosophy / Real world decision-making requires knowing what things in our environment are and knowing what they are worth. Uncertainty about what different options are, can affect the actions we take and the outcomes we expect. Uncertainty about the outcome of our choices or actions can also influence how attend to or consider certain options when making choices. We also know that context can affect our behavior. For example, when our environment changes frequently, choices that might have been good in the past, may not necessarily be the best course of action in the future. This can add further confusion about what to do. Across several mental health conditions, we see that problems arise when individuals need to take actions based on incomplete, uncertain, or frequently changing information. Real world decision-making requires knowing what things in our environment are and knowing what they are worth. Uncertainty about what different options are, can affect the actions we take and the outcomes we expect. Uncertainty about the outcome of our choices or actions can also influence how we attend to or consider options when making choices. We also know that context can affect our behavior. For example, when our environment changes frequently, choices that might have been good in the past, may not necessarily be the best course of action in the future. This can add further confusion about what to do. Across several mental health conditions, we see that problems arise when individuals need to take actions based on incomplete, uncertain, or frequently changing information. The first goal of this study was to better understand what healthy individuals do when they are faced with different levels of uncertainty around about what different options are (through changes in visual clarity), and what they are worth (through changes in probability of reward). A second goal was to see whether how the frequency at which choice clarity and outcomes change, effects the kinds of choices people make. Third, the study used a measure of brain activity, to determine what the brain is doing which participants make these complex decisions. Results showed that people's choices were best predicted by considering both clarity of the options and their outcome. Having certainty about the identity of the choices was more important than the value of those choices. Also, information about clarity and value of options were more likely to be considered when they were stable, versus when they were changing frequently. Decision-relevant regions of the brain seemed to respond most to overall information about uncertainty and stability of options rather than their clarity or outcome value per se. Future research should test these findings in a larger sample, further explore individual differences in how people respond to various types of uncertainty and determine how knowledge of these individual differences can inform personalized treatment for individuals with related mental health challenges.

To determine whether incomplete reward reduction and reward increment would influence the "frustration effect" (FE) in double alleyway, rats were given 153 trials with 50% partial reward in Goalbox 1 with 4 or 12 pellets in Goalbox 1 and 4 or 12 pellets in Goalbox 2. On trials 106-153, the postshift phase, half of the rats in each group were shifted to a reward magnitude opposite to that previously received in Goalbox 1. During preshift, Alley 1 start and running speeds were inversely related to Goalbox 2 reward magnitude. The size of the FE was directly related to Goalbox 1 and Goalbox 2 reward magnitude. During postshift, Alley 1 running were inversely related to preshift Goalbox 1 reward magnitude. The size of the FE was directly related to Goalbox 1 and Goalbox 2 reward magnitude. The results were discussed in terms of Amsel's theory of frustration. / Master of Science

Literature Review: Anhedonia, the loss of interest or pleasure in usually pleasurable activities, is a core symptom of depression and is associated with a reduction in positive affect (PA). Repetitive negative thought (RNT) is implicated in the development and maintenance of psychiatric disorders. It has been hypothesised that RNT causally contributes to anhedonia. The aim of this review was to explore this relationship to answer two questions: Is there a relationship between RNT and anhedonia? Does RNT causally contribute to anhedonia? Review inclusion criteria were: studies using standardised measures to report a relationship between RNT and anhedonia or reduced PA. Results suggest that cross-sectional and longitudinal studies identify a relationship between RNT and anhedonia. Preliminary evidence from experimental studies shows that RNT causally contributes to anhedonia. Limitations within the field are that anhedonia is rarely measured directly or behaviourally. Future research is warranted to explore the relationship between RNT and anhedonia with a particular focus on direct and behavioural measures of anhedonia. Empirical Paper: It is hypothesised that repetitive negative thought (RNT) causally contributes to anhedonia. There is cross-sectional and longitudinal evidence of this relationship, but it has not previously been investigated directly using experimental methods. In the present study, student participants were randomly assigned to an unresolved goal (RNT) manipulation (n = 43) or resolved goal (control) manipulation (n =41) prior to completing a reward sensitivity task. This task has been reliably found to train a response bias towards the stimuli that is differentially positively reinforced, with both depression and self-reported anhedonia associated with a reduced response bias. The unresolved goal versus resolved goal manipulation was effective, with the unresolved condition producing significantly higher levels of RNT during the reward sensitivity task relative to the resolved condition. Inconsistent with study predictions, there was no significant difference between the conditions on response bias, although there were trend findings, which tentatively suggest that RNT may influence anhedonia. Potential accounts for the null findings and future research are discussed.

The reward positivity is a component of the human event-related brain potential (ERP) elicited by feedback stimuli in trial-and-error learning and guessing tasks. A prominent theory holds that the reward positivity reflects a reward prediction error that is differentially sensitive to the valence of the outcomes, namely, larger for unexpected positive events relative to unexpected negative events (Holroyd & Coles, 2002). Although the theory has found substantial empirical support, most of these studies have utilized either monetary or performance feedback to test the hypothesis. However, in apparent contradiction to the theory, a recent study found that unexpected physical punishments (a shock to the finger) also elicit the reward positivity (Talmi, Atkinson, & El-Deredy, 2013). Accordingly, these investigators argued that this ERP component reflects a salience prediction error rather than a reward prediction error. To investigate this finding further, I adapted the task paradigm by Talmi and colleagues to a more standard guessing task often used to investigate the reward positivity. Participants navigated a virtual T-maze and received feedback on each trial under two conditions. In a reward condition the feedback indicated that they would either receive a monetary reward or not for their performance on that trial. In a punishment condition the feedback indicated that they would receive a small shock or not at the end of the trial. I found that the feedback stimuli elicited a typical reward positivity in the reward condition and an apparently delayed reward positivity in the punishment condition. Importantly, this signal was more positive to the stimuli that predicted the omission of a possible punishment relative to stimuli that predicted a forthcoming punishment, which is inconsistent with the salience hypothesis. / Graduate / 0633 / 0317 / heydari@uvic.ca

Firms have since long given their regular customers special treatment. With the help of IT, many firms have established formal ways to do this. An example is a so-called customer rewards program (CRP), by which the firm rewards the customer for repeated purchase. Firms allocate large resources in these programs with millions of customers enrolled. Hence, it seems important that the CRP works effectively. By effective we mean that it increases sales. Whether it is effective or not is a matter of how it is designed. A CRP typically comes with membership levels. We study how many membership levels the firm should offer in an effective program. We also study if customers prefer individual or group rewards and whether a CRP can break and create habitual purchasing behavior. In the study, we also analyze under what conditions the customer prefers a CRP over a sales promotion. In general, the study adds to the understanding of Customer Rewards Programs as an incentive structure. There are many different ways to design these incentives and especially the continuing development of IT is expected to influence the future design and role of these types of programs. This study is part of the Swedish Research School of Management and Information Technology (MIT) which is one of 16 national research schools supported by the Swedish Government. MIT is jointly operated by the following institutions: Blekinge Institute of Technology, Gotland University College, Jönköping International Business School, Karlstad University, Linköping University, Lund University, Mälardalen University College, Stockholm University, Växjö University, Örebro University, IT University of Göteborg, and Uppsala University, host to the research school. At the Swedish Research School of Management and Information Technology (MIT), research is conducted, and doctoral education provided, in three fields: management information systems, business administration, and informatics.