Memory plays an important role in defining how one behaves. The neurobiological mechanisms of memory have been studied extensively in animal models and the NMDA glutamate receptor has been identified to play an important role in the consolidation and reconsolidation of appetitive memories. Certain memories, depending on what was learned, can function differently and can be more difficult to disrupt based on a number of factors. Currently, no study has examined whether or not a reward-predictive stimulus attributed with incentive value is more difficult to disrupt than a stimulus that functions as a general reward-predictor. To determine the role of the NMDA receptor on memory consolidation with different functioning reward-predictive stimuli rats underwent a Pavlovian conditioned approach, where a post-session NMDA receptor antagonist was administered daily. Furthermore, to determine the role of the NMDA receptor on memory reconsolidation, another set of rats were trained on a Pavlovian conditioned approach task, after training was complete rats were presented with a reward-predictive stimuli followed by an administration of a NMDA receptor antagonist and then re-tested.

Caffeine is the most consumed psychoactive substance in the world, and most caffeine consumption in coffee and energy drinks is intended to produce a psychoactive effect. However, caffeine is not a primary reinforcer in preclinical paradigms – non-human species do not reliably take the drug to produce a psychoactive effect. However, caffeine is a ‘reinforcement enhancer’ in preclinical models; the effects of caffeine increase the motivation to obtain other non-drug reinforcers. The overall goal of this project was to determine if these reinforcement enhancing effects of caffeine could promote caffeine self-administration and to subsequently investigate the behavioral and neurochemical underpinnings of this effect. We hypothesized reliable caffeine self-administration would occur by adventitious pairing of caffeine with saccharin, a primary reinforcer. Second, we hypothesized that caffeine enhances reinforcement by increasing the salience of incentive stimuli, which are stimuli that come to evoke approach behaviors through associative learning (e.g., Pavlovian conditioning). Finally, incentive salience is moderated by dopamine release in the nucleus accumbens (NAc), an area highly involved in reward-learning and substance dependence. Therefore, we hypothesized that if caffeine enhanced control of approach behavior by incentives, then it would increase the ability of incentive stimuli to evoke dopamine in the NAc. These studies show that intravenous delivery of caffeine with oral saccharin increases operant relative to control groups responding for intravenous caffeine or oral saccharin. The effect was also dose-dependent, confirming that the psychoactive effects of caffeine increased behavior. We also extended this effect to an oral model of caffeine self-administration, which included a simple sweetener (saccharin) or a complex oral vehicle (saccharin with decaffeinated coffee) to mask the bitter taste of caffeine. Presenting caffeine with oral saccharin promoted self-administration, relative to saccharin alone and did not depend on the nature of the complexity of the vehicle. Caffeine also dose-dependently increased approach to an incentive stimulus and this effect was associated with increased extracellular dopamine in the NAc. These findings suggest caffeine enhances incentive motivation and that this effect may result from increases in CS-evoked striatal dopamine.

The pleasurable and euphoric effects of drugs play an important role in drug abuse; however, there are no established preclinical models for directly assessing the hedonic effects of drugs in rodents. The purpose of this dissertation was to investigate rat ultrasonic vocalizations (USVs) as a potential method for measuring positive affective states associated with amphetamine reinforcement. USVs are high-frequency social signals that rats use to communicate with one another. Calls in the 50-kHz range are thought to be a sign of positive affect, since they are elicited by naturally rewarding stimuli, and are modulated by mesolimbic dopamine activity. At the time this dissertation was begun, the majority of USV research focused on natural rewards, such as sex and social interactions, and the USVs associated with repeated exposure to a drug or appetitive desire for a drug had not been studied. Therefore, the objective of this dissertation was to characterize the production of 50-kHz USVs during repeated administration of amphetamine within different paradigms commonly used to study the behavioral and motivational effects of stimulants in rats. First, I found that the 50-kHz USVs elicited by amphetamine were sensitized by repeated exposure, showing that USV expression parallels the sensitization of mesolimbic circuitry that is involved in the development of addiction. Second, I found that rats produce conditioned anticipatory 50-kHz USVs during exposure to cues that predicted amphetamine, with the magnitude of anticipatory calling increasing as drug-cue associations were learned and strengthened. Third, I found that the number of unconditioned 50-kHz USVs produced during the initial amphetamine exposure predicted the subsequent expression of anticipatory 50-kHz USVs, the development of conditioned place preference for an amphetamine-paired environment, and corticosterone responses to the drug. Overall, these findings suggest that 50-kHz USVs are an expression of behavioral arousal associated with both the positive effects of amphetamine itself, and the incentive-motivational states elicited by drug-paired cues. In addition, they show that the intensity of the initial 50-kHz USV response to amphetamine reflects individual differences in sensitivity to drug reinforcement. / text

Psychological disorders such as autism, obsessive-compulsive disorder, drug addiction, and attention-deficit/hyperactivity disorder involve atypically persistent behavior and atypical activity of the neurotransmitter dopamine. Behavioral momentum theory states that the persistence of behavior in a context is determined by the reinforcement received previously in that context. Contexts previously associated with higher rates of reinforcement yield greater persistence of behavior than contexts previously associated with lower rates of reinforcement. According to a prominent hypothesis in behavioral neuroscience, dopamine mediates the incentive salience of a stimulus. A synthesis of behavioral momentum theory and the incentive salience hypothesis proposes similar roles for dopamine activity and reinforcement in determining the persistence of behavior in a context. The aim of this dissertation was to determine the extent to which a history of dopamine modulation in a context affects the subsequent persistence of behavior in extinction and relapse. Three groups of rats were trained to press a lever for food in two alternating contexts of a multiple schedule. Following a stable baseline, rats entered a treatment phase in which they received a drug or saline injection before and after sessions in each context. In the drug context, rats received the indirect dopamine agonist amphetamine, dopamine D1 antagonist SCH 23390, or a combination of amphetamine and SCH 23390 prior to the session and a saline injection following the session. The injection schedule was reversed for the saline context such that rats received a saline injection prior to each session in the saline context and a drug injection following the session. During an extinction phase, access to food was withheld. Response-independent food was then provided in each context to trigger reinstatement of responding. A history of dopamine agonism in a context increased the relative persistence of behavior, while a history of dopamine antagonism at D1 receptors and a combination of dopamine agonism and dopamine antagonism had little impact on the relative persistence of behavior. Likewise, reinstatement was relatively greater in a context previously associated with dopamine agonism. This effect was blocked when dopamine agonism was preceded by D1 antagonism. A history of D1 antagonism alone did not affect reinstatement. These results suggest that dopamine plays a role in the persistence of behavior in extinction and relapse, but that different dopamine receptors mediate these effects.

The aim of this thesis is to improve our understanding of the neural systems involved in schizophrenia by suggesting possible avenues for future computational modelling in an attempt to make sense of the vast number of studies relating to the symptoms and cognitive deficits relating to the disorder. This multidisciplinary research has covered three different levels of analysis: abnormalities in the microscopic brain structure, dopamine dysfunction at a neurochemical level, and interactions between cortical and subcortical brain areas, connected by cortico-basal ganglia circuit loops; and has culminated in the production of five models that provide useful clarification in this difficult field. My thesis comprises three major relevant modelling themes. Firstly, in Chapter 3 I looked at an existing neural network model addressing the Neurodevelopmental Hypothesis of Schizophrenia by Hoffman and McGlashan (1997). However, it soon became clear that such models were overly simplistic and brittle when it came to replication. While they focused on hallucinations and connectivity in the frontal lobes they ignored other symptoms and the evidence of reductions in volume of the temporal lobes in schizophrenia. No mention was made of the considerable evidence of dysfunction of the dopamine system and associated areas, such as the basal ganglia. This led to my second line of reasoning: dopamine dysfunction. Initially I helped create a novel model of dopamine neuron firing based on the Computational Substrate for Incentive Salience by McClure, Daw and Montague (2003), incorporating temporal difference (TD) reward prediction errors (Chapter 5). I adapted this model in Chapter 6 to address the ongoing debate as to whether or not dopamine encodes uncertainty in the delay period between presentation of a conditioned stimulus and receipt of a reward, as demonstrated by sustained activation seen in single dopamine neuron recordings (Fiorillo, Tobler & Schultz 2003). An answer to this question could result in a better understanding of the nature of dopamine signaling, with implications for the psychopathology of cognitive disorders, like schizophrenia, for which dopamine is commonly regarded as having a primary role. Computational modelling enabled me to suggest that while sustained activation is common in single trials, there is the possibility that it increases with increasing probability, in which case dopamine may not be encoding uncertainty in this manner. Importantly, these predictions can be tested and verified by experimental data. My third modelling theme arose as a result of the limitations to using TD alone to account for a reinforcement learning account of action control in the brain. In Chapter 8 I introduce a dual weighted artificial neural network, originally designed by Hinton and Plaut (1987) to address the problem of catastrophic forgetting in multilayer artificial neural networks. I suggest an alternative use for a model with fast and slow weights to address the problem of arbitration between two systems of control. This novel approach is capable of combining the benefits of model free and model based learning in one simple model, without need for a homunculus and may have important implications in addressing how both goal directed and stimulus response learning may coexist. Modelling cortical-subcortical loops offers the potential of incorporating both the symptoms and cognitive deficits associated with schizophrenia by taking into account the interactions between midbrain/striatum and cortical areas.

BACKGROUND: Alcohol use disorders (AUDs) affect 15 million people nationwide, 4% of which are adolescents (ages 12-17) and adolescents who binge drink significantly increase their likelihood of suffering from an AUD in adulthood. Research shows that cues (i.e. flavors) paired with alcohol (EtOH) produce significant cue-induced alcohol craving and contribute to relapse in adolescent and adult populations. However, there is a lack of research focused on how cues that accompany EtOH drinking during adolescence, affect EtOH craving later in life. The current study sought to examine the sex- and developmental-dependent effects of adolescent exposure to flavor cues associated with EtOH on operant-lick behavior and cue-induced dopamine (DA) levels within the nucleus accumbens shell (AcbSh; reward structure) in adulthood. METHODS: Adolescent alcohol-preferring (P) rats were randomly assigned to one of 4 groups and received 24 hr. access to three bottles on their home cage: Paired: 0.1% blueberry flavor extract (BB) + 15% v/v EtOH and 2 water bottles; Unpaired: 0.1% BB, 15% v/v EtOH, and water; 15% EtOH alone, and 2 water bottles; BB alone and 2 water bottles. Home cage fluid consumption was measured for 2-weeks. On the third week bottles were removed and all animals underwent 9 days of operant training using an operant sipper paradigm. This consisted of two sipper spouts connected to the computer by a lickometer, which registered tongue contacts with the sipper tube (Paired: BB+EtOH or water; Unpaired BB or EtOH; EtOH alone: EtOH or water; BB alone: BB or water). When the fixed ratio (FR) requirement for number of licks/tongue contacts was met, a liquid delivery solenoid dispensed 0.05 ml of fluid into the sipper tube. Following the final operant session all rats remained in their home-cage for approximately 40 days until adulthood at which point they were returned to the operant chambers and tested for appetitive and consummatory behavior in response to the flavor cue (all rats: BB or water; NO EtOH). Two weeks after the final operant session all rats underwent microdialysis testing to examine cue-induced DA levels in the AcbSh. RESULTS: Data indicated that animals in the paired group exhibited a significantly greater level of licking at the BB sipper and a significantly greater level of DA release in response to the flavor cue compared to the other groups. CONCLUSIONS: Overall, the data suggest that cues paired with EtOH during adolescence may produce persistent changes to the behavioral and neurobiological mechanisms that contribute to an increased risk of developing an AUD later in life.

Die Alkoholsucht ist ein global verbreitetes Phänomen und kennzeichnet sich durch eine Transition von kontrolliertem zu zwanghaftem Alkoholkonsum.Die Tendenz, einem neutralen Reiz eine Anreizwirkung zuzuschreiben, ist individuell unterschiedlich und stellt einen Risikofaktor für die Entwicklung einer Abhängigkeit dar.Zur Entwicklung spezifischer Präventionsstrategien ist ein besseres Verständnis der Zusammenhänge zwischen der Anreizsalienz und Alkoholabhängigkeit erforderlich.Der Übergang von mäßigem zu zwanghaftem Alkoholkonsum wurde durch das Modell des Alkoholentzugseffekts simuliert, das den menschlichen Alkoholrückfall nachahmt.Die Ratten erhielten freiwilligen Zugang zu verschieden Alkohollösungen mit wiederholten Deprivations- und Alkoholphasen.Die Ratten durchliefen zusätzlich den Pavlovian Conditioned Approach getestet, welcher die individuellen Tendenzen auf einen bedingten Reiz/Belohnung quantifiziert.Während des letzten ADE-Zyklus wurde mit Geschmacksverfälschung zwanghaftes Trinken ermittelt.Nach der Identifizierung zuverlässiger Prädiktoren für Suchtverhalten wurde präventive Neurostimulation durchgeführt, um die Tendenz der Tiere alkoholbezogenen Reizen eine motivationale Bedeutung beizumessen, zu beeinflussen, und die Manifestation eines Abhängigkeitsverhaltens zu verhindern. Weibchen tranken mehr Alkohol als Männchen und zeigten ST Verhalten im PavCA, während Männchen GT aufwiesen.Die Anwendung von transkranieller Gleichstromstimulation während PavCA führte zu mehr GT-Verhalten bei stimulierten Ratten.Frühe tDCS während des Trinkens hatte keinen Einfluss auf das akute und das Langzeit-Trinken. Die Ergebnisse zeigen eine komplexere Beziehung zwischen Anreizsalienz und Alkoholsucht und unterstreichen,individuelle Unterschiede und beide Geschlechter in der präklinischen Forschung zu berücksichtigen. / The consequences of alcohol dependence cause the global deaths of million people yearly.The ability of the environment can trigger dependent behavior and promote drinking.The tendency to attribute incentive salience to cues differs between subjects.By forming a cue-alcohol association, neutral cues receive motivational value,thereby predicting the likelihood of alcohol reward occurrence,known as Pavlovian learning.Understanding the relationship between incentive salience and alcohol addiction help inform treatment strategies.We study the relationship between incentive salience and alcohol addiction.The transition from moderate to compulsive alcohol intake can be captured by the alcohol deprivation effect rat model (mimics alcohol relapse in humans).Rats were given voluntary access to alcohol solutions with repeated abstinence/reintroduction phases.Further,rats were tested in the PavCA,which quantifies individual tendency toward a conditional cue and the reward, thus allowing to trace the process of attributing incentive salience to rewardcues.During the final ADE cycle,rats underwent a bitter taste adulteration test to assess for compulsive-like behavior.After identifying reliable predictors of addictive behavior,preventive tDCS was performed to influence the tendency of animals to attach motivational importance to alcohol-related stimuli,and to prevent the manifestation of alcohol addictive behavior.Females drank more alcohol than males and exhibited more ST behavior in the PavCA, whereas males showed GT behavior.PavCA phenotypes emerged early and remained stable.The application of tDCS during PavCA results in high GT numbers in stimulated rats.Early tDCS on drinking does not affect acute or long-term drinking.Our findings indicate a complex relationship between incentive salience and alcohol addiction and emphasize the importance of considering individual differences and both sexes in preclinical research.