Background: Disrupted reward learning has been suggested to contribute to the etiology and maintenance of depression. If deficits in reward learning are core to depression, we would expect that improving reward learning would decrease depression symptoms across time. Whereas previous studies have shown that changing reward learning can be done in a single study session, effecting clinically meaningful change in learning requires this change to endure beyond task completion and transfer to real world environments. With a longitudinal design, we investigate the potential for repeated sessions of behavioral training to create change in reward learning and decrease depression symptoms across time. Methods: 929 online participants (497 depression-present; 432 depression-absent) recruited from Amazon’s Mechanical Turk platform completed a behavioral training paradigm and clinical selfreport measures for up to eight total study visits. Participants were randomly assigned to one of 12 arms of the behavioral training paradigm, in which they completed a probabilistic reward learning task interspersed with queries about a feature of the task environment (11 learning arms) or a control query (1 control arm). Learning queries trained participants on one of four computational-based learning targets known to affect reinforcement learning (probability, average or extreme outcome values, and value comparison processes). A reinforcement learning model previously shown to distinguish depression related differences in learning was fit to behavioral responses using hierarchical Bayesian estimation to provide estimates of reward sensitivity and learning rate for each participant on each visit. Reward sensitivity captured participants’ value dissociation between high versus low outcome values, while learning rate informed how much participants learned from previously experienced outcomes. Mixed linear models assessed relationships between model-agnostic task performance, computational model-derived reinforcement learning parameters, depression symptoms, and study progression. Results: Across time, learning queries increased individuals’ reward sensitivities in depression-absent participants (β = 0.036, p =< 0.001, 95% CI (0.022, 0.049)). In contrast, control queries did not change reward sensitivities in depression-absent participants across time ((β = 0.016, p = 0.303, 95% CI (-0.015, 0.048)). Learning rates were not affected across time for participants receiving learning queries (β = 0.001, p = 0.418, 95% CI (-0.002, 0.004)) or control queries (β = 0.002, p = 0.558, 95% CI (-0.005, 0.009). Of the learning queries, those targeting value comparison processes improved depression symptoms (β = -0.509, p = 0.015, 95% CI (-0.912, - 0.106)) and increased reward sensitivities across time (β = 0.052, p =< 0.001, 95% CI (0.030, 0.075)) in depression-present participants. Increased reward sensitivities related to decreased depression symptoms across time in these participants (β = -2.905, p = 0.002, 95% CI (-4.75, - 1.114)). Conclusions: Multiple sessions of targeted behavioral training improved reward learning for participants with a range of depression symptoms. Improved behavioral reward learning was associated with improved clinical symptoms with time, possibly because learning transferred to real world scenarios. These results support disrupted reward learning as a mechanism contributing to the etiology and maintenance of depression and suggest the potential of repeated behavioral training to target deficits in reward learning. / Master of Science / Disrupted reward learning has been suggested to be central to depression. Work investigating how changing reward learning affects clinical symptoms has the potential to clarify the role of reward learning in depression. Here, we address this question by investigating if multiple sessions of behavioral training changes reward learning and decreases depression symptoms across time. We recruited 929 online participants to complete up to eight study visits. On each study visit participants completed a depression questionnaire and one of 12 arms of a behavioral training paradigm, in which they completed a reward learning task interspersed with queries about the task. Queries trained participants on one of four learning targets known to affect reward learning (probability, average or extreme outcome values, and value comparison processes). We used reinforcement learning to quantify specific reward learning processes, including how much participants valued high vs. low outcomes (reward sensitivity) and how much participants learned from previously experienced outcomes (learning rates). Across study visits, we found that participants without depression symptoms that completed the targeted behavioral training increased reward sensitivities (β = 0.036, p =< 0.001, 95% CI (0.022, 0.049)). Of the queries, those targeting value comparison processes improved both depression symptoms (β = -0.509, p = 0.015, 95% CI (-0.912, -0.106)) and reward sensitivities (β = 0.052, p =< 0.001, 95% CI (0.030, 0.075)) across study visits for participants with depression symptoms. These results suggest that multiple sessions of behavioral training can increase reward learning across time for participants with and without depression symptoms. Further, these results support the role of disrupted reward learning in depression and suggest the potential for behavioral training to improve both reward learning and symptoms in depression.

Psychotic symptoms are prevalent in a wide variety of psychiatric and neurological disorders. Yet, despite decades of research, the neurobiological mechanisms via which these symptoms come to manifest themselves remain to be elucidated. I argue in this thesis that using a mechanistic approach towards understanding psychosis that borrows heavily from the predictive coding framework, can help us understand the relationship between neurobiology and symptomology. In the first results chapter I present new data on a biomarker that has often been cited in relation to psychotic disorders, which is glutamate levels in the anterior cingulate cortex (ACC), as measured with magnetic resonance spectroscopy. In this chapter I aimed to replicate previous results that show differences in glutamate levels in psychosis and health. However, no statistically significant group differences and correlations with symptomology were found. In order to elucidate the potential mechanism underlying glutamate changes in the anterior cingulate cortex in psychosis, I tested whether a pharmacological challenge of Bromocriptine or Sulpiride altered glutamate levels in the anterior cingulate cortex. However, no significant group differences were found, between medication groups. In the second results chapter I aimed to address a long-standing question in the field of computational psychiatry, which is whether prior expectations have a stronger or weaker influence on inference in psychosis. I go on to show that this depends on the origin of the prior expectation and disease stage. That is, cognitive priors are stronger in first episode psychosis but not in people at risk for psychosis, whereas perceptual priors seem to be weakened in individuals at risk for psychosis compared to healthy individuals and individuals with first episode psychosis. Furthermore, there is some evidence that these alterations are correlated with glutamate levels. In the third results chapter I aimed to elucidate the nature of reward prediction error aberrancies in chronic schizophrenia. There has been some evidence suggesting that schizophrenia is associated with aberrant coding of reward prediction errors during reinforcement learning. However it is unclear whether these aberrancies are related to disease years and medication use. Here I provide evidence for a small but significant alteration in the coding of reward prediction errors that is correlated with medication use. In the fourth results chapter I aimed to study the influence of uncertainty on the coding of unsigned prediction errors during learning. It has been hypothesized by predictive coding theorists that dopamine plays a role in the precision-weighting of unsigned prediction error. This theory is of particular relevance to psychosis research, as this might provide a mechanism via which dopamine aberrancies, might lead to psychotic symptoms. I found that blocking dopamine using Sulpiride abolishes precision-weighting of unsigned prediction error, providing evidence for a dopamine mediated precision-weighting mechanism. In the fifth results chapter I aimed to extend this research into early psychosis, to elucidate whether psychosis is indeed associated with a failure to precision-weight prediction error. I found that first episode psychosis is indeed associated with a failure to precision-weight prediction errors, an effect that is explained by the experience of positive symptoms. In the sixth results chapter I explore whether the degree of precision-weighting of unsigned prediction errors is correlated with glutamate levels in the anterior cingulate cortex. Such a correlation might be plausible given that psychosis has been associated with both. However, I did not find such a relationship, even in a sample of 137 individuals. Thus I concluded that anterior cingulate glutamate levels might be more related to non-positive symptoms associated with psychotic disorders. In summary, a mechanistic approach towards understanding psychosis can give us valuable insights into the disease mechanisms at play. I have shown here that the influence of expectations on perception is different across disease stage in psychosis. Furthermore, aberrancies in prediction error mechanisms might explain positive symptoms in psychosis, a process likely mediated by dopaminergic mechanisms, whereas evidence for glutamatergic mediation remains absent.

Impaired social functioning conferred by mental illnesses has been constantly implicated in previous literatures. However, studies of social abnormalities in psychiatric conditions are often challenged by the difficulties of formalizing dynamic social exchanges and quantifying their neurocognitive underpinnings. Recently, the rapid growth of computational psychiatry as a new field along with the development of multiplayer economic paradigms provide powerful tools to parameterize complex interpersonal processes and identify quantitative indicators of social impairments. By utilizing these methodologies, the current set of studies aimed to examine social decision making during multiplayer economic games in participants diagnosed with depression (study 1) and combat-related post-traumatic stress disorder (PTSD, study 2), as well as an online population with elevated symptoms of borderline personality disorder (BPD, study 3). We then quantified and disentangled the impacts of multiple latent decision-making components, mainly social valuation and social learning, on maladaptive social behavior via explanatory modeling. Different underlying alterations were revealed across diagnoses. Atypical social exchange in depression and BPD were found attributed to altered social valuation and social learning respectively, whereas both social valuation and social learning contributed to interpersonal dysfunction in PTSD. Additionally, model-derived indices of social abnormalities positively correlated with levels of symptom severity (study 1 and 2) and exhibited a longitudinal association with symptom change (study 1). Our findings provided mechanistic insights into interpersonal difficulties in psychiatric illnesses, and highlighted the importance of a computational understanding of social function which holds potential clinical implications in differential diagnosis and precise treatment. / Doctor of Philosophy / People with psychiatric conditions often suffer from impaired social relationships due to an inability to engage in everyday social interactions. As different illnesses can sometimes produce the same symptoms, social impairment can also have different causes. For example, individuals who constantly avoid social activities may find them less interesting or attempt to avoid potential negative experiences. While those who display elevated aggression may have a strong desire for social dominance or falsely believe that others are also aggressive. However, it is hard to infer what drives these alterations by just observing the behavior. To address this question, we enrolled people with three different kinds of psychopathology to play an interactive game together with another player and mathematically modeled their latent decision-making processes. By comparing their model parameters to those of the control population, we were able to infer how people with psychopathology made the decisions and which part of the decision-making processes went wrong that led to disrupted social interactions. We found altered model parameters differed among people with major depression, post-traumatic stress disorder and borderline personality disorder, suggesting different causes underlying impaired social behavior observed in the game, the extent of which also positively correlated with their psychiatric symptom severity. Understanding the reasons behind social dysfunctions associated with psychiatric illnesses can help us better differentiate people with different diagnoses and design more effective treatments to restore interpersonal relationships.

Psychotic experiences (PEs) occur in the general population, beyond psychotic disorders. PEs are a risk factor for mental ill health in young people but can occur benignly in selected samples of adults. Environmental factors predispose to PEs but their underlying mechanisms are not well-understood. Progress in understanding PEs may be limited by diverse conceptualisations, imprecise measurement and a lack of explanatory frameworks that can bridge the gaps between aetiological factors, their effects on the brain and their behavioural manifestations. In this thesis, I undertook a comprehensive investigation of the measurement, health implications, aetiology and computational mechanisms of PEs in adolescents and young adults using data from two large cohort samples, supplemented with smaller-scale behavioural studies. I first investigated the measurement of PEs. I assessed and optimised the measurement of PEs in young people by two self-report instruments. I then used latent variable modelling to show that a self-report and interview instrument measured the same underlying psychotic phenomena. Both instruments were able to measure severe PEs, while the self-report questionnaire also measured more mild psychotic phenomena. I then investigated the health implications of PEs. Using cluster analysis in both cohorts, I found replicable patterns of PEs at similar levels of intensity and persistence but with and without depressive symptoms and with varying risk of mental disorder. Paranoid ideation was more associated with depressive symptoms than non-paranoid unusual perceptions and beliefs. Childhood adversity was associated with both PE-prone groups, but later social support from family and friends was far higher in those with PEs and low depressive symptoms than those with PEs and high depressive symptoms. Subsequently, I investigated the role of the social environment in the development of PEs and psychopathology using longitudinal structural equation modelling. I found that asocial dispositions increased or preceded increase in PEs over one year, mediated by detriment to social support. Conversely, PEs did not precede or increase asociality. I then showed that dimensions of PEs and depressive symptoms were promoted by childhood adversity but differentially affected by later social support, with paranoid ideation being more influenced by support than non-paranoid unusual perceptions/beliefs. Finally, I investigated specific mechanisms of PEs in two behavioural studies. In the seventh study, I used computational modelling of reward learning to link PEs to reduced ability to modulate learning by confidence, replicating computational effects of a pharmacological model of psychosis. I also used a novel visual task to show that the manifestation of PEs as anomalous perceptions versus anomalous beliefs might be explained by over-reliance on different types of prior knowledge in perceptual inference. These results suggest that different conceptual approaches to PEs might be synthesised despite issues with their measurement. PEs in young people, while not entirely benign, are heterogeneously associated with psychopathology. Importantly, they characterise a minority of young people who are at very high transdiagnostic risk of mental illness but also occur without distress in young people, often in the context of a supportive social environment. Health outcomes in young people with PEs are predicted and potentially modified by social functioning and social relationships. PEs might arise from atypicalities in how the influences of information sources on perception and belief-updating are modulated according to their reliabilities.

Computational psychiatry describes psychological phenomena as abnormalities in biological computations. Current available technologies span multiple organizational and temporal domains, but there remains a knowledge gap with respect to neuromodulator dynamics in humans. Recent efforts by members of the Montague Laboratory and collaborators adapted fast scan cyclic voltammetry (FSCV) from rodent experiments for use in human patients already receiving brain surgery. The process of modifying established FSCV methods for clinical application has led improved model building strategies, and a new "random burst" sensing protocol. The advent of random burst sensing raises questions about the capabilities of in-vivo electrochemistry techniques, while opening introducing possibilities for novel approaches. Through a series of in-vitro experiments, this study aims to explore and validate novel electrochemical sensing approaches. Initial expository experiments tested assumptions about waveform design to detect dopamine concentrations by reducing amplitude and duration of forcing functions, as well as distinguishing norepinephrine concentrations. Next, large data sets collected on mixtures of dopamine, serotonin and pH validated a newly proposed "low amplitude random burst sensing" protocol, for both within-probe and out-of-probe modeling. Data collected on the same set of solutions also attempted to establish an order-millisecond random burst sensing approach. Preliminary endeavors into using convolutional neural networks also provided an example of an alternative modeling strategy. The results of this work challenge existing assumptions of neurochemistry, while demonstrating the capabilities of new neurochemical sensing approaches. This study will also act as a springboard for emerging technological developments in human neurochemistry. / Doctor of Philosophy / Neuroscience characterizes nervous system functions from the cellular to the systems level. A gap in available technologies has prevented neuroscientist from studying how changes in the molecular dynamics in the brain relate to psychiatric conditions. Recent efforts by the Montague Laboratory have adapted neurochemistry techniques for use in human patients. Consequently, a new "random burst sensing" approach was developed that challenged existing assumptions about electrochemistry. In this study, in-vivo experiments were conducted to push the limits of electrochemical sensing by reducing the voltage amplitude range and increasing sensing temporal resolution of electrochemical sensing beyond previously established limits. The results of this study offer novel neurochemistry approaches and act as a jumping off point for future technological developments.

Major depressive disorder is a common, impairing disease, but current treatments are only moderately effective. Understanding how processes such as reward and punishment learning are disrupted in depression and how these disruptions are remediated through treatment is vital to improving outcomes for people with this disorder. In the present set of studies, computational reinforcement learning models and neuroimaging were used to understand how symptom clusters of depression (anhedonia and negative affect) were related to neural and behavioral measures of learning (Study 1, in Paper 1), how these alterations changed with improvement in symptoms after cognitive behavioral therapy (Study 2, in Paper 1), and how learning parameters could be directly altered in a learning retraining paradigm (Study 3, in Paper 2). Results showed that anhedonia and negative affect were uniquely related to changes in learning and that improvement in these symptoms correlated with changes in learning parameters; these parameters could also be changed through targeted queries based on reinforcement learning theory. These findings add important information to how learning is disrupted in depression and how current and novel treatments can remediate learning and improve symptoms. / Ph. D. / Major depression is very common and current treatments are sometimes helpful and sometimes not. In order to create more effective treatments, we need to better understand what exactly goes wrong when people are depressed. The present set of studies uses computational modeling and imaging of brain function to gain a clearer understanding of how people with depression learn from rewarding and punishing events differently, how these differences in learning improve with symptom improvement after receiving treatment for depression, and how learning differences can be directly targeted by teaching people to learn differently. I found that a reduced ability to experience pleasure, or anhedonia, in depression was related to differences learning from good outcomes while low mood was related to perceiving bad outcomes as worse. Both of these differences improved with successful treatment, and asking people questions related to learning also changed the way people learned in a way that may be useful for improving treatments.

No description available.

No description available.

When moving into a more connected world together with machines, a mutual understanding will be very important. With the increased availability in wear- able sensors, a better understanding of human needs is suggested. The Dart- mouth Research study at the Psychiatric Research Center has examined the viability of detecting and further on predicting human behaviour and complex tasks. The field of smoking detection was challenged by using the Q-sensor by Affectiva as a prototype. Further more, this study implemented a framework for future research on the basis for developing a low cost, connected, device with Thayer Engineering School at Dartmouth College. With 3 days of data from 10 subjects smoking sessions was detected with just under 90% accuracy using the Conditional Random Field algorithm. However, predicting smoking with Electrodermal Momentary Assessment (EMA) remains an unanswered ques- tion. Hopefully a tool has been provided as a platform for better understanding of habits and behaviour.

Background: Human and animal work suggests a shift from goal-directed to habitual decision-making in addiction. However, the evidence for this in human alcohol dependence is as yet inconclusive. Methods: Twenty-six healthy controls and 26 recently detoxified alcohol-dependent patients underwent behavioral testing with a 2-step task designed to disentangle goal-directed and habitual response patterns. Results: Alcohol-dependent patients showed less evidence of goal-directed choices than healthy controls, particularly after losses. There was no difference in the strength of the habitual component. The group differences did not survive controlling for performance on the Digit Symbol Substitution Task. Conclusion: Chronic alcohol use appears to selectively impair goal-directed function, rather than promoting habitual responding. It appears to do so particularly after nonrewards, and this may be mediated by the effects of alcohol on more general cognitive functions subserved by the prefrontal cortex.