Convergent evidence suggests that the dorsal medial prefrontal cortex (dmPFC) makes an important contribution to goal-directed action selection. The dmPFC is also part of a network of brain regions that becomes compromised in old age. It was hypothesized that during decision-making, some process of comparison takes place in the dmPFC between the representation of available actions and associated values, and that this process is changed with aging. These hypotheses were tested in aged and young adult rats performing a novel 3-choice, 2-cue decision task. Neuron and local field potential activity revealed that the dmPFC experienced different states during decision and outcome phases of the task, with increased local inhibition and oscillatory (gamma and theta) activity during cue presentation, and increased excitatory neuron activity (among regular firing neurons) at goal zones. Although excitatory and inhibitory activity appeared anti-correlated over phases of the decision task, cross-correlations and the prominent gamma oscillation revealed that excitation and inhibition were highly correlated on the millisecond scale. This "micro-scale" coupling between excitation and inhibition was altered in aged rats and the observed changes were correlated with changes in decision and movement speeds of the aged animals, suggesting a putative mechanism for age-related behavioral slowing. With respect to decision-making, both aged and young adult rats learned over multiple days to follow the rewarded cue in the 3-choice, 2-cue task. Support for the hypothesis that the dmPFC simultaneously represents alternative actions was not found; however, neuron activity selective for particular goal zones was observed. Interestingly, goal-selective neural activity during the decision period was more likely to take place on error trials, particularly on high-performing sessions and when rats exhibited a preference for a particular feeder. A possible interpretation of these patterns is that goal representations in the dmPFC might have sometimes overruled learned habits, which are likely to be involved in following the correct cue and which are known to be supported by other brain regions. These results describe fundamental properties of network dynamics and neural coding in the dmPFC, and have important implications for the neural basis of processing speed and goal-directed action.

No description available.

abstract: Animals learn to choose a proper action among alternatives according to the circumstance. Through trial-and-error, animals improve their odds by making correct association between their behavioral choices and external stimuli. While there has been an extensive literature on the theory of learning, it is still unclear how individual neurons and a neural network adapt as learning progresses. In this dissertation, single units in the medial and lateral agranular (AGm and AGl) cortices were recorded as rats learned a directional choice task. The task required the rat to make a left/right side lever press if a light cue appeared on the left/right side of the interface panel. Behavior analysis showed that rat's movement parameters during performance of directional choices became stereotyped very quickly (2-3 days) while learning to solve the directional choice problem took weeks to occur. The entire learning process was further broken down to 3 stages, each having similar number of recording sessions (days). Single unit based firing rate analysis revealed that 1) directional rate modulation was observed in both cortices; 2) the averaged mean rate between left and right trials in the neural ensemble each day did not change significantly among the three learning stages; 3) the rate difference between left and right trials of the ensemble did not change significantly either. Besides, for either left or right trials, the trial-to-trial firing variability of single neurons did not change significantly over the three stages. To explore the spatiotemporal neural pattern of the recorded ensemble, support vector machines (SVMs) were constructed each day to decode the direction of choice in single trials. Improved classification accuracy indicated enhanced discriminability between neural patterns of left and right choices as learning progressed. When using a restricted Boltzmann machine (RBM) model to extract features from neural activity patterns, results further supported the idea that neural firing patterns adapted during the three learning stages to facilitate the neural codes of directional choices. Put together, these findings suggest a spatiotemporal neural coding scheme in a rat AGl and AGm neural ensemble that may be responsible for and contributing to learning the directional choice task. / Dissertation/Thesis / Ph.D. Electrical Engineering 2014

Although research has highlighted the importance of decisions when learning and performing motor actions, few studies have focused on individuals’ ability to choose between potential motor actions. To help bridge this gap, this thesis presents a series of studies that investigate the behaviour of able-bodied individuals when attempting to choose movements based on a speed-accuracy tradeoff. In the first study, a two-alternative forced-choice task was used to determine whether people are consistent with Fitts’s law when choosing the movement they perceive to require the least movement duration. Participants performed almost perfectly when clear visual cues were available—when one of the targets was closer, wider, or both. Contrary to Fitts’s law, however, participants showed a preference for closer targets when visual cues were not informative—when one of the targets was closer and narrower. This study demonstrates that motor decisions are not always optimal, especially when participants are naïve at the task. To determine the basis of individuals’ preference for closer targets, a pair of studies explored the relation between motor decisions, imagined movements, and visual perception. Participants showed a similar deviation from Fitts’s law when imagining movements—believing that movement duration increased with distance within the same index of difficulty. Participants did not behave similarly, however, in a perceptual version of the decision task. These results suggest that imagined movements and motor decisions are linked, but they are not always based on veridical representations of actual movement. To further probe the origin of individuals’ erroneous belief about movement duration, the final study of this thesis measured movement duration for movements made at speeds other than ‘as fast as possible’. Movements made at more natural movement speeds shared important similarities with decisions and imagined movements. This study suggests that the biases seen in naïve motor decisions might originate from participants considering movements for which they have more experience, such as target-directed movements made at a naturally-selected pace. Together, the findings presented in this thesis may help to identify the ways that motor decisions can deviate from optimal, suggesting how those decisions must change with practice to better accomplish a task.

Although research has highlighted the importance of decisions when learning and performing motor actions, few studies have focused on individuals’ ability to choose between potential motor actions. To help bridge this gap, this thesis presents a series of studies that investigate the behaviour of able-bodied individuals when attempting to choose movements based on a speed-accuracy tradeoff. In the first study, a two-alternative forced-choice task was used to determine whether people are consistent with Fitts’s law when choosing the movement they perceive to require the least movement duration. Participants performed almost perfectly when clear visual cues were available—when one of the targets was closer, wider, or both. Contrary to Fitts’s law, however, participants showed a preference for closer targets when visual cues were not informative—when one of the targets was closer and narrower. This study demonstrates that motor decisions are not always optimal, especially when participants are naïve at the task. To determine the basis of individuals’ preference for closer targets, a pair of studies explored the relation between motor decisions, imagined movements, and visual perception. Participants showed a similar deviation from Fitts’s law when imagining movements—believing that movement duration increased with distance within the same index of difficulty. Participants did not behave similarly, however, in a perceptual version of the decision task. These results suggest that imagined movements and motor decisions are linked, but they are not always based on veridical representations of actual movement. To further probe the origin of individuals’ erroneous belief about movement duration, the final study of this thesis measured movement duration for movements made at speeds other than ‘as fast as possible’. Movements made at more natural movement speeds shared important similarities with decisions and imagined movements. This study suggests that the biases seen in naïve motor decisions might originate from participants considering movements for which they have more experience, such as target-directed movements made at a naturally-selected pace. Together, the findings presented in this thesis may help to identify the ways that motor decisions can deviate from optimal, suggesting how those decisions must change with practice to better accomplish a task.

Behavior is considered goal-directed when the actor integrates information about the subsequent outcome of an action (Balleine & O'Doherty, 2010; Dickinson & Balleine, 1994; Kiesel & Koch, 2012), potentially enabling the anticipation of consequences of an action. Thus, it requires prior acquisition of knowledge about the current contingencies between behavioral responses and their outcomes under certain stimulus conditions (J. Hoffmann & Engelkamp, 2013). This association chain enables events lying in the future to be mentally represented and assessed in terms of value and achievability. However, while neural correlates of instructed goal-directed action integration processes have already been examined in a functional magnetic resonance imaging (fMRI) study using this paradigm (Ruge & Wolfensteller, 2015), there has been no information if those processes are also reflected in Electroencephalography (EEG) and if so which specific EEG parameters are modulated by them. This dissertation set out to investigate neurocognitive mechanisms of instructed outcome response learning utilizing two different imaging methods, namely EEG and fMRI. Study 1 was an exploratory study to answer the question what kinds of learning-related EEG correlates were to expect. The O-R outcome integration specific EEG correlates identified in Study 1 served as regressors in a unified general linear model (EEG-informed fMRI analysis) in the co-registered EEG-fMRI study (Study 2). One of the key questions in this study was if the EEG signal could help to differentiate between BOLD pre-response activation associated with processes related to response preparation or initiation and activation associated with post-response outcome integration processes. The foundation to both studies of this work was an experimental paradigm of instructed S-R-O learning, which included a learning and a test phase. Stimuli were four abstract visual patterns that differed in each block. Each visual stimulus required a distinct manual response and was predictably followed by a distinct auditory outcome. Instructions were delivered via a “guided implementation” procedure in which the instruction was embedded within the first three successful behavioral implementation trials. In these first three trials, the visual stimulus was followed by an imperative stimulus highlighting the correct response. The guided implementation phase was followed by an unguided implementation phase where the correct response now had to be retrieved from memory. Behaviorally, the strength of acquired O-R associations can be analyzed via O-R compatibility effects measured in a subsequent outcome-priming test phase (Greenwald, 1970). In this test phase a previously learned outcome becomes an imperative stimulus that requires either the response, which produced that outcome in the preceding learning phase (O-R compatible), or a response, which produced a different outcome (O-R incompatible). The experimental design was embedded into an EEG recording setup in study 1 while study 2 comprised a simultaneous EEG-fMRI recording setup in which EEG scalp potentials were continuously recorded during the experimental session inside the MR scanner bore. Study 1 revealed various ERP markers correlated with outcome response learning. An ERP post-response anterior negativity following auditory outcomes was increasingly attenuated as a function of the acquired association strength. This suggests that previously reported action-induced sensory attenuation effects under extensively trained free choice conditions can be established within few repetitions of specific R-O pairings under forced choice conditions. Furthermore, an even more rapid development of a post-response but pre-outcome fronto-central positivity, which was reduced for high R-O learners, might indicate the rapid deployment of preparatory attention towards predictable outcomes. Finally, the study identified a learning-related stimulus-locked activity modulation within the visual P1-N1 latency range, which was thought to reflect the multi-sensory integration of the perceived antecedent visual stimulus with the anticipated auditory outcome. In general, study 2 was only partially able to replicate the EEG activity dynamics related to the formation of bidirectional R-O associations that were observed in study 1. Primarily, it was able to confirm the modulation in EEG negativity in the visual P1-N1 latency range over the learning course. The EEG-informed analysis revealed that learning-related modulations of the P1-N1 complex are functionally coupled to activation in the orbitofrontal cortex (OFC). More specifically, growing attenuation of the EEG negativity increase from early to late SRO repetition levels in high R-O learners was associated with an increase in activation in the OFC. An additional exploratory EEG analysis identified a recurring post outcome effect at central electrode sites expressed in a stronger negativity in late compared to early learning stages. This effect was present in both studies and showed no correlation with any of the behavioral markers of learning. The EEG-informed fMRI analysis resulted in a pattern of distinct functional couplings of this parameter with different brain regions, each correlated with different behavioral markers of S-R-O learning. First of all, increased coupling between the late EEG negativity and activation in the supplementary motor area (SMA) was positively correlated with the O-R compatibility effect. Thus, high R-O learners exhibited a stronger coupling than low R-O learners. Secondly, increased couplings between the late EEG negativity and activation in the somatosensory cortex as well as the dorsal caudate, on the other hand, were positively correlated with individual reaction time differences between early and late stages of learning. Regarding activation patterns prior to the behavioral response the results indicate that the OFC could serve as a (multimodal) hub for integrating stimulus information and information about its associated outcome in an early pre-stage of action selection and initiation. Learnt S-O contingencies would facilitate initiating the motor program of the action of choice. Hence, the earlier an outcome is anticipated (based on stimulus outcome associations), the better it will be associated with its response, eventually leading to stronger O-R compatibility effects later on. Thus, one could speculate that increased activation in response to S-R-O mappings possibly embodies a marker for the ongoing transition from mere stimulus-based behavior to a goal-directed behavior throughout the learning course. Post-response brain activation revealed a seemingly two-fold feedback integration stream of O-R contingencies. On one hand the SMA seems to be engaged in bidirectional encoding processes of O-R associations. The results promote the general idea that the SMA is involved in the acquisition of goal-directed behavior (Elsner et al., 2002; Melcher, Weidema, Eenshuistra, Hommel, & Gruber, 2008; Melcher et al., 2013). Together with prior research (Frimmel, Wolfensteller, Mohr, & Ruge, 2016) this notion can be generalized not only to extensive learning phases but also to learning tasks in which goal-directed behavior is acquired in only few practice trials. However, there is an ongoing debate on whether SMA activation can be clearly linked to sub-processes prior or subsequent to an agent’s action (Nachev, Kennard, & Husain, 2008). The results of this work provide additional evidence favoring an involvement of the SMA only following a performed action in response to an imperative stimulus and even more, subsequent to the perception of its ensuing effect. This may give rise to the interpretation that the SMA is associated with linking the motor program of the performed action to the sensory program of the perceived effect, hence establishing and strengthening O-R contingencies. Furthermore, the analysis identified an increased coupling of a late negativity in the EEG signal and activation in the dorsal parts of the caudate as well as the somatosensory cortex. The dorsal caudate has not particularly been brought into connection with O-R learning so far. I speculate that the coupling effect in this part of the caudate reflects an ongoing process of an early automatization of the acquired behavior. It has already be shown in a similar paradigm that behavior can be automatized within only few repetitions of novel instructed S-R mappings (Mohr et al., 2016).:Table of contents Table of contents II List of Figures IV List of Tables VI List of Abbreviations VII 1 Summary 1 1.1 Introduction 1 1.2 Study Objectives 2 1.3 Methods 3 1.4 Results 4 1.5 Discussion 4 2 Theoretical Background 7 2.1 Introduction 7 2.2 Theories of acquiring goal-directed behavior 9 2.2.1 Instrumental learning 9 2.2.1.1 Behavioral aspects 9 2.2.1.2 Neurophysiological correlates 14 2.2.2 Acquisition of goal-directed behavior according to ideomotor theory 16 2.2.2.1 Behavioral aspects 16 2.2.2.2 Neurophysiological correlates 22 2.3 Summary 25 2.4 Methodological background 26 2.4.1 Electroencephalography (EEG) 26 2.4.2 Functional magnetic resonance imaging (fMRI) 28 2.4.3 Co-registered EEG-fMRI 29 3 General objectives and research questions 34 4 Study 1 – Learning-related brain-electrical activity dynamics associated with the subsequent impact of learnt action-outcome associations 36 4.1 Introduction 36 4.2 Methods 39 4.3 Results 47 4.4 Discussion 60 5 Study 2 - Within trial distinction of O-R learning-related BOLD activity with the means of co-registered EEG information 64 5.1 Introduction 64 5.2 Methods 66 5.3 Results 86 5.4 Discussion 101 6 Concluding general discussion 109 6.1 Brief assessment of study objectives 109 6.2 Novel insights into rapid instruction based S-R-O learning? 109 6.2.1 Early stimulus outcome information retrieval indicates the transition from stimulus based behavior to goal-directed action 110 6.2.2 Post-response encoding and consolidation of O-R contingencies enables goal-directedness of behavior 112 6.3 Critical reflection of the methodology and outlook 116 6.3.1 Strengths and limitations of this work 116 6.3.2 Data quality assessment 117 6.3.3 A common neural foundation for EEG and fMRI? 119 6.3.4 How can co-registered EEG-fMRI contribute to a better understanding of the human brain? 121 6.4 General Conclusion 123 7 References 124 Danksagung Erklärung

Les processus de type “action-conséquence” (orienté vers un but) et stimulus-réponse sont deux composants importants du comportement. Le premier évalue le bénéfice d’une action pour choisir la meilleure parmi celles disponibles (sélection d’action) alors que le deuxième est responsable du comportement automatique, suscitant une réponse dès qu’un stimulus connu est présent. De telles habitudes sont généralement associées (et surtout opposées) aux actions orientées vers un but qui nécessitent un processus délibératif pour évaluer la meilleure option à prendre pour atteindre un objectif donné. En utilisant un modèle computationnel, nous avons étudié l’hypothèse classique de la formation et de l’expression des habitudes au niveau des ganglions de la base et nous avons formulé une nouvelle hypothèse quant aux rôles respectifs des ganglions de la base et du cortex. Inspiré par les travaux théoriques et expérimentaux de Leblois et al. (2006) et Guthrie et al. (2013), nous avons conçu un modèle computationnel des ganglions de la base, du thalamus et du cortex qui utilise des boucles distinctes (moteur, cognitif et associatif) ce qui nous a permis de poser l’hypothèse selon laquelle les ganglions de la base ne sont nécessaires que pour l’acquisition d’habitudes alors que l’expression de telles habitudes peut être faite par le cortex seul. En outre, ce modèle a permis de prédire l’existence d’un apprentissage latent dans les ganglions de la base lorsque leurs sorties (GPi) sont inhibées. En utilisant une tâche de bandit manchot à 2 choix, cette hypothèse a été expérimentalement testée et confirmée chez le singe; suggérant au final de rejeter l’idée classique selon laquelle l’automatisme est un trait subcortical. / Action-outcome and stimulus-response processes are two important components of behavior. The former evaluates the benefit of an action in order to choose the best action among those available (action selection) while the latter is responsible for automatic behavior, eliciting a response as soon as a known stimulus is present. Such habits are generally associated (and mostly opposed) to goal-directed actions that require a deliberative process to evaluate the best option to take in order to reach a given goal. Using a computational model, we investigated the classic hypothesis of habits formation and expression in the basal ganglia and proposed a new hypothesis concerning the respective role for both the basal ganglia and the cortex. Inspired by previous theoretical and experimental works (Leblois et al., 2006; Guthrie et al., 2013), we designed a computational model of the basal ganglia-thalamus-cortex that uses segregated loops (motor, cognitive and associative) and makes the hypothesis that basal ganglia are only necessary for the acquisition of habits while the expression of such habits can be mediated through the cortex. Furthermore, this model predicts the existence of covert learning within the basal ganglia ganglia when their output is inhibited. Using a two-armed bandit task, this hypothesis has been experimentally tested and confirmed in monkey. Finally, this works suggest to revise the classical idea that automatism is a subcortical feature.