Notre capacité à anticiper les événements futurs sur base de notre expérience nous permet d'interagir adéquatement avec notre environnement. Depuis presque deux siècles, cette observation a constitué une question centrale pour deux traditions de recherche différentes, à savoir la théorie idéomotrice de l'action et la théorie de l'apprentissage par renforcement. Alors que la première s'est concentrée sur les effets sensoriels associés avec une action donnée, la seconde tend à considérer que le comportement peut uniquement être influencé par les récompenses et punitions potentielles. Cette thèse de doctorat vise à lier ces deux approches, sur la base du modèle prédictif de la perception. À cette fin, nous avons conduit une série de six expériences explorant différents aspects de la relation entre les prédictions sensorielles et les prédictions de récompense au moyen de méthode de mesure comportementales, de techniques d'électroencéphalographie (EEG) et d'oculométrie. Nos résultats indiquent que (1) les contenus sensoriels et émotionnels appartenant à des stimuli complexes sont évalués ensembles, que (2) les différences inter-individuelles influencent la manière dont nous formons des prédictions et que (3) la mesure de la taille des pupilles est un outil prometteur pour l'étude des prédictions sensorielles. / Our ability to anticipate future events based on prior knowledge about our environment is shaping the way we engage with it. For almost two centuries, this observation has been a central topic for two different research traditions, namely the ideomotor theory and the reinforcement learning literature. Whereas the former focussed is inquiries on the sensory effects associated with a given action, the latter considered that behaviour was only influenced by potential rewards or punishments. This dissertation constitutes an attempt to bridge these two approaches, based on the predictive model theory. For this purpose, we conducted six experiments exploring various aspects of the relationship between reward and sensory prediction using classical behavioural and EEG methods, but also pupil size measurements. Our results suggest that (1) sensory and emotional features of complex stimuli are processed together, that (2) inter-individual differences influence prediction mechanisms and (3) pupil size measurement is a useful tool for sensory prediction.

選擇性模仿是指嬰兒在不同情境中選擇性地模仿他人的行為，為當今發展心理學備受矚目的議題。在Gergely、Bekkering與Király（2002）著名的研究中，當實驗者的雙手自由放在桌上，示範以額頭碰盒子開燈，嬰兒偏好模仿此新奇動作；但當實驗者雙手緊抓毛毯示範相同的動作則未發現嬰兒有顯著的模仿偏好。有些研究者認嬰兒可以透過目的推理理解他人的意圖，並認為模仿是建立在對意圖的解讀之上；相反地，動作－效果整合理論則認為動作本身的執行困難度與動作－效果連結的穩定性才是影響嬰兒選擇性模仿的主因。 為了解決兩者長久以來對於嬰兒模仿的爭議，本研究修訂Gergely等人（2002）的光盒研究派典，將焦點放在過去一直未受到重視的因果效能概念，降低示範動作的因果效能，嬰兒在實驗過程中會發現示範動作不是每次都能成功讓光盒發亮，藉此釐清目的推理論與動作－效果整合理論對嬰兒選擇性模仿的解釋適當性。實驗1的結果重製了Gergely等人（2002）的實驗結果，實驗2A與實驗2B皆發現18個月大的嬰兒在因果效能較低的情境中很少會模仿示範動作，顯示嬰兒的模仿行為較符合動作－效果整合理論的觀點，主要是受到動作執行的困難度與動作－效果聯結的穩定性的影響。 / Selective imitation refers to a phenomenon which infants differentially imitate the demonstrated action in different contexts. Recently, it has also become a popular research topic in developmental psychology. Gergely, Bekkering and Király (2002) uncovered a classic example of selective imitation. They found that infants tend to imitate the action of touching a light box with their forehead when they saw a model perform the action with her hands placed on the table, but not when her hands were restricted by a blanket. Some researchers claim that infants can interpret others’ intention through teleological reasoning, and they consider infants’ imitation is based on decoding of intentionality. Conversely, ideomotor theory argues that imitation depends on difficulty of the action execution and the stability of link between action and its effect. To address the long-standing dispute with infants’ imitation, we revised the paradigm in Gergely et al. (2002) and focused on the concept of causal efficacy which had been long ignored in the past. Infant would find that the demonstrated action, sometimes won’t turn on the light during the experimentation. In experiment 1, we replicated the results obtained in Gergely et al. (2002). Experiment 2A and 2B both found that 18-month-old infants rarely imitated the demonstrated action when the causal efficacy was relatively low. These results are closer to the ideomotor approach viewpoint of imitation, and it reveals that infants’ imitation depends on difficulty of the action execution and the stability of link between action and effect.

Behavior is considered goal-directed when the actor integrates information about the subsequent outcome of an action (Balleine & O'Doherty, 2010; Dickinson & Balleine, 1994; Kiesel & Koch, 2012), potentially enabling the anticipation of consequences of an action. Thus, it requires prior acquisition of knowledge about the current contingencies between behavioral responses and their outcomes under certain stimulus conditions (J. Hoffmann & Engelkamp, 2013). This association chain enables events lying in the future to be mentally represented and assessed in terms of value and achievability. However, while neural correlates of instructed goal-directed action integration processes have already been examined in a functional magnetic resonance imaging (fMRI) study using this paradigm (Ruge & Wolfensteller, 2015), there has been no information if those processes are also reflected in Electroencephalography (EEG) and if so which specific EEG parameters are modulated by them. This dissertation set out to investigate neurocognitive mechanisms of instructed outcome response learning utilizing two different imaging methods, namely EEG and fMRI. Study 1 was an exploratory study to answer the question what kinds of learning-related EEG correlates were to expect. The O-R outcome integration specific EEG correlates identified in Study 1 served as regressors in a unified general linear model (EEG-informed fMRI analysis) in the co-registered EEG-fMRI study (Study 2). One of the key questions in this study was if the EEG signal could help to differentiate between BOLD pre-response activation associated with processes related to response preparation or initiation and activation associated with post-response outcome integration processes. The foundation to both studies of this work was an experimental paradigm of instructed S-R-O learning, which included a learning and a test phase. Stimuli were four abstract visual patterns that differed in each block. Each visual stimulus required a distinct manual response and was predictably followed by a distinct auditory outcome. Instructions were delivered via a “guided implementation” procedure in which the instruction was embedded within the first three successful behavioral implementation trials. In these first three trials, the visual stimulus was followed by an imperative stimulus highlighting the correct response. The guided implementation phase was followed by an unguided implementation phase where the correct response now had to be retrieved from memory. Behaviorally, the strength of acquired O-R associations can be analyzed via O-R compatibility effects measured in a subsequent outcome-priming test phase (Greenwald, 1970). In this test phase a previously learned outcome becomes an imperative stimulus that requires either the response, which produced that outcome in the preceding learning phase (O-R compatible), or a response, which produced a different outcome (O-R incompatible). The experimental design was embedded into an EEG recording setup in study 1 while study 2 comprised a simultaneous EEG-fMRI recording setup in which EEG scalp potentials were continuously recorded during the experimental session inside the MR scanner bore. Study 1 revealed various ERP markers correlated with outcome response learning. An ERP post-response anterior negativity following auditory outcomes was increasingly attenuated as a function of the acquired association strength. This suggests that previously reported action-induced sensory attenuation effects under extensively trained free choice conditions can be established within few repetitions of specific R-O pairings under forced choice conditions. Furthermore, an even more rapid development of a post-response but pre-outcome fronto-central positivity, which was reduced for high R-O learners, might indicate the rapid deployment of preparatory attention towards predictable outcomes. Finally, the study identified a learning-related stimulus-locked activity modulation within the visual P1-N1 latency range, which was thought to reflect the multi-sensory integration of the perceived antecedent visual stimulus with the anticipated auditory outcome. In general, study 2 was only partially able to replicate the EEG activity dynamics related to the formation of bidirectional R-O associations that were observed in study 1. Primarily, it was able to confirm the modulation in EEG negativity in the visual P1-N1 latency range over the learning course. The EEG-informed analysis revealed that learning-related modulations of the P1-N1 complex are functionally coupled to activation in the orbitofrontal cortex (OFC). More specifically, growing attenuation of the EEG negativity increase from early to late SRO repetition levels in high R-O learners was associated with an increase in activation in the OFC. An additional exploratory EEG analysis identified a recurring post outcome effect at central electrode sites expressed in a stronger negativity in late compared to early learning stages. This effect was present in both studies and showed no correlation with any of the behavioral markers of learning. The EEG-informed fMRI analysis resulted in a pattern of distinct functional couplings of this parameter with different brain regions, each correlated with different behavioral markers of S-R-O learning. First of all, increased coupling between the late EEG negativity and activation in the supplementary motor area (SMA) was positively correlated with the O-R compatibility effect. Thus, high R-O learners exhibited a stronger coupling than low R-O learners. Secondly, increased couplings between the late EEG negativity and activation in the somatosensory cortex as well as the dorsal caudate, on the other hand, were positively correlated with individual reaction time differences between early and late stages of learning. Regarding activation patterns prior to the behavioral response the results indicate that the OFC could serve as a (multimodal) hub for integrating stimulus information and information about its associated outcome in an early pre-stage of action selection and initiation. Learnt S-O contingencies would facilitate initiating the motor program of the action of choice. Hence, the earlier an outcome is anticipated (based on stimulus outcome associations), the better it will be associated with its response, eventually leading to stronger O-R compatibility effects later on. Thus, one could speculate that increased activation in response to S-R-O mappings possibly embodies a marker for the ongoing transition from mere stimulus-based behavior to a goal-directed behavior throughout the learning course. Post-response brain activation revealed a seemingly two-fold feedback integration stream of O-R contingencies. On one hand the SMA seems to be engaged in bidirectional encoding processes of O-R associations. The results promote the general idea that the SMA is involved in the acquisition of goal-directed behavior (Elsner et al., 2002; Melcher, Weidema, Eenshuistra, Hommel, & Gruber, 2008; Melcher et al., 2013). Together with prior research (Frimmel, Wolfensteller, Mohr, & Ruge, 2016) this notion can be generalized not only to extensive learning phases but also to learning tasks in which goal-directed behavior is acquired in only few practice trials. However, there is an ongoing debate on whether SMA activation can be clearly linked to sub-processes prior or subsequent to an agent’s action (Nachev, Kennard, & Husain, 2008). The results of this work provide additional evidence favoring an involvement of the SMA only following a performed action in response to an imperative stimulus and even more, subsequent to the perception of its ensuing effect. This may give rise to the interpretation that the SMA is associated with linking the motor program of the performed action to the sensory program of the perceived effect, hence establishing and strengthening O-R contingencies. Furthermore, the analysis identified an increased coupling of a late negativity in the EEG signal and activation in the dorsal parts of the caudate as well as the somatosensory cortex. The dorsal caudate has not particularly been brought into connection with O-R learning so far. I speculate that the coupling effect in this part of the caudate reflects an ongoing process of an early automatization of the acquired behavior. It has already be shown in a similar paradigm that behavior can be automatized within only few repetitions of novel instructed S-R mappings (Mohr et al., 2016).:Table of contents Table of contents II List of Figures IV List of Tables VI List of Abbreviations VII 1 Summary 1 1.1 Introduction 1 1.2 Study Objectives 2 1.3 Methods 3 1.4 Results 4 1.5 Discussion 4 2 Theoretical Background 7 2.1 Introduction 7 2.2 Theories of acquiring goal-directed behavior 9 2.2.1 Instrumental learning 9 2.2.1.1 Behavioral aspects 9 2.2.1.2 Neurophysiological correlates 14 2.2.2 Acquisition of goal-directed behavior according to ideomotor theory 16 2.2.2.1 Behavioral aspects 16 2.2.2.2 Neurophysiological correlates 22 2.3 Summary 25 2.4 Methodological background 26 2.4.1 Electroencephalography (EEG) 26 2.4.2 Functional magnetic resonance imaging (fMRI) 28 2.4.3 Co-registered EEG-fMRI 29 3 General objectives and research questions 34 4 Study 1 – Learning-related brain-electrical activity dynamics associated with the subsequent impact of learnt action-outcome associations 36 4.1 Introduction 36 4.2 Methods 39 4.3 Results 47 4.4 Discussion 60 5 Study 2 - Within trial distinction of O-R learning-related BOLD activity with the means of co-registered EEG information 64 5.1 Introduction 64 5.2 Methods 66 5.3 Results 86 5.4 Discussion 101 6 Concluding general discussion 109 6.1 Brief assessment of study objectives 109 6.2 Novel insights into rapid instruction based S-R-O learning? 109 6.2.1 Early stimulus outcome information retrieval indicates the transition from stimulus based behavior to goal-directed action 110 6.2.2 Post-response encoding and consolidation of O-R contingencies enables goal-directedness of behavior 112 6.3 Critical reflection of the methodology and outlook 116 6.3.1 Strengths and limitations of this work 116 6.3.2 Data quality assessment 117 6.3.3 A common neural foundation for EEG and fMRI? 119 6.3.4 How can co-registered EEG-fMRI contribute to a better understanding of the human brain? 121 6.4 General Conclusion 123 7 References 124 Danksagung Erklärung

Les actions motrices humaines peuvent être envisagées comme étant soit volontaires, c'est-à-dire intérieurement déclenchées afin d’atteindre un certain but, soit réactives, c'est-à-dire extérieurement déclenchées par des stimuli environnementaux. Cette dissociation a notamment été proposée au sein de la théorie idéomotrice suggérant que la réalisation d'actions volontaires repose sur notre capacité à prédire les conséquences sensorielles de nos actions, grâce aux associations action-effet qui sont acquises avec l'expérience. Selon les modèles computationnels tels que le principe de minimisation de l’énergie libre, la prédiction sensorielle est également considérée comme un processus majeur de la perception et du contrôle moteur, indépendamment du type d’action. Dès lors, les études visant à explorer la prédiction sensorielle liée au contrôle moteur ont systématiquement minimisé la distinction potentielle entre deux types d'actions plus ou moins indépendantes. Dans la présente thèse, nous nous sommes principalement attelés à tester la théorie idéomotrice originale qui suggère une implication supérieure de la prédiction sensorielle dans les actions intentionnelles par rapport à des actions plus réactives. Nous avons réalisé ce travail selon trois axes : (1) À travers des expériences comportementales, nous avons cherché à préciser à quel(s) stade(s) moteur(s) la prédiction de l'effet de l'action pouvait être associée, dans les actions intentionnelles d’une part et dans les actions davantage réactives d’autre part, afin de pouvoir dissocier la dynamique temporelle de la prédiction sensorielle au sein de ces deux catégories d'actions. (2) En tirant parti des postulats dérivés des approches computationnelles, nous avons utilisé l'EEG pour explorer d'abord le niveau d'erreur de prédiction liée aux effets sensoriels imprévisibles ou mal-prédits afin de dissocier ces deux types d'événements non prédits au niveau neural. Par la suite, nous avons étudié si les marqueurs EEG de la prédiction sensorielle (c'est-à-dire l'erreur de prédiction et l'atténuation sensorielle) étaient modulés par le type d'action déclenchant l'effet sensoriel. (3) Enfin, nous avons examiné si des variations dans le processus de prédiction des effets de l'action pouvaient être associés à certains déficits moteurs dans la maladie de Parkinson et à des tendances impulsives mesurées chez des participants sains, pour éventuellement conférer une dimension clinique au processus de prédiction sensorielle. Nos résultats ont démontré (1) que la dynamique temporelle de la prédiction des effets de l'action semble effectivement dépendre du type d'action, en étant liée aux étapes précoces et tardives de la préparation motrice des actions intentionnelles, mais seulement aux étapes tardives de la préparation motrice des actions réactives. Nous avons également montré que (2) les événements mal prédits généraient une erreur de prédiction plus importante comparativement à des événements imprévisibles. Par ailleurs, les marqueurs EEG de la prédiction sensorielle étaient plus prononcés pour les effets auditifs déclenchés par des actions intentionnelles par rapport aux effets auditifs déclenchés par des actions réactives. Enfin, nos résultats ont permis de démontrer que (3) le processus de prédiction sensorielle semble être altéré lors de la réalisation d’actions intentionnelles chez des patients atteints de la maladie de Parkinson, et que les marqueurs EEG de la prédiction d’effets auditifs déclenchés par des actions intentionnelles sont modulés par les tendances impulsives d’individus sains. Dans l'ensemble, nos résultats soutiennent l’existence d’une dissociation fonctionnelle entre actions intentionnelle et réactive, et sont également cohérents avec la version originale de la théorie idéomotrice étant donné que la prédiction sensorielle semble être impliquée plus tôt et plus fortement dans les actions intentionnelles que dans les actions réactives. (…) / Motor actions can be classified as being either intention-based, i.e. internally triggered in order to reach a certain goal, or either stimulus-driven, i.e. externally triggered in order to accommodate to environmental events. This elementary dissociation was notably theorized within the original ideomotor theory stating that performing intention-based actions relies on our capacity to predict the sensory consequences of our actions, due to action-effect associations learnt through experience. In recent neurocomputational models such as the brain free-energy principle, this sensory prediction is considered as a key process of overall sensorium and motor control, regardless the action type. Henceforth, experiments studying sensory prediction related to motor control have systematically minimized the potential distinction between two more or less independent action types. In the current thesis, we mainly attempted to address this issue by testing the original ideomotor viewpoint, suggesting a superior involvement of action-effect prediction in intention-based actions compared to more reactive actions. We achieved this work according to three axes: (1) Through behavioural experiments, we aimed at clarifying which motor stage(s) action-effect prediction is related to, within intention-based actions and within stimulus-driven actions, in order to potentially dissociate the temporal dynamics of action-effect prediction in these two categories of actions. (2) Taking advantage from assumptions derived from neurocomputational approaches, we used EEG to first explore the level of prediction error related to unpredicted vs. mispredicted auditory events in order to dissociate these two types of nonpredicted events at a neural level. Then, we investigated whether EEG markers of sensory prediction (i.e., prediction error and sensory attenuation) were modulated by the kind of action triggering the sensory effect. (3) Finally, we intended to examine whether action-effect prediction variations could be linked to motor deficits in Parkinson's disease on the one hand, and to impulsivity tendencies in healthy participants on the other hand, for possibly yielding a clinical dimension to the sensory prediction process. Our findings demonstrated (1) the temporal dynamics of action-effect prediction seems to depend on the action kind, being linked to both early and late stages of motor preparation of intention-based actions and only to late stages of motor preparation of stimulus-driven actions. We also showed that (2) mispredicted events were linked to enhanced prediction error compared to unpredicted events, and that EEG markers of sensory prediction were more pronounced for auditory effects triggered by intention-based actions compared to auditory effects triggered by stimulus-driven actions. Then, our results sustained that (3) the action-effect prediction process seems to be impaired for intention-based actions in Parkinson's disease, and that EEG markers of sensory prediction for effects triggered by intention-based actions are modulated by impulsiveness tendencies in healthy participants. Altogether, our findings are consistent with the original version of the ideomotor theory given the action-effect prediction appeared to be earlier and stronger involved in intention-based actions compared to stimulus-driven actions. Our EEG data also modernized the ideomotor principle, reconciling it with neurocomputational approaches of sensory prediction. Finally, the clinical exploration of the action-effect prediction process in pathologies affecting motor control appeared promising to understand intermediate neurocognitive processes which are involved in motor symptoms or characteristics.