Despite the widespread use of caffeine as a central nervous stimulant, the central pharmacodynamic properties of the drug have not yet been conclusively evaluated in humans. The present study was undertaken to assess the acute effects of caffeine on measures of topographical quantitative electroencephalogram (EEG) in normal subjects. Ten healthy male volunteers (mean age ± SD 25 ± 4 years) received placebo and 200 mg of caffeine as powder with oral water solution (caffeine amount = 2 cups of coffee) under randomized, double-blind crossover conditions on two different occasions. Before administration and 30 min afterwards, a 17-channel quantitative EEG was recorded during relaxation with eyes open and closed (15 min each). Caffeine caused a significant reduction of total EEG power at fronto-parieto-occipital and central electrode positions of both hemispheres when the subjects kept their eyes open. Absolute power of the slow and fast alpha and slow beta activities was diminished in various regions of the brain (p < 0.05). The effect was more pronounced with the subjects keeping their eyes open than with eyes closed. It can be concluded that quantitative EEG is a sensitive method to assess the effects of psychostimulants on the human brain. Therefore, in pharmaco-EEG studies, environmental factors such as caffeine have to be excluded. / Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG-geförderten) Allianz- bzw. Nationallizenz frei zugänglich.

This project added to the sparse body of research on the neural underpinnings of turn-taking with an electroencephalography (EEG) investigation of spontaneous conversation. Eighteen participants (3 male, 15 female, mean age 29.79), recruited and participating in pairs, underwent EEG hyperscanning as they conversed on a freely chosen topic for 45 minutes. In line with previous research, it was predicted that a time-frequency analysis of the EEG might reveal either increased power at around 10 Hz (the location of one of two components of the mu rhythm, an oscillation possibly involved in motor preparation for speech), or reduced alpha (8-12 Hz) power (reflecting non-motor aspects of turn preparation) prior to taking one’s turn. Increased power between 8-12 Hz was observed around 1.5 and 1 second preceding turn-taking, but similar power increases also occurred prior to turn-yielding and the conversation partner continuing after a pause, and a reduction in alpha power was found in turn-taking relative to listening to the other speaker continue after a pause. It is unclear whether this activity reflected motor or non-motor aspects of turn preparation, but the spontaneous conversation paradigm proved feasible for investigating brain activity coupled to turn-taking despite the methodological obstacles. / Detta forskningsprojekt bedrar till ett ämne där relativt få studier har genomförts med en elektroencefalografi- (EEG-) undersökning av hjärnaktivitet som är kopplad till turtagning i spontant samtal. Arton deltagare (3 män, 15 kvinnor, medelålder 29,79) som rekryterades och deltog i par, genomgick EEG-hyperscanning medan de pratade om ett fritt valt ämne i 45 minuter. Det förutsades att en tidsfrekvensanalys av EEG kan avslöja antingen ökad effekt vid cirka 10 Hz (vilket motsvarar en av två komponenter i mu-rytmen, en oscillation som eventuellt är involverad i motoriska förberedelser för tal) eller reducerad alfaeffekt (8 -12 Hz) (vilket möjligen återspeglar icke-motoriska aspekter av turtagningsförberedelser) innan man tar sin tur. Ökad effekt mellan 8-12 Hz observerades ungefär 1,5 och 1 sekund före turtagning, men liknande ökningar inträffade också innan samtalspartnern tog sin tur eller fortsatte efter en paus, och en minskning av alfaeffekt observerades när turtagning jämfördes till kontexter där försökspersonerna lyssnade när den andra talaren fortsatte efter en paus. Det är oklart om denna aktivitet återspeglade motoriska eller icke-motoriska aspekter av turtagningsförberedelser, men det visar sig vara möjligt att undersöka hjärnaktivitet kopplad till spontant samtal på ett rimligt sätt trots paradigmens metodologiska svårigheter. / Hidden events in turn-taking

Studien implementerade ett hjärna-dator-gränssnitt med hjälp av EEG-instrumentet MindWave Mobile Headset. Vi undersökte om det var möjligt att utföra fyra operationer genom att använda tankemönster. Fyra försökspersoner deltog i studien. Deras uppgift var att tänka i två tankemönster i följd som resulterade i en operation. EEG-signalen förbehandlas så att en mönsterigenkänningsmetod (k-NN) lättare kunde urskilja två tankemönster ur signalen. Denna undersökning har till vår vetskap inte tidigare utförts och är därmed kunskapsluckan vi ämnar fylla. Att fylla denna kunskapslucka är av intresse för bland annat användargrupperna: rörelsehindrade, spelintresserade och Virtual Reality-användare. Vi tog fram en modell som modellerade det bästa möjliga utfallet av metodiken i föreliggande studie. Undersökningens resultat kunde inte användas för att göra slutsatser angående frågeställningen då detta skulle vara att post hoc-teoretisera. I modellen visades dock tre av fyra operationer vara genomförbara, med en indikation om att även den fjärde var möjlig att utföra. Resultatet indikerar att det finns anledning att utföra en fortsatt studie. Den föreslagna fortsatta studien bör innefatta nya mätningar som testas av modellen för att fullt ut besvara problemformuleringen. / This study implements a Brain-Computer-Interface using the EEG-instrument MindWave Mobile Headset. We studied the feasibility of performing four operations using thought patterns. Four test subjects participated in the study. Their task was to think in two subsequent thought patterns that resulted in an operation. The EEG-signal was pre-processed in such a way that a pattern recognition algorithm (k-NN) more easily could recognize two thought patterns in the signal. This study has to our knowledge not been done before and thus aims to fill this lack of knowledge in the scientific community. User groups that have an interest in filling this gap are, amongst others; disabled people, gamers, and Virtual Reality users. We created a model that modeled the best possible outcome of the method used in this study. Conclusions drawn from the result can not be used to fully answer the problem statement, since it would be to post hoc-theorize. However, three out of four operations were possible to perform in the model, with an indication that the fourth also was possible to perform. These results indicate that there are grounds to continue this study. The proposed continued study should include new measurements that are tested by the model to determine if it is feasible to distinguish all four operations.

Hjärndatorgränssnitt är enheter som fångar upp hjärnsignaler via elektroder på huvudet och översätter dem till datamängder och instruktioner mot externa enheter och applikationer. Gränssnitten har främst använts inom den medicinska domänen för att hjälpa personer med neurofysiologiska åkommor, men har även på senare tid börjat användas av ickemedicinska skäl av privatpersoner. I takt med att gränssnitten ökar i popularitet och når en bredare massa kommer det att innebära ett större informationsflöde av användardata som i sin tur kan bära på väldigt känslig information. Information såsom hälsodata och autentiseringsmetoder är några av flera informationstillgångar som ligger i farozonen enligt flera artiklar och kan råka ut för ett eller flera hot. För få en tydligare bild av de olika hoten samt dess konsekvens och sannolikhet har det genomförts en riskanalys gällande hemanvändares informationssäkerhet. För att få fram sårbarheter, hot och åtgärder som förekommer i riskanalysen har det utförts en tematisk analys. Genom den tematiska analysen visade det sig att det fanns flera hot mot hemanvändarnas konfidentialitet där användares PIN-koder, autentiseringsmetoder och hälsodata låg i farozonen. För att få en bättre förståelse kring hur gränssnitten fungerar samt hur stor sannolikhet det är för olika hot har det även genomförts en intervju med en lektor i kognitiv neurovetenskap, följande tillsammans med artiklarna från den tematiska analysen utgjorde därmed grunden för riskanalysen. Genom riskanalysen visade det sig att hoten mot hemanvändarnas möjlighet att använda gränsssnitten hade en ännu större sannolikhet att inträffa än hot mot användares konfidentialitet. / Brain- Computer Interfaces are devices that capture brain signals via electrodes on the head and then translates them into data sets and instructions to external devices and applications. The interfaces have mainly been used in the medical domain to help people with neurophysiological disorders but have also recently begun to be used for non-medical reasons by private persons. As the interfaces increase in popularity and reach a wider mass, it will mean a greater flow of information of user data that in turn can carry very sensitive information. Information such as health data and authentication methods are some of several information assets that are at risk according to multiple articles and may face one or more threats. To get a clearer picture of the various threats, their consequences and probabilities, a risk analysis has been carried out. In order to identify vulnerabilities, threats and measures that appear in the risk analysis, a thematic analysis has been performed. The thematic coding showed that there were several threats to the home user’s confidentiality where user’s PIN-codes and health data were at risk. In order to gain a better understanding of how the interfaces work and how likely it is for various threats to succeed, an interview was conducted with a senior lectrurer in cognitive neuroscience, the following together with the articles from the thematic analysis thus formed the basis for the risk analysis. The risk analysis showed that threats to home users' ability to use the interfaces were even more likely to occur than threats to user confidentiality.

Behavior is considered goal-directed when the actor integrates information about the subsequent outcome of an action (Balleine & O'Doherty, 2010; Dickinson & Balleine, 1994; Kiesel & Koch, 2012), potentially enabling the anticipation of consequences of an action. Thus, it requires prior acquisition of knowledge about the current contingencies between behavioral responses and their outcomes under certain stimulus conditions (J. Hoffmann & Engelkamp, 2013). This association chain enables events lying in the future to be mentally represented and assessed in terms of value and achievability. However, while neural correlates of instructed goal-directed action integration processes have already been examined in a functional magnetic resonance imaging (fMRI) study using this paradigm (Ruge & Wolfensteller, 2015), there has been no information if those processes are also reflected in Electroencephalography (EEG) and if so which specific EEG parameters are modulated by them. This dissertation set out to investigate neurocognitive mechanisms of instructed outcome response learning utilizing two different imaging methods, namely EEG and fMRI. Study 1 was an exploratory study to answer the question what kinds of learning-related EEG correlates were to expect. The O-R outcome integration specific EEG correlates identified in Study 1 served as regressors in a unified general linear model (EEG-informed fMRI analysis) in the co-registered EEG-fMRI study (Study 2). One of the key questions in this study was if the EEG signal could help to differentiate between BOLD pre-response activation associated with processes related to response preparation or initiation and activation associated with post-response outcome integration processes. The foundation to both studies of this work was an experimental paradigm of instructed S-R-O learning, which included a learning and a test phase. Stimuli were four abstract visual patterns that differed in each block. Each visual stimulus required a distinct manual response and was predictably followed by a distinct auditory outcome. Instructions were delivered via a “guided implementation” procedure in which the instruction was embedded within the first three successful behavioral implementation trials. In these first three trials, the visual stimulus was followed by an imperative stimulus highlighting the correct response. The guided implementation phase was followed by an unguided implementation phase where the correct response now had to be retrieved from memory. Behaviorally, the strength of acquired O-R associations can be analyzed via O-R compatibility effects measured in a subsequent outcome-priming test phase (Greenwald, 1970). In this test phase a previously learned outcome becomes an imperative stimulus that requires either the response, which produced that outcome in the preceding learning phase (O-R compatible), or a response, which produced a different outcome (O-R incompatible). The experimental design was embedded into an EEG recording setup in study 1 while study 2 comprised a simultaneous EEG-fMRI recording setup in which EEG scalp potentials were continuously recorded during the experimental session inside the MR scanner bore. Study 1 revealed various ERP markers correlated with outcome response learning. An ERP post-response anterior negativity following auditory outcomes was increasingly attenuated as a function of the acquired association strength. This suggests that previously reported action-induced sensory attenuation effects under extensively trained free choice conditions can be established within few repetitions of specific R-O pairings under forced choice conditions. Furthermore, an even more rapid development of a post-response but pre-outcome fronto-central positivity, which was reduced for high R-O learners, might indicate the rapid deployment of preparatory attention towards predictable outcomes. Finally, the study identified a learning-related stimulus-locked activity modulation within the visual P1-N1 latency range, which was thought to reflect the multi-sensory integration of the perceived antecedent visual stimulus with the anticipated auditory outcome. In general, study 2 was only partially able to replicate the EEG activity dynamics related to the formation of bidirectional R-O associations that were observed in study 1. Primarily, it was able to confirm the modulation in EEG negativity in the visual P1-N1 latency range over the learning course. The EEG-informed analysis revealed that learning-related modulations of the P1-N1 complex are functionally coupled to activation in the orbitofrontal cortex (OFC). More specifically, growing attenuation of the EEG negativity increase from early to late SRO repetition levels in high R-O learners was associated with an increase in activation in the OFC. An additional exploratory EEG analysis identified a recurring post outcome effect at central electrode sites expressed in a stronger negativity in late compared to early learning stages. This effect was present in both studies and showed no correlation with any of the behavioral markers of learning. The EEG-informed fMRI analysis resulted in a pattern of distinct functional couplings of this parameter with different brain regions, each correlated with different behavioral markers of S-R-O learning. First of all, increased coupling between the late EEG negativity and activation in the supplementary motor area (SMA) was positively correlated with the O-R compatibility effect. Thus, high R-O learners exhibited a stronger coupling than low R-O learners. Secondly, increased couplings between the late EEG negativity and activation in the somatosensory cortex as well as the dorsal caudate, on the other hand, were positively correlated with individual reaction time differences between early and late stages of learning. Regarding activation patterns prior to the behavioral response the results indicate that the OFC could serve as a (multimodal) hub for integrating stimulus information and information about its associated outcome in an early pre-stage of action selection and initiation. Learnt S-O contingencies would facilitate initiating the motor program of the action of choice. Hence, the earlier an outcome is anticipated (based on stimulus outcome associations), the better it will be associated with its response, eventually leading to stronger O-R compatibility effects later on. Thus, one could speculate that increased activation in response to S-R-O mappings possibly embodies a marker for the ongoing transition from mere stimulus-based behavior to a goal-directed behavior throughout the learning course. Post-response brain activation revealed a seemingly two-fold feedback integration stream of O-R contingencies. On one hand the SMA seems to be engaged in bidirectional encoding processes of O-R associations. The results promote the general idea that the SMA is involved in the acquisition of goal-directed behavior (Elsner et al., 2002; Melcher, Weidema, Eenshuistra, Hommel, & Gruber, 2008; Melcher et al., 2013). Together with prior research (Frimmel, Wolfensteller, Mohr, & Ruge, 2016) this notion can be generalized not only to extensive learning phases but also to learning tasks in which goal-directed behavior is acquired in only few practice trials. However, there is an ongoing debate on whether SMA activation can be clearly linked to sub-processes prior or subsequent to an agent’s action (Nachev, Kennard, & Husain, 2008). The results of this work provide additional evidence favoring an involvement of the SMA only following a performed action in response to an imperative stimulus and even more, subsequent to the perception of its ensuing effect. This may give rise to the interpretation that the SMA is associated with linking the motor program of the performed action to the sensory program of the perceived effect, hence establishing and strengthening O-R contingencies. Furthermore, the analysis identified an increased coupling of a late negativity in the EEG signal and activation in the dorsal parts of the caudate as well as the somatosensory cortex. The dorsal caudate has not particularly been brought into connection with O-R learning so far. I speculate that the coupling effect in this part of the caudate reflects an ongoing process of an early automatization of the acquired behavior. It has already be shown in a similar paradigm that behavior can be automatized within only few repetitions of novel instructed S-R mappings (Mohr et al., 2016).:Table of contents Table of contents II List of Figures IV List of Tables VI List of Abbreviations VII 1 Summary 1 1.1 Introduction 1 1.2 Study Objectives 2 1.3 Methods 3 1.4 Results 4 1.5 Discussion 4 2 Theoretical Background 7 2.1 Introduction 7 2.2 Theories of acquiring goal-directed behavior 9 2.2.1 Instrumental learning 9 2.2.1.1 Behavioral aspects 9 2.2.1.2 Neurophysiological correlates 14 2.2.2 Acquisition of goal-directed behavior according to ideomotor theory 16 2.2.2.1 Behavioral aspects 16 2.2.2.2 Neurophysiological correlates 22 2.3 Summary 25 2.4 Methodological background 26 2.4.1 Electroencephalography (EEG) 26 2.4.2 Functional magnetic resonance imaging (fMRI) 28 2.4.3 Co-registered EEG-fMRI 29 3 General objectives and research questions 34 4 Study 1 – Learning-related brain-electrical activity dynamics associated with the subsequent impact of learnt action-outcome associations 36 4.1 Introduction 36 4.2 Methods 39 4.3 Results 47 4.4 Discussion 60 5 Study 2 - Within trial distinction of O-R learning-related BOLD activity with the means of co-registered EEG information 64 5.1 Introduction 64 5.2 Methods 66 5.3 Results 86 5.4 Discussion 101 6 Concluding general discussion 109 6.1 Brief assessment of study objectives 109 6.2 Novel insights into rapid instruction based S-R-O learning? 109 6.2.1 Early stimulus outcome information retrieval indicates the transition from stimulus based behavior to goal-directed action 110 6.2.2 Post-response encoding and consolidation of O-R contingencies enables goal-directedness of behavior 112 6.3 Critical reflection of the methodology and outlook 116 6.3.1 Strengths and limitations of this work 116 6.3.2 Data quality assessment 117 6.3.3 A common neural foundation for EEG and fMRI? 119 6.3.4 How can co-registered EEG-fMRI contribute to a better understanding of the human brain? 121 6.4 General Conclusion 123 7 References 124 Danksagung Erklärung

Au cours des deux dernières décennies, l'intérêt porté aux interfaces cérébrales ou Brain Computer Interfaces (BCI) s’est considérablement accru, avec un nombre croissant de laboratoires de recherche travaillant sur le sujet. Depuis le projet Brain Computer Interface, où la BCI a été présentée à des fins de réadaptation et d'assistance, l'utilisation de la BCI a été étendue à d'autres applications telles que le neurofeedback et l’industrie du jeux vidéo. Ce progrès a été réalisé grâce à une meilleure compréhension de l'électroencéphalographie (EEG), une amélioration des systèmes d’enregistrement du EEG, et une augmentation de puissance de calcul.Malgré son potentiel, la technologie de la BCI n’est pas encore mature et ne peut être utilisé en dehors des laboratoires. Il y a un tas de défis qui doivent être surmontés avant que les systèmes BCI puissent être utilisés à leur plein potentiel. Ce travail porte sur des aspects importants de ces défis, à savoir la spécificité des systèmes BCI aux capacités physiques des utilisateurs, la robustesse de la représentation et de l'apprentissage du EEG, ainsi que la suffisance des données d’entrainement. L'objectif est de fournir un système BCI qui peut s’adapter aux utilisateurs en fonction de leurs capacités physiques et des variabilités dans les signaux du cerveau enregistrés.À ces fins, deux voies principales sont explorées : la première, qui peut être considérée comme un ajustement de haut niveau, est un changement de paradigmes BCI. Elle porte sur la création de nouveaux paradigmes qui peuvent augmenter les performances de la BCI, alléger l'inconfort de l'utilisation de ces systèmes, et s’adapter aux besoins des utilisateurs. La deuxième voie, considérée comme une solution de bas niveau, porte sur l’amélioration des techniques de traitement du signal et d’apprentissage statistique pour améliorer la qualité du signal EEG, la reconnaissance des formes, ainsi que la tache de classification.D'une part, une nouvelle méthodologie dans le contexte de la robotique d'assistance est définie : il s’agit d’une approche hybride où une interface physique est complémentée par une interface cérébrale pour une interaction homme-machine plus fluide. Ce système hybride utilise les capacités motrices résiduelles des utilisateurs et offre la BCI comme un choix optionnel : l'utilisateur choisit quand utiliser la BCI et peut alterner entre les interfaces cérébrales et musculaire selon le besoin.D'autre part, pour l’amélioration des techniques de traitement du signal et d'apprentissage statistique, ce travail utilise un cadre Riemannien. Un frein majeur dans le domaine de la BCI est la faible résolution spatiale du EEG. Ce problème est dû à l'effet de conductance des os du crâne qui agissent comme un filtre passe-bas non linéaire, en mélangeant les signaux de différentes sources du cerveau et réduisant ainsi le rapport signal-à-bruit. Par conséquent, les méthodes de filtrage spatial ont été développées ou adaptées. La plupart d'entre elles – à savoir la Common Spatial Pattern (CSP), la xDAWN et la Canonical Correlation Analysis (CCA) – sont basées sur des estimations de matrice de covariance. Les matrices de covariance sont essentielles dans la représentation d’information contenue dans le signal EEG et constituent un élément important dans leur classification. Dans la plupart des algorithmes d'apprentissage statistique existants, les matrices de covariance sont traitées comme des éléments de l'espace euclidien. Cependant, étant symétrique et défini positive (SDP), les matrices de covariance sont situées dans un espace courbe qui est identifié comme une variété riemannienne. Utiliser les matrices de covariance comme caractéristique pour la classification des signaux EEG, et les manipuler avec les outils fournis par la géométrie de Riemann, fournit un cadre solide pour la représentation et l'apprentissage du EEG. / In the last two decades, interest in Brain-Computer Interfaces (BCI) has tremendously grown, with a number of research laboratories working on the topic. Since the Brain-Computer Interface Project of Vidal in 1973, where BCI was introduced for rehabilitative and assistive purposes, the use of BCI has been extended to more applications such as neurofeedback and entertainment. The credit of this progress should be granted to an improved understanding of electroencephalography (EEG), an improvement in its measurement techniques, and increased computational power.Despite the opportunities and potential of Brain-Computer Interface, the technology has yet to reach maturity and be used out of laboratories. There are several challenges that need to be addresses before BCI systems can be used to their full potential. This work examines in depth some of these challenges, namely the specificity of BCI systems to users physical abilities, the robustness of EEG representation and machine learning, and the adequacy of training data. The aim is to provide a BCI system that can adapt to individual users in terms of their physical abilities/disabilities, and variability in recorded brain signals.To this end, two main avenues are explored: the first, which can be regarded as a high-level adjustment, is a change in BCI paradigms. It is about creating new paradigms that increase their performance, ease the discomfort of using BCI systems, and adapt to the user’s needs. The second avenue, regarded as a low-level solution, is the refinement of signal processing and machine learning techniques to enhance the EEG signal quality, pattern recognition and classification.On the one hand, a new methodology in the context of assistive robotics is defined: it is a hybrid approach where a physical interface is complemented by a Brain-Computer Interface (BCI) for human machine interaction. This hybrid system makes use of users residual motor abilities and offers BCI as an optional choice: the user can choose when to rely on BCI and could alternate between the muscular- and brain-mediated interface at the appropriate time.On the other hand, for the refinement of signal processing and machine learning techniques, this work uses a Riemannian framework. A major limitation in this filed is the EEG poor spatial resolution. This limitation is due to the volume conductance effect, as the skull bones act as a non-linear low pass filter, mixing the brain source signals and thus reducing the signal-to-noise ratio. Consequently, spatial filtering methods have been developed or adapted. Most of them (i.e. Common Spatial Pattern, xDAWN, and Canonical Correlation Analysis) are based on covariance matrix estimations. The covariance matrices are key in the representation of information contained in the EEG signal and constitute an important feature in their classification. In most of the existing machine learning algorithms, covariance matrices are treated as elements of the Euclidean space. However, being Symmetric and Positive-Definite (SPD), covariance matrices lie on a curved space that is identified as a Riemannian manifold. Using covariance matrices as features for classification of EEG signals and handling them with the tools provided by Riemannian geometry provide a robust framework for EEG representation and learning.

Many people experience discomfort during movement when using a knee prosthesis. As a result, a study is being conducted to see if biomedical models can be used to produce an optimal prosthetic socket to reduce discomfort. This has previously been accomplished using pressure sensors embedded in the sleeve to measure leg pressure at various stages of the gait cycle. To gather more information, an EEG should be performed to find where discomfort occurs throughout a gait cycle. The purpose of the EEG is to perform measurements for a certain time during walking to then be able to analyze the signals that occur from the brain's motor cortex. The signal analysis is performed by applying the Fast Fourier transform to convert EEG data from the time domain to the frequency domain. By examining frequency differences, it is demonstrated that additional measurements with more individuals are required to ensure that the results reflect discomfort, and that the EEG is useful in the applied field of use. / Vid användning av knäproteser upplever ett stort antal personer obehag vid rörelse. På grund av detta utförs det en studie där man genom biomedicinska modeller kan producera den optimala proteshylsan för att minska obehag. Detta har tidigare gjorts med trycksensorer placerade i hylsan, för att kunna mäta tryck från benet vid olika faser av en gångcykel. EEG ska användas för att få mer information var det uppstår obehag under en gångcykel. Syftet med EEG är att genomföra mätningar under en viss tid vid gång, för att sedan kunna analysera signalerna från hjärnans motoriska cortex. Signalanalysen sker genom att transformera EEG-data från tidsdomänen till frekvensdomänen med hjälp av Snabb Fourier transform. Genom att undersöka frekvensskillnader påvisas att ytterligare mätningar med fler individer krävs för att säkerställa att resultaten återspeglar obehag och att EEG är användbart inom det tillämpade användningsområdet. / This work was supported by the research project SocketSense (https://www.socketsense.eu/), funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825429

Diese Dissertation untersucht einige ungeklärten Aspekte der Sprachproduktion. Das erste Ziel war es zu klären, wie Komposita (z.B. Goldfisch) auf der lexikalisch-syntaktischen Ebene unseres Sprachproduktionssystems repräsentiert sind. Gibt es dort einen einzelnen lexikalischen Eintrag für das gesamte Kompositum (GOLDFISCH) oder mehrere Einträge für jedes seiner Konstituenten (GOLD und FISCH), welche beim Sprechen zusammengesetzt werden? Zur Beantwortung dieser Frage wurde die sogenannte kumulative semantische Interferenz (KSI) verwendet. Dieser semantische Kontexteffekt beschreibt die Beobachtung, dass die Benennlatenzen von Sprechern systematisch länger werden, wenn diese eine Reihe von semantisch verwandten Bildern benennen. Obwohl KSI bereits viel als Instrument in der Sprachproduktionsforschung genutzt wird, sind einige Fragen rund um den Effekt selbst noch offen. Das zweite Ziel dieser Dissertation war es daher einige dieser Fragen mit Hilfe von behavioralen und elektrophysiologischen Maßen zu beantworten, um so unser Verständnis von KSI zu erweitern. Die Ergebnisse deuten darauf hin, dass KSI ihren Ursprung auf der konzeptuellen Ebene des Sprachproduktionssystems hat und dass sie nicht von der morphologischen Komplexität der verwendeten Begriffe moduliert wird, aber davon, wie häufig diese benannt werden. Diese Erkenntnisse ermöglichen es in der Zukunft zielgenauere Vorhersagen zu machen, wenn KSI als Forschungsinstrument verwendet wird. Die Ergebnisse zeigen zudem, dass die Konstituenten von Komposita während deren Produktion aktiviert werden. Dies belegt, dass Komposita in einer komplexen Struktur repräsentiert sind, die aus einem Eintrag für das ganze Kompositum und zusätzlichen Einträgen für die Konstituenten besteht. Somit zeigen diese Ergebnisse, dass die Morphologie bereits die Repräsentationen auf der lexikalisch-syntaktischen Ebene beeinflusst und erweitern somit unser Wissen über den Aufbau unseres Sprachproduktionssystems. / This dissertation addresses unresolved issues concerning speech production processes and the cognitive architecture of our speech production system. The first aim was to answer the question how compounds (e.g., goldfish) are represented on the lexical-syntactic level of our speech production system. Is there a single entry for the whole compound (GOLDFISH) or multiple ones for each of its constituents (GOLD and FISH), which are assembled for each use? To investigate this question, we used the cumulative semantic interference (CSI) effect. This semantic context effect describes the observation that speakers’ naming latencies systematically increase when naming a sequence of semantically related pictures. Although CSI has been extensively used as a tool in language production research, several aspects of it are not fully understood. Thus, the second aim of this dissertation was to close some of these knowledge gaps and gain a more comprehensive understanding of CSI. In three studies, we first investigated the CSI effect, before using it as a tool to study the lexical representation of compounds. Behavioural and electrophysiological data from the first two studies point to a purely conceptual origin of CSI. Furthermore, they revealed that CSI is not influenced by the items’ morphological complexity but affected by item repetition. These findings advance our understanding of CSI and thus allow us to make more informed predictions when using CSI as a research tool. The last study showed that the compounds’ constituents are activated during compound production, which provides evidence for a complex lexical-syntactic representation of compounds, consisting of one entry for the holistic compound and additional entries for each of its constituents. This dissertation thus reveals that the morphological complexity of compounds affects the lexical-syntactic level during speech production and thus advances our understanding of the architecture of our speech production system.

Kognitive Kontrolle bezieht sich auf eine Vielzahl mentaler Fähigkeiten, die es uns erlauben im täglichen Leben zielgerichtete Entscheidungen zu treffen und sich flexibel an sich ständig ändernde Umweltanforderungen anzupassen. Das Ziel der vorliegenden Dissertation war es heraus zu finden, ob Kernfunktionen im Bereich der Konfliktüberwachung, Konfliktkontrolle, Fehlerverarbeitung und die daraus resultierenden Verhaltensanpassungen durch ein einheitliches Kontrollnetzwerk geleistet werden, oder ob spezifische Mechanismen die möglicherweise durch unabhängige neuronale Kontrollschleifen realisiert sind, die Flexibilität unserer Anpassungsfähigkeit steuern. Studie 1 und Studie 2 untersuchen sowohl generelle aus auch spezifische Aspekte der Konflikt- und Fehlerverarbeitung mit Hilfe klassischer Konfliktparadigmen und dem Einsatz von Zeit-Frequenz-analytischen Auswertungsmethoden. Studie 1 untersucht anhand 3 verschiedener Konfliktparadigmen (Simon, Flanker, NoGo) die Modulation der Theta Aktivität (4-8 Hz) und verortet diese grob innerhalb des medial frontalen Cortex (MFC), einer Struktur die durch eine Vielzahl von Studien als entscheidend bei der Konfliktverarbeitung angesehen wird. Die gefundene Theta Aktivität wurde in Studie 2 genutzt, um auch dynamische Netzwerkaktivierungen bei der Bearbeitung von Reiz- und Reaktionskonflikten zu beobachten. Es konnte gezeigt werden, dass ein vermutetes Netzwerk bestehend aus MFC, lateralen präfrontalen Cortices und motorischen Arealen bei der Lösung von Reaktionskonflikten beteiligt ist. In Studie 3 wird eine Simon-Aufgabe, die innerhalb von belohnenden oder bestrafenden Kontexten durchgeführt wurde, genutzt um zu zeigen, dass Konflikt- und Fehlerverarbeitung differentiell durch die Kontextmanipulation beeinflusst werden. Entgegen voriger Annahmen scheinen mehrere neuronale Kontrollsysteme an der Lösung von Konflikten und daraus resultierenden Verhaltensanpassungen beteiligt zu sein. / Cognitive control refers to a set of mental abilities that allow us goal-directed behavior in everyday life and to flexibly adapt to permanently changing environmental demands. The goal of the present dissertation was to investigate whether core functions in the area of conflict monitoring, conflict control, error processing and behavioral adjustments caused by these processes are enabled via a unitary control network or whether specific mechanisms that are possibly realized via independent control loops are responsible for the flexibility of our adaptability. Study 1 and 2 investigate general as well as specific aspects of conflict and error processing by using classic conflict paradigms and time-frequency-analytic methods. Study 1 compares the modulation of theta activity (4-8 Hz) across 3 conflict paradigms (Simon, Flanker, NoGo) and roughly situates it within medial frontal cortex (MFC), a structure which has been characterized as crucial for conflict processing in manifold studies. The found theta activity has been used in study 2, to observe dynamic network activations during processing of stimulus and response conflicts. Data confirmed that a hypothesized network consisting of MFC, lateral prefrontal cortices and motor areas is involved in conflict resolution. In study 3 we used a Simon task which was executed either during a rewarding or a punishing context assessing the influence of motivational contexts on conflict adaptation revealing that conflict and error processing were influenced differentially by the context manipulation. Against previous assumptions, several neuronal control systems seem to be engaged during conflict resolution and resulting behavioral adjustments.

Die langfristigen Folgen von psychischer Fehlbeanspruchung stellen ein beträchtliches Problem unserer modernen Gesellschaft dar. Zur Identifizierung derartiger Fehlbelastungen während der Mensch-Maschine-Interaktion (MMI) kann die objektive, kontinuierliche Messung der psychischen Beanspruchung einen wesentlichen Beitrag leisten. Neueste Entwicklungen in der Sensortechnologie und der algorithmischen Methodenentwicklung auf Basis von KI liefern die Grundlagen zu ihrer messtechnischen Bestimmung. Vorarbeiten zur Entwicklung einer Methode zur neuronalen Beanspruchungsdiagnostik sind bereits erfolgt (Radüntz, 2017). Eine praxisrelevante Nutzung dieser Ergebnisse ist erfolgsversprechend, wenn die Methode mit Wearables kombiniert werden kann. Gleichzeitig sind die Evaluation und bedingungsbezogene Reliabilitätsprüfung der entwickelten Methode zur neuronalen Beanspruchungsdiagnostik in realitätsnahen Umgebungen erforderlich. Im Rahmen von experimentellen Untersuchungen der Gebrauchstauglichkeit von kommerziellen EEG-Registrierungssystemen für den mobilen Feldeinsatz wird die darauf basierende Systemauswahl für die MMI-Praxis getroffen. Die Untersuchungen zur Validierung der kontinuierlichen Methode zur Beanspruchungsdetektion erfolgt am Beispiel des Fluglotsenarbeitsplatzes beim simulierten „Arrival Management“. / The long-term negative consequences of inappropriate mental workload on employee health constitute a serious problem for a digitalized society. Continuous, objective assessment of mental workload can provide an essential contribution to the identification of such improper load. Recent improvements in sensor technology and algorithmic methods for biosignal processing are the basis for the quantitative determination of mental workload. Neuronal workload measurement has the advantage that workload registration is located directly there where human information processing takes place, namely the brain. Preliminary studies for the development of a method for neuronal workload registration by use of the electroencephalogram (EEG) have already been carried out [Rad16, Rad17]. For the field use of these findings, the mental workload assess- ment on the basis of the EEG must be evaluated and its reliability examined with respect to several conditions in realistic environments. A further essential require-ment is that the method can be combined with the innovative technologies of gel free EEG registration and wireless signal transmission. Hence, the presented papers include two investigations. Main subject of the first investigation are experimental studies on the usability of commercially-oriented EEG systems for mobile field use and system selection for the future work. Main subject of the second investigation is the evaluation of the continuous method for neuronal mental workload registration in the field. Thereby, a challenging application was used, namely the arrival management of aircraft. The simulation of the air traffic control environment allows the realisation of realistic conditions with different levels of task load. Furthermore, the work is well contextualized in a domain which is very sensible to human-factors research.