The main objective of this dissertation is to investigate the effects of acquiring a second language (L2) at later periods of language development and native-like homonym processing in the first language (L1) from the perspective of Event-Related brain Potentials (ERP) using a cross-modal lexical decision task. To date, there is a lack of neurophysiological investigations into the effect that acquiring an L2 can have on processing strategies in the L1, and whether or not there is a precise age at which L2 exposure no longer affects native-like language processing. As such, my goal is to pinpoint this sensitive period specifically for homonym processing. To achieve this, I will present and discuss the results of two studies. The first study employs behavioural response measures using a cross-modal lexical decision task where participants simultaneously heard a sentence and made a decision to a visually-presented pseudoword or real word. The second study employs ERP measures using a novel ERP paradigm which investigates not only the main objective of this dissertation, but the second objective as well. This second objective is for this dissertation to become the first to evaluate the outcome of combining the cross-modal lexical decision task with ERPs. The behavioural and neurophysiological results for the monolingual group support the Reordered Access Model (Duffy, Morris, & Rayner, 1988) while the results for the bilingual groups do not. The results of the current studies indicate that those bilinguals who acquired French as an L2 rather than as a second native L1 show increasing divergence from monolingual native speakers in L1 homonym processing, with later acquirers exhibiting an exponentially marked divergence. This was found even though the task was carried out in English, the L1 (or one of the L1s) of all participants. The diverging performances of the bilinguals from the monolinguals were apparent in behavioural responses as well as in the amplitude, scalp distribution, and latency of ERP components, These differences were unique to each group, which supports the hypothesis that the acquisition of an L2 influences processing in the L1 (Dussias & Sagarra, 2007). Specifically, the early and late bilingual groups exhibited a marked divergence from the monolingual group as they revealed syntactic priming effects (p<.001) as well as lexical frequency effects (p<.001). They also revealed the greatest P600-like effect as they processed target words which were inappropriately- related to the priming homonyms (such as skin in Richard had a shed in the back of the garden). This suggests a heightened sensitivity to surface cues due to the L2 influencing homonym processing in the L1 (Cook, 2003; Dussias & Sagarra, 2007). Comparatively, the monolingual group revealed equal N400-like effects for lexical ambiguities overall compared to the unrelated conditions, and a context-by-frequency-interaction slowing their processing of the target word that is appropriately-related to the subordinate reading of the priming homonym, suggesting that they are not as sensitive to these same surface cues. Importantly, these results confirm that using ERPs along with a cross-modal lexical decision task is a promising paradigm to further study language processing.

No description available.

The main objective of this dissertation is to investigate the effects of acquiring a second language (L2) at later periods of language development and native-like homonym processing in the first language (L1) from the perspective of Event-Related brain Potentials (ERP) using a cross-modal lexical decision task. To date, there is a lack of neurophysiological investigations into the effect that acquiring an L2 can have on processing strategies in the L1, and whether or not there is a precise age at which L2 exposure no longer affects native-like language processing. As such, my goal is to pinpoint this sensitive period specifically for homonym processing. To achieve this, I will present and discuss the results of two studies. The first study employs behavioural response measures using a cross-modal lexical decision task where participants simultaneously heard a sentence and made a decision to a visually-presented pseudoword or real word. The second study employs ERP measures using a novel ERP paradigm which investigates not only the main objective of this dissertation, but the second objective as well. This second objective is for this dissertation to become the first to evaluate the outcome of combining the cross-modal lexical decision task with ERPs. The behavioural and neurophysiological results for the monolingual group support the Reordered Access Model (Duffy, Morris, & Rayner, 1988) while the results for the bilingual groups do not. The results of the current studies indicate that those bilinguals who acquired French as an L2 rather than as a second native L1 show increasing divergence from monolingual native speakers in L1 homonym processing, with later acquirers exhibiting an exponentially marked divergence. This was found even though the task was carried out in English, the L1 (or one of the L1s) of all participants. The diverging performances of the bilinguals from the monolinguals were apparent in behavioural responses as well as in the amplitude, scalp distribution, and latency of ERP components, These differences were unique to each group, which supports the hypothesis that the acquisition of an L2 influences processing in the L1 (Dussias & Sagarra, 2007). Specifically, the early and late bilingual groups exhibited a marked divergence from the monolingual group as they revealed syntactic priming effects (p<.001) as well as lexical frequency effects (p<.001). They also revealed the greatest P600-like effect as they processed target words which were inappropriately- related to the priming homonyms (such as skin in Richard had a shed in the back of the garden). This suggests a heightened sensitivity to surface cues due to the L2 influencing homonym processing in the L1 (Cook, 2003; Dussias & Sagarra, 2007). Comparatively, the monolingual group revealed equal N400-like effects for lexical ambiguities overall compared to the unrelated conditions, and a context-by-frequency-interaction slowing their processing of the target word that is appropriately-related to the subordinate reading of the priming homonym, suggesting that they are not as sensitive to these same surface cues. Importantly, these results confirm that using ERPs along with a cross-modal lexical decision task is a promising paradigm to further study language processing.

Substantial research has been done on the electroencephalogram (EEG) neural signals generated by feedback within a simple choice task, and there is much evidence for the existence of a reward prediction error signal generated in the anterior cingulate cortex of the brain when the outcome of this type of choice does not match expectations. However, less research has been done to date on the neural responses to intermediate outcomes in a multi-step choice task. Here, I investigated the neural signals generated by a complex, non-deterministic task that involved multiple choices before final win/loss feedback in order to see if the observed signals correspond to predictions made by reinforcement learning theory. In Experiment One, I conducted an EEG experiment to record neural signals while participants performed a computerized task designed to elicit the reward positivity, an event-related brain potential (ERP) component thought to be a biological reward prediction error signal. EEG results revealed a difference in amplitude of the reward positivity ERP component between experimental conditions comparing unexpected to expected feedback, as well as an interaction between valence and expectancy of the feedback. Additionally, results of an ERP analysis of the amplitude of the P300 component also showed an interaction between valence and expectancy. In Experiment Two, I used machine learning to classify epoched EEG data from Experiment One into experimental conditions to determine if individual states within the task could be differentiated based solely on the EEG data. My results showed that individual states could be differentiated with above-chance accuracy. I conclude by discussing how these results fit with the predictions made by reinforcement learning theory about the type of task investigated herein, and implications of those findings on our understanding of learning and decision-making in humans. / Graduate

This dissertation investigates how grammatical functions in transitive sentences (i.e., `subject' and `direct object') are distributed in written Swedish discourse with respect to morphosyntactic as well as semantic and referential (i.e., prominence-based) information. It also investigates how assignment of grammatical functions during on-line comprehension of transitive sentences in Swedish is influenced by interactions between morphosyntactic and prominence-based information. In the dissertation, grammatical functions are assumed to express role-semantic (e.g., Actor and Undergoer) and discourse-pragmatic (e.g., Topic and Focus) functions of NP arguments. Grammatical functions correlate with prominence-based information that is associated with these functions (e.g., animacy and definiteness). Because of these correlations, both prominence-based and morphosyntactic information are assumed to serve as argument interpretation cues during on-line comprehension. These cues are utilized in a probabilistic fashion. The weightings, interplay and availability of them are reflected in their distribution in language use, as shown in corpus data. The dissertation investigates these assumptions by using various methods in a triangulating fashion. The first contribution of the dissertation is an ERP (event-related brain potentials) experiment that investigates the ERP response to grammatical function reanalysis, i.e., a revision of a tentative grammatical function assignment, during on-line comprehension of transitive sentences. Grammatical function reanalysis engenders a response that correlates with the (re-)assignment of thematic roles to the NP arguments. This suggests that the comprehension of grammatical functions involves assigning role-semantic functions to the NPs. The second contribution is a corpus study that investigates the distribution of prominence-based, verb-semantic and morphosyntactic features in transitive sentences in written discourse. The study finds that overt morphosyntactic information about grammatical functions is used more frequently when the grammatical functions cannot be determined on the basis of word order or animacy. This suggests that writers are inclined to accommodate the understanding of their recipients by more often providing formal markers of grammatical functions in potentially ambiguous sentences. The study also finds that prominence features and their interactions with verb-semantic features are systematically distributed across grammatical functions and therefore can predict these functions with a high degree of confidence. The third contribution consists of three computational models of incremental grammatical function assignment. These models are based upon the distribution of argument interpretation cues in written discourse. They predict processing difficulties during grammatical function assignment in terms of on-line change in the expectation of different grammatical function assignments over the presentation of sentence constituents. The most prominent model predictions are qualitatively consistent with reading times in a self-paced reading experiment of Swedish transitive sentences. These findings indicate that grammatical function assignment draws upon statistical regularities in the distribution of morphosyntactic and prominence-based information in language use. Processing difficulties in the comprehension of Swedish transitive sentences can therefore be predicted on the basis of corpus distributions.

L'objectif de cette thèse est d'étudier le développement du système lexico-sémantique chez les enfants monolingues et bilingues. La question posée est la suivante : quand et comment les significations des mots commencent à être reliées entre elles et à s'intégrer dans un système sémantique interconnecté. Dans un premier temps, trois études ont été menées chez des enfants monolingues français. L'Etude 1, a pour but d'observer si les mots sont organisés selon des liens taxonomiques (e.g., cochon - cheval). L'Etude 2 explore si l'effet d'amorçage sémantique est sous-tendu par des mécanismes cognitifs, comme les processus d'activation automatique et contrôlé. Puis enfin, l'Etude 3 observe si les mots sont organisés en fonction de leur distance de similarité sémantique (e.g., vache - mouton versus vache - cerf). Dans un deuxième temps, deux études ont été conduites chez des enfants apprenant deux langues simultanément. L'Etude 4 vise à déterminer si les mots sont taxonomiquement liés dans chacune des langues. L'Etude 5 explore si les mots présentés dans une langue activent leurs représentations sémantiques dans l'autre langue et vice versa. Dans le but de répondre à ces questions, le traitement lexico-sémantique a été étudié en utilisant deux techniques : l'eye-tracking et les potentiels évoqués (PEs). Ces deux techniques enregistrent lors de la présentation des mots des réponses comportementales (Etude 3) et neuronales (Etude 1, 2, 4 et 5) de haute résolution temporelle. Les Etudes 1 et 2 montrent que chez les monolingues les mots sont liés taxonomiquement à l'âge de 18 et 24 mois. Durant le développement du langage, les deux processus d'activation automatique et contrôlé sont impliqués dans le traitement des mots (Etude 2). L'Etude 3 montre qu'à 24 mois, les mots sont organisés dans le système lexico-sémantique en développement selon la distance des similarités sémantiques. L'Etude 4 montre que chez les enfants bilingues, le traitement sémantique ne diffère pas selon les deux langues, mais la topographie des PEs varie selon la langue traitée. L'Etude 5 montre que les mots présentés dans une langue activent leurs représentations sémantiques dans la deuxième langue et vice versa. Toutefois, la topographie des PEs est modulée selon la direction de traduction. Ces résultats suggèrent que l'acquisition de deux langues, bien qu'elle soit très précoce, requière deux ressources neuronales bien distinctes, sous-tendant ainsi le traitement lexico-sémantique des langues dominante et non-dominante. / The present doctoral research explored the developing lexical-semantic system in monolingual and bilingual toddlers. The question of how and when word meanings are first related to each other and become integrated into an interconnected semantic system was investigated. Three studies were conducted with monolingual French learning children which aimed at exploring how words are organized, that is, according to taxonomic relationships (e.g., pig - horse) and to semantic similarity distances between words (e.g., cow - sheep versus cow - deer), and whether cognitive mechanisms, such as automatic activation and controlled processes, underlie priming effects. An additional two studies conducted with children learning two languages simultaneously, aimed at determining, first, whether taxonomically related word meanings, in each of the two languages, are processed in a similar manner. The second goal was to explore whether words presented in one language activate words in another language, and vice versa. In an attempt to answer these questions, lexical-semantic processing was explored by two techniques: eye-tracking and event-related potentials (ERPs) techniques. Both techniques provide high temporal resolution measures of word processing but differ in terms of responses. Eye-movement measurements (Study III) reflect looking preferences in response to spoken words and their time-course, whereas ERPs reflect implicit brain responses and their activity patterns (Study I, II, IV, and V). Study I and II revealed that words are taxonomically organized at 18 and 24-month-olds. Both automatic and controlled processes were shown to be involved in word processing during language development (Study II). Study III revealed that at 24-month-olds, categorical and feature overlap between items underpin the developing lexical-semantic system. That is, lexical-items in each semantic category are organized according to graded similarity distances. Productive vocabulary skills influenced word recognition and were related to underlying cognitive mechanisms. Study IV revealed no differences in terms of semantic processing in the bilinguals¿ two languages, but the ERP distribution across the scalp varied according to the language being processed. Study V showed that words presented in one language activate their semantic representations in the second language and the other way around. The distribution of the ERPs depended, however, on the direction of translation. The results suggest that even early dual language experience yields distinct neural resources underlying lexical-semantic processing in the dominant and non-dominant languages during language acquisition.

L'objectif de cette thèse est d'étudier le développement du système lexico-sémantique chez les enfants monolingues et bilingues. La question posée est la suivante : quand et comment les significations des mots commencent à être reliées entre elles et à s'intégrer dans un système sémantique interconnecté. Dans un premier temps, trois études ont été menées chez des enfants monolingues français. L'Etude 1, a pour but d'observer si les mots sont organisés selon des liens taxonomiques (e.g., cochon - cheval). L'Etude 2 explore si l'effet d'amorçage sémantique est sous-tendu par des mécanismes cognitifs, comme les processus d'activation automatique et contrôlé. Puis enfin, l'Etude 3 observe si les mots sont organisés en fonction de leur distance de similarité sémantique (e.g., vache - mouton versus vache - cerf). Dans un deuxième temps, deux études ont été conduites chez des enfants apprenant deux langues simultanément. L'Etude 4 vise à déterminer si les mots sont taxonomiquement liés dans chacune des langues. L'Etude 5 explore si les mots présentés dans une langue activent leurs représentations sémantiques dans l'autre langue et vice versa. Dans le but de répondre à ces questions, le traitement lexico-sémantique a été étudié en utilisant deux techniques : l'eye-tracking et les potentiels évoqués (PEs). Ces deux techniques enregistrent lors de la présentation des mots des réponses comportementales (Etude 3) et neuronales (Etude 1, 2, 4 et 5) de haute résolution temporelle. Les Etudes 1 et 2 montrent que chez les monolingues les mots sont liés taxonomiquement à l'âge de 18 et 24 mois. Durant le développement du langage, les deux processus d'activation automatique et contrôlé sont impliqués dans le traitement des mots (Etude 2). L'Etude 3 montre qu'à 24 mois, les mots sont organisés dans le système lexico-sémantique en développement selon la distance des similarités sémantiques. L'Etude 4 montre que chez les enfants bilingues, le traitement sémantique ne diffère pas selon les deux langues, mais la topographie des PEs varie selon la langue traitée. L'Etude 5 montre que les mots présentés dans une langue activent leurs représentations sémantiques dans la deuxième langue et vice versa. Toutefois, la topographie des PEs est modulée selon la direction de traduction. Ces résultats suggèrent que l'acquisition de deux langues, bien qu'elle soit très précoce, requière deux ressources neuronales bien distinctes, sous-tendant ainsi le traitement lexico-sémantique des langues dominante et non-dominante. / The present doctoral research explored the developing lexical-semantic system in monolingual and bilingual toddlers. The question of how and when word meanings are first related to each other and become integrated into an interconnected semantic system was investigated. Three studies were conducted with monolingual French learning children which aimed at exploring how words are organized, that is, according to taxonomic relationships (e.g., pig - horse) and to semantic similarity distances between words (e.g., cow - sheep versus cow - deer), and whether cognitive mechanisms, such as automatic activation and controlled processes, underlie priming effects. An additional two studies conducted with children learning two languages simultaneously, aimed at determining, first, whether taxonomically related word meanings, in each of the two languages, are processed in a similar manner. The second goal was to explore whether words presented in one language activate words in another language, and vice versa. In an attempt to answer these questions, lexical-semantic processing was explored by two techniques: eye-tracking and event-related potentials (ERPs) techniques. Both techniques provide high temporal resolution measures of word processing but differ in terms of responses. Eye-movement measurements (Study III) reflect looking preferences in response to spoken words and their time-course, whereas ERPs reflect implicit brain responses and their activity patterns (Study I, II, IV, and V). Study I and II revealed that words are taxonomically organized at 18 and 24-month-olds. Both automatic and controlled processes were shown to be involved in word processing during language development (Study II). Study III revealed that at 24-month-olds, categorical and feature overlap between items underpin the developing lexical-semantic system. That is, lexical-items in each semantic category are organized according to graded similarity distances. Productive vocabulary skills influenced word recognition and were related to underlying cognitive mechanisms. Study IV revealed no differences in terms of semantic processing in the bilinguals¿ two languages, but the ERP distribution across the scalp varied according to the language being processed. Study V showed that words presented in one language activate their semantic representations in the second language and the other way around. The distribution of the ERPs depended, however, on the direction of translation. The results suggest that even early dual language experience yields distinct neural resources underlying lexical-semantic processing in the dominant and non-dominant languages during language acquisition.

The present study intended to examine the neural basis of audiovisual integration, hypothetically achieved by synchronized gamma-band oscillations (30--80 Hz) that have been suggested to integrate stimulus features and top--down information. To that end, we studied the impact of visual symbolic information on early auditory sensory processing of upcoming sounds. In particular, we used a symbol-to-sound--matching paradigm in which simple score-like patterns predict corresponding sound patterns. Occasionally, a single sound is incongruent with the corresponding element of the visual pattern. In response to expected sounds congruent with the corresponding visual symbol, a power increase of phase-locked (evoked) activity in the 40-Hz band was observed peaking 42-ms poststimulus onset. Thus, for the first time, we demonstrated that the comparison process between a neural model, the expectation, and the current sensory input is implemented at very early levels of auditory processing. Subsequently, expected congruent sounds elicited a broadband power increase of non--phase-locked (induced)activity peaking 152-ms poststimulus onset, which might reflect the formation of a unitary event representation including both visual and auditory aspects of the stimulation. Gamma-band responses were not present for unexpected incongruent sounds. A model explaining the anticipatory activation of cortical auditory representations and the match of experience against expectation is presented

L'objectif de cette thèse est d'étudier le développement du système lexico-sémantique chez les enfants monolingues et bilingues. La question posée est la suivante : quand et comment les significations des mots commencent à être reliées entre elles et à s'intégrer dans un système sémantique interconnecté. Dans un premier temps, trois études ont été menées chez des enfants monolingues français. L'Etude 1, a pour but d'observer si les mots sont organisés selon des liens taxonomiques (e.g., cochon - cheval). L'Etude 2 explore si l'effet d'amorçage sémantique est sous-tendu par des mécanismes cognitifs, comme les processus d'activation automatique et contrôlé. Puis enfin, l'Etude 3 observe si les mots sont organisés en fonction de leur distance de similarité sémantique (e.g., vache - mouton versus vache - cerf). Dans un deuxième temps, deux études ont été conduites chez des enfants apprenant deux langues simultanément. L'Etude 4 vise à déterminer si les mots sont taxonomiquement liés dans chacune des langues. L'Etude 5 explore si les mots présentés dans une langue activent leurs représentations sémantiques dans l'autre langue et vice versa. Dans le but de répondre à ces questions, le traitement lexico-sémantique a été étudié en utilisant deux techniques : l'eye-tracking et les potentiels évoqués (PEs). Ces deux techniques enregistrent lors de la présentation des mots des réponses comportementales (Etude 3) et neuronales (Etude 1, 2, 4 et 5) de haute résolution temporelle. Les Etudes 1 et 2 montrent que chez les monolingues les mots sont liés taxonomiquement à l'âge de 18 et 24 mois. Durant le développement du langage, les deux processus d'activation automatique et contrôlé sont impliqués dans le traitement des mots (Etude 2). L'Etude 3 montre qu'à 24 mois, les mots sont organisés dans le système lexico-sémantique en développement selon la distance des similarités sémantiques. L'Etude 4 montre que chez les enfants bilingues, le traitement sémantique ne diffère pas selon les deux langues, mais la topographie des PEs varie selon la langue traitée. L'Etude 5 montre que les mots présentés dans une langue activent leurs représentations sémantiques dans la deuxième langue et vice versa. Toutefois, la topographie des PEs est modulée selon la direction de traduction. Ces résultats suggèrent que l'acquisition de deux langues, bien qu'elle soit très précoce, requière deux ressources neuronales bien distinctes, sous-tendant ainsi le traitement lexico-sémantique des langues dominante et non-dominante. / The present doctoral research explored the developing lexical-semantic system in monolingual and bilingual toddlers. The question of how and when word meanings are first related to each other and become integrated into an interconnected semantic system was investigated. Three studies were conducted with monolingual French learning children which aimed at exploring how words are organized, that is, according to taxonomic relationships (e.g., pig - horse) and to semantic similarity distances between words (e.g., cow - sheep versus cow - deer), and whether cognitive mechanisms, such as automatic activation and controlled processes, underlie priming effects. An additional two studies conducted with children learning two languages simultaneously, aimed at determining, first, whether taxonomically related word meanings, in each of the two languages, are processed in a similar manner. The second goal was to explore whether words presented in one language activate words in another language, and vice versa. In an attempt to answer these questions, lexical-semantic processing was explored by two techniques: eye-tracking and event-related potentials (ERPs) techniques. Both techniques provide high temporal resolution measures of word processing but differ in terms of responses. Eye-movement measurements (Study III) reflect looking preferences in response to spoken words and their time-course, whereas ERPs reflect implicit brain responses and their activity patterns (Study I, II, IV, and V). Study I and II revealed that words are taxonomically organized at 18 and 24-month-olds. Both automatic and controlled processes were shown to be involved in word processing during language development (Study II). Study III revealed that at 24-month-olds, categorical and feature overlap between items underpin the developing lexical-semantic system. That is, lexical-items in each semantic category are organized according to graded similarity distances. Productive vocabulary skills influenced word recognition and were related to underlying cognitive mechanisms. Study IV revealed no differences in terms of semantic processing in the bilinguals¿ two languages, but the ERP distribution across the scalp varied according to the language being processed. Study V showed that words presented in one language activate their semantic representations in the second language and the other way around. The distribution of the ERPs depended, however, on the direction of translation. The results suggest that even early dual language experience yields distinct neural resources underlying lexical-semantic processing in the dominant and non-dominant languages during language acquisition.

Die Blickrichtung und ihre Richtung sind wichtige nonverbale Hinweise für die Etablierung von sozialen Interaktionen und die Wahrnehmung von emotionalen Gesichtsausdrücken bei anderen. Ob der Blick direkt auf den Betrachter gerichtet ist (direkter Blick) oder abgewendet (abgewandter Blick), beeinflusst unsere soziale Aufmerksamkeit und emotionale Reaktionen. Dies deutet darauf hin, dass Emotionen und Blickrichtung informative Werte haben, die sich möglicherweise in frühen oder späteren Stadien der neurokognitiven Verarbeitung interagieren. Trotz theoretischer Grundlage, der geteilten Signal-Hypothese (Adams & Kleck, 2003), gibt es einen Mangel an strukturierten elektrophysiologischen Untersuchungen zu den Wechselwirkungen zwischen Emotionen und Blickrichtung sowie ihren neuronalen Korrelaten und wie sie sich in verschiedenen Bevölkerungsgruppen unterscheiden. Um diese Lücke zu schließen, verwendete diese Doktorarbeit ereigniskorrelierte Hirnpotentiale (ERPs), um die Reaktionen auf emotionale Ausdrücke und Blickrichtung in einem neuen Paradigma zu untersuchen, das statischen und dynamischen Blick mit Gesichtsausdrücken kombiniert. Es wurden drei verschiedene Populationen untersucht. Studie 1 untersuchte in einer normalen Stichprobe die Amplituden der ERP-Komponenten, die durch die erstmalige Präsentation von Gesichtern und nachfolgende Änderungen der Blickrichtung in der Hälfte der Durchgänge ausgelöst wurden. In Studie 2 wurden aufgrund der atypischen Gesichtsverarbeitung und verminderten Reaktionen auf Augenblick beim Autismus die ERPs und Augenbewegungen bei zwei Stichproben von Kindern mit unterschiedlichem Schweregrad ihrer Autismusmerkmale untersucht. In Studie 3 wurde in einer großen Stichprobe die vermutlich erhöhte Sensitivität bei der Emotionsverarbeitung und Reaktion auf Augenblick bei Müttern im postpartalen Zeitraum mit besonderem Fokus auf die Gesichter von Säuglingen untersucht. Zusammenfassend zeigen die Ergebnisse der drei Studien, dass in sozialen Interaktionen die emotionalen Effekte von Gesichtern durch die dynamische Blickrichtung moduliert werden. / The eye gaze and its direction are important and relevant non-verbal cues for the establishment of social interactions and the perception of others’ emotional facial expressions. Gaze direction itself, whether eyes are looking straight at the viewer (direct gaze) or whether they look away (averted gaze), affects our social attention and emotional response. This implies that both emotion and gaze have informational values, which might interact at early or later stages of neurocognitive processing. Despite the suggestion of a theoretical basis for this interaction, the shared signal hypothesis (Adams & Kleck, 2003), there is a lack of structured electrophysiological investigations into the interactions between emotion and gaze and their neural correlates, and how they vary across populations. Addressing this need, the present doctoral dissertation used event-related brain potentials (ERPs) to study responses to emotional expressions and gaze direction in a novel paradigm combining static and dynamic gaze with facial expressions. The N170 and EPN were selected as ERP components believed to reflect gaze perception and reflexive attention, respectively. Three different populations were investigated. Study 1, in a normal sample, investigated the amplitudes of the ERP components elicited by the initial presentation of faces and subsequent changes of gaze direction in half of the trials. In Study 2, based on the atypical face processing and diminished responses to eye gaze in autism, the ERPs and eye movements were examined in two samples of children varying in the severity of their autism traits. In Study 3, In a large sample, I addressed the putatively increased sensitivity in emotion processing and response to eye gaze in mothers during their postpartum period with a particular focus on infant's faces. Taken together, the results from three studies demonstrate that in social interactions, the emotional effects of faces are modulated by dynamic gaze direction.