Brain-computer interfaces (BCIs) might restore communication to people severely disabled by amyotrophic lateral sclerosis (ALS) or other disorders. We sought to: 1) define a protocol for determining whether a person with ALS can use a visual P300-based BCI; 2) determine what proportion of this population can use the BCI; and 3) identify factors affecting BCI performance. Twenty-five individuals with ALS completed an evaluation protocol using a standard 6 × 6 matrix and parameters selected by stepwise linear discrimination. With an 8-channel EEG montage, the subjects fell into two groups in BCI accuracy (chance accuracy 3%). Seventeen averaged 92 (± 3)% (range 71-100%), which is adequate for communication (G70 group). Eight averaged 12 (± 6)% (range 0-36%), inadequate for communication (L40 subject group). Performance did not correlate with disability: 11/17 (65%) of G70 subjects were severely disabled (i.e. ALSFRS-R < 5). All L40 subjects had visual impairments (e.g. nystagmus, diplopia, ptosis). P300 was larger and more anterior in G70 subjects. A 16-channel montage did not significantly improve accuracy. In conclusion, most people severely disabled by ALS could use a visual P300-based BCI for communication. In those who could not, visual impairment was the principal obstacle. For these individuals, auditory P300-based BCIs might be effective.

The world in which we live is filled with sensory experiences. Language provides us with a manner in which to communicate these experiences with one another. In order to partake in this communication, it is necessary to acquire labels for things we see, hear, feel, smell, and taste. Much is known about how we learn words for things we can see, but this bias in the literature leaves many open questions about words attributed to other modalities. This cumulative dissertation aims to close this gap by investigating how both 10- to-12-month old infants and adults map novel pseudowords onto environmental sounds in an auditory associative word learning task with the aim to explore how humans learn words for things that cannot be seen, such as thunder, siren, and, lullaby. Infants were found, via event-related potentials (ERPs), to be successful at auditory associative word learning, while adults are much stronger learners in multimodal audio-visual conditions. Across the lifespan, sensory modality was found to affect word learning differently in infants than in adults. Where infants benefitted from unimodal auditory word learning, adults were more successful in multimodal audiovisual paradigms. Furthermore, the modality of the object being labelled modulated the temporal onset and the topological distribution of the N400 ERP component of violated lexical-semantic expectation. Lastly, the temporal congruency of presented stimuli affected word learning in adults in an inverted manner to other forms of statistical learning. Word learning is sensitive to age, modality, and means of presentation, providing evidence for various intertwined learning mechanisms and bringing us a step closer towards understanding human linguistic cognition.

La reconnaissance experte de l'expression faciale est cruciale pour l'interaction et la communication sociale. Le comportement, les potentiels évoqués (ERP), et les techniques d’oculométrie peuvent être utilisés pour étudier les mécanismes cérébraux qui participent au traitement visuel automatique. La reconnaissance d'expressions faciales implique non seulement l'extraction d'informations à partir de caractéristiques faciales diagnostiques, stratégie qualifiée de traitement local, mais aussi l'intégration d'informations globales impliquant des traitements configuraux. Des nombreuses recherches concernant le traitement des informations faciales émotionnelles il apparaît que l’interaction des traitements locaux et configuraux pour la reconnaissance des émotions est mal comprise. La complexité inhérente à l'intégration de l'information faciale est mise en lumière lorsque l'on compare la performance de sujets sains et d’individus atteints de schizophrénie, car ces derniers ont tendance à s’attarder sur quelques éléments locaux, parfois peu informatifs. Les différentes façons d'examiner les visages peuvent avoir un impact sur la capacité socio-cognitive de reconnaître les émotions. Pour ces raisons, cette thèse étudie le rôle des caractéristiques diagnostiques et configurales dans la reconnaissance de l'expression faciale. En plus des aspects comportementaux, nous avons donc examiné la dynamique spatiale et temporelle des fixations à l’aide de mesures oculométriques, ainsi que l’activité électrophysiologique précoce considérant plus particulièrement les composantes P100 et N170. Nous avons créé de nouveaux stimuli des esquisses par une transformation numérique de portraits photos en esquisses, pour des visages exprimant colère, tristesse, peur, joie ou neutralité, issus de la base Radboud Faces Database, en supprimant les informations de texture du visage et ne conservant que les caractéristiques diagnostiques (yeux et sourcils, nez, bouche). Ces esquisses altèrent le traitement configural en comparaison avec les visages photographiques, ce qui augmente le traitement des caractéristiques diagnostiques par traitement élémentaire, en contrepartie. La comparaison directe des mesures neurocognitives entre les esquisses et les visages photographiques exprimant des émotions de base n'a jamais été testée, à notre connaissance. Dans cette thèse, nous avons examiné (i) les fixations oculaires en fonction du type de stimulus, (ii) la réponse électrique aux manipulations expérimentales telles que l'inversion et la déconfiguration du visage. Concernant, les résultats comportementaux montrent que les esquisses de visage transmettent suffisamment d'information expressive (compte tenu de la présence des caractéristiques diagnostiques) pour la reconnaissance des émotions en comparaison des visages photographiques. Notons que, comme attendu, il y avait un net avantage de la reconnaissance des émotions pour les expressions heureuses par rapport aux autres émotions. En revanche, reconnaître des visages tristes et en colère était plus difficile. Ayant analysé séparément les fixations successives, les résultats indiquent que les participants ont adopté un traitement plus local des visages croqués et photographiés lors de la deuxième fixation. Néanmoins, l'extraction de l'information des yeux est nécessaire lorsque l'expression transmet des informations émotionnelles plus complexes et lorsque les stimuli sont simplifiés comme dans les esquisses. Les résultats de l’électroencéphalographie suggèrent également que les esquisses ont engendré plus de traitement basé sur les parties. Les éléments transmis par les traits diagnostiques pourraient avoir fait l'objet d'un traitement précoce, probablement dû à des informations de bas niveau durant la fenêtre temporelle de la P100, suivi d'un décodage ultérieur de la structure faciale dans la fenêtre temporelle de la N170. / Proficient recognition of facial expression is crucial for social interaction. Behaviour, event-related potentials (ERPs), and eye-tracking techniques can be used to investigate the underlying brain mechanisms supporting this seemingly effortless processing of facial expression. Facial expression recognition involves not only the extraction of expressive information from diagnostic facial features, known as part-based processing, but also the integration of featural information, known as configural processing. Despite the critical role of diagnostic features in emotion recognition and extensive research in this area, it is still not known how the brain decodes configural information in terms of emotion recognition. The complexity of facial information integration becomes evident when comparing performance between healthy subjects and individuals with schizophrenia because those patients tend to process featural information on emotional faces. The different ways in examining faces possibly impact on social-cognitive ability in recognizing emotions. Therefore, this thesis investigates the role of diagnostic features and face configuration in the recognition of facial expression. In addition to behavior, we examined both the spatiotemporal dynamics of fixations using eye-tracking, and early neurocognitive sensitivity to face as indexed by the P100 and N170 ERP components. In order to address the questions, we built a new set of sketch face stimuli by transforming photographed faces from the Radboud Faces Database through the removal of facial texture and retaining only the diagnostic features (e.g., eyes, nose, mouth) with neutral and four facial expressions - anger, sadness, fear, happiness. Sketch faces supposedly impair configural processing in comparison with photographed faces, resulting in increased sensitivity to diagnostic features through part-based processing. The direct comparison of neurocognitive measures between sketch and photographed faces expressing basic emotions has never been tested. In this thesis, we examined (i) eye fixations as a function of stimulus type, and (ii) neuroelectric response to experimental manipulations such face inversion and deconfiguration. The use of these methods aimed to reveal which face processing drives emotion recognition and to establish neurocognitive markers of emotional sketch and photographed faces processing. Overall, the behavioral results showed that sketch faces convey sufficient expressive information (content of diagnostic features) as in photographed faces for emotion recognition. There was a clear emotion recognition advantage for happy expressions as compared to other emotions. In contrast, recognizing sad and angry faces was more difficult. Concomitantly, results of eye-tracking showed that participants employed more part-based processing on sketch and photographed faces during second fixation. The extracting information from the eyes is needed when the expression conveys more complex emotional information and when stimuli are impoverished (e.g., sketch). Using electroencephalographic (EEG), the P100 and N170 components are used to study the effect of stimulus type (sketch, photographed), orientation (inverted, upright), and deconfiguration, and possible interactions. Results also suggest that sketch faces evoked more part-based processing. The cues conveyed by diagnostic features might have been subjected to early processing, likely driven by low-level information during P100 time window, followed by a later decoding of facial structure and its emotional content in the N170 time window. In sum, this thesis helped elucidate elements of the debate about configural and part-based face processing for emotion recognition, and extend our current understanding of the role of diagnostic features and configural information during neurocognitive processing of facial expressions of emotion.

Spoken language is produced with a great deal of variability with which listeners must be able to cope. One source of variation is coarticulation, which is due to articulatory planning and transitions between segments. Recently, the temporal features of coarticulation were investigated during a picture/spoken-word matching task by using spliced stimuli carrying either congruent or incongruent subphonemic cues at the CV juncture (Archibald & Joanisse, 2011). ERPs were recorded with attention paid to the phonological mapping negativity (PMN) (Connolly & Phillips, 1994; Newman & Connolly, 2004) – a prelexical response sensitive to violations of phonological expectations. Results found that the PMN varied in response to coarticulation violations and concluded that phonetic features in spoken words influence prelexical processing during word recognition. Using a written-/spoken-word paradigm, Arbour, 2012 controlled phonological shape by using onsets that were either fricatives or stops, hypothesizing that coarticulatory information would be differentially processed due to their temporal differences. Findings supported the PMN’s sensitivity to coarticulation but also showed that temporal and physical differences between onsets modulated the effect. These results raise the question of whether acoustic distance between vowels will modulate prelexical processing of speech as reflected by the PMN amplitude: the focus of the current study. Words were organized into minimal sets such that all onset/coda combinations appeared with each vowel provided that English words resulted. Vowels were one of /i, u, æ, ɑ/, maximizing acoustic distance (height and backness). Data from 20 subjects indicate that the PMN is sensitive to the degree of difference between the original and post-splice vowels. When the number of distinctive features changing is greater, the result is an earlier, more robust PMN. This suggests that the rate of speech recognition is not static but dynamic, and is dependent on likeness of subphonemic features. / Thesis / Master of Science (MSc)

The default mode network (DMN) is a highly cited neural network whose functional roles are not well understood. Until recently, event related fMRI experiments used to study the DMN could only be conducted in an open-loop format. The purpose of this study was to demonstrate the potential statistical advantages of real-time fMRI studies to conduct closed-loop experiments to directly test putative DMN functions. Using both fMRI simulations and large archival datasets, we demonstrate that open-loop designs are less statistically powerful than closed-loop experiments that can trigger stimuli at controlled levels of brain activity. When simulating event scheduling on resting state data, DMN levels were normally distributed, but the event timing proved to be ineffective in capturing the highest and lowest DMN values on average across subjects. Statistical differences in DMN levels collected by the Human Connectome Project-Aging (HCP-A) during a Go/NoGo task were also reported, along with the network's distributional effects across subjects. When examining DMN levels in 136 subjects more prone to commission errors the mean DMN levels were reported to be higher during and prior to incorrect NoGo responses. Exploring DMN levels in these same individuals reacting to a Go task also revealed differing measurement patterns when compared to all 711 subjects in the study. Additionally, the distribution of total DMN levels across all participants, as well as during a Go or NoGo trial, showed a shift in the mean towards deactivation. Furthermore, the peak at this location was greater and revealed that increased sampling occurred at the mean and under sampling at the tails. Overall, the cumulative findings in this study were successful in providing statistical arguments to support propositions for more powerful closed-loop experimentation in fMRI. / Master of Science / Activity in a neural network is observed through the use of functional MRI (fMRI) by tracking higher levels of oxygenated blood to that region when active and lower quantities when inactive. Neural networks vary in their responsibilities, thus fMRI tasks are designed to trigger a response based on the functional role of the network. This can be exemplified by studying the blood flow to default mode network (DMN), a network responsible for mind wandering, during a task that requires focus. Researchers can then correlate moments of high activity, which indicates a greater degree of mind wandering, or low activity to a correct or incorrect response to the task. Unfortunately, the timing in which a task is presented to the participant is predetermined prior to the subject entering the MRI making it difficult to capture a correct or incorrect response at the precise moment of activation or deactivation. This concept is known as open-loop and often collects data at moments of neutral activity, neither high nor low. In contrast, a closed-loop design allows a researcher to monitor the DMN's activation levels in real time and present the task at a desired time. This provides more useful data to the experimenter as all recorded responses to the task correlate with exact moments of high and low activation. This makes claims about the neural network's role statistically more powerful as there is a greater quantity of data at these moments rather than during a neutral activation state. The purpose of this thesis is to provide statistical arguments that support propositions for more powerful closed-loop experimentation in fMRI.

Individuals who suffer from severe motor disabilities face the possibility of the loss of speech. A Brain-Computer Interface (BCI) can provide a means for communication through non-muscular control. Current BCI systems use characters that flash from gray to white (GW), making adjacent character difficult to distinguish from the target. The current study implements two types of color stimulus (grey to color [GC] and color intensification [CI]) and I hypotheses that color stimuli will; (1) reduce distraction of nontargets (2) enhance target response (3) reduce eye strain. Online results (n=21) show that GC has increased information transfer rate over CI. Mean amplitude revealed that GC had earlier positive latency than GW and greater negative amplitude than CI, suggesting a faster perceptual process for GC. Offline performance of individual optimal channels revealed significant improvement over online standardized channels. Results suggest the importance of a color stimulus for enhanced response and ease of use.

The critical bandwidth (CBW) is an auditory phenomenon that has been used to study various aspects of auditory processing, including auditory masking, complex tone processing, and loudness perception. Although the psychoacoustic aspects of the CBW have been well studied, the underlying neurophysiology of the CBW has not been as thoroughly examined. The current study examined the neurophysiology of the CBW in young adults, as well as loudness perception in response to the CBW. Auditory stimuli consisting of complex tones of varying bandwidths were presented to 12 individuals (6 male and 6 female, ages 18-26 years). Complex tones were presented around center frequencies (CFs) of 250, 500, 1000, and 3000 Hz at bandwidths of 2, 5, 8, 10, 20, 50, 100, 200, 500, 1000, and 2000 Hz. Participants made loudness perception judgments while electroencephalography measured and recorded components of the event related potentials (ERPs) in response to the acoustic stimuli. Reaction time (RT) was recorded for each behavioral response, and the latencies of the N1, P2, C3, and C4 components of the ERPs were obtained. The results showed that RT increased with increasing bandwidth followed by a decrease in RT corresponding approximately with the CBW. This indicated that participants perceived a change in loudness at bandwidths greater than the CBW. Significant differences, p < .05, in RT were observed in bandwidths of 5 Hz and greater, although there was not complete consistency in this observation across all CFs and bandwidths. No significant critical band-like behavior amongst ERP latencies was observed. The results indicated that responses to acoustic stimuli originating in the superior temporal gyrus progressed to areas of higher neural function in the mid-temporal lobe. It was observed that each response must be processed temporally and independently to determine if a frequency difference is present for each stimulus. This observation is significant because this type of processing had not been identified prior to the current study.

Because event-related potentials (ERPs) can reflect individual differences in intellectual ability, individual differences in college grade-point average (GPA) may be associated with specific individual ERP waves, such as the P300. However, P300 amplitude is higher in women than in men and varies across the menstrual cycle, factors that could confound the association between GPA and ERPs. In this regard, our objective was to determine whether differences in GPA are reflected in ERPs while standardling for sex and menstrual phase. After participants provided informed consent, we obtained GPAs from 22 right-handed college students (11 male, age range 22 to 26 and 10 female, age range 17 to 24) at a university with high admission and retention standards. We assessed menstrual phase by measuring luteinizing hormone levels across the cycle. We then obtained ERPs for each male participant and ERPs during each phase of the menstrual cycle for each female participant in an object-recognition visual pop-out protocol using Net Station Software (Electrical Geodesics, Inc., Eugene, Oregon) and E-prime Software (Psychology Software Tools, Inc., Sharpsburg, Pennsylvania). Males had larger P300s than females. The male and female high GPA was significantly different from the low GPA male and female groups. High GPA in females and males were associated with a positive peak at approximately 689 ms that was not present in the low-GPA male group and was significantly diminished in low-GPA females. Electro-cortical processing of cognitive stimuli differs between college students with high and low GPAs.

The mismatch negativity (MMN) response, a passively-elicited component of the auditory event-related potential (ERP), reflects preattentive identification of infrequent changes in acoustic stimuli. In the current study, the MMN response was examined closely to determine what extent natural speech sounds evoke the MMN. It was hypothesized that a significant MMN response results during the presentation of deviant stimuli from which spectral energy within formant bands critical to vowel identification has been removed. Localizations of dipoles within the cortex were hypothesized to yield information pertaining to the processing of formant-specific linguistic information. A same/different discrimination task was administered to 20 adult participants (10 female and 10 male) between the ages of 18 and 26 years. Data from behavioral responses and ERPs were recorded. Results demonstrated that the MMN may be evoked by natural speech sounds. Grand-averaged brain maps of ERPs created for all stimulus pairs showed a large preattentive negativity. Additionally, amplitudes of the MMN were greatest for pairs of auditory stimuli in which spectral energy not corresponding to formant frequencies was digitally eliminated. Dipoles reconstructed from temporal ERP data were located in cortical areas known to support language and auditory processing. Significant differences between stimulus type and reaction time were also noted. The current investigation confirms that the MMN response is evoked by natural speech sounds and provides evidence for a theory of preattentive formant-based processing of speech sounds.

Moderate-to-severe traumatic brain injury (M/S TBI) can affect an individual's ability to perform daily tasks. For example, individuals with M/S TBI are more likely to commit errors on tasks such as making a meal or wrapping a present. The neural processes involved in such errors are poorly understood. Studies suggest that neurophysiologic markers of cognitive control and error processing may be helpful in gaining additional insight into errors on naturalistic action tasks. Unfortunately, previous experimental methods left a methodological gap which limited the use of neurophysiological markers in the study of naturalistic action. Several recent studies in healthy adults have suggested one method of bridging the gap by having individuals observe another person's errors. The current study was the first study to employ the method in a TBI population as a possible means of gaining additional insight into the detrimental effects of M/S TBI on the performance of naturalistic actions. In order to gain additional insight into the effects of M/S TBI on the completion of naturalistic tasks I used two neurophysiologic markers of cognitive control and error processing. They were the observer error related negativity (oERN) and the P300 components of the scalp-recorded event-related potential (ERP). I hypothesized that individuals with M/S TBI would demonstrate error-specific changes in the two oERN and P300 that would correlate with self-reported difficulties in daily functioning. The study consisted of two experiments. One compared 15 individuals with M/S TBI to 17 demographically similar healthy controls on an error related naturalistic action based picture task. The second compared an overlapping sample of 16 individuals with M/S TBI to 16 demographically similar controls as they watched a confederate complete the Erikson flanker task, a commonly used task in the study of electrophysiological markers. Accuracy (error vs. correct) and group (M/S TBI vs. control) effects were analyzed using 2 x 2 repeated measures ANOVAs on ERP amplitude and latency. Pearson product-moment correlations were calculated to evaluate the relationship between the P300 and oERN and measures of self-reported executive functioning (Frontal Systems Behavior Scale, FrSBe) and neuropsychological measures. Findings supported a difference between the control and M/S TBI groups in how errors were processed during the naturalistic actions based picture task. There was an interaction between group membership and response accuracy (error vs. correct) on P300 amplitude and P300 latency. Controls demonstrated reduced P300 amplitude and latency on error trials compared to correct trials. Individuals with M/S TBI did not demonstrate a significant difference between correct trials and error trials on P300 amplitude and latency. The amplitude and latency of the P300 were correlated with self-reported functional difficulties in individuals with M/S TBI but not control participants. A Fisher's r -- z analysis indicated that correlations differed significantly between groups; however, an outlier was identified in the correlational data. Removal of the outlier data led to non-significant results in the Fisher's r -- z analysis. Taken together, results of the picture task supplied evidence that for individuals with M/S TBI differences in neurophysiologic markers between groups could be explained by reduced adaptation to complexity or by possible deficits in a secondary error processing pathway for complex errors. Future research could focus on better defining the functional relationship between P300 amplitude and latency and increased errors in naturalistic actions following M/S TBI. Observation of the flanker task did not elicit oERN waveforms from either healthy controls or from individuals with M/S TBI. The results could be due to problems with the current task, but also raised some concerns about previous studies using the flanker task which employed a slightly different methodology requiring participants to count errors. The current study did not require participant to count errors. As a whole, the study supplied partial support for using electrophysiological markers of error processing to gain additional understanding increased errors in the performance of naturalistic actions following M/S TBI.