Brain neural activity generates electrical discharges, which manifest as electrical and magnetic potentials around the scalp. Those potentials can be registered with magnetoencephalography (MEG) and electroencephalography (EEG) devices. Data acquired by M/EEG is extremely difficult to work with due to the inherent complexity of underlying brain processes and low signal-to-noise ratio (SNR). Machine learning techniques have to be employed in order to reveal the underlying structure of the signal and to understand the brain state. This thesis explores a diverse range of machine learning techniques which model the structure of M/EEG data in order to decode the mental state. It focuses on measuring a subject's variability and on modeling intrasubject variability. We propose to measure subject variability with a spectral clustering setup. Further, we extend this approach to a unified classification framework based on Laplacian regularized support vector machine (SVM). We solve the issue of intrasubject variability by employing a model with latent variables (based on a latent SVM). Latent variables describe transformations that map samples into a comparable state. We focus mainly on intrasubject experiments to model temporal misalignment.

Attention Deficit Hyperactivity Disorder (ADHD) is a disorder affecting cognitive control. These functions are important to achieve goals when different actions need to be executed in close succession. This type of multi-component behavior, which often further requires the processing of information from different modalities, is important for everyday activities. Yet, possible changes in neurophysiological mechanisms have not been investigated in adolescent ADHD. We examined N = 31 adolescent ADHD patients and N = 35 healthy controls (HC) in two Stop-Change experiments using either uni-modal or bi-modal stimuli to trigger stop and change processes. These stimuli were either presented together (SCD0) or in close succession of 300 milliseconds (SCD300). Using event-related potentials (ERP), EEG data decomposition and source localization we analyzed neural processes and functional neuroanatomical correlates of multicomponent behavior. Compared to HCs, ADHD patients had longer reaction times and higher error rates when Stop and Change stimuli were presented in close succession (SCD300), but not when presented together (SCD0). This effect was evident in the uni-modal and bi-modal experiment and is reflected by neurophysiological processes reflecting response selection mechanisms in the inferior parietal cortex (BA40). These processes were only detectable after accounting for intra-individual variability in neurophysiological data; i.e. there were no effects in standard ERPs. Multi-component behavior is not always deficient in ADHD. Rather, modulations in multi-component behavior depend on a critical temporal integration window during response selection which is associated with functioning of the inferior parietal cortex. This window is smaller than in HCs and independent of the complexity of sensory input.

Attention Deficit Hyperactivity Disorder (ADHD) is a disorder affecting cognitive control. These functions are important to achieve goals when different actions need to be executed in close succession. This type of multi-component behavior, which often further requires the processing of information from different modalities, is important for everyday activities. Yet, possible changes in neurophysiological mechanisms have not been investigated in adolescent ADHD. We examined N = 31 adolescent ADHD patients and N = 35 healthy controls (HC) in two Stop-Change experiments using either uni-modal or bi-modal stimuli to trigger stop and change processes. These stimuli were either presented together (SCD0) or in close succession of 300 milliseconds (SCD300). Using event-related potentials (ERP), EEG data decomposition and source localization we analyzed neural processes and functional neuroanatomical correlates of multicomponent behavior. Compared to HCs, ADHD patients had longer reaction times and higher error rates when Stop and Change stimuli were presented in close succession (SCD300), but not when presented together (SCD0). This effect was evident in the uni-modal and bi-modal experiment and is reflected by neurophysiological processes reflecting response selection mechanisms in the inferior parietal cortex (BA40). These processes were only detectable after accounting for intra-individual variability in neurophysiological data; i.e. there were no effects in standard ERPs. Multi-component behavior is not always deficient in ADHD. Rather, modulations in multi-component behavior depend on a critical temporal integration window during response selection which is associated with functioning of the inferior parietal cortex. This window is smaller than in HCs and independent of the complexity of sensory input.

Le but principal de cette thèse était de mieux comprendre la relation entre le déploiement de l’attention visuo-spatiale et la réponse lors de la recherche visuelle. Nous avons combiné des mesures de chronométrie mentale (les temps de réponse; TR) et d’électrophysiologie, permettant le suivi du décours temporel des évènements et la séparation des processus en sous-phases. Trois études ont été effectuées. Dans les deux premières études, la N2pc, un potentiel relié à un événement (PRE), a été utilisé comme indice électrophysiologique de l’attention visuelle vers un stimulus latéral lors d’une tâche de recherche visuelle ayant une cible facilement repérable (qui saute aux yeux) parmi des distracteurs. Cette composante est caractérisée, en électrophysiologie, par une plus grande négativité environ 200 ms suivant la présentation de la stimulation aux sites postérieurs et controlatéraux du champ visuel ciblé. Dans la première étude, la relation entre la composante N2pc et la réponse a été examinée. Pour ce faire, les données électrophysiologiques ont été scindées selon le TR médian. La N2pc était plus ample pour les essais et les participants plus rapides comparés à ceux qui étaient plus lents, suggérant qu’un déploiement attentionnel plus efficace (N2pc plus ample) avait accéléré les processus subséquents (TR plus court). Dans l’étude 2, le traitement depuis le moment du déploiement attentionnel jusqu’à la réponse a été élucidé en examinant le déclenchement de la N2pc par rapport à la réponse et non au stimulus (ce qui est plus utilisé dans les paradigmes classiques). Nous avons été les premiers à utiliser la RLpcN (response-locked posterior contralateral negativity), composée de la composante N2pc et la SPCN (une composante suivant la N2pc reflétant le traitement en mémoire visuelle à court terme). Les résultats ont démontré une augmentation du temps entre le début de la RLpcN et la réponse pour les TR plus lents, reflétant un traitement plus long suivant le déploiement de l’attention spatiale. Nous avons par la suite utilisé cette composante (la RLpcN) pour étudier la recherche visuelle difficile (Étude 3), où la cible n’était pas facilement identifiable parmi des distracteurs, à l’aide de manipulations expérimentales modulant la RLpcN, soit le nombre d’items et de réponses possibles. Plus il y avait d’items dans le champ visuel ou de possibilités de réponses, plus longue était la durée entre le début de la RLpcN et la réponse, démontrant ainsi qu’il est possible d’observer l’activité associée avec les processus sous-jacents à ces manipulations lors d’une recherche difficile. En somme, nous avons montré (1) qu’un déploiement attentionnel plus efficace accélère les processus subséquents, (2) que le traitement déterminant les TR se produit, en majorité, suivant le déploiement de l’attention et, enfin, (3) qu’il est possible d’identifier des marqueurs électrophysiologiques de la sélection de la cible ainsi que de la réponse lors d’une recherche difficile. Dans l’ensemble, les résultats des études constituant la présente thèse vont au-delà des études électrophysiologiques de recherche visuelle typiques, qui utilisent généralement des cibles qui sautent aux yeux, et élucident le décours temporel du traitement lors de recherches plus complexes. / The overarching thesis was to understand better the relationship between the deployment of visual spatial attention and the eventual response during visual search. We combined mental chronometry with electrophysiological measures, allowing us to track the temporal sequence of events and bisect processing into sub-phases. The two first studies used the N2pc, an event-related potential (ERP) component, as an electrophysiological marker of visual attention to laterally presented stimuli using a pop-out (i.e., the target stands out) visual search task. The N2pc is characterised by an enhanced negativity emerging around 200 ms after the display onset at posterior sites contralateral to the attended visual field. In Study 1, we first evaluated the relationship between the N2pc component and the moment of response. Electrophysiological data were split according to the response time (RT) median. The N2pc was larger for both fast compared with slow trials and participants, suggesting that a more efficient deployment of attention (larger N2pc) sped up downstream processing (shorter RTs). In Study 2, processing from the deployment of attention to the response was elucidated by examining N2pc onset relative to the response instead of to the display onset. We pioneered the use of the RLpcN (response-locked contralateral negativity), composed of the N2pc and the SPCN (a component following the N2pc and reflecting processing of task-relevant stimuli in visual short-term memory). Importantly, more time passed between RLpcN onset and the response for longer RTs, reflecting more processing time following the onset of visual spatial attention. We then used this component to study difficult search (Study 3), where the target was not easily located, using experimental manipulations designed to modulate the RLpcN. We showed that when there were more items present in the visual field or response selection was more difficult, there was a longer duration between RLpcN onset and the response, demonstrating that it is possible to observe activity associated with specific processes during difficult visual search. In summary, we provide evidence that (1) a more efficient deployment of attention speeds up downstream processing, (2) processing determining RTs occurs, in majority, following the deployment of attention, and (3) it is possible to identify electrophysiological markers of target and response selection during difficult search. Together, the results of these experiments go beyond typical electrophysiological experiments of visual search, which use pop-out targets, and elucidate the time course of processing during more complex search.

本研究包含兩個實驗，分別在再認作業的學習及測驗階段呈現選自國際情緒圖片系統（IAPS）的情緒性及非情緒性圖片作為情境操弄，進而了解情緒性情境對記憶歷程中，入碼及提取階段的影響。行為資料、腦電波相關電位及情緒的生理反應指標皆收集用作分析。實驗一中，中性及負向圖片在記憶目標字出現之前呈現，藉以操弄受試者記憶入碼的情緒狀態。行為結果並未顯示情緒效果，表示入碼的情緒情境不能影響記憶的正確率。事件相關腦電位的資料亦沒有在記憶相關的效果顯現情緒圖片的影響。實驗二中，中性及負向的圖片在提取階段呈現於目標字之前，藉以影響受是者記憶提取時的情緒狀態。相似於實驗一的結果，受試者的行為表現並未受情緒圖片的影響。但在事件相關電位方面，300至500毫秒間，靠近中間前腦區的新舊效果受到情緒的影響。觀看負向圖片後，受試者記憶提取時的新舊效果消失。此結果指出記憶提取時的情緒情境可能造成提取的認知歷程的改變。 / In two experiments, emotional or non-emotional pictures chosen from the IAPS (International Affective Pictures System) were presented before target words in the study phase or test phase of a recognition task to examine the effect of emotional context on memory encoding and retrieval. Behavioral data, ERP (Event-Related Potential) and physical indicators were adopted. In experiment one; neutral or negative pictures were presented before target in study phase in order to modify the emotional state of encoding. Behavioral data showed no emotional effect. Emotion of encoding context did not affect memory in accuracy. ERP data did not show a significant emotional effect on memory related components. In experiment two, pictures were presented before retrieval to modulate the emotional states during memory retrieving. Similar to experiment one, behavioral data was not affected in all indicators. ERP data in experiment two showed an emotional effect on mid-frontal old/new during 300 to 500ms that the old/new difference was eliminated when retrieved under a negative context. The result provided a possible emotional effect on response bias during retrieval.

睡眠是否能鞏固陳述性記憶目前尚無定論。過去研究一致較支持睡眠能增進相關字詞配對的學習，但睡眠是否能增進無關字詞配對的學習，目前仍存在不一致的發現。造成該差異的原因可能是：過去研究多採用的行為測量指標，或許無法充分反映出睡眠促進記憶新聯結（new association）產生的效果。事件關聯電位（Event-related potential, ERP）的N400反映出語意記憶系統內每個字詞彼此的相關程度，因此本研究使用N400來探討睡眠強化無關字詞配對形成新聯結的電生理歷程。30名健康受試者（15位男性與15位女性，平均年齡為20.7歲） 隨機分派至睡眠組或清醒組，第一晚在學習80組無關字詞配對後，接受第一次再認記憶測驗，同時進行ERP的記錄。隨後睡眠組接受睡眠記錄（PSG），清醒組則接受整晚的睡眠剝奪，兩組受試者皆在第二晚給予8小時的躺床時間，使他們有機會充足睡眠以恢復精神，於第三天早上接受第二次再認記憶測驗及ERP記錄。在記憶測驗時，無關字詞配對分別組成促發字（prime）與目標字（target）先後出現，受試者需判斷先後出現的字詞是否為先前學過的完整配對，在測試階段同時記錄腦電波訊號。行為測量結果顯示睡眠過後，再認表現的正確率顯著提高且反應時間明顯縮短，但在睡眠剝奪後則顯示相反的結果。電生理測量發現睡眠組的N400振幅在睡眠過後較清醒組明顯降低。另外，睡眠組較清醒組有較高的正確率與較短的反應時間。睡眠組再認測驗的進步量與慢波睡眠呈現負相關，而慢波睡眠與第一次再認測驗的正確率呈現正相關，根據二階段睡眠記憶鞏固理論，慢波睡眠涉及重新組織記憶的歷程（系統性固化），因此學習表現較佳的受試者出現較多的深度睡眠，可能反應其經歷系統性固化。本研究結果顯示睡眠對於產生新聯結有明顯的增強效果，而且慢波睡眠可能參與了記憶表徵重新分配的歷程。 / The effect of sleep on declarative memory remains contradictory. Prior studies show that sleep benefits the learning of related word pairs consistently, while the learning of unrelated word pairs, however, show mixed results. It is possible that the behavioral measures used in previous studies are not sensitive enough to reveal subtle effects of sleep on new associations. N400, an event-related potential (ERP) component reflecting relatedness among words in semantic memory, was used in the present study to investigate the effect of sleep on the physiological process underlying new associations of unrelated word pairs. Participants were randomly assigned to either a Sleep group or a Wakefulness group. In the learning phase, participants were asked to memorize 80 visually presented unrelated word-pairs, followed by a pre-test phase with a recognition task. The participants then underwent either a night of nocturnal sleep (Sleep group) or sleep deprivation (Wakefulness group). A post-test was conducted after subjects had one night of recovery sleep. During both pre-test and post-test sessions, prime and target words were presented successively for the subjects to judge whether they were among the original pairs or new pairs. ERPs were recorded during both test phases. The behavioral data show that differences in improvement of recognition and decreases in reaction time from pre-test to post-test are significant between Sleep and Wakefulness groups. N400 peak amplitude attenuated significantly after sleep but not after wakefulness. The improvement of recognition negatively correlates with slow wave sleep (SWS). The number of word-pairs acquired in the learning phase, however, correlates positively with SWS. According to the two-stage memory consolidation theory of sleep, SWS involves in redistribution of memory (systematic consolidation). Therefore, that the participants with high performance showed more SWS may reflect the process of systematic consolidation. These results suggest that the sleep has an enhancing effect on the formation of novel association, and SWS may be involved in the process of redistributing memory representations.

The aim of this thesis was to help elucidate the mechanisms that underlie age-related decline in visual selective attention and working memory (WM). Older and younger adults completed a behavioural WM task, after which electroencephalogram (EEG) was recorded as participants perform a localized attentional interference (LAI) task – competition/attentional interference was manipulated by systematically altering the distance between targets and distractors. Older adults showed impaired accuracy and reaction time on the WM and LAI tasks. Two event-related-potentials, indexing spatial attention (N2pc) and target processing (Ptc), displayed attenuated amplitude and increased latency in older adults. Thus, spatial selection, target enhancement and processing speed deficits may contribute to age-related attentional impairments. Furthermore, an unexpected component was found between the N2pc and Ptc in the older adult waveforms. Preliminary analyses suggest this may be the PD, implicated in distractor suppression, which may be differentially contributing to older and younger adults’ electrophysiology and attentional processing.

The aim of this thesis was to help elucidate the mechanisms that underlie age-related decline in visual selective attention and working memory (WM). Older and younger adults completed a behavioural WM task, after which electroencephalogram (EEG) was recorded as participants perform a localized attentional interference (LAI) task – competition/attentional interference was manipulated by systematically altering the distance between targets and distractors. Older adults showed impaired accuracy and reaction time on the WM and LAI tasks. Two event-related-potentials, indexing spatial attention (N2pc) and target processing (Ptc), displayed attenuated amplitude and increased latency in older adults. Thus, spatial selection, target enhancement and processing speed deficits may contribute to age-related attentional impairments. Furthermore, an unexpected component was found between the N2pc and Ptc in the older adult waveforms. Preliminary analyses suggest this may be the PD, implicated in distractor suppression, which may be differentially contributing to older and younger adults’ electrophysiology and attentional processing.