Through linguistic abilities, our brain can comprehend and produce an infinite number of new sentences constructed from a finite set of words. Although recent research has uncovered the neural representation of semantics during comprehension of isolated words or adjective-noun phrases, the neural representation of the words during utterance planning is less understood. We apply existing machine learning methods to Magnetoencephalography (MEG) data recorded during a picture naming experiment, and predict the semantic properties of uttered words before they are said. We explore the representation of concepts over time, under controlled tasks, with varying compositional requirements. Our results imply that there is enough information in brain activity recorded by MEG to decode the semantic properties of the words during utterance planning. Also, we observe a gradual improvement in the semantic decoding of the first uttered word, as the participant is about to say it. Finally, we show that, compared to non-compositional tasks, planning to compose an adjective-noun phrase is associated with an enhanced and sustained representation of the noun. Our results on the neural mechanisms of basic compositional structures are a small step towards the theory of language in the brain. / Graduate

Brain decoding involves the determination of a subject’s cognitive state or an associated stimulus from functional neuroimaging data measuring brain activity. In this setting the cognitive state is typically characterized by an element of a finite set, and the neuroimaging data comprise voluminous amounts of spatiotemporal data measuring some aspect of the neural signal. The associated statistical problem is one of classification from high-dimensional data. We explore the use of functional principal component analysis, mutual information networks, and persistent homology for examining the data through exploratory analysis and for constructing features characterizing the neural signal for brain decoding. We review each approach from this perspective, and we incorporate the features into a classifier based on symmetric multinomial logistic regression with elastic net regularization. The approaches are illustrated in an application where the task is to infer from brain activity measured with magnetoencephalography (MEG) the type of video stimulus shown to a subject. / Graduate

Les études de décodage cérébral visent à entrainer un modèle d'activité cérébrale qui reflète l'état cognitif du participant. Des variations interindividuelles substantielles dans l'organisation fonctionnelle du cerveau représentent un défi pour un décodage cérébral précis. Dans cette thèse, nous évaluons si des modèles de décodage cérébral précis peuvent être entrainés avec succès entièrement au niveau individuel. Nous avons utilisé un ensemble de données individuel dense d'imagerie par résonance magnétique fonctionnelle (IRMf) pour lequel six participants ont terminé l'ensemble de la batterie de tâches du “Human Connectome Project” > 13 fois sur dix sessions d'IRMf distinctes. Nous avons implémenté plusieurs méthodes de décodage, des simples machines à vecteurs de support aux réseaux complexes de neurones à convolution de graphes. Tous les décodeurs spécifiques à l'individu ont été entrainés pour classifier simultanément les volumes d'IRMf simples (TR = 1,49) entre 21 conditions expérimentales, en utilisant environ sept heures de données d'IRMf par participant. Les meilleurs résultats de prédiction ont été obtenus avec notre modèle de machine à vecteurs de support avec une précision de test allant de 64 à 79 % (niveau de la chance environ 7%). Les perceptrons multiniveaux et les réseaux convolutionnels de graphes ont également obtenu de très bons résultats (63-78% et 63-77%, respectivement). Les cartes d'importance des caractéristiques dérivées du meilleur modèle (SVM) ont révélé que la classification utilise des régions pertinentes pour des domaines cognitifs particuliers, sur la base d’a priori neuro-anatomique. En appliquant un modèle individuel aux données d’un autre sujet (classification inter-sujets), on observe une précision nettement inférieure à celle des modèles spécifiques au sujet, ce qui indique que les décodeurs cérébraux individuels ont appris des caractéristiques spécifiques à chaque individu. Nos résultats indiquent que des ensembles de données de neuroimagerie profonde peuvent être utilisés pour former des modèles de décodage cérébral précis au niveau individuel. Les données de cette étude sont partagées librement avec la communauté (https://cneuromod.ca), et pourront servir de benchmark de référence, pour l’entrainement de modèles de décodage cérébral individuel, ou bien des études de “transfert learning” à partir de l’échantillon collecté par le human connectome project. / Brain decoding studies aim to train a pattern of brain activity that reflects the cognitive state of the participant. Substantial inter-individual variations in functional organization represent a challenge to accurate brain decoding. In this thesis, we assess whether accurate brain decoding models can be successfully trained entirely at the individual level. We used a dense individual functional magnetic resonance imaging (fMRI) dataset for which six participants completed the entire Human Connectome Project (HCP) task battery>13 times across ten separate fMRI sessions. We assessed several decoding methods, from simple support vector machines to complex graph convolution neural networks. All individual-specific decoders were trained to classify single fMRI volumes (TR = 1.49) between 21 experimental conditions simultaneously, using around seven hours of fMRI data per participant. The best prediction accuracy results were achieved with our support vector machine model with test accuracy ranging from 64 to 79% (chance level of about 7%). Multilevel perceptrons and graph convolutional networks also performed very well (63-78% and 63-77%, respectively). Best Model Derived Feature Importance Maps (SVM) revealed that the classification uses regions relevant to particular cognitive domains, based on neuroanatomical priors. Applying an individual model to another subject's data (across-subject classification) yields significantly lower accuracy than subject-specific models, indicating that individual brain decoders have learned characteristics specific to each individual. Our results indicate that deep neuroimaging datasets can be used to train accurate brain decoding models at the individual level. The data from this study is shared freely with the community (https://cneuromod.ca) and can be used as a reference benchmark, for training individual brain decoding models, or for “transfer learning” studies from the sample collected by the human connectome project.

Brain neural activity generates electrical discharges, which manifest as electrical and magnetic potentials around the scalp. Those potentials can be registered with magnetoencephalography (MEG) and electroencephalography (EEG) devices. Data acquired by M/EEG is extremely difficult to work with due to the inherent complexity of underlying brain processes and low signal-to-noise ratio (SNR). Machine learning techniques have to be employed in order to reveal the underlying structure of the signal and to understand the brain state. This thesis explores a diverse range of machine learning techniques which model the structure of M/EEG data in order to decode the mental state. It focuses on measuring a subject's variability and on modeling intrasubject variability. We propose to measure subject variability with a spectral clustering setup. Further, we extend this approach to a unified classification framework based on Laplacian regularized support vector machine (SVM). We solve the issue of intrasubject variability by employing a model with latent variables (based on a latent SVM). Latent variables describe transformations that map samples into a comparable state. We focus mainly on intrasubject experiments to model temporal misalignment.