Cognitive processes associated with spatial orientation can use different reference frames: egocentric, centered on observer and allocentric, centered on objects in the environment. In this thesis, we use EEG to investigate the dynamics of brain processes accompanying spatial orientation based on these reference frames. Participants were instructed to estimate distances between objects or themselves and objects located in a virtual circular arena; this task was presented in both 2D and 3D displays. Task-related EEG changes were analyzed using a time-frequency analysis and event-related potential analysis of 128-channel EEG recordings. Through time-frequency analysis we found significant power differences in delta, theta, alpha, beta and gamma bands amongst the control, egocentric and allocentric testing conditions. We noted a decrease in alpha power in occipital and parietal regions, while a significantly stronger decrease was observed for the allocentric condition compared to both egocentric and control conditions. A similar pattern was also detectable for the beta band. We also report an increase in theta and delta power in temporal, fronto-temporal and lateral frontal regions that was significantly stronger for the egocentric condition compared to control and, in some electrodes, even...

Brain-machine interfaces (BMIs) are the only currently viable means of communication for many individuals suffering from locked-in syndrome (LIS) – profound paralysis that results in severely limited or total loss of voluntary motor control. By inferring user intent from task-modulated neurological signals and then translating those intentions into actions, BMIs can enable LIS patients increased autonomy. Significant effort has been devoted to developing BMIs over the last three decades, but only recently have the combined advances in hardware, software, and methodology provided a setting to realize the translation of this research from the lab into practical, real-world applications. Non-invasive methods, such as those based on the electroencephalogram (EEG), offer the only feasible solution for practical use at the moment, but suffer from limited communication rates and susceptibility to environmental noise. Maximization of the efficacy of each decoded intention, therefore, is critical. This thesis addresses the challenge of implementing a BMI intended for practical use with a focus on an autonomous assistive robot application. First an adaptive EEG- based BMI strategy is developed that relies upon code-modulated visual evoked potentials (c-VEPs) to infer user intent. As voluntary gaze control is typically not available to LIS patients, c-VEP decoding methods under both gaze-dependent and gaze- independent scenarios are explored. Adaptive decoding strategies in both offline and online task conditions are evaluated, and a novel approach to assess ongoing online BMI performance is introduced. Next, an adaptive neural network-based system for assistive robot control is presented that employs exploratory learning to achieve the coordinated motor planning needed to navigate toward, reach for, and grasp distant objects. Exploratory learning, or “learning by doing,” is an unsupervised method in which the robot is able to build an internal model for motor planning and coordination based on real-time sensory inputs received during exploration. Finally, a software platform intended for practical BMI application use is developed and evaluated. Using online c-VEP methods, users control a simple 2D cursor control game, a basic augmentative and alternative communication tool, and an assistive robot, both manually and via high-level goal-oriented commands.

No description available.

No description available.

<p>Previous human behavioral research has provided support for the existence of different frames of reference utilized during spatial processing that can be dependent or independent of the observer. These are known respectively as egocentric and allocentric frames of reference. However, it has been difficult to dissociate these two different processes under realistic conditions. Importantly, how these frames of reference are influenced by the visual and non-visual information is not well understood. Therefore, the studies of this thesis evaluated spatial processing utilizing realistic and ecologically valid stimuli in environments of different scales, while systematically manipulating the visual and non-visual information available during learning. We demonstrated that non-visual information generated by actively walking through an environment leads to more egocentric processing, whereas the same visual motion information presented passively via a video leads to more allocentric processing (Chapter 2). Further, characteristics of the visual scene can also influence how it is processed, dependent on the strength of the verbal identity of the features in the environment (Chapter 3). Specifically, in a small room environment subject’s representations of corners-to-corners (corners do not have an obvious verbal component) were not as strongly encoded relative to each other in comparison to objects-to-objects (objects with an obvious verbal identity ). Finally, we demonstrated differential influences of non-visual information dependent on whether the features in the visual scene were more allocentrically processed or egocentrically processed (Chapter Four). Specifically, when different features of layouts are made distinguishable by their identity, this lead to more allocentric processing whereas when different features are made distinguishable by their relative position, this lead to more egocentric processing. Further, non-visual information made available during spatial updating when the observer is changing viewpoints benefitted tasks focused on differentiating changes to objects’ identity and less so for differentiating changes in relative object position.</p> / Doctor of Philosophy (PhD)

<p dir="ltr">This study investigates healthcare decision-making dynamics among adults diagnosed with autism spectrum disorder (ASD) without intellectual disability (ID), focusing on the influence of the egocentric norm concept and comparing it to neurotypical (NT) individuals. Employing a quasi-experimental design and meticulously developed vignettes, the research gathers quantitative and qualitative data from Prolific survey responses, exploring nuances in decision processes. The findings reveal substantial disparities in response patterns between individuals with ASD without ID and NT counterparts, emphasizing the significance of the egocentric norm. Factor analysis and structural equation modeling support the validity of the questions and highlight egocentric norms’ impactful role in health-related decisions.</p><p dir="ltr">Qualitatively, themes within both groups illuminate unique decision-making processes, including enduring support from significant others, autonomy trends, and reliance on digital resources. The study contributes nuanced insights for healthcare practitioners, policymakers, and educators, emphasizing the need for tailored interventions that resonate with the distinctive decision-making dynamics of individuals with ASD without ID. While acknowledging limitations related to sample characteristics and reliance on self-reported data, the study provides an in-depth understanding of the interplay between the egocentric norm, Reasoned Action Approach (RAA) concepts, and other influential factors in healthcare decision-making.</p>

Environments with people are complex, with many activities and events that need to be represented and explained. The goal of scene understanding is to either determine what objects and people are doing in such complex and dynamic environments, or to know the overall happenings, such as the highlights of the scene. The context within which the activities and events unfold provides key insights that cannot be derived by studying the activities and events alone. \emph{In this thesis, we show that this rich contextual information can be successfully leveraged, along with the video data, to support dynamic scene understanding}. We categorize and study four different types of contextual cues: (1) spatio-temporal context, (2) egocentric context, (3) geographic context, and (4) environmental context, and show that they improve dynamic scene understanding tasks across several different application domains. We start by presenting data-driven techniques to enrich spatio-temporal context by augmenting Bag-of-Words models with temporal, local and global causality information and show that this improves activity recognition, anomaly detection and scene assessment from videos. Next, we leverage the egocentric context derived from sensor data captured from first-person point-of-view devices to perform field-of-view localization in order to understand the user's focus of attention. We demonstrate single and multi-user field-of-view localization in both indoor and outdoor environments with applications in augmented reality, event understanding and studying social interactions. Next, we look at how geographic context can be leveraged to make challenging ``in-the-wild" object recognition tasks more tractable using the problem of food recognition in restaurants as a case-study. Finally, we study the environmental context obtained from dynamic scenes such as sporting events, which take place in responsive environments such as stadiums and gymnasiums, and show that it can be successfully used to address the challenging task of automatically generating basketball highlights. We perform comprehensive user-studies on 25 full-length NCAA games and demonstrate the effectiveness of environmental context in producing highlights that are comparable to the highlights produced by ESPN.

Episodic memory is processed by the extended hippocampal system, and pathology or injury to individual components of this system can result in deficits in spatial learning and memory (Aggleton & Brown, 1999). Extensive research regarding spatial memory has been carried out on the anterior thalamic nuclei, a component of the extended hippocampal system, but the contribution of the laterodorsal thalamic nuclei, an adjacent structure with similar neural connections, is less clear. The purpose of the present study was to compare the effects of selective anterior thalamic nuclei lesions (AT) with selective laterodorsal thalamic nuclei lesions (LD) in a novel land-based spatial reference memory task. This assessed the use of proximal and distal visual cues on the propensity to use allocentric or egocentric navigation strategies to locate a specific place in space, as well as the temporal evolution of these navigation strategies. AT lesion impairments were observed in the acquisition trials in both proximal and distal cue conditions. LD lesion rats were unimpaired in the acquisition trials in both visual cue conditions. Across the probe trials, lesion effects were not observed when tested for general navigation, egocentric or allocentric strategies, and there was no clear improvement in performance over the four weeks of probe trials. However, performance was consistently poorer for all groups when proximal cues facilitated navigation compared to distal cues. Performance differences related to cue proximity may reflect the influence of motion parallax, the perceived displacement rate of visual cues. The absence of lesion effects across probes were thought to be due to the preferential use of cued navigation, which was reliant on a single salient beacon, and the lack of integration between cued and place navigation, which was reliant on the formation of a spatial representation.

Depuis les vingt dernières années, l'industrie a développé plusieurs produits commerciaux dotés de capacités auditives et visuelles. La grand majorité de ces produits est composée d'un caméscope et d'un microphone embarqué (téléphones portables, tablettes, etc). D'autres, comme la Kinect, sont équipés de capteurs de profondeur et/ou de petits réseaux de microphones. On trouve également des téléphones portables dotés d'un système de vision stéréo. En même temps, plusieurs systèmes orientés recherche sont apparus (par exemple, le robot humanoïde NAO). Du fait que ces systèmes sont compacts, leurs capteurs sont positionnés près les uns des autres. En conséquence, ils ne peuvent pas capturer la scène complète, mais qu'un point de vue très particulier de l'interaction sociale en cours. On appelle cela "Analyse Égocentrique de Scènes Audio-Visuelles''.Cette thèse contribue à cette thématique de plusieurs façons. D'abord, en fournissant une base de données publique qui cible des applications comme la reconnaissance d'actions et de gestes, localisation et suivi d'interlocuteurs, analyse du tour de parole, localisation de sources auditives, etc. Cette base a été utilisé en dedans et en dehors de cette thèse. Nous avons aussi travaillé le problème de la détection d'événements audio-visuels. Nous avons montré comme la confiance en une des modalités (issue de la vision en l'occurrence), peut être modélisée pour biaiser la méthode, en donnant lieu à un algorithme d'espérance-maximisation visuellement supervisé. Ensuite, nous avons modifié l'approche pour cibler la détection audio-visuelle d'interlocuteurs en utilisant le robot humanoïde NAO. En parallèle aux travaux en détection audio-visuelle d'interlocuteurs, nous avons développé une nouvelle approche pour la reconnaissance audio-visuelle de commandes. Nous avons évalué la qualité de plusieurs indices et classeurs, et confirmé que l'utilisation des données auditives et visuelles favorise la reconnaissance, en comparaison aux méthodes qui n'utilisent que l'audio ou que la vidéo. Plus tard, nous avons cherché la meilleure méthode pour des ensembles d'entraînement minuscules (5-10 observations par catégorie). Il s'agit d'un problème intéressant, car les systèmes réels ont besoin de s'adapter très rapidement et d'apprendre de nouvelles commandes. Ces systèmes doivent être opérationnels avec très peu d'échantillons pour l'usage publique. Pour finir, nous avons contribué au champ de la localisation de sources sonores, dans le cas particulier des réseaux coplanaires de microphones. C'est une problématique importante, car la géométrie du réseau est arbitraire et inconnue. En conséquence, cela ouvre la voie pour travailler avec des réseaux de microphones dynamiques, qui peuvent adapter leur géométrie pour mieux répondre à certaines tâches. De plus, la conception des produits commerciaux peut être contrainte de façon que les réseaux linéaires ou circulaires ne sont pas bien adaptés. / Along the past two decades, the industry has developed several commercial products with audio-visual sensing capabilities. Most of them consists on a videocamera with an embedded microphone (mobile phones, tablets, etc). Other, such as Kinect, include depth sensors and/or small microphone arrays. Also, there are some mobile phones equipped with a stereo camera pair. At the same time, many research-oriented systems became available (e.g., humanoid robots such as NAO). Since all these systems are small in volume, their sensors are close to each other. Therefore, they are not able to capture de global scene, but one point of view of the ongoing social interplay. We refer to this as "Egocentric Audio-Visual Scene Analysis''.This thesis contributes to this field in several aspects. Firstly, by providing a publicly available data set targeting applications such as action/gesture recognition, speaker localization, tracking and diarisation, sound source localization, dialogue modelling, etc. This work has been used later on inside and outside the thesis. We also investigated the problem of AV event detection. We showed how the trust on one of the modalities (visual to be precise) can be modeled and used to bias the method, leading to a visually-supervised EM algorithm (ViSEM). Afterwards we modified the approach to target audio-visual speaker detection yielding to an on-line method working in the humanoid robot NAO. In parallel to the work on audio-visual speaker detection, we developed a new approach for audio-visual command recognition. We explored different features and classifiers and confirmed that the use of audio-visual data increases the performance when compared to auditory-only and to video-only classifiers. Later, we sought for the best method using tiny training sets (5-10 samples per class). This is interesting because real systems need to adapt and learn new commands from the user. Such systems need to be operational with a few examples for the general public usage. Finally, we contributed to the field of sound source localization, in the particular case of non-coplanar microphone arrays. This is interesting because the geometry of the microphone can be any. Consequently, this opens the door to dynamic microphone arrays that would adapt their geometry to fit some particular tasks. Also, because the design of commercial systems may be subject to certain constraints for which circular or linear arrays are not suited.

Pour pouvoir restituer des informations qui correspondent aux besoins de l'utilisateur, les mécanismes d'adaptation doivent disposer de métadonnées sur celui-ci telles que ses caractéristiques personnelles, ses préférences générales, ses centres d'intérêt. De ce fait, le profil utilisateur construit à partir de celles-ci devient central dans tout système basé sur la personnalisation. Dans cette thèse, nous nous focalisons sur l'approche qui consiste à s'appuyer sur le réseau social de l'utilisateur pour enrichir le profil de cet utilisateur, les métadonnées explicites étant complétées par les informations issues de notre processus d'analyse. Nous appelons les techniques ou processus associés à cette approche " profilage social ". Le terme " profil social " désigne un profil construit à l'aide du réseau social de l'utilisateur. Un profil social contient les métadonnées traduisant les intérêts de l'utilisateur extraits à partir des informations partagées par les individus de son réseau social. Les intérêts de l'utilisateur évoluant au fil du temps dans la vie réelle, il en est de même pour ceux extraits depuis son réseau social : pertinents à un moment donné, ils peuvent ne plus être significatifs ultérieurement. Partant de ce constat, les principales informations que nous souhaitons étudier pour détecter un changement de centres d'intérêt ne sont pas ciblées sur l'utilisateur lui-même mais sur les éléments de son réseau social (liens entre les membres, informations qui circulent entre eux) : l'évolution du profil social de l'utilisateur est donc liée à l'évolution de son réseau social. Nous proposons une démarche générique de profilage social efficace permettant de construire un profil social représentatif de l'utilisateur prenant en compte différents types de réseaux ainsi que leurs caractéristiques évolutives. Pour prendre en compte l'évolution des intérêts dans le profil social, nous avons proposé d'améliorer l'efficacité des processus de construction du profil social existants en intégrant la prise en compte de l'évolution du réseau social de l'utilisateur. Nous proposons d'intégrer un facteur temporel dans ces processus (approche basée sur des individus et approche basée sur les communautés). La solution permet de privilégier les intérêts provenant d'informations significatives et à jour. Il s'agit donc d'intégrer une mesure temporelle dans l'étape d'extraction et pondération des intérêts. Cette mesure est calculée d'une part, à partir de la pertinence temporelle des informations utilisées pour extraire cet intérêt et d'autre part, à partir de la pertinence temporelle de l'individu qui partage ces informations. Nous mettons en œuvre la méthode proposée au travers d'expérimentations dans deux réseaux sociaux différents : DBLP, un réseau de publications scientifiques et Twitter, un réseau de micro-blogs. Les résultats de ces expérimentations nous ont permis de montrer l'efficacité de la méthode temporelle proposée par rapport aux processus de construction du profil social qui ne prennent pas en compte des critères temporels. En étudiant les résultats en fonction des techniques de pondération des intérêts ou fonctions temporelles utilisées, nous constatons que la fonction temporelle et la technique utilisées donnant les meilleurs résultats varient selon l'approche de construction du profil social choisie, selon la taille et la densité du réseau étudié mais aussi selon sur le type de réseau. La problématique abordée dans cette thèse est relativement nouvelle dans le contexte des systèmes de personnalisation de l'information et ouvre de nombreuses perspectives : évaluation du profil social dans un système de recommandation par exemple, application de la méthode proposée dans d'autres types de réseaux sociaux, application de techniques de mise à jour du profil, conception d'une plateforme de construction du profil social selon les caractéristiques du réseau. / User profiling is essential for personalization systems (e.g. personalized information retrieval systems, recommendation systems) to identify user information (preference, interests...), in order to propose relevant content based on his/her specific needs and requirements. Many works have shown that user's social neighbors can be a meaningful source to infer his/her interests. Besides, sociology works have shown that the user is better described by people around him/her, especially the people that are directly connected to him/her (his egocentric network). In this work, the term "social profiling" is considered as the interest extraction approach that consists in extracting user interests from information of his/her social neighbors. The user's profile built within this approach is called "social profile". As user behaviors evolve over time, it is necessary to take into consideration the evolution of user interests in user profiling process. In the case of social profile, user interests are extracted from the information shared by his/her social neighbors. Hence, the evolution of extracted interests is related to the evolution of information shared on user social network and to the evolution of relationships between the user and his/her social neighbors. This issue becomes particularly important in the Online Social Networks (OSNs) context where user behavior changes quickly. For a user, the relationships and information in his/her social network can evolve and become obsolete for him/her overtime. Two users creating a relationship are not required to know each other in real life. Thus, the relationship persistence is not always maintained in this case. Social events or viral marketing (buzz) are also factors that enhance online social content sharing. In this work, we propose a generic approach that considers the evolution in user's social network in the social profiling process and can be applied in different types of social network. To handle this, we propose to apply a time-aware method into existing social profile building process (individual based and community based approaches). This strategy aims at weighting user's interests in the social profile based on their temporal score. The temporal score of an interest is computed by combining the temporal score of information used to extract the interests (computed by considering their freshness) with the temporal of individuals who share the information in the network (computed by considering the freshness of the interaction with the user). The technique and temporal function used to compute the temporal score are customizable. Thus, we can find out the most appropriate technique or temporal function depending on the types or characteristics of the adopted social network. The experiments conducted on DBLP and Twitter showed that the so-called time-aware social profiling process applying our proposed time-aware method outperforms the existing time-agnostic social profiling process. We also found that the most appropriate technique, temporal function and social profiling approach vary depending on the network characteristics (size, density) and to the social network type. Our approach opens many opportunities for future studies in social information filtering and many application domains as well as on the Web (e.g. evolution of social profile in personalization of search engines, recommender systems in e-commerce,). Our long-term perspective consists in the proposal of a generic platform that extracts the information and builds the user social profile based on the type and the specific characteristics of the underlying social network. Such a platform would be parameterized by the characteristics of the targeted social network using a machine learning approach.