Thesis (Ph.D.)--Computing, Georgia Institute of Technology, 2009. / Committee Chair: Abowd, Gregory; Committee Member: Edwards, Keith; Committee Member: Grinter, Rebecca; Committee Member: LaMarca, Anthony; Committee Member: Starner, Thad.

Thesis (Ph. D.)--State University of New York at Binghamton, Thomas J. Watson School of Engineering and Applied Science, Department of Electrical and Computer Engineering, 2008. / Includes bibliographical references.

Thesis (Ph. D.), Memorial University of Newfoundland, 1999. / Bibliography: p. 131-137.

Thesis (Ph. D.)--Lehigh University, 2006. / Includes vita. Includes bibliographical references (leaves 114-124).

Orientador: Carlos Alberto dos Reis Filho / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de Computação / Made available in DSpace on 2018-08-05T17:36:56Z (GMT). No. of bitstreams: 1 Salustiano_RogerioEsteves_M.pdf: 3698724 bytes, checksum: a5c6d59ec19db38f5a0324243ddb1eb5 (MD5) Previous issue date: 2006 / Resumo: Este trabalho propõe um sistema computacional no qual são aplicadas técnicas de Fusão de Sensores no monitoramento de ambientes. O sistema proposto permite a utilização e incorporação de diversos tipos de dados, incluindo imagens, sons e números em diferentes bases. Dentre os diversos algoritmos pertinentes a um sistema como este, foram implementados os de Sensores em Consenso que visam a combinação de dados de uma mesma natureza. O sistema proposto é suficientemente flexível, permitindo a inclusão de novos tipos de dados e os correspondentes algoritmos que os processem. Todo o processo de recebimento dos dados produzidos pelos sensores, configuração e visualização dos resultados é realizado através da Internet / Abstract: This work proposes a computer system in which Sensor Fusion techniques are applied to monitoring the environment. The proposed system allows the use and incorporation of different data types, including images, sounds and numbers in different bases. Among the existing algorithms that pertain to a system like this, those, which aim to combine data of the same nature, called Consensus Sensors, have been particularly implemented. The proposed system is flexible enough and allows the inclusion of new data types and their corresponding algorithms. The whole process of receiving the data produced by the sensors, configuration of produced results as well as their visualization is performed through the Internet / Mestrado / Eletrônica, Microeletrônica e Optoeletrônica / Mestre em Engenharia Elétrica

Cette thèse a pour but d'étudier et de quantifier la contribution de la perception multimodale assistée par le contexte pour détecter et suivre des objets en mouvement. Cette étude sera appliquée à la détection et la reconnaissance des objets pertinents dans les environnements de la circulation pour les véhicules intelligents (VI). Les résultats à obtenir devront permettre de transposer le concept proposé à un ensemble plus large de capteurs et de classes d'objets en utilisant une approche système intégrative qui implique des méthodes d'apprentissage. En particulier, ces méthodes d'apprentissage vont examiner comment l'implantation dans un système intégré, qui prévoie une multitude des sources de données différentes, peut conduire à apprendre 1) sans ou avec une supervision limitée, réduite en exploitant des corrélations 2) de façon incrémentale à la connaissance stockée au lieu de faire un entraînement complet à chaque fois qu’une nouvelle donnée arrive 3) collectivement à chaque instant d'apprentissage dans le système entraîné d'une manière qui assure approximativement une fusion optimale. Concrètement, le couplage fort entre les classifier des objets en modalités multiples aussi bien que l'extraction du contexte de la géométrie de la scène sont à étudier: d'abord en théorie, après en application du trafic routier. La nouveauté de l'approche d'intégration envisagée se pose dans le couplage fort entre les composants du système, tels que la segmentation, le suivi des objets, l'estimation de la géométrie de la scène et la catégorisation des objets basée sur la stratégie de l'inférence probabiliste. Une telle stratégie caractérise des systèmes où toutes les composants de perception émettent et reçoivent les distributions des résultats possibles avec leur score de croyance probabiliste attribué. De cette façon, chaque composant de traitement peut prendre en compte les résultats des autres composants au niveau plus bas par rapport aux combinaisons des résultats finaux. Cela diminue beaucoup le temps et les ressources pour le calcul, quand les techniques de l'application de l'inférence Bayésienne garantissent que les données d'entrée peu plausible n'apportent pas des impacts négatifs. / This thesis project will investigate and quantify the contribution of context-aided multimodal perception for detecting and tracking moving objects. This research study will be applied to the detection and recognition ofrelevant objects in road traffic environments for Intelligent Vehicles (IV). The results to be obtained will allow us to transpose the proposed concept to a wide range of state-of-the-art sensors and object classes by means of an integrative system approach involving learning methods. In particular, such learning methods will investigate how the embedding into an embodied system providing a multitude of different data sources, can be harnessed to learn 1) without, or with reduced, explicit supervision by exploiting correlations 2) incrementally, by adding to existing knowledge instead of complete retraining every time new data arrive 3) collectively, each learning instance in the system being trained in a way that ensures approximately optimal fusion. Concretely, a tight coupling between object classifiers in multiple modalities as well as geometric scene context extraction will be studied, first in theory, then in the context of road traffic. The novelty of the envisioned integration approach lies in the tight coupling between system components such as object segmentation, object tracking, scene geometry estimation and object categorization based on a probabilistic inference strategy. Such a strategy characterizes systems where all perception components broadcast and receive distributions of multiple possible results together with a probabilistic belief score. In this way, each processing component can take into account the results of other components at a much earlier stage (as compared to just combining final results), thus hugely increasing its computation power, while the application of Bayesian inference techniques will ensure that implausible inputs do not cause negative effects.

Indiana University-Purdue University Indianapolis (IUPUI) / A key component in tracking is identifying relevant data and combining the data in an effort to provide an accurate estimate of both the location and the orientation of an object marker as it moves through an environment. This thesis proposes an enhancement to an existing tracking system, the enhanced distributed tracking system (e-DTS), in the form of the e-DTS 2.0 and provides an empirical analysis of these enhancements. The thesis also provides suggestions on future enhancements and improvements. When a Camera identifies an object within its frame of view, it communicates with a JINI-based service in an effort to expose this information to any client who wishes to consume it. This aforementioned communication utilizes the JINI Multicast Lookup Protocol to provide the means for a dynamic discovery of any sensors as they are added or removed from the environment during the tracking process. The client can then retrieve this information from the service and perform a fusion technique in an effort to provide an estimation of the marker's current location with respect to a given coordinate system. The coordinate system handoff and transformation is a key component of the e-DTS 2.0 tracking process as it improves the agility of the system.

For complex satellite missions that rely on agile and high-precision manoeuvres, the low-friction aspect of the space environment is a critical component in understanding the attitude control dynamics of the spacecraft. The Kinesthetic Node and Autonomous Table-Top Emulator (KNATTE) is a three-degree-of-freedom frictionless vehicle that serves as the foundation of a multipurpose platform for real-time spacecraft hardware-in-the-loop experiments, and allows emulation of these conditions in two dimensions with the purpose of validating various guidance, navigation, and control algorithms. The data acquisition of the vehicle depends on a computer vision system (CVS) that yields position and attitude data, but also suffers from unpredictable blackout events. To complement such measurements, KNATTE incorporates an inertial measurement unit (IMU) that yields accelerometer, gyroscope, and magnetometer data. This study describes a multisensor data fusion approach to obtain accurate attitude information by combining the measurements from the CVS and the IMU using nonlinear Kalman filter algorithms. To do this, the data fusion algorithms are developed and tested in a Matlab/Simulink environment. After that, the algorithms are adapted to the KNATTE platform and their performance is confirmed in various conditions. Through this work, the accuracy and efficiency of the approach can be checked by numerical simulation and real-time experiments. In addition, the quality of the CVS measurements are further improved by the introduction of a neural network to the image processing pipeline of the original system.

Après la démocratisation des robots industriels, la tendance actuelle est au développement de robots sociaux dont la fonction principale est l'interaction avec ses utilisateurs. Le déploiement de telles plate-formes dans des boutiques, des musées ou des gares relance différentes problématiques dont celle de la localisation pour les robots mobiles. Cette thèse traite ainsi de la localisation du robot Pepper en milieu intérieur non-contraint. Présent dans de nombreuses boutiques au Japon, Pepper est utilisé par des personnes non-expertes et doit donc être le plus autonome possible. Cependant, les solutions de localisation autonome de la littérature souffrent des limitations de la plate-forme. Les travaux de cette thèse s'articulent autour de deux grands axes. D'abord, le problème de la relocalisation dans un environnement visuellement redondant est étudié. La solution proposée consiste à combiner la vision et le Wi-Fi dans une approche probabiliste basée sur l'apparence. Ensuite, la question de la création d'une carte métrique cohérente est approfondie. Pour compenser les nombreuses pertes de suivi d'amers visuels causées par une fréquence d'acquisition basse, des contraintes odométriques sont ajoutées à une optimisation par ajustement de faisceaux. Ces solutions ont été testées et validées sur plusieurs robots Pepper à partir de données collectées dans différents environnements intérieurs sur plus de 7 km. / After the democratization of industrial robots, the current trend is the development of social robots that create strong interactions with their users. The deployment of such platforms in shops, museums or train stations raises various issues including the autonomous localization of mobile robots. This thesis focuses on the localization of Pepper robots in a non-constrained indoor environment. Pepper robots are daily used in many shops in Japan and must be as autonomous as possible. However, localization solutions in the literature suffer from the limitations of the platform. This thesis is split into two main themes. First, the problem of relocalization in a visually redundant environment is studied. The proposed solution combines vision and Wi-Fi in a probabilistic approach based on the appearance. Then, the question of a consistent metrical mapping is examined. In order to compensate the numerous losses of tracking caused by the low acquisition frequency, odometric constraints are added to a bundle adjustment optimization. These solutions have been tested and validated on several Pepper robots, from data collected in different indoor environments over more than 7 km.

At the start of the 21st century, the topic of complexity remains a formidable challenge in engineering, science and other aspects of our world. It seems that when disaster strikes it is because some complex and unforeseen interaction causes the unfortunate outcome. Why did the financial system of the world meltdown in 2008-2009? Why are global temperatures on the rise? These questions and other ones like them are difficult to answer because they pertain to contexts that require lengthy descriptions. In other words, these contexts are complex. But we as human beings are able to observe and recognize this thing we call 'complexity'. Furthermore, we recognize that there are certain elements of a context that form a system of complex interactions - i.e., a complex system. Many researchers have even noted similarities between seemingly disparate complex systems. Do sub-atomic systems bear resemblance to weather patterns? Or do human-based economic systems bear resemblance to macroscopic flows? Where do we draw the line in their resemblance? These are the kinds of questions that are asked in complex systems research. And the ability to recognize complexity is not only limited to analytic research. Rather, there are many known examples of humans who, not only observe and recognize but also, operate complex systems. How do they do it? Is there something superhuman about these people or is there something common to human anatomy that makes it possible to fly a plane? - Or to drive a bus? Or to operate a nuclear power plant? Or to play Chopin's etudes on the piano? In each of these examples, a human being operates a complex system of machinery, whether it is a plane, a bus, a nuclear power plant or a piano. What is the common thread running through these abilities? The study of situational awareness (SA) examines how people do these types of remarkable feats. It is not a bottom-up science though because it relies on finding general principles running through a host of varied human activities. Nevertheless, since it is not constrained by computational details, the study of situational awareness provides a unique opportunity to approach complex tasks of operation from an analytical perspective. In other words, with SA, we get to see how humans observe, recognize and react to complex systems on which they exert some control. Reconciling this perspective on complexity with complex systems research, it might be possible to further our understanding of complex phenomena if we can probe the anatomical mechanisms by which we, as humans, do it naturally. At this unique intersection of two disciplines, a hybrid approach is needed. So in this work, we propose just such an approach. In particular, this research proposes a computational approach to the situational awareness (SA) of complex systems. Here we propose to implement certain aspects of situational awareness via a biologically-inspired machine-learning technique called Hierarchical Temporal Memory (HTM). In doing so, we will use either simulated or actual data to create and to test computational implementations of situational awareness. This will be tested in two example contexts, one being more complex than the other. The ultimate goal of this research is to demonstrate a possible approach to analyzing and understanding complex systems. By using HTM and carefully developing techniques to analyze the SA formed from data, it is believed that this goal can be obtained.