Artificial intelligent (AI) systems today are booming and they are used to solve new tasks or improve the performance on existing ones. However, most AI systems work in a black-box fashion, which prevents the users from accessing the inner modules. This leads to two major problems: (i) users have no idea when the underlying system will fail and thus it could fail abruptly without any warning or explanation, and (ii) users' lack of proficiency about the system could fail pushing the AI progress to its state-of-the-art. In this work, we address these problems in the following directions. First, we develop a failure prediction system, acting as an input filter. It raises a flag when the system is likely to fail with the given input. Second, we develop a portfolio computer vision system. It is able to predict which of the candidate computer vision systems perform the best on the input. Both systems have the benefit of only looking at the inputs without running the underlying vision systems. Besides, they are applicable to any vision system. By equipped such systems on different applications, we confirm the improved performance. Finally, instead of identifying errors, we develop more interpretable AI systems, which reveal the inner modules directly. We take two tasks as examples, words semantic matching and Visual Question Answering (VQA). In VQA, we take binary questions on abstract scenes as the first stage, then we extend to all question types on real images. In both cases, we take attention as an important intermediate output. By explicitly forcing the systems to attend correct regions, we ensure the correctness in the systems. We build a neural network to directly learn the semantic matching, instead of using the relation similarity between words. Across all the above directions, we show that by diagnosing errors and making more interpretable systems, we are able to improve the performance in the current models. / Ph. D.

The Roobots are a robot soccer team which participated in the RoboCup small-sized robot league competition in the years 2000, 2001 and 2002, when they finished in fourth place. This thesis describes the design of the robots in the 2002 team. Design issues for mobile robots in the RoboCup small-sized robot league are reviewed. The design decisions are presented. Finally, some lessons learnt for system design and project management from the three years of competition are presented.

ITC/USA 2006 Conference Proceedings / The Forty-Second Annual International Telemetering Conference and Technical Exhibition / October 23-26, 2006 / Town and Country Resort & Convention Center, San Diego, California / This paper describes the concept of wide-angle coverage optical vision system integrated with guidance support of microwave or acoustical imaging arrays. The objective is to provide the capability of effective high-resolution full-view monitoring and sensing. The optical component, formed by a multi-camera array, is responsible for the main interface with human users. The acoustical and microwave arrays are integrated, allowing the system to function in the event-triggered modality for optimal efficiency. In this paper, the arrays discussed are in circular configurations. With minor modification, the system can also function with linear array configurations.

The main purpose of this thesis was to develop a printed circuit board for Autoliv Electronics AB. This circuit board should be placed in their test equipment to support some of their camera vision systems used in cars. The main task was to combine the existing hardware into one module. To be able to achieve this, the most important factors in designing a printed circuit board was considered. A satisfying power distribution network is the most crucial one. This was accomplished by using decoupling capacitors to achieve low enough impedance for all circuits. Calculations and simulations were executed for all integrated circuits to find the correct size and numbers of capacitors. The impedance of the circuit board was tested with a network analyzer to confirm that the impedance were low enough, which was the case. System functionality was never tested completely, due to delivery problems with some external equipment.

<p> </p><p>In this thesis a method to compensate camera distortion is developed for an FPGA platform as part of a complete vision system. Several methods and models is presented and described to give a good introduction to the complexity of the problems that is overcome with the developed method. The solution to the core problem is shown to have a good precision on a sub-pixel level.</p><p> </p>

The representation and manipulation of visual content in a computer vision system requires a suitable abstraction of raw visual content such as pixels in an image. In this thesis, we study region-based feature representations and in particular, hierarchical segmentations because they do make no assumptions about region granularity. Hierarchical segmentations create a large feature space that increases the cost of subsequent processing in computer vision systems. We introduce a segment filter to reduce the feature space of hierarchical segmentations by identifying unique regions in the images. The filter uses appearance-based properties of the regions and the structure of the segmentation for the selection of a small set of descriptive regions. The filter works in two phases: selection with a criteria based on relative region size and a sorting based on a variational criteria. The filter is applicable to any hierarchical segmentation algorithm, in particular to bottom-up and region growing approaches. We evaluate the filter's performance against an extensive set of ground-truth regions from a dataset containing image sequences with scenes of different complexity. We demonstrate a novel region-to-region image matching approach as a possible application of our segment filter. A reduced segmentation tree is reconstructed based on the set of regions provided by the filtering. The reduction of the feature space by the segment filter simplifies our region-to-region matching approach. The correspondences between regions from two different images is established by a similarity measure. We use a modified mutual information measurement to compute the similarity of regions. The identified region correspondences are refined using the reduced segmentation tree. Our region-to-region matching approach is evaluated with an extensive set of ground-truth correspondences. This evaluation shows the large potential of both, our filtering and our matching approach.

The representation and manipulation of visual content in a computer vision system requires a suitable abstraction of raw visual content such as pixels in an image. In this thesis, we study region-based feature representations and in particular, hierarchical segmentations because they do make no assumptions about region granularity. Hierarchical segmentations create a large feature space that increases the cost of subsequent processing in computer vision systems. We introduce a segment filter to reduce the feature space of hierarchical segmentations by identifying unique regions in the images. The filter uses appearance-based properties of the regions and the structure of the segmentation for the selection of a small set of descriptive regions. The filter works in two phases: selection with a criteria based on relative region size and a sorting based on a variational criteria. The filter is applicable to any hierarchical segmentation algorithm, in particular to bottom-up and region growing approaches. We evaluate the filter's performance against an extensive set of ground-truth regions from a dataset containing image sequences with scenes of different complexity. We demonstrate a novel region-to-region image matching approach as a possible application of our segment filter. A reduced segmentation tree is reconstructed based on the set of regions provided by the filtering. The reduction of the feature space by the segment filter simplifies our region-to-region matching approach. The correspondences between regions from two different images is established by a similarity measure. We use a modified mutual information measurement to compute the similarity of regions. The identified region correspondences are refined using the reduced segmentation tree. Our region-to-region matching approach is evaluated with an extensive set of ground-truth correspondences. This evaluation shows the large potential of both, our filtering and our matching approach.

Human welders have long been able to monitor a weld pool and adjust welding parameters accordingly. Automated welding robots can provide consistent movement during the welding process, but lack the ability to monitor the weld pool. A vision system attached to the welding robot could provide a way to monitor the weld pool substantially faster than a human being. Previous vision systems to monitor weld pool surfaces have been developed, but their uses are limited since the system is fixed in place. The compact vision system developed in this research attaches directly to the welding torch, which provides no limitations in weld pool monitoring. This system takes advantage of the specular surface of a molten weld pool by reflecting a pattern of laser beams from the weld pool surface. The deformation of the laser beam after it reflects from the weld pool surface can provide clues to the weld pool shape, and thus the penetration of the weld. Image processing techniques and geometric optics are used to reconstruct a weld pool surface shape based on the image captured of the deformed laser pattern.

The representation and manipulation of visual content in a computer vision system requires a suitable abstraction of raw visual content such as pixels in an image. In this thesis, we study region-based feature representations and in particular, hierarchical segmentations because they do make no assumptions about region granularity. Hierarchical segmentations create a large feature space that increases the cost of subsequent processing in computer vision systems. We introduce a segment filter to reduce the feature space of hierarchical segmentations by identifying unique regions in the images. The filter uses appearance-based properties of the regions and the structure of the segmentation for the selection of a small set of descriptive regions. The filter works in two phases: selection with a criteria based on relative region size and a sorting based on a variational criteria. The filter is applicable to any hierarchical segmentation algorithm, in particular to bottom-up and region growing approaches. We evaluate the filter's performance against an extensive set of ground-truth regions from a dataset containing image sequences with scenes of different complexity. We demonstrate a novel region-to-region image matching approach as a possible application of our segment filter. A reduced segmentation tree is reconstructed based on the set of regions provided by the filtering. The reduction of the feature space by the segment filter simplifies our region-to-region matching approach. The correspondences between regions from two different images is established by a similarity measure. We use a modified mutual information measurement to compute the similarity of regions. The identified region correspondences are refined using the reduced segmentation tree. Our region-to-region matching approach is evaluated with an extensive set of ground-truth correspondences. This evaluation shows the large potential of both, our filtering and our matching approach.

Un système de vision embarquée multi-capteurs est doté de plusieurs capteurs d'images de technologie différente.Il peut être un capteur couleur, un capteur infrarouge ou encore un capteur bas niveau de lumière.Les caractéristiques de ces capteurs sont également hétérogènes.Nous avons différentes fréquences trames, résolutions et dynamiques de pixels.Cette multiplicité et cette hétérogénéité des capteurs d'images permet à un système de vision de mieux répondre à ses besoins.En fait, un système de vision multi-capteurs doit fonctionner dans plusieurs milieux opérationnels (urbain, marin, boisé).Il doit également s'adapter à plusieurs conditions de luminosité (jour, nuit, faible éclairage).Enfin, la multiplicité des capteurs permet d'offrir des fonctionnalités intéressantes à l'utilisateur final : fusion multispectrale, vision panoramique, vision multi-champs.Le défi de conception est que l'ensemble de ces paramètres environnementaux et opérationnels peuvent varier dynamiquement au cours de l'utilisation du système de vision.Il est nécessaire que la conception de l'architecture tienne compte de cette variabilité dynamique du contexte d'utilisation.L'architecture doit présenter la flexibilité dynamique suffisante afin de s'adapter aux variations de contexte.Elle doit également pouvoir prendre conscience de l'évolution du contexte.La solution architecturale doit tout de même satisfaire les contraintes de surface et de consommation énergétique d'un système embarqué et portable.Nous proposons dans cette thèse un moniteur permettant à l'architecture actuelle de Safran de s'auto-adapter dynamiquement.Ce moniteur joue deux rôles dans l'auto-adaptation de l'architecture.D'une part, il observe en permanence les changements de contexte.D'autre part, il décide et pilote en conséquence les adaptations à effectuer sur l'architecture.L'observation porte sur l'environnement opérationnel et sur le système de vision multi-capteurs (y compris l'architecture).Le moniteur analyse les données d'observation et prend des décisions sur l'adaptation.Enfin, il commande les différents contrôleurs de l'architecture afin d'exécuter les adaptations requises par le changement de contexte.Nous introduisons un réseau de routeurs qui a pour principal objectif l'acheminement des données de monitoring.Le réseau proposé permet d'accéder à l'architecture sans pour autant compromettre le traitement des flux d'images.Ce réseau s'inspire de nos précédents travaux pour la mise en place d'un système de paquets de données cite{Ng2011}.Un dernier volet de notre proposition porte sur la gestion de la mémoire trames.Avec les changements de contexte permanents, le besoin en ressources de mémoire évolue dynamiquement.Pour une utilisation économique et optimale des ressources, il est nécessaire d'adapter l'attribution des ressources au fil des variations des besoins.Nous présentons un contrôleur mémoire permettant l'allocation dynamique de l'espace mémoire et la régulation dynamique de la distribution de la bande passante mémoire.Nous évaluons les différents volets de notre proposition à l'aide d'une implémentation sur un FPGA Cyclone V de chez ALTERA (5CGX).Nous présentons les validations progressivement au fur et à mesure que nous abordons chaque volet de notre proposition.Chaque validation présente les performances en temps et en surface / An embedded multi-sensor vision system involves several types of image sensors such as colour, infrared or low-light sensor.Characteristics of the sensors are often various (different resolution, frame rate and pixel depth).Hence, the vision system has to deal with several heterogeneous image streams.That multiplicity and the heterogeneity of the sensors help to face various environmental contexts.We consider a multi-sensor vision system that has to work in different area (city, sea, forest) and handle several operations (multispectral fusion, panoramic, multifocus).The vision system has to also face various luminosity conditions : day, night or low-light condition.The challenge of designing architecture for such a vision system is that the working context can dynamically vary.The designer has to take in account this dynamic variation of the working context.The architecture should be enough flexible to adapt its processing to the requirements of the context.It also has to be able to detect any variation of the context and adapt itself according to the context.Above all, the design should satisfy area and power constraints of an embedded and portable system.In this thesis, we propose an embedded monitor enabling dynamic auto-adaptation of the current multi-stream architecture of Safran.The monitor accomplishes two tasks for the auto-adaptation of the architecture.First, he continuously observes changes of both external and internal contexts.Then, he decides the adaptation that the architecture needs in response to the context variation.Observation of the external context is about the type of the area and the luminosity conditions.While, observation of the internal context focuses on the current status of the vision system and its architecture.To perform the adaptation, the monitor sends adaptation commands toward controllers of the architecture.We introduce a Network-on-Chip (NoC) based interconnexion layer to fulfill monitoring communication.This NoC is inspired from our previous work cite{Ng2011}.This layer allows observing and commanding the processing stages without compromising the existing pixels streams.Routers of the NoC are responsible for routing observation data from processing stages to the monitor and adaptation commands from the monitor toward processing stages.The proposed NoC takes in account the heterogeneity of working frequencies.Finally, we present a memory controller that enables dynamic allocation of the frame memory.When the working context changes, memory resources requirements change too.For an optimised and economical resources utilisation, we propose to dynamically adapt the frame buffer allocation.Also, the proposed has the possibility to dynamically manage the bandwidth of the frame memory.We introduce a pondered round robin-based method with the ability to adapt the weights on-the-fly.Our proposition has been evaluated with a typical Safran multi-stream architecture.It has been implemented in a FPGA target.Area performances have been evaluated through synthesis for a ALTERA Cyclone V FPGA (5CGX).Latency performances have been evaluated thanks to ModelSim simulations