Modern automobiles and particularly those with off-road lineage possess subsystems that can be configured to better negotiate certain terrain types. Different terrain classes amount to different adherence (or surface grip) and compressibility properties that impact vehicle ma-noeuvrability and should therefore incur a tailored throttle response, suspension stiffness and so on. This thesis explores prospective terrain recognition for an anticipating terrain response driver assistance system. Recognition of terrain and road terrain is cast as a semantic segmen-tation task whereby forward driving images or point clouds are pre-segmented into atomic units and subsequently classified. Terrain classes are typically of amorphous spatial extent con-taining homogenous or granularly repetitive patterns. For this reason, colour and texture ap-pearance is the saliency of choice for monocular vision. In this work, colour, texture and sur-face saliency of atomic units are obtained with a bag-of-features approach. Five terrain classes are considered, namely grass, dirt, gravel, shrubs and tarmac. Since colour can be ambiguous among terrain classes such as dirt and gravel, several texture flavours are explored with scalar and structured output learning in a bid to devise an appropriate visual terrain saliency and predictor combination. Texture variants are obtained using local binary patters (LBP), filter responses (or textons) and dense key-point descriptors with daisy. Learning algorithms tested include support vector machine (SVM), random forest (RF) and logistic regression (LR) as scalar predictors while a conditional random field (CRF) is used for structured output learning. The latter encourages smooth labelling by incorporating the prior knowledge that neighbouring segments with similar saliency are likely segments of the same class. Once a suitable texture representation is devised the attention is shifted from monocular vision to stereo vision. Sur-face saliency from reconstructed point clouds can be used to enhance terrain recognition. Pre-vious superpixels span corresponding supervoxels in real world coordinates and two surface saliency variants are proposed and tested with all predictors: one using the height coordinates of point clouds and the other using fast point feature histograms (FPFH). Upon realisation that road recognition and terrain recognition can be assumed as equivalent problems in urban en-vironments, the top most accurate models consisting of CRFs are augmented with composi-tional high order pattern potentials (CHOPP). This leads to models that are able to strike a good balance between smooth local labelling and global road shape. For urban environments the label set is restricted to road and non-road (or equivalently tarmac and non-tarmac). Ex-periments are conducted using a proprietary terrain dataset and a public road evaluation da-taset.

L'objectif de cette thèse est de proposer une solution au problème de la constance des couleurs entre les images d'une même scène acquises selon un même point de vue ou selon différents points de vue. Ce problème constitue un défi majeur en vision par ordinateur car d'un point de vue à l'autre, on peut être confronté à des variations des conditions d'éclairage (spectre de l'éclairage, intensité de l'éclairage) et des conditions de prise de vue (point de vue, type de caméra, paramètres d'acquisition tels que focus, exposition, balance des blancs, etc.). Ces variations induisent alors des différences d'apparence couleur entre les images acquises qui touchent soit sur l'ensemble de la scène observée soit sur une partie de celle-ci. Dans cette thèse, nous proposons une solution à ce problème qui permet de modéliser puis de compenser, de corriger, ces variations de couleur à partir d'une méthode basée sur quatre étapes : (1) calcul des correspondances géométriques à partir de points d'intérêt (SIFT et MESR) ; (2) calculs des correspondances couleurs à partir d'une approche locale; (3) modélisation de ces correspondances par une méthode de type RANSAC; (4) compensation des différences de couleur par une méthode polynomiale à partir de chacun des canaux couleur, puis par une méthode d'approximation linéaire conjuguée à une méthode d'estimation de l'illuminant de type CAT afin de tenir compte des intercorrélations entre canaux couleur et des changements couleur dus à l'illuminant. Cette solution est comparée aux autres approches de l'état de l'art. Afin d'évaluer quantitativement et qualitativement la pertinence, la performance et la robustesse de cette solution, nous proposons deux jeux d'images spécialement conçus à cet effet. Les résultats de différentes expérimentations que nous avons menées prouvent que la solution que nous proposons est plus performante que toutes les autres solutions proposées jusqu'alors / The objective of this thesis is to propose a solution to the problem of color consistency between images originate from the same scene irrespective of acquisition conditions. Therefore, we present a new color mapping framework that is able to compensate color differences and achieve color consistency between views of the same scene. Our proposed, new framework works in two phases. In the first phase, we propose a new method that can robustly collect color correspondences from the neighborhood of sparse feature correspondences, despite the low accuracy of feature correspondences. In the second phase, from these color correspondences, we introduce a new, two-step, robust estimation of the color mapping model: first, nonlinear channel-wise estimation; second, linear cross-channel estimation. For experimental assessment, we propose two new image datasets: one with ground truth for quantitative assessment; another, without the ground truth for qualitative assessment. We have demonstrated a series of experiments in order to investigate the robustness of our proposed framework as well as its comparison with the state of the art. We have also provided brief overview, sample results, and future perspectives of various applications of color mapping. In experimental results, we have demonstrated that, unlike many methods of the state of the art, our proposed color mapping is robust to changes of: illumination spectrum, illumination intensity, imaging devices (sensor, optic), imaging device settings (exposure, white balance), viewing conditions (viewing angle, viewing distance)

O objetivo deste trabalho é reconhecer gestos em tempo real apenas com o uso de câmeras, sem marcadores, roupas ou qualquer outro tipo de sensor. A montagem do ambiente de captura é simples, com apenas duas câmeras e um computador. O fundo deve ser estático, e contrastar com o usuário. A ausência de marcadores ou roupas especiais dificulta a tarefa de localizar os membros. A motivação desta tese é criar um ambiente de realidade virtual para treino de goleiros, que possibilite corrigir erros de movimentação, posicionamento e de escolha do método de defesa. A técnica desenvolvida pode ser aplicada para qualquer atividade que envolva gestos ou movimentos do corpo. O reconhecimento de gestos começa com a detecção da região da imagem onde se encontra o usuário. Nessa região, localizamos as regiões mais salientes como candidatas a extremidades do corpo, ou seja, mãos, pés e cabeça. As extremidades encontradas recebem um rótulo que indica a parte do corpo que deve representar. Um vetor com as coordenadas das extremidades é gerado. Para descobrir qual a pose do usuário, o vetor com as coordenadas das suas extremidades é classificado. O passo final é a classificação temporal, ou seja, o reconhecimento do gesto. A técnica desenvolvida é robusta, funcionando bem mesmo quando o sistema foi treinado com um usuário e aplicado a dados de outro. / Our aim in this work is to recognize gestures in real time with cameras, without markers or special clothes. The capture environment setup is simple, uses just two cameras and a computer. The background must be static, and its colors must be different the users. The absence of markers or special clothes difficults the location of the users limbs. The motivation of this thesis is to create a virtual reality environment for goalkeeper training, but the technique can be applied in any activity that involves gestures or body movements. The recognition of gestures starts with the background subtraction. From the foreground, we locate the more proeminent regions as candidates to body extremities, that is, hands, feet and head. The found extremities receive a label that indicates the body part it may represent. To classify the users pose, the vector with the coordinates of his extremities is compared to keyposes and the best match is selected. The final step is the temporal classification, that is, the gesture recognition. The developed technique is robust, working well even when the system was trained with an user and applied to another users data.

[pt] Escoamentos intermitentes gás-líquido são encontrados em diversas aplicações de engenharia, como em linhas de produção de petróleo e em sistemas de refrigeração de reatores nucleares. O conhecimento detalhado deste padrão de escoamento é indispensável para a construção de modelos matemáticos representativos deste fenômeno físico complexo. No presente trabalho, foi desenvolvido um estudo experimental com o objetivo de fornecer informações que possibilitem a melhoria dos modelos existentes para o escoamento bifásico intermitente em duto horizontal. A técnica de velocimetria por imagem de partículas estereoscópica (SPIV) de alta frequência foi empregada, em conjunto com a técnica de fluorescência induzida por laser (LIF), permitindo a obtenção de campos de velocidade na região do pistão de líquido, a jusante da bolha alongada, e na região do filme líquido, a montante da bolha. O arranjo experimental possibilitou a medição das três componentes do vetor velocidade do escoamento bifásico água-glicerina e ar na seção transversal de uma tubulação horizontal de 40mm de diâmetro interno. Os resultados obtidos, originais na literatura, mostraram a evolução detalhada das três componentes dos campos médios do vetor velocidade ao longo da passagem da bolha alongada durante o escoamento intermitente. / [en] Gas-liquid intermittent flows can be found in many engineering applications, such as oil production lines and nuclear reactors cooling systems. The detailed knowledge of this flow pattern is necessary for the development of accurate mathematical models of this complex phenomenon. In this work, an experimental study was developed with the goal to provide data that could be used to improve the existing models for two-phase intermittent flow in horizontal pipes. The high-frequency stereoscopic particle image velocimetry (SPIV) technique was employed along with the laser induced fluorescence (LIF) technique, allowing the measurement of velocity fields in the liquid slug, downstream of the elongated bubble, and in the liquid film, upstream of the elongated bubble. The experimental arrangement allowed the measurement of the three components of the velocity vector in an intermittent two-phase water-glycerine and air flow, in a transversal sections of a 40mm internal diameter horizontal pipe. The results obtained revealed original detailed information on the evolution of the three components of the averaged velocity field associated with the passage of the elongated bubble characteristic of the intermittent flow.

Object recognition is a skill we as humans often take for granted. Due to our formidable object learning, recognition and generalisation skills, it is sometimes hard to see the multitude of obstacles that need to be overcome in order to replicate this skill in an artificial system. Object recognition is also one of the classical areas of computer vision, and many ways of approaching the problem have been proposed. Recently, visually capable robots and autonomous vehicles have increased the focus on embodied recognition systems and active visual search. These applications demand that systems can learn and adapt to their surroundings, and arrive at decisions in a reasonable amount of time, while maintaining high object recognition performance. Active visual search also means that mechanisms for attention and gaze control are integral to the object recognition procedure. This thesis describes work done on the components necessary for creating an embodied recognition system, specifically in the areas of decision uncertainty estimation, object segmentation from multiple cues, adaptation of stereo vision to a specific platform and setting, and the implementation of the system itself. Contributions include the evaluation of methods and measures for predicting the potential uncertainty reduction that can be obtained from additional views of an object, allowing for adaptive target observations. Also, in order to separate a specific object from other parts of a scene, it is often necessary to combine multiple cues such as colour and depth in order to obtain satisfactory results. Therefore, a method for combining these using channel coding has been evaluated. Finally, in order to make use of three-dimensional spatial structure in recognition, a novel stereo vision algorithm extension along with a framework for automatic stereo tuning have also been investigated. All of these components have been tested and evaluated on a purpose-built embodied recognition platform known as Eddie the Embodied. / Embodied Visual Object Recognition

This thesis investigates the use of Generative Adversarial Networks (GANs) for detecting images containing non-natural objects in natural environments and if the introduction of stereo data can improve the performance. The state-of-the-art GAN-based anomaly detection method presented by A. Berget al. in [5] (BergGAN) was the base of this thesis. By modifiying BergGAN to not only accept three channel input, but also four and six channel input, it was possible to investigate the effect of introducing stereo data in the method. The input to the four channel network was an RGB image and its corresponding disparity map, and the input to the six channel network was a stereo pair consistingof two RGB images. The three datasets used in the thesis were constructed froma dataset of aerial video sequences provided by SAAB Dynamics, where the scene was mostly wooded areas. The datasets were divided into training and validation data, where the latter was used for the performance evaluation of the respective network. The evaluation method suggested in [5] was used in the thesis, where each sample was scored on the likelihood of it containing anomalies, Receiver Operating Characteristics (ROC) analysis was then applied and the area under the ROC-curve was calculated. The results showed that BergGAN was successfully able to detect images containing non-natural objects in natural environments using the dataset provided by SAAB Dynamics. The adaption of BergGAN to also accept four and six input channels increased the performance of the method, showing that there is information in stereo data that is relevant for GAN-based anomaly detection. There was however no substantial performance difference between the network trained with two RGB images versus the one trained with an RGB image and its corresponding disparity map.

Spatialization of audio in the vertical plane has historically been limited. Instead, sound designers have used basic DSP to create pseudo height effects to explain the positions of corresponding objects. In recent years, binaural synthesis has become more widespread following an increase in the use of software rendering methods. With these advancements, uncertainty seems to be present around best practices when combining sound design with binaural synthesis for vertical placement of audio cues in games. This thesis compares the vertical localization performance between head related transfer functions (HRTFs) and stereo interaural level difference (ILD), when sounds have been designed with basic DSP to have auditory spatial schema (ASC). A sort of embedment of positional information. It was found that there was no significant time difference between the conditions, while hitcount, the number of correct directions selected, displayed a significant difference in some of the comparisons.

Recent advancements in audio technology for computer games has made possible for implementations with binaural audio. Compared to regular stereo sound, binaural audio offers possibilities for a player to experience spatial sound, including sounds along the vertical plane, using their own headphones. A computer game prototype called “Crystal Gatherer” was created for this study to explore the possibilities of binaural audio imple- mentation regarding localization and perception of objects that make sound in a 3D game. The game featured two similar game levels, with the difference that one used binaural sound, and the other stereo sound. The levels consisted of a dark space that the player could navigate freely with the objective to find objects that make sound, called “crystals”, as fast as they could. An experiment was conducted with 14 test sub- jects that played the game, qualitative and quantitative data was collected, including the time the players took to complete the game levels, respectively, and answers about how they experienced the levels. A majority of test subjects reported that they per- ceived a difference between the levels. No significant difference was found between the levels in terms of efficacy of finding the objects that made sound. Some test subjects stated that they found localization was better in the binaural level of the game, others found the stereo level to be better in this respect. The study shows that there can exist possibilities for binaural audio to change the perception of audio in computer games.

No description available.

The aim of this project is to examine the feasibility and reliability of the use of a low cost computer vision system to track and intercept a thrown object. A stereo vision system tracks the object using color recognition and then guides a mobile wheeled robot towards an interception point in order to capture it. Two different trajectory prediction models are compared. One model fits a second degree polynomial to the collected positional measurements of the object and the other uses the Forward Euler Method to construct the objects flight path. To accurately guide the robot, the angular position of the robot must also be measured. Two different methods of measuring the angular position are presented and their respective reliability are measured. A calibrated magnetometer is used as one method while pure computer vision is implemented as the alternative method. A functional object tracking and interception system that was able to intercept the thrown object was constructed using both the polynomial fitting trajectory prediction model as well as the one based on the Forward Euler Method. The magnetometer and pure computer vision are both viable methods of determining the angular position of the robot with an error of less than 1.5°. / I detta projekt behandlas konstruktionen av och pålitligheten i en bollfånganderobot och dess bakomliggande lågbudgetkamerasystem. För att fungera i tre dimensioner används en stereokameramodul som spårar bollen med hjälp av färgigenkänning och beräknar bollbanan samt förutspår nedslaget för att ge god tid till roboten att genskjuta bollen. Två olika bollbanemodeller testas, där den ena tar hänsyn till luftmotståndet och nedslaget beräknas numeriskt och den andra anpassar en andragradspolynom till de observerade datapunkterna. För att styra roboten till den tänkta uppfångningspunkten behövs både robotens position, vilket bestäms med kameramodulen, och robotens riktning.Riktningen bestäms medbåde en magnetometer och med kameramodulen, för att undersöka vilken metod som passar bäst. Den förslagna konstruktionen för roboten och kamerasystemet kan spåra och fånga objekt med bådadera de testade modellerna för att beräkna bollbana, dock så är tillförlitligheten i den numeriska metoden betydligt känsligare för dåliga mätvärden. Det är även möjligt att använda sig av både magnetometern eller endast kameramodulen för att bestämma robotens riktning då båda ger ett fel under 1.5°.