Finding salient features in an image, and matching them to their corresponding features in another image is an important step for many vision-based applications. Feature description plays an important role in the feature matching process. A robust feature descriptor must works with a number of image deformations and should be computationally efficient. For resource-limited systems, floating point and complex operations such as multiplication and square root are not desirable. This research first introduces a robust and efficient feature descriptor called PRObability (PRO) descriptor that meets these requirements without sacrificing matching accuracy. The PRO descriptor is further improved by incorporating only affine features for matching. While performing well, PRO descriptor still requires larger descriptor size, higher offline computation time, and more memory space than other binary feature descriptors. SYnthetic BAsis (SYBA) descriptor is developed to overcome these drawbacks. SYBA is built on the basis of a new compressed sensing theory that uses synthetic basis functions to uniquely encode or reconstruct a signal. The SYBA descriptor is designed to provide accurate feature matching for real-time vision applications. To demonstrate its performance, we develop algorithms that utilize SYBA descriptor to localize the soccer ball in a broadcast soccer game video, track ground objects for unmanned aerial vehicle, and perform motion analysis, and improve visual odometry accuracy for advanced driver assistance systems. SYBA provides high feature matching accuracy with computational simplicity and requires minimal computational resources. It is a hardware-friendly feature description and matching algorithm suitable for embedded vision applications.

[pt] À medida que o processo de digitalização avança em diversos setores, a criação de modelos digitais 3D torna-se cada vez mais necessária. Normalmente, esses modelos são construídos por designers 3D, exigindo um esforço manual considerável quando o objeto modelado é complexo. Além disso, como o designer não tem uma referência precisa na maioria dos casos, o modelo resultante está sujeito a erros de medição. No entanto, é possível minimizar o esforço de construção e o erro de medição usando técnicas de aquisição 3D e modelos CAD previamente construídos. A saída típica de uma técnica de aquisição 3D é uma nuvem de pontos 3D bruta, que precisa de processamento para reduzir o ruído inerente e a falta de informações topológicas. Os modelos CAD são normalmente usados para documentar um processo de projeto de engenharia, apresentando alta complexidade e muitos detalhes irrelevantes para muitos processos de visualização. Portanto, dependendo da aplicação, devemos simplificar bastante o modelo CAD para atender aos seus requisitos. Nesta tese, nos concentramos na construção de modelos digitais 3D a partir dessas fontes. Mais precisamente, apresentamos um conjunto de algoritmos de processamento de geometria para automatizar diferentes etapas de um fluxo de trabalho típico usado para esta construção. Primeiro, apresentamos um algoritmo de redução de ruído de nuvem de pontos que visa preservar as feições nítidas da superfície subjacente. Este algoritmo inclui soluções para a estimativa normal e problemas de detecção de feições nítidas. Em segundo lugar, apresentamos uma extensão do algoritmo de redução de ruído de nuvem de pontos para processar malhas triangulares, onde tiramos proveito da topologia explícita definida pela malha. Por fim, apresentamos um algoritmo para a simplificação extrema de modelos CAD complexos, que tendem a se aproximar da superfície externa do objeto modelado. Os algoritmos propostos são comparados com métodos de última geração, apresentando resultados competitivos e superando-os na maioria dos casos de teste. / [en] As the digitalization process advances in several industries, the creation of 3D digital models is becoming more and more required. Commonly, these models are constructed by 3D designers, requiring considerable manual effort when the modeled object is complex. In addition, since the designer does not have an accurate reference in most cases, the resulting model is prone to measurement errors. However, it is possible to minimize the construction effort and the measurement error by using 3D acquisition techniques and previously constructed CAD models. The typical output of a 3D acquisition technique is a raw 3D point cloud, which needs processing to reduce the inherent noise and lack of topological information. CAD models are typically used to document an engineering design process, presenting high complexity and too many details irrelevant to many visualization processes. So, depending on the application, we must severely simplify the CAD model to meet its requirements. In this thesis, we focus on the construction of 3D digital models from these sources. More precisely, we present a set of geometry processing algorithms to automatize different stages of a typical workflow used for this construction. First, we present a point cloud denoising algorithm that seeks to preserve the sharp features of the underlying surface. This algorithm includes solutions for the normal estimation and sharp feature detection problems. Second, we present an extension of the point cloud denoising algorithm to process triangle meshes, where we take advantage of the explicit topology defined by the mesh. Finally, we present an algorithm for the extreme simplification of complex CAD models, which tends to approximate the outer surface of the modeled object. The proposed algorithms are compared with state-of-the-art methods, showing competitive results and outperforming them in most test cases.

Augmented reality has been on the rise for some time now and begun making its way onto the mobile market for both IOS and Android. In 2017 Apple released ARKit for IOS which is a software development kit for developing augmented reality applications. To counter this, Google released their own variant called ARCore on the 1st of march 2018. ARCore is also a software development kit for developing augmented reality applications but made for the Android, Unity and Unreal platforms instead. Since ARCore is released recently it is still unknown what particular limitations may exist for it. The purpose of this paper is give an indication to companies and developers about ARCore’s potential limitations. The goal with this paper and work is to map how well ARCore works during different circumstances, and in particular, how its feature detection works and behaves. A quantitative research was done with the usage of the case study method. Various tests were performed with a modified test-application supplied by Google. The tests included testing how ARCore’s feature detection, the process that analyzes the environment presented to the application. This which enables the user of an application to place a virtual object on the physical environment. The tests were done to see how ARCore works during different light levels, different types of surfaces, different angles and the difference between having the device stationary or moving. From the testing that were done some conclusions could be drawn about the light levels, surfaces and differences between a moving and stationary device. More research and testing following these principles need to be done to draw even more conclusions of the system and its limitations. How these should be done is presented and discussed. / Forstarkt verklighet (augmented reality) har stigit under en tid och börjat ta sig in på mobilmarknaden for både IOS och Android. År 2017 släppte Apple ARKit för IOS vilket är en utvecklingsplattform för att utveckla applikationer inom förstärkt verklighet. Som svar på detta så slappte Google sin egna utvecklingsplattform vid namn ARCore, som släpptes den 1 mars 2018. ARCore är också en utvecklingsplattform för utvecklandet av applikationer inom förstarkt verklighet men istället inom Android, Unity och Unreal. Sedan ARCore släpptes nyligen är det fortfarande okant vilka särskilda begränsningar som kan finnas för det. Syftet med denna avhandling är att ge företag och utvecklare en indikation om ARCores potentiella begränsningar. Målet med denna avhandling och arbete är att kartlägga hur väl ARCore fungerar under olika omstandigheter, och i synnerhet hur dess struktursdetektor fungerar och beter sig. En kvantitativ forskning gjordes med användning av fallstudie metoden. Olika tester utfördes med en modifierad test-applikation från Google. Testerna inkluderade testning av hur ARCores struktursdetektor, processen som analyserar miljön runt om sig, fungerar. Denna teknik möjliggor att användaren av en applikation kan placera ett virtuellt objekt på den fysiska miljön. Testen innebar att se hur ARCore arbetar under olika ljusnivåer, olika typer av ytor, olika vinklar och skillnaden mellan att ha enheten stationär eller rör på sig. Från testningen som gjordes kan man dra några slutsatser om ljusnivåer, ytor och skillnader mellan en rörlig och stationar enhet. Mer forskning och testning enligt dessa principer måste göras för att dra ännu mer slutsatser av systemet och dess begränsningar. Hur dessa ska göras presenteras och diskuteras.

Computer Aided Design (CAD) models often need to be processed due to data translation issues and requirements of the downstream applications like computational field simulation, rapid rototyping, computer graphics,computational manufacturing, and real-time rendering before they can be used. Automatic CAD model processing tools can significantly reduce the amount of time and cost associated with the manual processing.In this dissertaion, automated topology generation and feature removal techniques are developed to prepare suitable models with mimunum user interaction. A topology generation algorithm, commonly known as CAD repairing/healing, is presented to detect commonly found geometrical and topological issues like cracks, gaps, overlaps, intersections, T-connections, and no/invalid topology in the model, process them and build correct topological information. The present algorithm is based on the iterative vertex pair contraction and expansion operations called stitching and filling respectively. The algorithm closes small gaps/overlaps via the stitching operation and fills larger gaps by adding faces through the filling operation to process the model accurately. Processed models are guaranteed to be free of intersecting faces or surfaces. Moreover, the topology generation algorithm can process manifold as well as non-manifold models, which makes the procedure more general and flexible. This algorithm uses an automatic and adaptive distance threshold that enhances reliability of the process and preserves small features in the model. In addition, a spatial data structure, the octree, is used for searching and neighbor finding to process large models efficiently. In this way, the combination of generality, accuracy, reliability, and efficiency of this algorithm seems to be a significant improvement over existing techniques. Results are presented showing the effectiveness of the algorithm to process two- and three-dimensional configurations. Feature detection and removal and feature collapse algorithms are presented to detect and remove small features from CAD models automatically. The feature detection and removal algorithm uses a feature size measure based on the surface area and perimeter to detect small features accurately and remove them from the model. Small feature removal may create holes in the model that are post-processed using the stitching and/or filling operations of the topology generation algorithm. The feature collapse algorithm is based on the iterative vertex pair contraction operation, which is a generalization of an edge-collapse operation, to collapse small features. Unlike previous efforts that use edge-collapse as a dimension reduction operator, the feature collapse algorithm can pair up any arbitrary vertices and perform iterative vertex pair contraction to collapse small features as well as glue unconnected regions. Results showing the automatic detection and removal of most commonly found small features like small edges/faces, fillets, chamfers, nuts, and bolts from real mechanical parts are presented.

Scale-invariant interest points have found several highly successful applications in computer vision, in particular for image-based matching and recognition. This paper presents a theoretical analysis of the scale selection properties of a generalized framework for detecting interest points from scale-space features presented in Lindeberg (Int. J. Comput. Vis. 2010, under revision) and comprising: an enriched set of differential interest operators at a fixed scale including the Laplacian operator, the determinant of the Hessian, the new Hessian feature strength measures I and II and the rescaled level curve curvature operator, as well as an enriched set of scale selection mechanisms including scale selection based on local extrema over scale, complementary post-smoothing after the computation of non-linear differential invariants and scale selection based on weighted averaging of scale values along feature trajectories over scale. A theoretical analysis of the sensitivity to affine image deformations is presented, and it is shown that the scale estimates obtained from the determinant of the Hessian operator are affine covariant for an anisotropic Gaussian blob model. Among the other purely second-order operators, the Hessian feature strength measure I has the lowest sensitivity to non-uniform scaling transformations, followed by the Laplacian operator and the Hessian feature strength measure II. The predictions from this theoretical analysis agree with experimental results of the repeatability properties of the different interest point detectors under affine and perspective transformations of real image data. A number of less complete results are derived for the level curve curvature operator. / <p>QC 20121003</p> / Image descriptors and scale-space theory for spatial and spatio-temporal recognition

In this paper, a method for determining ground plane features in a sequence of images captured by a mobile camera is presented. The hardware of the mobile system consists of a monocular camera that is mounted on an inertial measurement unit (IMU). An image processing procedure is proposed, first to extract image features and match them across consecutive image frames, and second to detect the ground plane features using a two-step algorithm. In the first step, the planar homography of the ground plane is constructed using an IMU-camera motion estimation approach. The obtained homography constraints are used to detect the most likely ground features in the sequence of images. To reject the remaining outliers, as the second step, a new plane normal vector computation approach is proposed. To obtain the normal vector of the ground plane, only three pairs of corresponding features are used for a general camera transformation. The normal-based computation approach generalizes the existing methods that are developed for specific camera transformations. Experimental results on real data validate the reliability of the proposed method. / <p>QC 20121107</p>

This thesis represents Bayesian joint audio-visual tracking for the 3D locations of multiple people and a current speaker in a real conference environment. To achieve this objective, it focuses on several different research interests, such as acoustic-feature detection, visual-feature detection, a non-linear Bayesian framework, data association, and sensor fusion. As acoustic-feature detection, time-delay-of-arrival~(TDOA) estimation is used for multiple source detection. Localization performance using TDOAs is also analyzed according to different configurations of microphones. As a visual-feature detection, Viola-Jones face detection is used to initialize the locations of unknown multiple objects. Then, a corner feature, based on the results from the Viola-Jones face detection, is used for motion detection for robust objects. Simple point-to-line correspondences between multiple cameras using fundamental matrices are used to determine which features are more robust. As a method for data association and sensor fusion, Monte-Carlo JPDAF and a data association with IPPF~(DA-IPPF) are implemented in the framework of particle filtering. Three different tracking scenarios of acoustic source tracking, visual source tracking, and joint acoustic-visual source tracking are represented using the proposed algorithms. Finally the real-time implementation of this joint acoustic-visual tracking system using a PC, four cameras, and six microphones is addressed with two parts of system implementation and real-time processing.

Etude bibliographique sur le recalage d'images de visage et sur le recalage d'images et travail en collaboration avec Son VuS, pour définir la précision nécessaire du recalage en fonction des exigences des méthodes de reconnaissance de visages. / Face alignment is an important step in a typical automatic face recognition system.This thesis addresses the alignment of faces for face recognition applicationin video surveillance context. The main challenging factors of this research includethe low quality of images (e.g., low resolution, motion blur, and noise), uncontrolledillumination conditions, pose variations, expression changes, and occlusions. In orderto deal with these problems, we propose several face alignment methods using differentstrategies. The _rst part of our work is a three-stage method for facial pointlocalization which can be used for correcting mis-alignment errors. While existingalgorithms mostly rely on a priori knowledge of facial structure and on a trainingphase, our approach works in an online mode without requirements of pre-de_nedconstraints on feature distributions. The proposed method works well on images underexpression and lighting variations. The key contributions of this thesis are aboutjoint image alignment algorithms where a set of images is simultaneously alignedwithout a biased template selection. We respectively propose two unsupervised jointalignment algorithms : \Lucas-Kanade entropy congealing" (LKC) and \gradient correlationcongealing" (GCC). In LKC, an image ensemble is aligned by minimizing asum-of-entropy function de_ned over all images. GCC uses gradient correlation coef-_cient as similarity measure. The proposed algorithms perform well on images underdi_erent conditions. To further improve the robustness to mis-alignments and thecomputational speed, we apply a multi-resolution framework to joint face alignmentalgorithms. Moreover, our work is not limited in the face alignment stage. Since facealignment and face acquisition are interrelated, we develop an adaptive appearanceface tracking method with alignment feedbacks. This closed-loop framework showsits robustness to large variations in target's state, and it signi_cantly decreases themis-alignment errors in tracked faces.

The energy consumption to make the off-chip memory writing and readings are aknown problem. In the image processing field structure from motion simpler compressiontechniques could be used to save energy. A balance between the detected features suchas corners, edges, etc., and the degree of compression becomes a big issue to investigate.In this thesis a deeper study of this balance are performed. A number of more advancedcompression algorithms for processing of still images such as JPEG is used for comparisonwith a selected number of simpler compression algorithms. The simpler algorithms canbe divided into two categories: individual block-wise compression of each image andcompression with respect to all pixels in each image. In this study the image sequences arein grayscale and provided from an earlier study about rolling shutters. Synthetic data setsfrom a further study about optical flow is also included to see how reliable the other datasets are. / Energikonsumtionen för att skriva och läsa till off-chip minne är ett känt problem. Inombildbehandlingsområdet struktur från rörelse kan enklare kompressionstekniker användasför att spara energi. En avvägning mellan detekterade features såsom hörn, kanter, etc.och grad av kompression blir då en fråga att utreda. I detta examensarbete har en djuparestudie av denna avvägning utförts. Ett antal mer avancerade kompressionsalgoritmer förbearbetning av stillbilder som tex. JPEG används för jämförelse med ett antal utvaldaenklare kompressionsalgoritmer. De enklare algoritmerna kan delas in i två kategorier:individuell blockvis kompression av vardera bilden och kompression med hänsyn tillsamtliga pixlar i vardera bilden. I studien är bildsekvenserna i gråskala och tillhandahållnafrån en tidigare studie om rullande slutare. Syntetiska data set från ytterligare en studie om’optical flow’ ingår även för att se hur pass tillförlitliga de andra dataseten är.

This thesis investigates the problem of detecting and tracking sticky notes on Kanban boards using classical computer vision techniques. Currently, there exists some alternatives for digitizing sticky notes, but none keep track of notes that have already been digitized, allowing for duplicate notes to be created when scanning multiple images of the same Kanban board. Kanban boards are widely used in various industries, and being able to recognize, and possibly in the future even digitize entire Kanban boards could provide users with extended functionality. The implementation presented in this thesis is able to, given two images, detect the Kanban boards in each image and rectify them. The rectified images are then sent to the Google Cloud Vision API for text detection. Then, the rectified images are used to detect all the sticky notes. The positional information of the notes and columns of the Kanban boards are then used to filter the text detection to find the text inside each note as well as the header text for each column. Between the two images, the columns are compared and matched, as well as notes of the same color. If columns or notes in one image do not have a match in the second image, it is concluded that the boards are different, and the user is informed of why. If all columns and notes in one image have matches in the second image but some notes have moved, the user is informed of which notes that have moved, and how they have moved as well. The different experiments conducted in this thesis on the implementation show that it works well, but it is very confined to strict requirements, making it unsuitable for commercial use. The biggest problem to solve is to make the implementation more general, i.e. the Kanban board layout, sticky note shapes and colors as well as their actual content. / Denna avhandling undersöker problemet med att upptäcka och spåra klisterlappar och Kanban-tavlor med hjälp av klassiska datorseendetekniker. För närvarande finns det några alternativ för att digitalisera klisterlappar, men ingen håller reda på anteckningar som redan har digitaliserats, vilket gör att duplicerade anteckningar kan skapas när du skannar flera bilder av samma Kanban-kort. Kanban-kort används flitigt i olika branscher och att kunna känna igen, och eventuellt i framtiden även digitalisera hela Kanban-tavlor, skulle kunna ge användarna utökad funktionalitet. Implementeringen som presenteras i denna avhandling kan, givet två bilder, upptäcka Kanban-brädorna i varje bild och korrigera dem. De korrigerade bilderna skickas sedan till Google Cloud Vision API för textidentifiering. Sedan används de korrigerade bilderna för att upptäcka alla klisterlappar. Positionsinformationen för anteckningarna och kolumnerna på Kanban-tavlan används sedan för att filtrera textdetekteringen för att hitta texten i varje anteckning såväl som rubriktexten för varje kolumn. Mellan de två bilderna jämförs och matchas kolumnerna, samt anteckningar av samma färg. Om kolumner eller anteckningar i en bild inte har en matchning i den andra bilden dras slutsatsen att brädorna är olika och användaren informeras om varför. Om alla kolumner och anteckningar i en bild har matchningar i den andra bilden men några anteckningar har flyttats, informeras användaren om vilka anteckningar som har flyttats och hur de har flyttats. De olika experiment som genomförs i denna avhandling om implementering visar att den fungerar bra, men den är mycket begränsad till strikta krav, vilket gör den olämplig för kommersiellt bruk. Det största problemet att lösa är att göra implementeringen mer generell, d.v.s. Kanban-tavlans layout, klisterlapparnas former och färger samt deras faktiska innehåll.