Recognizing generic objects by single or multi view range images is a contemporary popular problem in 3D object recognition area with developing technology of scanning devices such as laser range scanners. This problem is vital to current and future vision systems performing shape based matching and classification of the objects in an arbitrary scene. Despite improvements on scanners, there are still imperfections on range scans such as holes or unconnected parts on images. This studyobjects at proposing and comparing algorithms that match a range image to complete 3D models in a target database.The study started with a baseline algorithm which usesstatistical representation of 3D shapesbased on 4D geometricfeatures, namely SURFLET-Pair relations.The feature describes the geometrical relationof a surface-point pair and reflects local and the global characteristics of the object. With the desire of generating solution to the problem,another algorithmthat interpretsSURFLET-Pairslike in the baseline algorithm, in which histograms of the features are used,isconsidered. Moreover, two other methods are proposed by applying 2D space filing curves on range images and applying 4D space filling curves on histograms of SURFLET-Pairs. Wavelet transforms are used for filtering purposes in these algorithms. These methods are tried to be compact, robust, independent on a global coordinate frame and descriptive enough to be distinguish queries&rsquo / categories.Baseline and proposed algorithms are implemented on a database in which range scans of real objects with imperfections are queries while generic 3D objects from various different categories are target dataset.

[Truncated abstract] The aim of visual recognition is to identify objects in a scene and estimate their pose. Object recognition from 2D images is sensitive to illumination, pose, clutter and occlusions. Object recognition from range data on the other hand does not suffer from these limitations. An important paradigm of recognition is model-based whereby 3D models of objects are constructed offline and saved in a database, using a suitable representation. During online recognition, a similar representation of a scene is matched with the database for recognizing objects present in the scene . . . The tensor representation is extended to automatic and pose invariant 3D face recognition. As the face is a non-rigid object, expressions can significantly change its 3D shape. Therefore, the last part of this thesis investigates representations and matching techniques for automatic 3D face recognition which are robust to facial expressions. A number of novelties are proposed in this area along with their extensive experimental validation using the largest available 3D face database. These novelties include a region-based matching algorithm for 3D face recognition, a 2D and 3D multimodal hybrid face recognition algorithm, fully automatic 3D nose ridge detection, fully automatic normalization of 3D and 2D faces, a low cost rejection classifier based on a novel Spherical Face Representation, and finally, automatic segmentation of the expression insensitive regions of a face.

Apresentamos neste trabalho o desenvolvimento de um método para o reconhecimento do Formato de Objetos Tri-dimensionais. Os sistemas tradicionais de Visão Computacional empregam imagens bi-dimensionais obtidos através de câmeras de TV, ricas em detalhes necessários a visão humana. Estes detalhes em grande parte das aplicações industriais de Robôs são supérfluos. Os algoritmos tradicionais de classificação consomem portanto muito tempo no processamento deste excesso de informação. Para este trabalho, desenvolvemos um sistema dedicado para reconhecimento que utiliza um feixe de Laser defletido sobre um objeto e a digitalização da Luminância em cada ponto de sua superfície. A intensidade luminosa refletida e proporcional a distância do ponto ao observador. É, portanto, possível determinar parâmetros que classifiquem cada objeto. A inclinação de cada face de um poliedro, o comportamento de suas fronteiras e também a existência de arestas internas, são as características adotadas. Estas características são então rotuladas, permitindo que o programa de classificação busque em um \"banco de conhecimento\" previamente estabelecido, a descrição dos objetos. Uma mesa giratória permite a rotação do modele fornecendo novas vistas ao observador, determinando sua classificação. Todo o sistema é controlado por um microcomputador cujo programa reconhece em tempo real o objeto em observação. Para o protótipo construído, utilizamos um Laser de HeNe sendo a recepção do raio refletido realizada por um fototransistor. Os objetos reconhecíveis pelo programa são poliedros regulares simples, compondo o seguinte conjunto: 1 prisma de base triangular, 1 cubo, 1 pirâmide de base triangular, 1 pirâmide de base retangular. O tratamento matemático empregado visa a comprovação da tecnologia proposta, podendo, na continuação de trabalhos futuros, ser efetivamente estendido a diversos outros objetos como, por exemplo, os de superfícies curvas. / We present in this work a new method for three dimensional Shape Recognition. Traditional Computer Vision systems use bi-dimensional TV camera images. In most of the industrial Robotic applications, the excess of detail obtained by the TV camera is needless. Traditional classification algorithms spend a lot of time to process the excess of information. For the present work we developed a dedicated recognition system, which deflects a Laser beam over an object and digitizes the Reflected beam point by point over the surface. The intensity of the reflected beam is proportional to the observer distance. Using this technique it was possible to establish features to classify various objects. These features are the slope of the polyhedral surfaces, the boundary type and the inner edges. For each object the features are labeled and the classification algorithm searches in a \"knowledge data base\" for the object description. The recognition system used a He-Ne Laser and the reflected signal was captured by a photo-transistor. The object to be recognized is placed over a rotating table which can be rotated, supplying a new view for the classification. A microcomputer controls the system operation and the object is recognized in real time. The recognized objects were simple regular polyhedral, just as: 1 triangular base prism, 1 cube, 1 triangular base pyramid, 1 rectangular base pyramid. To check that the proposed technology was correct, we used a dedicated mathematical approach, which can be extended to other surfaces, such as curves, in future works.

abstract: Autonomous Vehicles (AVs), or self-driving cars, are poised to have an enormous impact on the automotive industry and road transportation. While advances have been made towards the development of safe, competent autonomous vehicles, there has been inadequate attention to the control of autonomous vehicles in unanticipated situations, such as imminent crashes. Even if autonomous vehicles follow all safety measures, accidents are inevitable, and humans must trust autonomous vehicles to respond appropriately in such scenarios. It is not plausible to program autonomous vehicles with a set of rules to tackle every possible crash scenario. Instead, a possible approach is to align their decision-making capabilities with the moral priorities, values, and social motivations of trustworthy human drivers.Toward this end, this thesis contributes a simulation framework for collecting, analyzing, and replicating human driving behaviors in a variety of scenarios, including imminent crashes. Four driving scenarios in an urban traffic environment were designed in the CARLA driving simulator platform, in which simulated cars can either drive autonomously or be driven by a user via a steering wheel and pedals. These included three unavoidable crash scenarios, representing classic trolley-problem ethical dilemmas, and a scenario in which a car must be driven through a school zone, in order to examine driver prioritization of reaching a destination versus ensuring safety. Sample human driving data in CARLA was logged from the simulated car’s sensors, including the LiDAR, IMU and camera. In order to reproduce human driving behaviors in a simulated vehicle, it is necessary for the AV to be able to identify objects in the environment and evaluate the volume of their bounding boxes for prediction and planning. An object detection method was used that processes LiDAR point cloud data using the PointNet neural network architecture, analyzes RGB images via transfer learning using the Xception convolutional neural network architecture, and fuses the outputs of these two networks. This method was trained and tested on both the KITTI Vision Benchmark Suite dataset and a virtual dataset exclusively generated from CARLA. When applied to the KITTI dataset, the object detection method achieved an average classification accuracy of 96.72% and an average Intersection over Union (IoU) of 0.72, where the IoU metric compares predicted bounding boxes to those used for training. / Dissertation/Thesis / Masters Thesis Mechanical Engineering 2020

Esta tese descreve uma abordagem para a utilização da informação de cores no sistema de visão artificial com inspiração biológica denominada Cyvis-1. Considerando-se que grande parte da literatura sobre segmentação de imagens se refere a imagens em níveis de cinza, informações cromáticas na segmentação permanecem uma área que ainda deve ser mais bem explorada e para a qual se direcionou o interesse da presente pesquisa. Neste trabalho, o subsistema de cor do Cyvis-1 é definido, mantendo-se o vínculo com os princípios que inspiram o sistema de visão como um todo: hierarquia, modularidade, especialização do processamento, integração em vários níveis, representação efetiva da informação visual e integração com conhecimento de nível alto. O subsistema de cor se insere neste escopo, propondo uma técnica para segmentação de imagens coloridas baseada em mapas auto-organizáveis para a classificação dos pontos da imagem. A segmentação incorpora a determinação do número de classes sem supervisão, tornando o processo mais independente de intervenção humana. Por este processo de segmentação, são produzidos mapas das regiões encontradas e um mapa de bordas, derivado das regiões. Uma segunda proposta do trabalho é um estudo comparativo do desempenho de técnicas de segmentação por bordas. A comparação é feita em relação a um mapa de bordas de referência e o comportamento de várias técnicas é analisado segundo um conjunto de atributos locais baseados em contrastes de intensidade e cor. Derivada desta comparação, propõe-se também uma combinação para a geração de um mapa de bordas a partir da seleção das técnicas segundo seus desempenhos locais. Finalmente, integrando os aspectos anteriores, é proposta urna estruturação do módulo de cor, adicionalmente com a aquisição de imagens, a análise de formas e o reconhecimento de objetos poliédricos. Há, neste contexto, a integração ao módulo de estéreo, que proporciona o cálculo de dados tridimensionais, essenciais para o reconhecimento dos objetos. Para cada parte deste trabalho são propostas formas de avaliação para a validação dos resultados, demonstrando e caracterizando a eficiência e as limitações de cada uma. / This thesis describes an approach to color information processing in the biologically-inspired artificial vision system named Cyvis-1. Considering that most of the current literature in image segmentation deals with gray level images, color information remains an incipient area, which has motivated this research. This work defines the color subsystem within the Cyvis-1 underlying phylosophy, whose main principles include hierarchy, modularity, processing specialization, multilevel integration, effective representation of visual information, and high-level knowledge integration. The color subsystem is then introduced according to this framework, with a proposal of a segmentation technique based on self-organizing maps. The number of regions in the image is achieved through a unsupervised clustering approach, so no human interaction is needed. Such segmentation technique produces region oriented representation of the classes, which are used to derive an edge map. Another main topic in this work is a comparative study of the edge maps produced by several edge-oriented segmentation techniques. A reference edge map is used as standard segmentation, to which other edge maps are compared. Such analysis is carried out by means of local attributes (local gray level and \"color\" contrasts). As a consequence of the comparison, a combination edge map is also proposed, based on the conditional selection of techniques considering the local attributes. Finally, the integration of two above topics is proposed, which is characterized by the design of the color subsystem of Cyvis-1, altogether with the modules for image acquisition, shape analysis and polyhedral object recognition. In such a context, the integration with the stereo subsystem is accomplished, allowing the evaluation of the three-dimensional data needed for object recognition. Assessment and validation of the three proposals were carried out, providing the means for analyzing their efficiency and limitations.

Στη παρούσα εργασία, δημιουργήθηκε ένα εκπαιδευτικό περιβάλλον εικονικής πραγματικότητας για προσομοίωση σεισμού σε σχολική τάξη. Πρόκειται για ένα ασφαλές περιβάλλον, εφικτού κόστους, που προσομοιώνει αρκετά καλά το φυσικό. Παρουσιάζεται ο τρόπος κατασκευής των αντικειμένων της τάξης με τη χρήση λογισμικού μοντελοποίησης και η δημιουργία διαφόρων συμβάντων που λαμβάνουν χώρα κατά τη διάρκεια του σεισμού, με τη χρήση λογισμικού εικονικής πραγματικότητας. Η εφαρμογή δίνει τη δυνατότητα στο μαθητή να αποκτήσει ψυχολογική εξοικείωση και να ενισχύσει την αντιληπτικότητά του σχετικά με το φαινόμενο, να βελτιώσει την απόδοσή του αποκομίζοντας καινούργιες εμπειρίες και ταυτόχρονα να εφαρμόζει όσα έχει μάθει για τους τρόπους αντίδρασης σε περίπτωση σεισμού σε συνθήκες πραγματικού γεγονότος. / An educational environment of virtual reality was designed in order to simulate an earthquake occurring while students are present in a classroom. Such a virtual environment has many advantages: it is secure for students, not costly and very similar to the real life one that is to the actual classroom in the sense that students are free to interact within its confines. The present dissertation attempts to present the ways through which school elements can be produced using a modelling tool. Additionally, we have created a variety of incidents taking place while the earthquake is occurring using a virtual reality software. Summing up, this application offers students the opportunity to psychologically familiarize themselves with the phenomenon of an earthquake while at the same time reinforces their awareness of it. It offers students the possibility to acquire new experiences and improve their performance in crisis management (e.g. earthquake), and it simultaneously sets while an example-environment to apply their theoretical knowledge in real life situations.

In this thesis we study a perception problem in the context of autonomous driving. Specifically, we study the computer vision problem of 3D object detection, in which objects should be detected from various sensor data and their position in the 3D world should be estimated. We also study the application of Generative Adversarial Networks in domain adaptation techniques, aiming to improve the 3D object detection model's ability to transfer between different domains. The state-of-the-art Frustum-PointNet architecture for LiDAR-based 3D object detection was implemented and found to closely match its reported performance when trained and evaluated on the KITTI dataset. The architecture was also found to transfer reasonably well from the synthetic SYN dataset to KITTI, and is thus believed to be usable in a semi-automatic 3D bounding box annotation process. The Frustum-PointNet architecture was also extended to explicitly utilize image features, which surprisingly degraded its detection performance. Furthermore, an image-only 3D object detection model was designed and implemented, which was found to compare quite favourably with current state-of-the-art in terms of detection performance. Additionally, the PixelDA approach was adopted and successfully applied to the MNIST to MNIST-M domain adaptation problem, which validated the idea that unsupervised domain adaptation using Generative Adversarial Networks can improve the performance of a task network for a dataset lacking ground truth annotations. Surprisingly, the approach did however not significantly improve upon the performance of the image-based 3D object detection models when trained on the SYN dataset and evaluated on KITTI.

Esta tese descreve uma abordagem para a utilização da informação de cores no sistema de visão artificial com inspiração biológica denominada Cyvis-1. Considerando-se que grande parte da literatura sobre segmentação de imagens se refere a imagens em níveis de cinza, informações cromáticas na segmentação permanecem uma área que ainda deve ser mais bem explorada e para a qual se direcionou o interesse da presente pesquisa. Neste trabalho, o subsistema de cor do Cyvis-1 é definido, mantendo-se o vínculo com os princípios que inspiram o sistema de visão como um todo: hierarquia, modularidade, especialização do processamento, integração em vários níveis, representação efetiva da informação visual e integração com conhecimento de nível alto. O subsistema de cor se insere neste escopo, propondo uma técnica para segmentação de imagens coloridas baseada em mapas auto-organizáveis para a classificação dos pontos da imagem. A segmentação incorpora a determinação do número de classes sem supervisão, tornando o processo mais independente de intervenção humana. Por este processo de segmentação, são produzidos mapas das regiões encontradas e um mapa de bordas, derivado das regiões. Uma segunda proposta do trabalho é um estudo comparativo do desempenho de técnicas de segmentação por bordas. A comparação é feita em relação a um mapa de bordas de referência e o comportamento de várias técnicas é analisado segundo um conjunto de atributos locais baseados em contrastes de intensidade e cor. Derivada desta comparação, propõe-se também uma combinação para a geração de um mapa de bordas a partir da seleção das técnicas segundo seus desempenhos locais. Finalmente, integrando os aspectos anteriores, é proposta urna estruturação do módulo de cor, adicionalmente com a aquisição de imagens, a análise de formas e o reconhecimento de objetos poliédricos. Há, neste contexto, a integração ao módulo de estéreo, que proporciona o cálculo de dados tridimensionais, essenciais para o reconhecimento dos objetos. Para cada parte deste trabalho são propostas formas de avaliação para a validação dos resultados, demonstrando e caracterizando a eficiência e as limitações de cada uma. / This thesis describes an approach to color information processing in the biologically-inspired artificial vision system named Cyvis-1. Considering that most of the current literature in image segmentation deals with gray level images, color information remains an incipient area, which has motivated this research. This work defines the color subsystem within the Cyvis-1 underlying phylosophy, whose main principles include hierarchy, modularity, processing specialization, multilevel integration, effective representation of visual information, and high-level knowledge integration. The color subsystem is then introduced according to this framework, with a proposal of a segmentation technique based on self-organizing maps. The number of regions in the image is achieved through a unsupervised clustering approach, so no human interaction is needed. Such segmentation technique produces region oriented representation of the classes, which are used to derive an edge map. Another main topic in this work is a comparative study of the edge maps produced by several edge-oriented segmentation techniques. A reference edge map is used as standard segmentation, to which other edge maps are compared. Such analysis is carried out by means of local attributes (local gray level and \"color\" contrasts). As a consequence of the comparison, a combination edge map is also proposed, based on the conditional selection of techniques considering the local attributes. Finally, the integration of two above topics is proposed, which is characterized by the design of the color subsystem of Cyvis-1, altogether with the modules for image acquisition, shape analysis and polyhedral object recognition. In such a context, the integration with the stereo subsystem is accomplished, allowing the evaluation of the three-dimensional data needed for object recognition. Assessment and validation of the three proposals were carried out, providing the means for analyzing their efficiency and limitations.

This thesis is dedicated to the problem of 3D face capture. A majority of current face capture systems exploits a pattern projection. The goal of thesis was to develop the 3D face capture system based on the passive stereo photogrammetry. The high-resolution images of a face are captured by one pair of the calibrated digital still cameras. The image pre-processing consists of the removal of lens distortion, rectification of image pair and face extraction. The stereo image matching is accentuated. Three different approaches to the stereo correspondence search are presented - the local technique, the global technique using a graph cut and the hybrid technique merging previous two. At last, the 3D model is reconstructed according to the found correspondence. The results show that the high-quality 3D face models can be obtained despite traditional difficulties with matching pattern-free face images. Also it is possible to keep the computational demands on the acceptable level, although the high-resolution images are processed.

Recent advancements in the field of Computer Vision are a by-product of breakthroughs in the domain of Artificial Intelligence. Object detection in monocular images is now realized by an amalgamation of Computer Vision and Deep Learning. While most approaches detect objects as a mere two dimensional (2D) bounding box, there are a few that exploit rather traditional representation of the 3D object. Such approaches detect an object either as a 3D bounding box or exploit its shape primitives using active shape models which results in a wireframe-like detection. Such a wireframe detection is represented as combinations of detected keypoints (or landmarks) of the desired object. Apart from a faithful retrieval of the object’s true shape, wireframe based approaches are relatively robust in handling occlusions. The central task of this thesis was to find such an approach and to implement it with the goal of its performance evaluation. The object of interest is the vehicle class (cars, mini vans, trucks etc.) and the evaluation data is monocular traffic surveillance videos collected by the supervising chair. A wireframe type detection can aid several facets of traffic analysis by improved (compared to 2D bounding box) estimation of the detected object’s ground plane. The thesis encompasses the process of implementation of the chosen approach called Occlusion-Net [40], including its design details and a qualitative evaluation on traffic surveillance videos. The implementation reproduces most of the published results across several occlusion categories except the truncated car category. Occlusion-Net’s erratic detections are mostly caused by incorrect detection of the initial region of interest. It employs three instances of Graph Neural Networks for occlusion reasoning and localization. The thesis also provides a didactic introduction to the field of Machine and Deep Learning including intuitions of mathematical concepts required to understand the two disciplines and the implemented approach.:Contents 1 Introduction 1 2 Technical Background 7 2.1 AI, Machine Learning and Deep Learning 7 2.1.1 But what is AI ? 7 2.1.2 Representational composition by Deep Learning 10 2.2 Essential Mathematics for ML 14 2.2.1 Linear Algebra 15 2.2.2 Probability and Statistics 25 2.2.3 Calculus 34 2.3 Mathematical Introduction to ML 39 2.3.1 Ingredients of a Machine Learning Problem 39 2.3.2 The Perceptron 40 2.3.3 Feature Transformation 46 2.3.4 Logistic Regression 48 2.3.5 Artificial Neural Networks: ANN 53 2.3.6 Convolutional Neural Network: CNN 61 2.3.7 Graph Neural Networks 68 2.4 Specific Topics in Computer Vision 72 2.5 Previous work 76 3 Design of Implemented Approach 81 3.1 Training Dataset 81 3.2 Keypoint Detection : MaskRCNN 83 3.3 Occluded Edge Prediction : 2D-KGNN Encoder 84 3.4 Occluded Keypoint Localization : 2D-KGNN Decoder 86 3.5 3D Shape Estimation: 3D-KGNN Encoder 88 4 Implementation 93 4.1 Open-Source Tools and Libraries 93 4.1.1 Code Packaging: NVIDIA-Docker 94 4.1.2 Data Processing Libraries 94 4.1.3 Libraries for Neural Networks 95 4.1.4 Computer Vision Library 95 4.2 Dataset Acquisition and Training 96 4.2.1 Acquiring Dataset 96 4.2.2 Training Occlusion-Net 96 4.3 Refactoring 97 4.3.1 Error in Docker File 97 4.3.2 Image Directories as Input 97 4.3.3 Frame Extraction in Parallel 98 4.3.4 Video as Input 100 4.4 Functional changes 100 4.4.1 Keypoints In Output 100 4.4.2 Mismatched BB and Keypoints 101 4.4.3 Incorrect Class Labels 101 4.4.4 Bounding Box Overlay 101 5 Evaluation 103 5.1 Qualitative Evaluation 103 5.1.1 Evaluation Across Occlusion Categories 103 5.1.2 Performance on Moderate and Heavy Vehicles 105 5.2 Verification of Failure Analysis 106 5.2.1 Truncated Cars 107 5.2.2 Overlapping Cars 108 5.3 Analysis of Missing Frames 109 5.4 Test Performance 110 6 Conclusion 113 7 Future Work 117 Bibliography 119