Workplace injuries are common in today's society due to a lack of adequately worn safety equipment. A system that only admits appropriately equipped personnel can be created to improve working conditions and worker safety. The goal is thus to develop a system that will improve construction workers' safety. Building such a system necessitates computer vision, which entails object recognition, facial recognition, and human recognition, among other things. The basic idea is first to detect the human and remove the background to speed up the process and avoid potential interferences. After that, the cropped image is subjected to facial and object recognition. The code is written in Python and includes libraries such as OpenCV, face_recognition, and CVZone. Some of the different algorithms chosen were YOLOv4 and Histogram of Oriented Gradients. The results were measured at three respectively five-meter distances. As a result of the system’s pipeline, algorithms, and software, a mean average precision of 99% and 89% was achieved at the respective distances. At three and five meters, the model achieved a precision rate of 100%. The recall rates were 96% - 100% at 3m and 54% - 100% at 5m. Finally, the fps was measured at 1.2 on a system without GPU. / Skador på arbetsplatsen är vanliga i dagens samhälle på grund av att skyddsutrustning inte används eller används felaktigt. Målet är därför att bygga ett robust system som ska förbättra säkerhet. Ett system som endast ger tillträde till personal med rätt skyddsutrustning kan skapas för att förbättra arbetsförhållandena och arbetarsäkerheten. Att bygga ett sådant system kräver datorseende, vilket bland annat innebär objektigenkänning, ansiktsigenkänning och mänsklig igenkänning. Grundidén är att först upptäcka människan och ta bort bakgrunden för att göra processen mer effektiv och undvika potentiella störningar. Därefter appliceras ansikts- och objektigenkänning på den beskurna bilden. Koden är skriven i Python och inkluderar bland annat bibliotek som: OpenCV, face_recognition och CVZone. Några av de algoritmer som valdes var YOLOv4 och Histogram of Oriented Gradients. Resultatet mättes på tre, respektive fem meters avstånd. Systemets pipeline, algoritmer och mjukvara gav en medelprecision för alla klasser på 99%, och 89% för respektive avstånd. För tre och fem meters avstånd uppnådde modellen en precision på 100%. Recall uppnådde värden mellan 96% - 100% vid 3 meters avstånd och 54% - 100% vid 5 meters avstånd. Avslutningsvis uppmättes antalet bilder per sekund till 1,2 på ett system utan GPU.

Using computer vision to automatically process and understand images is becoming increasingly popular. One frequently used technique in this area is object detection, where the goal is to both localize and classify objects in images. Today's detection models are accurate, but there is still room for improvement. Most models process objects independently and do not take any contextual information into account in the classification step. This thesis will therefore investigate if a performance improvement can be achieved by classifying all objects jointly with the use of contextual information. An architecture that has the ability to learn relationships of this type of information is the transformer. To investigate what performance that can be achieved, a new architecture is constructed where the classification step is replaced by a transformer block. The model is trained and evaluated on document images and shows promising results with a mAP score of 87.29. This value is compared to a mAP of 88.19, which was achieved by the object detector, Mask R-CNN, that the new model is built upon.  Although the proposed model did not improve the performance, it comes with some benefits worth exploring further. By using contextual information the proposed model can eliminate the need for Non-Maximum Suppression, which can be seen as a benefit since it removes one hand-crafted process. Another benefit is that the model tends to learn relatively quickly and a single pass over the dataset seems sufficient. The model, however, comes with some drawbacks, including a longer inference time due to the increase in model parameters. The model predictions are also less secure than for Mask R-CNN. With some further investigation and optimization, these drawbacks could be reduced and the performance of the model be improved.

Perception for autonomous drive systems is the most essential function for safe and reliable driving. LiDAR sensors can be used for perception and are vying for being crowned as an essential element in this task. In this thesis, we present a novel real-time solution for detection and tracking of moving objects which utilizes deep learning based 3D object detection. Moreover, we present a joint solution which utilizes the predictability of Kalman Filters to infer object properties and semantics to the object detection algorithm, resulting in a closed loop of object detection and object tracking.On one hand, we present YOLO++, a 3D object detection network on point clouds only. A network that expands YOLOv3, the latest contribution to standard real-time object detection for three-channel images. Our object detection solution is fast. It processes images at 20 frames per second. Our experiments on the KITTI benchmark suite show that we achieve state-of-the-art efficiency but with a mediocre accuracy for car detection, which is comparable to the result of Tiny-YOLOv3 on the COCO dataset. The main advantage with YOLO++ is that it allows for fast detection of objects with rotated bounding boxes, something which Tiny-YOLOv3 can not do. YOLO++ also performs regression of the bounding box in all directions, allowing for 3D bounding boxes to be extracted from a bird's eye view perspective. On the other hand, we present a Multi-threaded Object Tracking (MTKF) solution for multiple object tracking. Each unique observation is associated to a thread with a novel concurrent data association process. Each of the threads contain an Extended Kalman Filter that is used for predicting and estimating an associated object's state over time. Furthermore, a LiDAR odometry algorithm was used to obtain absolute information about the movement of objects, since the movement of objects are inherently relative to the sensor perceiving them. We obtain 33 state updates per second with an equal amount of threads to the number of cores in our main workstation.Even if the joint solution has not been tested on a system with enough computational power, it is ready for deployment. Using YOLO++ in combination with MTKF, our real-time constraint of 10 frames per second is satisfied by a large margin. Finally, we show that our system can take advantage of the predicted semantic information from the Kalman Filters in order to enhance the inference process in our object detection architecture.

Detecção de objetos é um problema clássico em visão computacional, presente em aplicações como vigilância automatizada, análise de imagens médicas e recuperação de informação. Dentre as abordagens existentes na literatura para resolver esse problema, destacam-se métodos baseados em reconhecimento de pontos-chave que podem ser interpretados como diferentes implementações de um mesmo arcabouço. O objetivo desta pesquisa de doutorado é desenvolver e avaliar uma versão generalizada desse arcabouço, na qual reconhecimento de pontos-chave é substituído por reconhecimento de grafos-chave. O potencial da pesquisa reside na riqueza de informação que um grafo pode apresentar antes e depois de ser reconhecido. A dificuldade da pesquisa reside nos problemas que podem ser causados por essa riqueza, como maldição da dimensionalidade e complexidade computacional. Três contribuições serão incluídas na tese: a descrição detalhada de um arcabouço para detecção de objetos baseado em grafos-chave, implementações fiéis que demonstram sua viabilidade e resultados experimentais que demonstram seu desempenho. / Object detection is a classic problem in computer vision, present in applications such as automated surveillance, medical image analysis and information retrieval. Among the existing approaches in the literature to solve this problem, we can highlight methods based on keypoint recognition that can be interpreted as different implementations of a same framework. The objective of this PhD thesis is to develop and evaluate a generalized version of this framework, on which keypoint recognition is replaced by keygraph recognition. The potential of the research resides in the information richness that a graph can present before and after being recognized. The difficulty of the research resides in the problems that can be caused by this richness, such as curse of dimensionality and computational complexity. Three contributions are included in the thesis: the detailed description of a keygraph-based framework for object detection, faithful implementations that demonstrate its feasibility and experimental results that demonstrate its performance.

Les systèmes de cartographie mobile sont de plus en plus utilisés pour la cartographie des scènes urbaines. La technologie de scan laser mobile (où le scanner est embarqué sur un véhicule) en particulier permet une cartographie précise de la voirie, la compréhension de la scène, la modélisation de façade, etc. Dans cette thèse, nous nous concentrons sur la détection de changement entre des nuages de points laser de cartographie mobile. Tout d'abord, nous étudions la détection des changements a partir de données RIEGL (scanner laser plan) pour la mise à jour de bases de données géographiques et l'identification d'objet temporaire. Nous présentons une méthode basée sur l'occupation de l'espace qui permet de surmonter les difficultés rencontrées par les méthodes classiques fondées sur la distance et qui ne sont pas robustes aux occultations et à l'échantillonnage anisotrope. Les zones occultées sont identifiées par la modélisation de l'état d'occupation de l'espace balayé par des faisceaux laser. Les écarts entre les points et les lignes de balayage sont interpolées en exploitant la géométrie du capteur dans laquelle la densité d'échantillonnage est isotrope. Malgré quelques limites dans le cas d'objets pénétrables comme des arbres ou des grilles, la méthode basée sur l'occupation est en mesure d'améliorer la méthode basée sur la distance point à triangle de façon significative. La méthode de détection de changement est ensuite appliquée à des données acquises par différents scanners laser et à différentes échelles temporelles afin de démontrer son large champs d'application. La géométrie d'acquisition est adaptée pour un scanner dynamique de type Velodyne. La méthode basée sur l'occupation permet alors la détection des objets en mouvement. Puisque la méthode détecte le changement en chaque point, les objets en mouvement sont détectés au niveau des points. Comme le scanner Velodyne scanne l'environnement de façon continue, les trajectoires des objets en mouvement peut être extraite. Un algorithme de détection et le suivi simultané est proposé afin de retrouver les trajectoires de piétons. Cela permet d'estimer avec précision la circulation des piétons des circulations douces dans les lieux publics. Les changements peuvent non seulement être détectés au niveau du point, mais aussi au niveau de l'objet. Ainsi nous avons pu étudier les changements entre des voitures stationnées dans les rues à différents moments de la journée afin d'en tirer des statistiques utiles aux gestionnaires du stationnement urbain. Dans ce cas, les voitures sont détectés en premier lieu, puis les voitures correspondantes sont comparées entre des passages à différents moments de la journée. Outre les changements de voitures, l'offre de stationnement et les types de voitures l'utilisant sont également des informations importantes pour la gestion du stationnement. Toutes ces informations sont extraites dans le cadre d'un apprentissage supervisé. En outre, une méthode de reconstruction de voiture sur la base d'un modèle déformable générique ajusté aux données est proposée afin de localiser précisément les voitures. Les paramètres du modèle sont également considérés comme caractéristiques de la voiture pour prendre de meilleures décisions. De plus, ces modèles géométriquement précis peuvent être utilisées à des fins de visualisation. Dans cette thèse, certains sujets liés à la détection des changements comme par exemple, suivi, la classification, et la modélisation sont étudiés et illustrés par des applications pratiques. Plus important encore, les méthodes de détection des changements sont appliquées à différentes géométries d'acquisition de données et à de multiples échelles temporelles et au travers de deux stratégies: “bottom-up” (en partant des points) et “top-down” (en partant des objets) / Mobile mapping systems are increasingly used for street environment mapping, especially mobile laser scanning technology enables precise street mapping, scene understanding, facade modelling, etc. In this research, the change detection from laser scanning point clouds is investigated. First of all, street environment change detection using RIEGL data is studied for the purpose of database updating and temporary object identification. An occupancy-based method is presented to overcome the challenges encountered by the conventional distance-based method, such as occlusion, anisotropic sampling. Occluded areas are identified by modelling the occupancy states within the laser scanning range. The gaps between points and scan lines are interpolated under the sensor reference framework, where the sampling density is isotropic. Even there are some conflicts on penetrable objects, e.g. trees, fences, the occupancy-based method is able to enhance the point-to-triangle distance-based method. The change detection method is also applied to data acquired by different laser scanners at different temporal-scales with the intention to have wider range of applications. The local sensor reference framework is adapted to Velodyne laser scanning geometry. The occupancy-based method is implemented to detection moving objects. Since the method detects the change of each point, moving objects are detect at point level. As the Velodyne scanner constantly scans the surroundings, the trajectories of moving objects can be detected. A simultaneous detection and tracking algorithm is proposed to recover the pedestrian trajectories in order to accurately estimate the traffic flow of pedestrian in public places. Changes can be detected not only at point level, but also at object level. The changes of cars parking on street sides at different times are detected to help regulate on-street car parking since the parking duration is limited. In this case, cars are detected in the first place, then they are compared with corresponding ones. Apart from car changes, parking positions and car types are also important information for parking management. All the processes are solved in a supervised learning framework. Furthermore, a model-based car reconstruction method is proposed to precisely locate cars. The model parameters are also treated as car features for better decision making. Moreover, the geometrically accurate models can be used for visualization purposes. Under the theme of change detection, related topics, e.g. tracking, classification, modelling, are also studied for the reason of practical applications. More importantly, the change detection methods are applied to different data acquisition geometries at multiple temporal-scales. Both bottom-up (point-based) and top-down (object-based) change detection strategies are investigated

La détection d'objets en mouvement représente une étape très importante pour de nombreuses applications telles que l'analyse du comportement humain pour la surveillance visuelle, la reconnaissance d'action par modèle, le suivi du trafic routier, etc. La soustraction d'arrière-plan est une approche populaire, mais difficile étant donnée qu'elle doit surmonter de nombreux obstacles, comme l'évolution dynamique du fond, les variations de luminosité, les occlusions, etc. Dans les travaux présentés, nous nous sommes intéressés à ce problème de segmentation objets/fond, avec une modélisation floue de type-2 pour gérer l'imprécision du modèle et des données. La méthode proposée modélise l'état de chaque pixel à l'aide d'un modèle de mélange de gaussiennes imprécis et évolutif, qui est exploité par plusieurs classifieurs flous pour finalement estimer la classe du pixel à chaque image. Plus précisément, cette décision prend en compte l'historique de son évolution, mais aussi son voisinage spatial et ses éventuels déplacements dans les images précédentes. Puis nous avons comparé la méthode proposée avec d'autres méthodes proches, notamment des méthodes basées sur un modèle de mélanges gaussiens, des méthodes basées floues, ou de type ACP. Cette comparaison nous a permis de situer notre méthode par rapport à l'existant et de proposer quelques perspectives à ce travail. / Moving objects detection is a very important step for many applications such as human behavior analysis surveillance, model-based action recognition, road traffic monitoring, etc. Background subtraction is a popular approach, but difficult given that it must overcome many obstacles, such as dynamic background changes, brightness variations, occlusions, and so on. In the presented works, we focused on this problem of objects/background segmentation, using a type-2 fuzzy modeling to manage the inaccuracy of the model and the data. The proposed method models the state of each pixel using an imprecise and scalable Gaussian mixture model, which is exploited by several fuzzy classifiers to ultimately estimate the pixel class at each image. More precisely, this decision takes into account the history of its evolution, but also its spatial neighborhood and its possible displacements in the preceding images. Then we compared the proposed method with other close methods, including methods based on a gaussian mixture model, fuzzy based methods, or ACP type methods. This comparison allowed us to assess its good performances, and to propose some perspectives to this work.

<div>Plant Phenotyping is a set of methodologies for measuring and analyzing characteristic traits of a plant. While traditional plant phenotyping techniques are labor-intensive and destructive, modern imaging technologies have provided faster, non-invasive, and more cost-effective capabilities for plant phenotyping. Among different image-based phenotyping platforms, I focus on phenotyping with image data captured by Unmanned Aerial Vehicle (UAV) and ground vehicles. The crop plant used in my study is sorghum [Sorghum bicolor (L.) Moench]. In this thesis, I present multiple methods to estimate plot-level and plant-level plant traits from data collected by various platforms, including UAV and ground vehicles. I propose an image plant phenotyping system that provides end-to-end RGB data analysis for plant scientists. I describe a plant segmentation method using HSV color information. I introduce two methods to locate the center of the plants using Multiple Instance Learning (MIL) and Convolutional Neural Networks (CNN). I present three methods to segment individual leaves by shape-based approaches in both Cartesian coordinates and Polar coordinates. I propose a method to estimate leaf length and width for overhead leaf images. I describe a method to estimate leaf angle from data collected by a modified wheel-based sprayer with a sensor boom vehicle, Phenorover. Methods are tested and verified on image data collected by UAV and ground vehicle platforms in sorghum fields in West Lafayette, Indiana, USA. Estimated phenotypic traits include plant locations, the number of plants per plot, leaf area, canopy cover, Leaf Area Index (LAI), leaf count, leaf angle, leaf length, and leaf width.</div>

The organization International Aid Services (IAS) provides people in East Africawith clean water through well drilling. The wells are located in surroundingsfar away for the investors to inspect and therefore IAS wishes to be able to monitortheir wells to get a better overview if different types of improvements needto be made. To see the load on different water sources at different times of theday and during the year, and to know how many people that are visiting thewells, is of particular interest. In this paper, a method is proposed for countingpeople around the wells. The goal is to choose a suitable method for detectinghumans in images and evaluate how it performs. The area of counting humansin images is not a new topic, though it needs to be taken into account that thesituation implies some restrictions. A Raspberry Pi with an associated camerais used, which is a small embedded system that cannot handle large and complexsoftware. There is also a limited amount of data in the project. The methodproposed in this project uses a pre-trained convolutional neural network basedobject detector called the Single Shot Detector, which is adapted to suit smallerdevices and applications. The pre-trained network that it is based on is calledMobileNet, a network that is developed to be used on smaller systems. To see howgood the chosen detector performs it will be compared with some other models.Among them a detector based on the Inception network, a significantly larger networkthan the MobileNet. The base network is modified by transfer learning.Results shows that a fine-tuned and modified network can achieve better result,from a F1-score of 0.49 for a non-fine-tuned model to 0.66 for the fine-tuned one.

Cette thèse est centrée sur le problème de la détection d'objets saillants et de leur segmentation dans une vidéo en vue de détecter les objets les plus attractifs ou d'affecter des identités cohérentes d'objets à chaque pixel d'une séquence vidéo. Concernant la détection d'objets saillants dans vidéo, outre une revue des techniques existantes, une nouvelle approche et l'extension d'un modèle sont proposées; de plus une approche est proposée pour la segmentation d'instances d'objets vidéo. Pour la détection d'objets saillants dans une vidéo, nous proposons : (1) une approche traditionnelle pour détecter l'objet saillant dans sa totalité à l'aide de la notion de "bordures virtuelles". Un filtre guidé est appliqué sur la sortie temporelle pour intégrer les informations de bord spatial en vue d'une meilleure détection des bords de l'objet saillants. Une carte globale de saillance spatio-temporelle est obtenue en combinant la carte de saillance spatiale et la carte de saillance temporelle en fonction de l'entropie. (2) Une revue des développements récents des méthodes basées sur l'apprentissage profond est réalisée. Elle inclut les classifications des méthodes de l'état de l'art et de leurs architectures, ainsi qu'une étude expérimentale comparative de leurs performances. (3) Une extension d'un modèle de l'approche traditionnelle proposée en intégrant un procédé de détection d'objet saillant d'image basé sur l'apprentissage profond a permis d'améliorer encore les performances. Pour la segmentation des instances d'objets dans une vidéo, nous proposons une approche d'apprentissage profond dans laquelle le calcul de la confiance de déformation détermine d'abord la confiance de la carte masquée, puis une sélection sémantique est optimisée pour améliorer la carte déformée, où l'objet est réidentifié à l'aide de l'étiquettes sémantique de l'objet cible. Les approches proposées ont été évaluées sur des jeux de données complexes et de grande taille disponibles publiquement et les résultats expérimentaux montrent que les approches proposées sont plus performantes que les méthodes de l'état de l'art. / This thesis focuses on the problem of video salient object detection and video object instance segmentation which aim to detect the most attracting objects or assign consistent object IDs to each pixel in a video sequence. One approach, one overview and one extended model are proposed for video salient object detection, and one approach is proposed for video object instance segmentation. For video salient object detection, we propose: (1) one traditional approach to detect the whole salient object via the adjunction of virtual borders. A guided filter is applied on the temporal output to integrate the spatial edge information for a better detection of the salient object edges. A global spatio-temporal saliency map is obtained by combining the spatial saliency map and the temporal saliency map together according to the entropy. (2) An overview of recent developments for deep-learning based methods is provided. It includes the classifications of the state-of-the-art methods and their frameworks, and the experimental comparison of the performances of the state-of-the-art methods. (3) One extended model further improves the performance of the proposed traditional approach by integrating a deep-learning based image salient object detection method For video object instance segmentation, we propose a deep-learning approach in which the warping confidence computation firstly judges the confidence of the mask warped map, then a semantic selection is introduced to optimize the warped map, where the object is re-identified using the semantics labels of the target object. The proposed approaches have been assessed on the published large-scale and challenging datasets. The experimental results show that the proposed approaches outperform the state-of-the-art methods.

A suite of new collision avoidance systems (CAS) is presented for use in heavy vehicles whose structure and size necessarily impede driver visibility is introduced. The main goal of the project is to determine the appropriate use of each of the commercially available technologies and, where possible, produce a low cost variant suitable for use in proximity detection on large mining industry haul trucks. CAS variants produced were subjected to a field demonstration and, linked to the output from the earlier CAS 1 project, (a production high-definition in-cabin video monitor and r/f tagging system). The CAS 2 system used low cost Doppler continuous wave radar antennae coupled to the CAS 1 monitor to indicate the presence of an object moving at any speed above 3 Km/h relative to the antennae. The novelty of the CAS 3 system lies in the design of 3 interconnected, modules. The modules are 8 radar antennae (as used in CAS 2) modules located on the truck, software to interface with the end user (i.e. the drivers of the trucks) and a display unit. Modularisation enables the components to be independently tested, evaluated and replaced when in use. The radar antennae modules and the system as a whole are described together with the empirical tests conducted and results obtained. The tests, drawing on Monte-Carlo simulation techniques, demonstrate both the 'correctness' of the implementations and the effectiveness of the system. The results of the testing of the final prototype unit were highly successful both as a computer simulation level and in practical tests on light vehicles. A number of points, (as a consequence of the field test), are reviewed and their application to future projects discussed.