Harness racing horses are exposed to high workload and consequently, they are at risk of joint injuries and lameness. In recent years, the interest in applications to improve animal welfare has increased and there is a demand for objective assessment methods that can enable early and robust diagnosis of injuries. In this thesis, experiments were conducted on video recordings collected by a helmet camera mounted on the driver of a sulky. The aim was to take the first steps toward equine gait analysis by investigating how semantic segmentation and 3D reconstruction of such data could be performed. Since these were the first experiments made on this data, no expectations of the results existed in advance. Manual pixel-wise annotations were created on a small set of extracted frames and a deep learning model for semantic segmentation was trained to localize the horse, as well as the sulky and reins. The results are promising and could probably be further improved by expanding the annotated dataset and using a larger image resolution. Structure-from-motion using COLMAP was performed to estimate the camera motion in part of a video recording. A method to filter out dynamic objects based on masks created from predicted segmentation maps was investigated and the results showed that the reconstruction was part-wise successful, but struggled when dynamic objects were not filtered out and when the equipage was moving at high speed along a straight stretch. Overall the results are promising, but further development needs to be conducted to ensure robustness and conclude whether data collected by the investigated helmet camera configuration is suitable for equine gait analysis.

This thesis addresses the problem of improving the delineation of skylines, also referred to as skyline detection, in occluded and challenging environments where existing skyline delineation methods may struggle or fail. Delineated skylines can be used in monocular camera localization methods by comparing delineated skylines to digital elevation model data to estimate a position based on known terrain. This is particularly useful in GPS-denied environments in which active sensing is either impractical or undesirable for various reasons, so that passive sensing using monocular cameras is necessary and/or strategically advantageous. This thesis presents a novel method of skyline delineation using deep learning-based semantic segmentation of monocular camera images to detect natural skylines of distant landscapes in the presence of occlusions. Skylines are extracted from semantic segmentation predictions as the boundary between pixel clusters labeled as terrain to those labeled as sky, with additional segmentation classes representing the known set of potential occlusions in a given environment. Additionally, each pixel in the detected skyline contours are assigned a confidence score based on local intensity gradients to reduce the potential impacts of erroneous skyline contours on position estimation. The utility of these delineated skylines is demonstrated by obtaining orientation and position estimates using existing methods of skyline-based localization. In these methods, the delineated natural skyline is compared to rendered skylines using digital elevation model data and the position estimate is obtained by finding the closest match. Results from the proposed skyline delineation method using semantic segmentation, with accompanying localization demonstration, is presented on two distinct data sets. The first is obtained from the Perseverance Rover operating in the Jezero Crater region of Mars, and the second is obtained from an uncrewed surface vessel operating in the Gulf of Koper, Slovenia. / Denna avhandling tar upp problemet med att förbättra avgränsningen av skylines, även kallad skylinedetektion, i tilltäppta och utmanande miljöer där befintliga skylineavgränsningsmetoder kan kämpa eller misslyckas. Avgränsade skylines kan användas i monokulära kameralokaliseringsmetoder genom att jämföra avgränsade skylines med digitala höjdmodelldata för att uppskatta en position baserat på känd terräng. Detta är särskilt användbart i GPS-nekas miljöer där aktiv avkänning är antingen opraktisk eller oönskad av olika skäl, så att passiv avkänning med användning av monokulära kameror är nödvändig och/eller strategiskt fördelaktig. Denna avhandling presenterar en ny metod för skylineavgränsning med användning av djupinlärningsbaserad semantisk segmentering av monokulära kamerabilder för att detektera naturliga skylines av avlägsna landskap i närvaro av ocklusioner. Horisonter extraheras från semantiska segmenteringsförutsägelser som gränsen mellan pixelkluster märkta som terräng till de märkta som himmel, med ytterligare segmenteringsklasser som representerar den kända uppsättningen potentiella ocklusioner i en given miljö. Dessutom tilldelas varje pixel i de detekterade skylinekonturerna ett konfidenspoäng baserat på lokala intensitetsgradienter för att minska den potentiella påverkan av felaktiga skylinekonturer på positionsuppskattning. Användbarheten av dessa avgränsade skylines demonstreras genom att erhålla orienterings- och positionsuppskattningar med hjälp av befintliga metoder för skylinebaserad lokalisering. I dessa metoder jämförs den avgränsade naturliga horisonten med renderade silhuetter med hjälp av digitala höjdmodelldata och positionsuppskattningen erhålls genom att hitta den närmaste matchningen. Resultat från den föreslagna metoden för skylineavgränsning med semantisk segmentering, med tillhörande lokaliseringsdemonstration, presenteras på två distinkta datamängder. Den första kommer från Perseverance Rover som verkar i Jezero Crater-regionen på Mars, och den andra erhålls från ett obemannat ytfartyg som verkar i Koperbukten, Slovenien.

Learning from unlabeled data is a topic of critical significance in machine learning, as the large datasets required to train ever-growing models are costly and impractical to annotate. Semi-Supervised Learning (SSL) methods aim to learn from a few labels and a large unlabeled dataset. In another approach, Domain Adaptation (DA) leverages data from a similar source domain to train a model for a target domain. This thesis focuses on Semi-Supervised Domain Adaptation (SSDA) for the dense task of semantic segmentation, where labels are particularly costly to obtain. SSDA has not received much attention yet, even though it has a great potential and represents a realistic scenario. The few existing SSDA methods for semantic segmentation reuse ideas from Unsupervised DA, despite the di↵erences between the two settings. This thesis proposes a new semantic segmentation framework designed particularly for the SSDA setting. The approach followed was to forego domain alignment and focus instead on enhancing clusterability of target domain features, an idea from SSL. The method is based on consistency regularization, combined with pixel contrastive learning and self-training. The proposed framework is found to be e↵ective not only in SSDA, but also in SSL. Ultimately, a unified solution for SSL and SSDA semantic segmentation is presented. Experiments were conducted on the target dataset of Cityscapes and source dataset of GTA5. The method proposed is competitive in both SSL and SSDA, and sets a new state-of-the-art for SSDA achieving a 65.6% mIoU (+4.4) on Cityscapes with 100 labeled samples. This thesis has an immediate impact on practical applications by proposing a new best-performing framework for the under-explored setting of SSDA. Furthermore, it also contributes towards the more ambitious goal of designing a unified solution for learning from unlabeled data. / Inlärning med hjälp av omärkt data är ett område av stor vikt inom maskininlärning. Detta på grund av att de stora datamängder som blivit nödvändiga för att träna konstant växande modeller både är kostsamma och opraktiska att implementera. Målet med Semi-Supervised Learning (SSL) är att kombinera ett fåtal etiketter med en stor mängd omärkt data för inlärning. Som ett annat tillvägagångssätt använder Domain Adaptation (DA) data från en liknande domän för att träna en annan måldomän. I Denna avhandling används Semi-Supervised Domain Adaptation (SSDA) för att utföra sådan semantisk segmentering, i vilken etiketter är särskilt kostsamma att erhålla. SSDA är ännu inte genererat mycket uppmärksamhet, även om det har en stor potential och representerar ett realistiskt scenario. De få metoder av SSDA som existerar för semantisk segmentering återanvänder idéer från Unsupervised DA, trots de olikheter som finns mellan de två modellerna. Denna avhandling föreslår ett nytt ramverk för semantisk segmentering, designat speciellt för SSDA modellen. Detta genom att försaka domänanpassning och i stället fokusera på att förbättra klusterbarheten av måldomänens egenskaper, en idé tagen från SSL. Metoden är baserad på konsistensregularisering, i kombination med pixelkontrastinlärning och självinlärning. Det föreslagna ramverket visar sig vara effektivt, inte bara för SSDA, men även för SSL. Till slut presenteras en enad lösning för semantisk segmentering med SLL och SSDA. Experiment utfördes på måldata från Cityscapes samt källdata från GTA5. Den föreslagna metoden är konkurrenskraftig både för SSL och SSDA, och blir världsledande för SSDA genom att uppnå 65,6% mIoU (+4,4) för Cityscapes med 100 märkta testdata. Denna avhandling har en omedelbar effekt gällande praktiska applikationer genom att föreslå ett nytt ”bäst resulterande” ramverk för dåligt utforskade inställningar av SSDA. Till yttermera visso bidrar avhandlingen även till det mer ambitiösa målet att designa en enad lösning för maskininlärning från omärkta data.

La télédétection des forêts est devenue de plus en plus accessible grâce à l'utilisation de véhicules aériens inoccupés (UAV) et à l'apprentissage profond, ce qui permet d'obtenir des images répétées à haute résolution et d’observer les changements phénologiques à des échelles spatiales et temporelles plus importantes. Dans les forêts tempérées, à l'automne, la sénescence des feuilles se produit lorsque les feuilles changent de couleur et tombent. Cependant, l'influence de la sénescence foliaire sur la segmentation des espèces d'arbres à l'aide d'un réseau neuronal convolutif (CNN) n'a pas encore été évaluée. Nous avons acquis de l’imagerie haute résolution par UAV au-dessus d’une forêt tempérée au Québec à sept reprises entre mai et octobre 2021. Nous avons segmenté et identifié 23 000 couronnes d'arbres de 14 classes différentes pour entraîner et valider un CNN pour chaque acquisition d'imagerie. La meilleure segmentation (F1-score le plus élevé) était au début de la coloration des feuilles (début septembre) et le F1-score le plus bas au pic de la coloration automnale (début octobre). La chronologie de la sénescence varie considérablement d’une espèce à l’autre et au sein d’une même espèce, ce qui entraîne une grande variabilité du signal télédétecté. Les espèces d'arbres à feuilles caduques et à feuilles persistantes qui présentaient des traits distinctifs et moins variables dans le temps entre les individus ont été mieux classées. Bien que la segmentation des arbres dans une forêt hétérogène demeure un défi, l'imagerie UAV et l'apprentissage profond démontrent un grand potentiel pour la cartographie des espèces d'arbres. Les résultats obtenus dans une forêt tempérée où la couleur des feuilles change fortement pendant la sénescence automnale montrent que la meilleure performance pour la segmentation des espèces d'arbres se produit au début de ce changement de couleur. / Remote sensing of forests has become increasingly accessible with the use of unoccupied aerial vehicles (UAV), along with deep learning, allowing for repeated high-resolution imagery and the capturing of phenological changes at larger spatial and temporal scales. In temperate forests during autumn, leaf senescence occurs when leaves change colour and drop. However, the influence of leaf senescence in temperate forests on tree species segmentation using a Convolutional Neural Network (CNN) has not yet been evaluated. Here, we acquired high-resolution UAV imagery over a temperate forest in Quebec, Canada on seven occasions between May and October 2021. We segmented and labelled 23,000 tree crowns from 14 different classes to train and validate a CNN for each imagery acquisition. The CNN-based segmentation showed the highest F1-score (0.72) at the start of leaf colouring in early September and the lowest F1-score (0.61) at peak fall colouring in early October. The timing of the events occurring during senescence, such as leaf colouring and leaf fall, varied substantially between and within species and according to environmental conditions, leading to higher variability in the remotely sensed signal. Deciduous and evergreen tree species that presented distinctive and less temporally-variable traits between individuals were better classified. While tree segmentation in a heterogenous forest remains challenging, UAV imagery and deep learning show high potential in mapping tree species. Our results from a temperate forest with strong leaf colour changes during autumn senescence show that the best performance for tree species segmentation occurs at the onset of this colour change.

The information era has led the manufacturer of trucks and logistics solution providers are inclined towards software as a service (SAAS) based solutions. With advancements in software technologies like artificial intelligence and deep learning, the domain of computer vision has achieved significant performance boosts that it competes with hardware based solutions. Firstly, data is collected from a large number of sensors which can increase production costs and carbon footprint in the environment. Secondly certain useful physical quantities/variables are impossible to measure or turns out to be very expensive solution. So in this dissertation, we are investigating the feasibility of providing the similar solution using a single sensor (dashboard- camera) to measure multiple variables. This provides a sustainable solution even when scaled up in huge fleets. The video frames that can be collected from the visual perception of the truck (i.e. the on-board camera of the truck) is processed by the deep learning techniques and operational data can be extracted. Certain techniques like the image classification and semantic segmentation outputs were experimented and shows potential to replace costly hardware counterparts like Lidar or radar based solutions. / Informationstiden har lett till att tillverkare av lastbilar och logistiklösningsleve -rantörer är benägna mot mjukvara som en tjänst (SAAS) baserade lösningar. Med framsteg inom mjukvaruteknik som artificiell intelligens och djupinlärnin har domänen för datorsyn uppnått betydande prestationsförstärkningar att konkurrera med hårdvarubaserade lösningar. För det första samlas data in från ett stort antal sensorer som kan öka produktionskostnaderna och koldioxidavtry -cket i miljön. För det andra är vissa användbara fysiska kvantiteter / variabler omöjliga att mäta eller visar sig vara en mycket dyr lösning. Så i denna avhandling undersöker vi möjligheten att tillhandahålla liknande lösning med hjälp av en enda sensor (instrumentbrädkamera) för att mäta flera variabler. Detta ger en hållbar lösning även när den skalas upp i stora flottor. Videoramar som kan samlas in från truckens visuella uppfattning (dvs. lastbilens inbyggda kamera) bearbetas av djupinlärningsteknikerna och operativa data kan extraher -as. Vissa tekniker som bildklassificering och semantiska segmenteringsutgång -ar experimenterades och visar potential att ersätta dyra hårdvaruprojekt som Lidar eller radarbaserade lösningar.

No description available.

[pt] A eficiência de culturas pode ser aprimorada monitorando-se suas condições de forma contínua e tomando-se decisões baseadas em suas análises. Os dados para análise podem ser obtidos através de sensores de imagens e o processo de monitoramento pode ser automatizado utilizando-se algoritmos de reconhecimento de imagem com diferentes níveis de complexidade. Alguns dos algoritmos de maior êxito estão relacionados a abordagens supervisionadas de aprendizagem profunda (Deep Learning) as quais utilizam formas de Redes Neurais de Convolucionais (CNNs). Nesta dissertação de mestrado, empregaram-se modelos de aprendizagem profunda supervisionados para classificação, regressão, detecção de objetos e segmentação semântica em tarefas de monitoramento de culturas, utilizando-se amostras de imagens obtidas através de três níveis distintos: Satélites, Veículos Aéreos Não Tripulados (UAVs) e Robôs Terrestres Móveis (MLRs). Ambos satélites e UAVs envolvem o uso de imagens multiespectrais. Para o primeiro nível, implementou-se um modelo CNN baseado em Transfer Learning para a classificação de espécies vegetativas. Aprimorou-se o desempenho de aprendizagem do transfer learning através de um método de análise estatística recentemente proposto. Na sequência, para o segundo nível, implementou-se um algoritmo segmentação semântica multitarefa para a detecção de lavouras de cana-de-açúcar e identificação de seus estados (por exemplo, saúde e idade da cultura). O algoritmo também detecta a vegetação ao redor das lavouras, sendo relevante na busca por ervas daninhas. No terceiro nível, implementou-se um algoritmo Single Shot Multibox Detector para detecção de cachos de tomate. De forma a avaliar o estado dos cachos, utilizaram-se duas abordagens diferentes: uma implementação baseada em segmentação de imagens e uma CNN supervisionada adaptada para cálculos de regressão capaz de estimar a maturação dos cachos de tomate. De forma a quantificar cachos de tomate em vídeos para diferentes estágios de maturação, empregou-se uma implementação de Região de Interesse e propôs-se um sistema de rastreamento o qual utiliza informações temporais. Para todos os três níveis, apresentaram-se soluções e resultados os quais superam as linhas de base do estado da arte. / [en] Crop efficiency can be improved by continually monitoring their state and making decisions based on their analysis. The data for analysis can be obtained through images sensors and the monitoring process can be automated by using image recognition algorithms with different levels of complexity. Some of the most successful algorithms are related to supervised Deep Learning approaches which use a form of Convolutional Neural Networks (CNNs). In this master s dissertation, we employ supervised deep learning models for classification, regression, object detection, and semantic segmentation in crop monitoring tasks, using image samples obtained through three different levels: Satellites, Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs). Both satellites and UAVs levels involve the use of multispectral images. For the first level, we implement a CNN model based on transfer learning to classify vegetative species. We also improve the transfer learning performance by a newly proposed statistical analysis method. Next, for the second level, we implement a multi-task semantic segmentation algorithm to detect sugarcane crops and infer their state (e.g. crop health and age). The algorithm also detects the surrounding vegetation, being relevant in the search for weeds. In the third level, we implement a Single Shot Multibox detector algorithm to detect tomato clusters. To evaluate the cluster s state, we use two different approaches: an implementation based on image segmentation and a supervised CNN regressor capable of estimating their maturity. In order to quantify the tomato clusters in videos at different maturation stages, we employ a Region of Interest implementation and also a proposed tracking system which uses temporal information. For all the three levels, we present solutions and results that outperform state-of-the art baselines.

<p>Emerging mobile mapping systems include a wide range of platforms, for instance, manned aircraft, unmanned aerial vehicles (UAV), terrestrial systems like trucks, tractors, robots, and backpacks, that can carry multiple sensors including LiDAR scanners, cameras, and georeferencing units. Such systems can maneuver in the field to quickly collect high-resolution data, capturing detailed information over an area of interest. With the increased volume and distinct characteristics of the data collected, practical quality control procedures that assess the agreement within/among datasets acquired by various sensors/systems at different times are crucial for accurate, robust interpretation. Moreover, the ability to derive semantic information from acquired data is the key to leveraging the complementary information captured by mobile mapping systems for diverse applications. This dissertation addresses these challenges for different systems (airborne and terrestrial), environments (urban and rural), and applications (agriculture, archaeology, hydraulics/hydrology, and transportation).</p> <p>In this dissertation, quality control procedures that utilize features automatically identified and extracted from acquired data are developed to evaluate the relative accuracy between multiple datasets. The proposed procedures do not rely on manually deployed ground control points or targets and can handle challenging environments such as coastal areas or agricultural fields. Moreover, considering the varying characteristics of acquired data, this dissertation improves several data processing/analysis techniques essential for meeting the needs of various applications. An existing ground filtering algorithm is modified to deal with variation in point density; digital surface model (DSM) smoothing and seamline control techniques are proposed for improving the orthophoto quality in agricultural fields. Finally, this dissertation derives semantic information for diverse applications, including 1) shoreline retreat quantification, 2) automated row/alley detection for plant phenotyping, 3) enhancement of orthophoto quality for tassel/panicle detection, and 4) point cloud semantic segmentation for mapping transportation corridors. The proposed approaches are tested using multiple datasets from UAV and wheel-based mobile mapping systems. Experimental results verify that the proposed approaches can effectively assess the data quality and provide reliable interpretation. This dissertation highlights the potential of modern mobile mapping systems to map challenging environments for a variety of applications.</p>

Hand segmentation is a fundamental part of many computer vision systems aimed at gesture recognition or hand tracking. In particular, augmented reality solutions need a very accurate gesture analysis system in order to satisfy the end consumers in an appropriate manner. Therefore the hand segmentation step is critical. Segmentation is a well-known problem in image processing, being the process to divide a digital image into multiple regions with pixels of similar qualities. Classify what pixels belong to the hand and which ones belong to the background need to be performed within a real-time performance and a reasonable computational complexity. While in the past mainly light-weight probabilistic and machine learning approaches were used, this work investigates the challenges of real-time hand segmentation achieved through several deep learning techniques. Is it possible or not to improve current state-of-theart segmentation systems for smartphone applications? Several models are tested and compared based on accuracy and processing speed. Transfer learning-like approach leads the method of this work since many architectures were built just for generic semantic segmentation or for particular applications such as autonomous driving. Great effort is spent on organizing a solid and generalized dataset of hands, exploiting the existing ones and data collected by ManoMotion AB. Since the first aim was to obtain a really accurate hand segmentation, in the end, RefineNet architecture is selected and both quantitative and qualitative evaluations are performed, considering its advantages and analysing the problems related to the computational time which could be improved in the future. / Handsegmentering är en grundläggande del av många datorvisionssystem som syftar till gestigenkänning eller handspårning. I synnerhet behöver förstärkta verklighetslösningar ett mycket exakt gestanalyssystem för att tillfredsställa slutkonsumenterna på ett lämpligt sätt. Därför är handsegmenteringssteget kritiskt. Segmentering är ett välkänt problem vid bildbehandling, det vill säga processen att dela en digital bild i flera regioner med pixlar av liknande kvaliteter. Klassificera vilka pixlar som tillhör handen och vilka som hör till bakgrunden måste utföras i realtidsprestanda och rimlig beräkningskomplexitet. Medan tidigare använts huvudsakligen lättviktiga probabilistiska metoder och maskininlärningsmetoder, undersöker detta arbete utmaningarna med realtidshandsegmentering uppnådd genom flera djupinlärningstekniker. Är det möjligt eller inte att förbättra nuvarande toppmoderna segmenteringssystem för smartphone-applikationer? Flera modeller testas och jämförs baserat på noggrannhet och processhastighet. Transfer learning-liknande metoden leder metoden för detta arbete eftersom många arkitekturer byggdes bara för generisk semantisk segmentering eller för specifika applikationer som autonom körning. Stora ansträngningar läggs på att organisera en gedigen och generaliserad uppsättning händer, utnyttja befintliga och data som samlats in av ManoMotion AB. Eftersom det första syftet var att få en riktigt exakt handsegmentering, väljs i slutändan RefineNetarkitekturen och både kvantitativa och kvalitativa utvärderingar utförs med beaktande av fördelarna med det och analys av problemen relaterade till beräkningstiden som kan förbättras i framtiden.

In recent years the wide availability of high-resolution satellite images has made the remote monitoring of water resources all over the world possible. While the detection of open water from satellite images is relatively easy, a significant percentage of the water extent of wetlands is covered by vegetation. Convolutional Neural Networks have shown great success in the task of detecting wetlands in satellite images. However, these models require large amounts of manually annotated satellite images, which are slow and expensive to produce. In this paper we use self-supervised training methods to train a Convolutional Neural Network to detect water from satellite images without the use of annotated data. We use a combination of deep clustering and negative sampling based on the paper ”Unsupervised Single-Scene Semantic Segmentation for Earth Observation”, and we expand the paper by changing the clustering loss, the model architecture and implementing an ensemble model. Our final ensemble of self-supervised models outperforms a single supervised model, showing the power of self-supervision. / Under de senaste åren har den breda tillgången på högupplösta satellitbilder möjliggjort fjärrövervakning av vattenresurser över hela världen. Även om det är relativt enkelt att upptäcka öppet vatten från satellitbilder, täcks en betydande andel av våtmarkernas vattenutbredning av vegetation. Lyckligtvis kan radarsignaler tränga igenom vegetation, vilket gör det möjligt för oss att upptäcka vatten gömt under vegetation från satellitradarbilder. Under de senaste åren har Convolutional Neural Networks visat stor framgång i denna uppgift. Tyvärr kräver dessa modeller stora mängder manuellt annoterade satellitbilder, vilket är långsamt och dyrt att producera. Självövervakad inlärning är ett område inom maskininlärning som syftar till att träna modeller utan användning av annoterade data. I den här artikeln använder vi självövervakad träningsmetoder för att träna en Convolutional Neural Network-baserad modell för att detektera vatten från satellitbilder utan användning av annoterade data. Vi använder en kombination av djup klustring och kontrastivt lärande baserat på artikeln ”Unsupervised Single-Scene Semantic Segmentation for Earth Observation”. Dessutom utökar vi uppsatsen genom att modifiera klustringsförlusten och modellarkitekturen som används. Efter att ha observerat hög varians i våra modellers prestanda implementerade vi också en ensemblevariant av vår modell för att få mer konsekventa resultat. Vår slutliga ensemble av självövervakade modeller överträffar en enda övervakad modell, vilket visar kraften i självövervakning.