Autonomous driving requires lightweight and robust perception systems that can rapidly and accurately interpret the complex driving environment. This dissertation investigates the transformative capacity of discrete wavelet transform (DWT), inverse DWT, CNNs, and transformers as foundational elements to develop lightweight perception architectures for autonomous vehicles. The inherent properties of DWT, including its invertibility, sparsity, time-frequency localization, and ability to capture multi-scale information, present an inductive bias. Similarly, transformers capture long-range dependency between features. By harnessing these attributes, novel wavelet-enhanced deep learning architectures are introduced. The first contribution is introducing a lightweight backbone network that can be employed for real-time processing. This network balances processing speed and accuracy, outperforming established models like ResNet-50 and VGG16 in terms of accuracy while remaining computationally efficient. Moreover, a multiresolution attention mechanism is introduced for CNNs to enhance feature extraction. This mechanism directs the network's focus toward crucial features while suppressing less significant ones. Likewise, a transformer model is proposed by leveraging the properties of DWT with vision transformers. The proposed wavelet-based transformer utilizes the convolution theorem in the frequency domain to mitigate the computational burden on vision transformers caused by multi-head self-attention. Furthermore, a proposed wavelet-multiresolution-analysis-based 3D object detection model exploits DWT's invertibility, ensuring comprehensive environmental information capture. Lastly, a multimodal fusion model is presented to use information from multiple sensors. Sensors have limitations, and there is no one-fits-all sensor for specific applications. Therefore, multimodal fusion is proposed to use the best out of different sensors. Using a transformer to capture long-range feature dependencies, this model effectively fuses the depth cues from LiDAR with the rich texture derived from cameras. The multimodal fusion model is a promising approach that integrates backbone networks and transformers to achieve lightweight and competitive results for 3D object detection. Moreover, the proposed model utilizes various network optimization methods, including pruning, quantization, and quantization-aware training, to minimize the computational load while maintaining optimal performance. The experimental results across various datasets for classification networks, attention mechanisms, 3D object detection, and multimodal fusion indicate a promising direction in developing a lightweight and robust perception system for robotics, particularly in autonomous driving.

Tato práce se zabývá návrhem nových algoritmů pro zpracování řídkých 3D dat senzorů LiDAR, včetně kompletního návrhu batohovího mobilního mapovacího řešení. Tento výzkum byl motivován potřebou takových řešení v oblasti geodézie, mobilního průzkumu a výstavby. Nejprve je prezentován iterační algoritmus pro spolehlivou registraci mračen bodů a odhad odometrie z měření 3D LiDARu. Problém řídkosti a velikosti těchto dat je řešen pomocí náhodného vzorkování pomocí Collar Line Segments (CLS). Vyhodnocení na standardní datové sadě KITTI ukázalo vynikající přesnost oproti známému algoritmu General ICP. Konvoluční neuronové sítě hrají důležitou roli ve druhé metodě odhadu odometrie, která zpracovává kódovaná data LiDARu do 2D matic. Metoda je schopna online výkonu, zatímco je zachována přesnost, když požadujeme pouze parametry posunu. To může být užitečné v situacích, kdy je vyžadován online náhled mapování a parametry rotace mohou být spolehlivě poskytnuty např. senzorem IMU. Na základě algoritmu CLS bylo navrženo a implementováno batohové mobilní mapovací řešení 4RECON. S využitím kalibrovaného a synchronizovaného páru LiDARů Velodyne a s nasazením řešení GNSS/INS s duální anténou, byl vyvinut univerzální systém poskytující přesné 3D modelování malých vnitřních i velkých otevřených prostředí. Naše hodnocení prokázalo, že požadavky stanovené pro tento systém byly splněny -- relativní přesnost do $5$~cm a průměrná chyba georeferencí pod $12$~cm. Poslední stránky obsahují popis a vyhodnocení další metody založené na konvolučních neuronových sítích -- navržených pro segmentaci země v mračnech bodů 3D LiDARu. Tato metoda překonala současný stav techniky v této oblasti a představuje způsob, jakým může být sémantická informace vložena do 3D laserových dat.

No description available.

This thesis investigates the use of sampled CAD models for training and calibrating a semantic segmentation model, RandLA-Net, with the ultimate goal of localizing modules for digital twinning (the process of creating digital twins). A significant contribution is the development of the Random Placement of Component Generator (RPCG), a synthetic dataset generator that randomly places CAD models within scenes while preserving contextual information such as typical height above ground. Training and testing on datasets generated by RPCG demonstrated its ability to recognize class modules in various randomly generated scenes. Various hyperparameters related to the loss function and pre-processing steps were explored to improve RandLA-Net’s generalization to different contextual settings. Notably, using a class-weighted α in the focal loss showed promise in correctly classifying infrequent classes and reducing network overconfidence under domain shifts with similar prior probability distributions. The semantic segmentation results were promising for the RPCG test set, achieving a mean True Positive Rate (mTPR) of 98% and a mean Intersection over Union(mIoU) of 93.6%. However, the performance on a sampled version of a CAD model representing an installation named Undercentral was comparatively lower, with a mTPR of 41.1% and a mIoU of 33.4%, indicating the need for further adaptation to varied contextual environments. Proposed improvements include enhancing RPCG with an occupancy grid to better simulate compact scenes and evaluating different subsampling rates in RandLA-Net’s random sampling layers. For confidence calibration, the thesis finds that averaging multiple Monte Carlo (MC) dropout evaluations effectively reduces network overconfidence and improves model reliability. Although this work addresses only a portion of the overall digital twinning process, it highlights the potential of synthetic data generation in enhancing semantic segmentation models and contributes towards the localization of modules in digital twin creation.

[pt] No presente trabalho propomos uma nova forma de extrair a silhueta de uma nuvem de pontos, via reconstrução local de uma superfície descrita implicitamente por uma função polinomial. Esta reconstrução é baseada nos métodos Gradient one fitting e Ridge regression. A curva silhueta fica definida implicitamente por um sistema de equações não-lineares e sua geração é feita por continuação numérica. Como resultado, verificamos que nosso método se mostrou adequado para tratar dados com ruídos. Além disso, apresentamos um método para a extração local de linhas laplacianas de uma nuvem de pontos baseado na reconstrução local utilizando a triangulação de Delaunay. / [en] In this work we propose a new method for silhouette extraction of a point cloud, via local reconstruction of a surface described implicitly by a polynomial function. This reconstruction is based on the Gradient one fitting and Ridge regression methods. The curve silhouette is implicitly defined by a system of nonlinear equations, and is obtained using numerical continuation. As a result, we observe that our method is suitable to handle noisy data. In addition, we present a method for extracting Laplacian Lines of a point cloud based on local reconstruction using the Delaunay triangulation.

Semantic segmentation is a key approach to comprehensive image data analysis. It can be applied to analyze 2D images, videos, and even point clouds that contain 3D data points. On the first two problems, CNNs have achieved remarkable progress, but on point cloud segmentation, the results are less satisfactory due to challenges such as limited memory resource and difficulties in 3D point annotation. One of the research studies carried out by the Computer Vision Lab at Linköping University was aiming to ease the semantic segmentation of 3D point cloud. The idea is that by first projecting 3D data points to 2D space and then focusing only on the analysis of 2D images, we can reduce the overall workload for the segmentation process as well as exploit the existing well-developed 2D semantic segmentation techniques. In order to improve the performance of CNNs for 2D semantic segmentation, the study has used input data derived from different modalities. However, how different modalities can be optimally fused is still an open question. Based on the above-mentioned study, this thesis aims to improve the multistream framework architecture. More concretely, we investigate how different singlestream architectures impact the multistream framework with a given fusion method, and how different fusion methods contribute to the overall performance of a given multistream framework. As a result, our proposed fusion architecture outperformed all the investigated traditional fusion methods. Along with the best singlestream candidate and few additional training techniques, our final proposed multistream framework obtained a relative gain of 7.3\% mIoU compared to the baseline on the semantic3D point cloud test set, increasing the ranking from 12th to 5th position on the benchmark leaderboard.

Nos últimos anos, acompanhar o desenvolvimento de uma cultura tem se tornado cada vez mais imprescindível para a tomada de decisões. Sistemas aéreos remotamente pilotados são muito promissores em aplicações de monitoramento. Sua flexibilidade, facilidade de operação e construção relativamente barata os tornam os melhores candidatos para monitorar atividades na agricultura de precisão, onde as reações imediatas de manejo às doenças das plantas, à falta de nutrientes das plantas e às mudanças ambientais são o ponto focal para eficiência e produtividade das plantações. No entanto, no Brasil a utilização desta tecnologia ainda é limitada e o número de publicações científicas sobre o assunto é escasso. No caso específico da cana-de-açúcar, a utilização de aeronave remotamente pilotada (RPA) é bastante promissora e publicações científicas internacionais são limitadas. O objetivo deste trabalho foi avaliar a potencialidade de imagens obtidas a partir de câmeras com diferentes bandas espectrais embarcadas em RPA para obtenção de modelos tridimensionais para estimativa de altura, produtividade e variabilidade espacial. As coletas foram realizadas ao longo da safra 2014/2015, durante o período de um ano. Foi utilizada uma aeronave remotamente pilotada equipada com uma câmera digital com sensibilidade na região espectral do visível (RGB) e outra na região espectral do infravermelho próximo (IVP) sincronizadas com um sistema de navegação global por satélite (GNSS). Este sistema possibilitou a aquisição de imagens com altíssima resolução (3 cm pixel-1) e permitiu a geração de orto-mosaicos e modelos digitais de superfícies (MDS) através de métodos de reconstrução automática em 3D, ajustados por pontos de controle em solo. O RPA seguiu um plano de voo pré-determinado sobre o local do estudo para garantir a aquisição de imagens com cruzamento e sobreposição superior a 90%. O método de validação foi conduzido a partir das medidas de altura obtidas a campo com o auxílio de régua topográfica. Após o processamento das imagens aéreas foi possível a identificação das áreas com ausência de fechamento de dossel, observando também a relação desses locais com o baixo desenvolvimento da altura das plantas ao longo de seu ciclo. A regressão entre os valores da estimativa de altura obtidas com as simulações apresentou erro relativo inferior a 13%, já a estimativa da produtividade apresentou erro na faixa de 6%. A estimativa de altura e produtividade demonstram o alto potencial para o monitoramento e avaliação de talhões de cana-de-açúcar, podendo ser uma ferramenta utilizada no apoio a gestão destas áreas. / In the last few years, monitoring the development of a culture has become increasingly imperative for decision-making. Remotely piloted aircraft systems (RPA) are very promising in monitoring applications. Their flexibility, ease of operation, and relatively inexpensive construction make them the best candidates to monitor precision farming activities where immediate management responses to plant diseases, lack of plant nutrients, and environmental changes are the focal point for efficiency And productivity of plantations. However in Brazil the use of this technology is still limited and the number of scientific publications on the subject is scarce. In the specific case of sugarcane the use of RPA is very promising and international scientific publications are limited. The objective of this work was to evaluate the potentiality of images obtained from cameras with different spectral bands embedded in RPA to obtain three - dimensional models for estimation of height, productivity and spatial variability. The collections were carried out during the 2014/2015 harvest, during a period of one year, using a remotely piloted aircraft equipped with a digital camera with sensitivity in the visible spectral region (RGB) and another in the near infrared spectral region (NIR) Synchronized with a GNSS. This system allowed the acquisition of images with very high resolution (3 cm pixel-1) allowing the generation of ortho-mosaics and digital surface models (DSM), through automatic 3D reconstruction methods adjusted by control points in soil. The RPA followed a pre-determined flight plan on the study site to ensure cross-over and overlapping acquisition of over 90%. The validation method was carried out from the height measurements obtained in the field with the aid of topography. After the aerial images processing, it was possible to identify the areas of crop failure, also observing the relation of these locations with the low development of plant height throughout its cycle. The regression between the values of the height estimation obtained with the simulations resulted in a relative error of less than 13%. The results obtained demonstrate the high potential of this technique for monitoring and evaluation of sugarcane fields, and can be a tool used to support the management of these areas.

Submitted by Maicon Juliano Schmidt (maicons) on 2015-03-16T13:44:12Z No. of bitstreams: 1 00000AC7.pdf: 4983854 bytes, checksum: 354b89932347d08ba4263cd1c545ef4f (MD5) / Made available in DSpace on 2015-03-16T13:44:12Z (GMT). No. of bitstreams: 1 00000AC7.pdf: 4983854 bytes, checksum: 354b89932347d08ba4263cd1c545ef4f (MD5) Previous issue date: 2014-01-01 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Diversas áreas do conhecimento utilizam técnicas de coleta de dados espaciais com grande resolução e precisão. Em especial, a Geologia tem utilizado novos equipamentos para a automatização de levantamentos e mapeamentos geológicos. A coleta digital feita com lasers scanners terrestres registra não só posicionamento e cor, como também a intensidade de retorno do sinal emitido. Ao utilizar as informações de intensidade de retorno do laser, pode ocorrer o efeito de borda, que é quando o laser colide parcialmente com o alvo e parte do sinal é perdido ou, ainda, colide com outros objetos não desejados ao fundo. Esse efeito faz com que a intensidade de retorno seja uma intensidade incorreta, quando analisa-se a reflectância dos materiais que compõem o alvo escaneado. Para resolver este problema, um novo algoritmo foi desenvolvido utilizando os dados conhecidos do laser, como a posição e a divergência do sinal para recuperar a intensidade de retorno do pulso laser correta quando o efeito de borda é detectado nos dados coletados. Os resultados mostram que esta técnica é uma possível solução para recuperar a intensidade de retorno do pulso de acordo com a reflectância dos materiais que compõem o afloramento. Um estudo adicional é necessário para realizar a otimização do algoritmo e para realizar uma análise estatística das intensidades corrigidas. / Several areas of knowledge use digital techniques to collect spacial data with higher resolution and precision. In special, Geology is using new equipaments to automate geological surveys and mappings. The digital acquisition made with terrestrial laser scanners records not only the target’s position and color, but also the return of the emitted signal’s intensity. When using the laser’s intensity return information, it may occur the edge effect, that is when the laser collides parcially with the target and part of the signal is lost or, also, collides with other undesired objects in the background. This effect makes the laser’s return intensity to be incorrect, when the reflectance of the materials that compose the target being analized. To solve this issue, a new algorithm was developed using the known data as the laser scanner’s position and signal’s divergence to recover the correct laser’s intensity when an edge effect is detected in the collected data. The results show that this technique is a possible solution to recover the correct laser’s return intensity according to the reflectance of the outcrop materials. Additional research is needed to optimize the algorithm and make a statistical analysis of the corrected laser intensity data.

3D point cloud processing has been a critical task due to the increasing demand of a variety of applications such as urban planning and management, as-built mapping of industrial sites, infrastructure monitoring, and road safety inspection. Point clouds are mainly acquired from two sources, laser scanning and optical imaging systems. However, the original point clouds usually do not provide explicit semantic information, and the collected data needs to undergo a sequence of processing steps to derive and extract the required information. Moreover, according to application requirements, the outcomes from the point cloud processing could be different. This dissertation presents two tiers of data processing. The first tier proposes an adaptive data processing framework to deal with multi-source and multi-platform point clouds. The second tier introduces two point clouds processing strategies targeting applications mainly from urban environments and transportation corridors.<div><br></div><div>For the first tier of data processing, the internal characteristics (e.g., noise level and local point density) of data should be considered first since point clouds might come from a variety of sources/platforms. The acquired point clouds may have a large number of points. Data processing (e.g., segmentation) of such large datasets is time-consuming. Hence, to attain high computational efficiency, this dissertation presents a down-sampling approach while considering the internal characteristics of data and maintaining the nature of the local surface. Moreover, point cloud segmentation is one of the essential steps in the initial data processing chain to derive the semantic information and model point clouds. Therefore, a multi-class simultaneous segmentation procedure is proposed to partition point cloud into planar, linear/cylindrical, and rough features. Since segmentation outcomes could suffer from some artifacts, a series of quality control procedures are introduced to evaluate and improve the quality of the results.<br></div><div><br></div><div>For the second tier of data processing, this dissertation focuses on two applications for high human activity areas, urban environments and transportation corridors. For urban environments, a new framework is introduced to generate digital building models with accurate right-angle, multi-orientation, and curved boundary from building hypotheses which are derived from the proposed segmentation approach. For transportation corridors, this dissertation presents an approach to derive accurate lane width estimates using point clouds acquired from a calibrated mobile mapping system. In summary, this dissertation provides two tiers of data processing. The first tier of data processing, adaptive down-sampling and segmentation, can be utilized for all kinds of point clouds. The second tier of data processing aims at digital building model generation and lane width estimation applications.<br></div>

In this thesis a methodology for representing 3D subjects and their deformations in adverse situations is studied. The study is focused in providing methods based on registration techniques to improve the data in situations where the sensor is working in the limit of its sensitivity. In order to do this, it is proposed two methods to overcome the problems which can difficult the process in these conditions. First a rigid registration based on model registration is presented, where the model of 3D planar markers is used. This model is estimated using a proposed method which improves its quality by taking into account prior knowledge of the marker. To study the deformations, it is proposed a framework to combine multiple spaces in a non-rigid registration technique. This proposal improves the quality of the alignment with a more robust matching process that makes use of all available input data. Moreover, this framework allows the registration of multiple spaces simultaneously providing a more general technique. Concretely, it is instantiated using colour and location in the matching process for 3D location registration.