Yes / This article proposes an approach named SafeML II, which applies empirical cumulative distribution function-based statistical distance measures in a designed human-in-the loop procedure to ensure the safety of machine learning-based classifiers in autonomous vehicle software. The application of artificial intelligence (AI) and data-driven decision-making systems in autonomous vehicles is growing rapidly. As autonomous vehicles operate in dynamic environments, the risk that they can face an unknown observation is relatively high due to insufficient training data, distributional shift, or cyber-security attack. Thus, AI-based algorithms should make dependable decisions to improve their interpretation of the environment, lower the risk of autonomous driving, and avoid catastrophic accidents. This paper proposes an approach named SafeML II, which applies empirical cumulative distribution function (ECDF)-based statistical distance measures in a designed human-in-the-loop procedure to ensure the safety of machine learning-based classifiers in autonomous vehicle software. The approach is model-agnostic and it can cover various machine learning and deep learning classifiers. The German Traffic Sign Recognition Benchmark (GTSRB) is used to illustrate the capabilities of the proposed approach. / This work was supported by the Secure and Safe MultiRobot Systems (SESAME) H2020 Project under Grant Agreement 101017258.

<p dir="ltr">Introduces a new framework and state-of-the-art algorithm in closed-loop prediction for motion planning under differential constraints. More specifically, this work introduces the idea of sampling on specific "sampling regions" rather than the entire workspace to speed-up the motion planning process by orders of magnitude.</p>

The utilization of image analysis and object detection spans various industries, serving purposes such as anomaly detection, automated workflows, and monitoring tool wear and tear. This thesis addresses the challenge of achieving precise robot navigation between fixed start and end points by combining GPS and image analysis. The underlying motivation for tackling this issue lies in facilitating the creation of immersive videos, mainly aimed at individuals with disabilities, enabling them to virtually explore diverse locations through a compilation of shorter video clips.  The research delves into diverse models for object detection frameworks and tools, including NVIDIA Detectnet, and YOLOv5. Through a comprehensive evaluation of their performance and accuracy, the thesis proceeds to implement a prototype system utilizing an Elegoo Smart Robot Car, a camera, a GPS module, and an embedded NVIDIA Jetson Nano system.  Performance metrics such as precision, recall, and map are employed to assess the models' effectiveness. The findings indicate that the system demonstrates high accuracy and speed in detection, exhibiting robustness across varying lighting conditions and camera settings

Visual recognition tasks have witnessed vast improvements in performance since the advent of deep learning. Despite the gains in performance, image understanding algorithms are still not completely robust to partial occlusion. In this work, we propose a novel object classification method based on compositional modeling and explore its effect in the context of the newly introduced panoptic segmentation task. The panoptic segmentation task combines both semantic and instance segmentation to perform labelling of the entire image. The novel classification method replaces the object detection pipeline in UPSNet, a Mask R-CNN based design for panoptic segmentation. We also discuss an issue with the segmentation mask prediction of Mask R-CNN that affects overlapping instances. We perform extensive experiments and showcase results on the complex COCO and Cityscapes datasets. The novel classification method shows promising results for object classification on occluded instances in complex scenes. / Master of Science / Visual recognition tasks have witnessed vast improvements in performance since the advent of deep learning. Despite making significant improvements, algorithms for these tasks still do not perform well at recognizing partially visible objects in the scene. In this work, we propose a novel object classification method that uses compositional models to perform part based detection. The method first looks at individual parts of an object in the scene and then makes a decision about its identity. We test the proposed method in the context of the recently introduced panoptic segmentation task. The panoptic segmentation task combines both semantic and instance segmentation to perform labelling of the entire image. The novel classification method replaces the object detection module in UPSNet, a Mask R-CNN based algorithm for panoptic segmentation. We also discuss an issue with the segmentation mask prediction of Mask R-CNN that affects overlapping instances. After performing extensive experiments and evaluation, it can be seen that the novel classification method shows promising results for object classification on occluded instances in complex scenes.

The proposed master's thesis investigates the adaptability and efficacy of self-supervised representation learning (SSL) in medical image analysis, focusing on Mammographic Imaging to develop robust representation learning models. This research will build upon existing studies in Mammographic Imaging that have utilized contrastive learning and knowledge distillation-based self-supervised methods, focusing on SimCLR (Chen et al 2020) and SimSiam (Chen et al 2020) and evaluate approaches to increase the classification performance in a transfer learning setting. The thesis will critically evaluate and integrate recent advancements in these SSL paradigms (Chhipa 2023, chapter 2), and incorporating additional SSL approaches. The core objective is to enhance robust generalization and label efficiency in medical imaging analysis, contributing to the broader field of AI-driven diagnostic methodologies. The proposed master's thesis will not only extend the current understanding of SSL in medical imaging but also aims to provide actionable insights that could be instrumental in enhancing breast cancer detection methodologies, thereby contributing significantly to the field of medical imaging and cancer research.

This thesis explores the potential of Neural Radiance Fields (NeRF) for generating orthophotos in forestry applications. Traditional orthophoto production methods, such as those implemented in Pix4D, require high image overlap and significant data collection. NeRF, a novel 3D scene reconstruction technique, shows potential for reducing these requirements by effectively reconstructing scenes with lower image overlaps. This study compares the orthophotos produced by NeRF and Pix4D using various degrees of image overlap, evaluating the results based on geometric accuracy, image quality, and robustness to data variations. The findings indicate that NeRF can produce orthophotos from low-overlap images with geometric accuracy comparable to orthophotos produced by Pix4D from high-overlap images, though with some trade-offs in image sharpness. These results suggest potential cost savings and operational efficiencies in forestry applications, providing a viable alternative to traditional photogrammetric techniques.

Robots are operating for longer times and collecting much more data than just a few years ago. In this setting we are interested in exploring ways of modeling the environment, segmenting out areas of interest and keeping track of the segmentations over time, with the purpose of building 4D models (i.e. space and time) of the relevant parts of the environment. Our approach relies on repeatedly observing the environment and creating local maps at specific locations. The first question we address is how to choose where to build these local maps. Traditionally, an operator defines a set of waypoints on a pre-built map of the environment which the robot visits autonomously. Instead, we propose a method to automatically extract semantically meaningful regions from a point cloud representation of the environment. The resulting segmentation is purely geometric, and in the context of mobile robots operating in human environments, the semantic label associated with each segment (i.e. kitchen, office) can be of interest for a variety of applications. We therefore also look at how to obtain per-pixel semantic labels given the geometric segmentation, by fusing probabilistic distributions over scene and object types in a Conditional Random Field. For most robotic systems, the elements of interest in the environment are the ones which exhibit some dynamic properties (such as people, chairs, cups, etc.), and the ability to detect and segment such elements provides a very useful initial segmentation of the scene. We propose a method to iteratively build a static map from observations of the same scene acquired at different points in time. Dynamic elements are obtained by computing the difference between the static map and new observations. We address the problem of clustering together dynamic elements which correspond to the same physical object, observed at different points in time and in significantly different circumstances. To address some of the inherent limitations in the sensors used, we autonomously plan, navigate around and obtain additional views of the segmented dynamic elements. We look at methods of fusing the additional data and we show that both a combined point cloud model and a fused mesh representation can be used to more robustly recognize the dynamic object in future observations. In the case of the mesh representation, we also show how a Convolutional Neural Network can be trained for recognition by using mesh renderings. Finally, we present a number of methods to analyse the data acquired by the mobile robot autonomously and over extended time periods. First, we look at how the dynamic segmentations can be used to derive a probabilistic prior which can be used in the mapping process to further improve and reinforce the segmentation accuracy. We also investigate how to leverage spatial-temporal constraints in order to cluster dynamic elements observed at different points in time and under different circumstances. We show that by making a few simple assumptions we can increase the clustering accuracy even when the object appearance varies significantly between observations. The result of the clustering is a spatial-temporal footprint of the dynamic object, defining an area where the object is likely to be observed spatially as well as a set of time stamps corresponding to when the object was previously observed. Using this data, predictive models can be created and used to infer future times when the object is more likely to be observed. In an object search scenario, this model can be used to decrease the search time when looking for specific objects. / <p>QC 20171009</p>

In the field of autonomous driving a common scenario is to apply deep learningmodels on camera feeds to provide information about the surroundings. A recenttrend is for such vision-based methods to be centralized, in that they fuse imagesfrom all cameras in one big model for a single comprehensive output. Designingand tuning such models is hard and time consuming, in both development andtraining. This thesis aims to reproduce the results of a paper about a centralizedvision-based model performing 3D object detection, called BEVDet. Additionalgoals are to ablate the technique of class balanced grouping and sampling usedin the model, to tune the model to improve generalization, and to change thedetection head of the model to a Transformer decoder-based head. The findings include a successful reproduction of the results of the paper,while adding depth supervision to BEVDet establishes a baseline for the subsequentexperiments. An increasing validation loss during most of the training indicatesthat there is room for improvement in the generalization of the model. Severaldifferent methods are tested in order to resolve the increasing validation loss,but they all fail to do so. The ablation study shows that the class balanced groupingis important for the performance of the chosen configuration of the model,while the class balanced sampling does not contribute significantly. Without extensivetuning the replacement head gives performance similar to the PETR, themodel that the head is adapted from, but fails to match the performance of thebaseline model. In addition, the model with the Transformer decoder-based headshows a converging validation loss, unlike the baseline model.

Over the past decade the capabilities of autonomous systems have been steadily increasing. Unmanned systems are moving from systems that are predominantly remotely operated, to systems that include a basic decision making capability. This is a trend that is expected to continue with autonomous systems making decisions in increasingly complex environments, based on more abstract, higher-level missions and goals. These changes have significant implications for how these systems should be designed and engineered. Indeed, as the goals and tasks these systems are to achieve become more abstract, and the environments they operate in become more complex, are current approaches to verification and validation sufficient? Domain Specific Modelling is a key technology for the verification of autonomous systems. Verifying these systems will ultimately involve understanding a significant number of domains. This includes goals/tasks, environments, systems functions and their associated performance. Relational Model Transformations provide a means to utilise, combine and check models for consistency across these domains. In this thesis an approach that utilises relational model transformation technologies for systems verification, Systems MDD, is presented along with the results of a series of trials conducted with an existing relational model transformation language (QVT-Relations). These trials identified a number of problems with existing model transformation languages, including poorly or loosely defined semantics, differing interpretations of specifications across different tools and the lack of a guarantee that a model transformation would generate a model that was compliant with its associated meta-model. To address these problems, two related solvers were developed to assist with realising the Systems MDD approach. The first solver, MMCS, is concerned with partial model completion, where a partial model is defined as a model that does not fully conform with its associated meta-model. It identifies appropriate modifications to be made to a partial model in order to bring it into full compliance. The second solver, TMPT, is a relational model transformation engine that prioritises target models. It considers multiple interpretations of a relational transformation specification, chooses an interpretation that results in a compliant target model (if one exists) and, optionally, maximises some other attribute associated with the model. A series of experiments were conducted that applied this to common transformation problems in the published literature.

Thesis (PhD)--Stellenbosch University, 2012. / ENGLISH ABSTRACT: This research focuses on the development of new efficient algorithms for multi-path planning and multi-rigid body constrained attitude control. The work is motivated by current and future applications of these algorithms in: intelligent control of multiple autonomous aircraft and spacecraft systems; control of multiple mobile and industrial robot systems; control of intelligent highway vehicles and traffic; and air and sea traffic control. We shall collectively refer to the class of mobile autonomous systems as “agents”. One of the challenges in developing and applying such algorithms is that of complexity resulting from the nontrivial agent dynamics as agents interact with other agents, and their environment. In this work, some of the current approaches are studied with the intent of exposing the complexity issues associated them, and new algorithms with reduced computational complexity are developed, which can cope with interaction constraints and yet maintain stability and efficiency. To this end, this thesis contributes the following new developments to the field of multipath planning and multi-body constrained attitude control: • The introduction of a new LMI-based approach to collision avoidance in 2D and 3D spaces. • The introduction of a consensus theory of quaternions by applying quaternions directly with the consensus protocol for the first time. • A consensus and optimization based path planning algorithm for multiple autonomous vehicle systems navigating in 2D and 3D spaces. • A proof of the consensus protocol as a dynamic system with a stochastic plant matrix. • A consensus and optimization based algorithm for constrained attitude synchronization of multiple rigid bodies. • A consensus and optimization based algorithm for collective motion on a sphere. / AFRIKAANSE OPSOMMING: Hierdie navorsing fokus op die ontwikkeling van nuwe koste-effektiewe algoritmes, vir multipad-beplanning en veelvuldige starre-liggaam beperkte standbeheer. Die werk is gemotiveer deur huidige en toekomstige toepassing van hierdie algoritmes in: intelligente beheer van veelvuldige outonome vliegtuig- en ruimtevaartuigstelsels; beheer van veelvuldige mobiele en industrile robotstelsels; beheer van intelligente hoofwegvoertuie en verkeer; en in lug- en see-verkeersbeheer. Ons sal hier “agente” gebruik om gesamentlik te verwys na die klas van mobiele outonome stelsels. Een van die uitdagings in die ontwikkeling en toepassing van sulke algoritmes is die kompleksiteit wat spruit uit die nie-triviale agentdinamika as gevolg van die interaksie tussen agente onderling, en tussen agente en hul omgewing. In hierdie werk word sommige huidige benaderings bestudeer met die doel om die kompleksiteitskwessies wat met hulle geassosieer word, bloot te l^e. Verder word nuwe algoritmes met verminderde berekeningskompleksiteit ontwikkel. Hierdie algoritmes kan interaksie-beperkings hanteer, en tog stabiliteit en doeltreffendheid behou. Vir hierdie doel dra die proefskrif die volgende nuwe ontwikkelings by tot die gebied van multipad-beplanning van multi-liggaam beperkte standbeheer: • Die voorstel van ’n nuwe LMI-gebasseerde benadering tot botsingsvermyding in 2D en 3D ruimtes. • Die voorstel van ’n konsensus-teorie van “quaternions” deur “quaternions” vir die eerste keer met die konsensusprotokol toe te pas. • ’n Konsensus- en optimeringsgebaseerde padbeplanningsalgoritme vir veelvoudige outonome voertuigstelsels wat in 2D en 3D ruimtes navigeer. • Die bewys van ’n konsensusprotokol as ’n dinamiese stelsel met ’n stochastiese aanlegmatriks. • ’n Konsensus- en optimeringsgebaseerde algoritme vir beperkte stand sinchronisasie van veelvoudige starre liggame. • ’n Konsensus- en optimeringsgebaseerde algoritme vir kollektiewe beweging op ’n sfeer.