Three-dimensional information is often used in robotics and 3D-mapping. There exist several ways to obtain a three-dimensional map. However, the time of flight used in the laser scanners or the structured light utilized by Kinect-like sensors sometimes are not sufficient. In this thesis, we investigate two CNN based stereo matching methods for obtaining 3D-information from a grayscaled pair of rectified images.While the state-of-the-art stereo matching method utilize a Siamese architecture, in this project a two-channel and a two stream network are trained in an attempt to outperform the state-of-the-art. A set of experiments were performed to achieve optimal hyperparameters. By changing one parameter at the time, the networks with architectures mentioned above are trained. After a completed training the networks are evaluated with two criteria, the error rate, and the runtime.Due to time limitations, we were not able to find optimal learning parameters. However, by using settings from [17] we train a two-channel network that performed almost on the same level as the state-of-the-art. The error rate on the test data for our best architecture is 2.64% while the error rate for the state-of-the-art Siamese network is 2.62%. We were not able to achieve better performance than the state-of-the-art, but we believe that it is possible to reduce the error rate further. On the other hand, the state-of-the-art Siamese stereo matching network is more efficient and faster during the disparity estimation. Therefore, if the time efficiency is prioritized, the Siamese based network should be considered.

This Master thesis tackles the problem of measuring objects’ directions from a motionless observation point. A new method based on a single rotating camera requiring the knowledge of only two (or more) landmarks’ direction is proposed. In a first phase, multi-view geometry is used to estimate camera rotations and key elements’ direction from a set of overlapping images. Then in a second phase, the direction of any object can be estimated by resectioning the camera associated to a picture showing this object. A detailed description of the algorithmic chain is given, along with test results on both synthetic data and real images taken with an infrared camera. / Detta masterarbete behandlar problemet med att mäta objekts riktningar från en fast observationspunkt. En ny metod föreslås, baserad på en enda roterande kamera som kräver endast två (eller flera) landmärkens riktningar. I en första fas används multiperspektivgeometri, för att uppskatta kamerarotationer och nyckelelements riktningar utifrån en uppsättning överlappande bilder. I en andra fas kan sedan riktningen hos vilket objekt som helst uppskattas genom att kameran, associerad till en bild visande detta objekt, omsektioneras. En detaljerad beskrivning av den algoritmiska kedjan ges, tillsammans med testresultat av både syntetisk data och verkliga bilder tagen med en infraröd kamera.

3D semantic segmentation is an expanding topic within the field of computer vision, which has received more attention in recent years due to the development of more powerful GPUs and the newpossibilities offered by deep learning techniques. Simultaneously, the amount of available spatial LiDAR data over Sweden has also increased. This work combines these two advances and investigates if a 3D deep learning model for semantic segmentation can learn to detect terrain roughness in airborne LiDAR data. The annotations for terrain roughness used in this work are taken from SGUs 2D soil type map. Other airborne data sources are also used to filter the annotations and see if additional information can boost the performance of the model.  Since this is the first known attempt at terrain roughness classification from 3D data, an initial test was performed where fields were classified. This ensured that the model could process airborne LiDAR data and work for a terrain classification task. The classification of fields showed very promising results without any fine-tuning. The results for the terrain roughness classification task show that the model could find a pattern in the validation data but had difficulty generalizing it to the test data. The filtering methods tested gave an increased mIoU and indicated that better annotations might be necessary to distinguish terrain roughness from other terrain types. None of the features obtained from the other data sources improved the results and showed no discriminating abilities when examining their individual histograms. In the end, more research is needed to determine whether terrain roughness can be detected from LiDAR data or not.

The term hydrobatics refers to the agile maneuvering of underwater vehicles. Hydrobatic capabilities in autonomous underwater vehicles (AUVs) can enable increased maneuverability without a sacrifice in efficiency and speed. This means innovative robot designs and new use case scenarios are possible. Benefits and technical challenges related to hydrobatic AUVs are explored in this thesis. The dissertation contributes to new knowledge in simulation, control and field applications, and provides a structured approach to realize hydrobatic capabilities in real world impact areas. Three impact areas are considered - environmental monitoring, ocean production and security. A combination of agility in maneuvering and efficiency in performance is crucial for successful AUV applications. To achieve such performance, two technical challenges must be solved. First, these AUVs have fewer control inputs than degrees of freedom, which leads to the challenge of underactuation. The challenge is described in detail and solution strategies that use optimal control and model predictive control (MPC) are highlighted. Second, the flow around an AUV during hydrobatic maneuvers transitions from laminar to turbulent flow at high angles of attack. This renders flight dynamics modelling difficult. A full 0-360 degree envelope flight dynamics model is therefore derived, which combines a multi-fidelity hydrodynamic database with a generalized component-buildup approach. Such a model enables real-time (or near real-time) simulations of hydrobatic maneuvers including loops, helices and tight turns. Next, a cyber-physical system (CPS) is presented -- it safely transforms capabilities derived in simulation to real-world use cases in the impact areas described. The simulator environment is closely integrated with the robotic system, enabling pre-validation of controllers and software before hardware deployment. The small and hydrobatic SAM AUV (developed in-house at KTH as part of the Swedish Maritime Robotics Center) is used as a test platform. The CPS concept is validated by using the SAM AUV for the search and detection of a submerged target in field operating conditions. Current research focuses on further exploring underactuated control and motion planning. This includes development of real-time nonlinear MPC implementations running on AUV hardware, as well as intelligent control through feedback motion planning, system identification and reinforcement learning. Such strategies can enable real-time robust and adaptive control of underactuated systems. These ideas will be applied to demonstrate new capabilities in the three impact areas. / Termen hydrobatik avser förmåga att utföra avancerade manövrer med undervattensfarkoster. Syftet är att, med bibehållen fart och räckvidd, utvigda den operationella förmågan i manövrering, vilket möjliggör helt nya användningsområden för maximering av kostnadseffektivitet. I denna avhandling undersöks fördelar och tekniska utmaningar relaterade till hydrobatik som tillämpas på undervattensrobotar, vanligen kallade autonoma undervattensfarkoster (AUV). Avhandlingen bidrar till ny kunskap i simulering, reglering samt tillämpning i experiment av dessa robotar genom en strukturerad metod för att realisera hydrobatisk förmåga i realistiska scenarier.  Tre nyttoområden beaktas - miljöövervakning, havsproduktion och säkerhet. Inom dessa nyttoområden har ett antal scenarios identifierats där en kombination av smidighet i manövrerbarhet samt effektivitet i prestanda är avgörande för robotens förmåga att utföra sin uppgift. För att åstadkomma detta måste två viktiga tekniska utmaningar lösas. För det första har dessa AUVer färre styrytor/trustrar än frihetsgrader, vilket leder till utmaningen med underaktuering. Utmaningen beskrivs i detalj och lösningsstrategier som använder optimal kontroll och modellprediktiv kontroll belyses. För det andra är flödet runt en AUV som genomför hydrobatiska manövrar komplext med övergång från laminär till stark turbulent flöde vid höga anfallsvinklar. Detta gör flygdynamikmodellering svår. En full 0-360 graders flygdynamikmodell härleds därför, vilken kombinerar en multi-tillförlitlighets hydrodynamisk databas med en generaliserad strategi för komponentvis-superpositionering av laster. Detta möjliggör prediktering av hydrobatiska manövrar som t.ex. utförande av looping, roll, spiraler och väldigt snäva svängradier i realtids- eller nära realtids-simuleringar. I nästa steg presenteras ett cyber-fysikaliskt system (CPS) – där funktionalitet som härrör från simuleringar kan överföras till de verkliga användningsområdena på ett effektivt och säkert sätt. Simulatormiljön är nära integrerad i robot-miljön, vilket möjliggör förvalidering av reglerstrategier och mjukvara innan hårdvaruimplementering. En egenutvecklad hydrobatisk AUV (SAM) används som testplattform. CPS-konceptet valideras med hjälp av SAM i ett realistiskt sceanrio genom att utföra ett sökuppdrag av ett nedsänkt föremål under fältförhållanden. Resultaten av arbetet i denna licentiatavhandling kommer att användas i den fortsatta forskningen som fokuserar på att ytterligare undersöka och utveckla ny metodik för reglering av underaktuerade AUVer. Detta inkluderar utveckling av realtidskapabla ickelinjära MPC-implementeringar som körs ombord, samt AI-baserade reglerstrategier genom ruttplaneringsåterkoppling, autonom systemidentifiering och förstärkningsinlärning. Sådan utveckling kommer att tillämpas för att visa nya möjligheter inom de tre nyttoområdena. / SMaRC

Background It can be observed that our society is heading more and more towards automation. Autonomous machines show large potential and are being used progressively often in a range of different areas and tasks. They are increasing the productivity and transforming jobs and industries. However, the implemented systems of autonomous machines are usually specified for certain conditions, in structured and static environments. Making the implementation very contextual to the environment it is in. Dynamic environments, is something that is continuously changing or being changed, meaning a lot of challenges for the implementation and operation of something autonomous. Objectives The purpose of this study is to investigate how to help the conditions for implementation of autonomous machines in dynamic environments. The sites and machines in the construction industry fulfill the described context well and is therefore chosen as the main field of study for this thesis. Method A main case study exploration has been used to disclose the result. Including different methods of data gathering such as literature research, interviews, observations, field visits, and workshops. Data has also been collected in form of learnings from prototypes and experiments conducted throughout the study. Results The results evaluate how the aiding of the implementation and operation of autonomous machines could be done in dynamic environments such as the construction sites. It considers working at remote areas without human assistance, the external information needed for the autonomous machines, the different technologies that could be used, and how to take a first step towards an autonomous future. A concept solution is presented, which could be implemented today and used to help the implementation and operation of autonomous machines. Conclusion The findings in this study indicates that the machines need to understand elements in dynamic environments to be able to conduct meaningful tasks. For this there is a need for external information through different technologies, making element visible in a continuously changing structure. Material management is one of the essential elements that needs to be made visible for the machines. The results can be introduced today through the concept and be developed along with the rest of the technology to make the adaptation and implementation easier. / Bakgrund Det kan observeras att vårt samhälle går alltmer mot automatisering. Autonoma maskiner visar stor potential och används successivt mer för en rad olika områden och uppgifter. De ökar produktiviteten och omvandlar jobb och industrier. De implementerade systemen för autonoma maskiner är oftast specialiserade för vissa förhållanden, i strukturerade och statiska miljöer, vilket leder till att implementeringen är mycket kontextuellt för miljön. Dynamiska miljöer är något som ständigt ändras, vilket innebär en hel del utmaningar för implementeringen och driften av något autonomt och självständigt. Mål Syftet med denna studie är att undersöka hur man hjälper förutsättningarna för implementeringen av autonoma maskiner i dynamiska miljöer. Byggarbetsplatser och maskiner inom konstruktionsbranschen uppfyller det beskrivna kontexten väl och väljs därför som huvudområde för denna avhandling. Metod En explorativt fallstudie har använts för att komma fram till resultatet, tillsammans med olika metoder för datainsamling såsom litteraturundersökning, intervjuer, observationer, fältbesök och workshops. Insamling av data har även skett i form av lärdomar från prototyper och experiment som genomförts under studien. Resultat Resultaten utvärderar hur implementationen och driften av autonoma maskiner kan hjälpas  i dynamiska miljöer såsom konstruktion lägen. Vidare utreds de autonoma maskinernas arbete i avlägsna områden utan mänskligt bistånd och den externa informationen som behövs för maskinerna i det sammanhanget. De olika teknologierna som kan användas är utvärderade tillsammans med hur ett första steg kan tas mot en självständig framtid. En konceptlösning presenteras, som skulle kunna implementeras idag och användas för att hjälpa till med implementering och driften av autonoma maskiner. Slutsats Resultaten i denna studie visar att maskinerna måste förstå element i dynamiska miljöer för att kunna genomföra meningsfulla uppgifter. Därför finns det behov av extern information genom olika teknologier, vilka synliggör elementet i en ständigt varierande struktur. Materialhantering är en av de väsentliga delarna som måste synliggöras för maskinerna. Resultaten kan introduceras idag genom konceptet och utvecklas tillsammans med resten av tekniken för att göra anpassningen till tekniken och implementationen enklare.

This thesis studies motion planning for future autonomous vehicles with main focus on passenger cars. By having automatic steering and braking together with information about the environment, such as other participants in the traffic or obstacles, it would be possible to perform autonomous maneuvers while taking limitations of the vehicle and road–tire interaction into account. Motion planning is performed to find such maneuvers that bring the vehicle from the current state to a desired future state, here by formulating the motion-planning problem as an optimal control problem. There are a number of challenges for such an approach to motion planning; some of them are how to formulate the criterion in the motion planning (objective function in the corresponding optimal control problem), and how to make the solution of motion-planning problems efficient to be useful in online applications. These challenges are addressed in this thesis. As a criterion for motion-planning problems of passenger vehicles on doublelane roads, it is investigated to use a lane-deviation penalty function to capture the observation that it is dangerous to drive in the opposing lane, but safe to drive in the original lane after the obstacle. The penalty function is augmented with certain additional terms to address also the recovery behavior of the vehicle. The resulting formulation is shown to provide efficient and steady maneuvers and gives a lower time in the opposing lane compared to other objective functions. Under varying parameters of the scenario formulation, the resulting maneuvers are changing in a way that exhibits structured characteristics. As an approach to improve efficiency of computations for the motion-planning problem, it is investigated to segment motion planning of the full maneuver into several smaller maneuvers. A way to extract segments is considered from a vehicle dynamics point of view, and it is based on extrema of the vehicle orientation and the yaw rate. The segmentation points determined using this approach are observed to allow efficient splitting of the optimal control problem for the full maneuver into subproblems. Having a method to segment maneuvers, this thesis further studies methods to allow parallel computation of these maneuvers. One investigated method is based on Lagrange relaxation and duality decomposition. Smaller subproblems are formulated, which are governed by solving a low-complexity coordination problem. Lagrangian relaxation is performed on a subset of the dynamic constraints at the segmentation points, while the remaining variables are predicted. The prediction is possible because of the observed structured characteristics resulting from the used lane-deviation penalty function. An alternative approach is based on adoption of the alternating augmented Lagrangian method. Augmentation of the Lagrangian allows to apply relaxation for all dynamic constraints at the segmentation points, and the alternating approach makes it possible to decompose the full problem into subproblems and coordinating their solutions by analytically solving an overall coordination problem. The presented decomposition methods allow computation of maneuvers with high correspondence and lower computational times compared to the results obtained for solving the full maneuver in one step.

As the field of Artificial Intelligence continues to evolve into a tool of societal impact, a need of breaking its initial boundaries as a computer science discipline arises to also include different humanistic fields. The work presented in this thesis revolves around the role that explainable artificial intelligence has in human-robot interaction through the study of poetry generators. To better understand the scope of the project, a poetry generators study presents the steps involved in the development process and the evaluation methods. In the algorithmic development of poetry generators, the shift from traditional disciplines to transdisciplinarity is identified. In collaboration with researchers from the Research Institutes of Sweden, state-of-the-art generators are tested to showcase the power of artificially enhanced artifacts. A development plateau is discovered and with the inclusion of Design Thinking methods potential future human-robot interaction development is identified. A physical prototype capable of verbal interaction on top of a poetry generator is created with the new feature of changing the corpora to any given audio input. Lastly, the strengths of transdisciplinarity are connected with the open-sourced community in regards to creativity and self-expression, producing an online tool to address future work improvements and introduce nonexperts to the steps required to self-build an intelligent robotic companion, thus also encouraging public technological literacy. Explainable AI is shown to help with user involvement in the process of creation, alteration and deployment of AI enhanced applications.

Designing cutting tools for the turning industry, providing optimal cutting parameters is of importance for both the client, and for the company's own research. By examining the metal chips that form in the turning process, operators can recommend optimal cutting parameters. Instead of doing manual classification of metal chips that come from the turning process, an automated approach of detecting chips and classification is preferred. This thesis aims to evaluate if such an approach is possible using either a Convolutional Neural Network (CNN) or a CNN feature extraction coupled with machine learning (ML). The thesis started with a research phase where we reviewed existing state of the art CNNs, image processing and ML algorithms. From the research, we implemented our own object detection algorithm, and we chose to implement two CNNs, AlexNet and VGG16. A third CNN was designed and implemented with our specific task in mind. The three models were tested against each other, both as standalone image classifiers and as a feature extractor coupled with a ML algorithm. Because the chips were inside a machine, different angles and light setup had to be tested to evaluate which setup provided the optimal image for classification. A top view of the cutting area was found to be the optimal angle with light focused on both below the cutting area, and in the chip disposal tray. The smaller proposed CNN with three convolutional layers, three pooling layers and two dense layers was found to rival both AlexNet and VGG16 in terms of both as a standalone classifier, and as a feature extractor. The proposed model was designed with a limited system in mind and is therefore more suited for those systems while still having a high accuracy. The classification accuracy of the proposed model as a standalone classifier was 92.03%. Compared to the state of the art classifier AlexNet which had an accuracy of 92.20%, and VGG16 which had an accuracy of 91.88%. When used as a feature extractor, all three models paired best with the Random Forest algorithm, but the accuracy between the feature extractors is not that significant. The proposed feature extractor combined with Random Forest had an accuracy of 82.56%, compared to AlexNet with an accuracy of 81.93%, and VGG16 with 79.14% accuracy. / DIGICOGS

Image segmentation through neural networks and deep learning have, in the recent decade, become a successful tool for automated decision-making. For Luossavaara-Kiirunavaara Aktiebolag (LKAB), this means identifying the amount of slag inside a furnace through computer vision.  There are many prominent convolutional neural network architectures in the literature, and this thesis explores two: a modified U-Net and the PSPNet. The architectures were combined with three loss functions and three class weighting schemes resulting in 18 model configurations that were evaluated and compared. This thesis also explores transfer learning techniques for neural networks tasked with identifying slag in images from inside a furnace. The benefit of transfer learning is that the network can learn to find features from already labeled data of another context. Finally, the thesis explored how temporal information could be utilised by adding an LSTM layer to a model taking pairs of images as input, instead of one. The results show (1) that the PSPNet outperformed the U-Net for all tested configurations in all relevant metrics, (2) that the model is able to find more complex features while converging quicker by using transfer learning, and (3) that utilising temporal information reduced the variance of the predictions, and that the modified PSPNet using an LSTM layer showed promise in handling images with outlying characteristics.

Reconstruction of sonar images is an inverse problem, which is normally solved with model-based methods. These methods may introduce undesired artifacts called angular and range leakage into the reconstruction. In this thesis, a method called Learned Primal-Dual Reconstruction, which combines a data-driven and a model-based approach, is used to investigate the use of data-driven methods for reconstruction within sonar imaging. The method uses primal and dual variables inspired by classical optimization methods where parts are replaced by convolutional neural networks to iteratively find a solution to the reconstruction problem. The network is trained and validated with synthetic data on eight models with different architectures and training parameters. The models are evaluated on measurement data and the results are compared with those from a purely model-based method. Reconstructions performed on synthetic data, where a ground truth image is available, show that it is possible to achieve reconstructions with the data-driven method that have less leakage than reconstructions from the model-based method. For reconstructions performed on measurement data where no ground truth is available, some variants of the learned model achieve a good result with less leakage.