In this work, we develop the transfer learning (TL) of reinforcement learning (RL) for the robotic skill of throwing a ball into a basket, from a computer simulated environment to a real-world implementation. Whereas learning of the same skill has been previously explored by using a Programming by Demonstration approach directly on the real-world robot, for our work, the model-based RL algorithm PILCO was employed as an alternative as it provides the robot with no previous knowledge or hints, i.e. the robot begins learning from a tabula rasa state, PILCO learns directly on the simulated environment, and as part of its procedure, PILCO models the dynamics of the inflatable, plastic ball used to perform the task. The robotic skill is represented as a Markov Decision Process, the robotic arm is a Kuka LWR4+, RL is enabled by PILCO, and TL is achieved through policy adjustments. Two learned policies were transferred, and although the results show that no exhaustive policy adjustments are required, large gaps remain between the simulated and the real environment in terms of the ball and robot dynamics. The contributions of this thesis include: a novel TL of RL framework for teaching the basketball skill to the Kuka robotic arm; the development of a pythonised version of PILCO; robust and extendable ROS packages for policy learning and adjustment in a simulated or real robot; a tracking-vision package with a Kinect camera; and an Orocos package for a position controller in the robotic arm.

The goal of this project was to create a self-fixing system for a mo-bile robot as a part of an assisted living system in a home. To accom-plish that, a new system has been generated in this project called SAS, comprising three steps: self-detection, anomaly-recognition, and self-modification. To be more detailed, the robot should 1) find itself in a mirror using vision sensor data; 2) recognize the presence of flaws in its detected image; and 3) act to deal with the flaws. To demonstrate this approach a specific scenario for the current work was chosen, involving a robot which consisted of a moving base, a camera, a robot arm, and a target to fix. The robot was designed to move around in a simplified intelligent home-like environment outfitted with a mirror and take advantage of the mirror to performself-fixing. As a conclusion, the proposed system (SAS) allowed a mobile robotsome ability to fix its exterior in one simplified context.

Industrial robots are increasingly used within the manufacturing industry, especially in collaborative applications, where robots and operators are intended to work together in certain tasks. This collaboration needs to be safe, to ensure that an operator does not get injured in any way. One of several solutions to this is to use virtual safety zones, which limits the robots working range and area to operate within, and may be more flexible than physical fences. When the robot exceeds the allowed limit of the virtual safety zone, a control system that monitors the robot position, forces to robot to stop. Depending on the current speed and payload of the robot, the initialized stop has a braking distance until the robot has completely stopped. How far the separation distance between human and robot must be, is calculated using ISO-standard guidelines when doing risk assessments. To support affected personnel in their work, an investigation and experimentation of braking distances among several robots has been conducted. These testing experiments have been designed to simulate a collaborative operation which is an excessive risk in a robot cell. The tests have been performed with various speeds and payloads, for comparison between the robot models and for validation against already existing data. The difference with this study compared to existing ones is that several robot axis’ are used simultaneously in the testing movements, which is a benefit since a robot rarely operates with only one axis at a time.  Main results of the performed tests are that the robot doesn’t obtain speeds over 2000 mm/s when axis 1 is not involved, before the virtual safety zone is reached. Axis 1 can generate the highest speeds overall, and is therefore a significant factor of the braking distance. The results and conclusions from this thesis states that these kinds of tests give useful information to the industry when it comes to safety separation distance and risk assessments. When applying the information in a correct way, the benefits are that a shorter safety separation distance can be used without compromising on safety. This leads to great advantages in robot cell design, because space is limited on the factory floor.

Eelume AS is a norwegian company that develops autonomous underwater vehicles. Their flagship model is an underwater snake robot that performs inspection, maintenance and repair operations. For the time being, Eelume has been using acoustic communications between their AUV's and the docking station, but it has a big drawback: bandwidth. Eelume is interested in streaming live high-quality video from their AUV's to the docking station, which requires several megabits per second. As underwater radio frequency communications are not possible, wireless optical communications seem to be the best available alternative.   The focus of this Thesis is to design an underwater wireless optical communications system that could be implemented on the Eelume AUV, although it is designed as a standalone embedded system that could be integrated into any other platform. Two prototypes were designed and tested through-air: a low-cost system featuring a PIN photodiode that can stream a 1.5 Mbps video signal over 0.5 meters and a high-sensitivity system featuring an avalanche photodiode that can stream a 2.5 Mbps video signal over 10.5 meters.   Even if further underwater testing is needed and some inherent limitations in the design like the precise calibration or the ambient light noise effects could be mitigated. The results achieved by this high-sensitivity system demonstrates that a high-bandwidth mid-range underwater wireless optical communication system featuring a blue/green LED array as the light source and an avalanche photodiode as the photodetector is a viable solution for streaming live high-quality video over several meters even in very turbid seawaters.

One longstanding challenge in the field of robotics has been the robust and reliable grasping of objects of unknown shape. Part of that challenge lies in reconstructing the object’s shape using only limited observations. Most approaches use either visual or tactile information to reconstruct the shape, having to face issues resulting from the limitations of the chosen modality. This thesis tries to combine the strengths of visual and tactile observations by taking the result from an existing visual approach and refining that result through sparse tactile glances. The existing approach produces potential shape hypotheses in voxel space which get combined into one final shape. This thesis takes that final shape and determines voxels of interest using either entropy or variance. These voxels will be targeted by the exploration, providing information about these voxels. This information will be used to assign weights to the original hypotheses in order for the combined shape to better fit the observations. All explorations are simulated and evaluated in MATLAB. The resulting shapes are evaluated based on their Jaccard Index with the ground truth model. The algorithm leads to improvements in the Jaccard Index, but not to drastically different looking shapes.

Friction Stir Welding has seen a fast uptake in many industry segments. Mechanical properties superior to fusion welding, the ability to weld "unweldable" aluminium alloys and low distortion are often described as the main reasons for the fast industrial implementation of FSW. Most existing applications consist of long straight welding joints. Applications with complex weld geometries, however, are rarely produced by FSW. These geometries can induce thermal variations during the welding process, thus making it challenging to maintain a consistent weld quality. In-process adaptation of weld parameters to respond to geometrical variations and other environmental variants allow new design opportunities for FSW. Weld quality has been shown to be reliant on the welding temperature. However, the optimal methodology to control the temperature is still under development.The research work presented in this thesis focuses on some steps to take in order to reach the improvement of the FSW temperature controller, thus reach a better and consistent weld quality. In the present work different temperature methods were evaluated. Temperature measurements acquired by the tool-workpiece thermocouple (TWT) method were accurate and fast, and thereby enhanced suitable for the controller. Different environmental conditions influencing the material heat dissipation were imposed in order to verify the controller effect on the joint quality. In comparison with no controlled weld, the use of the controller enabled a fast optimization of welding parameters for the different conditions, leading to an improvement of the mechanical properties of the joint.For short weld lengths, such as stitch welds, the initial plunge and dwell stages occupy a large part of the total process time. In this work temperature control was applied during these stages. This approach makes the plunge and dwell stages more robust by preventing local material overheating, which could lead to a tool meltdown. The TWT method was demonstrated to allow a good process control during plunging and continuous welding. The approach proposed for control offers weld quality consistency and improvement. Also, it allows a reduction of the time required for the development of optimal parameters, providing a fast adaptation to disturbances during welding and, by decreasing the plunge time, provides a significant decrease on the process time for short welds.

Deploying robots in unknown and complex areas for inspection tasks is becoming a real need for various application scenarios. Recently, there has been an increasing interest to develop and use autonomous aerial robots in environments such as urban voids and subterranean mine tunnels, aiming to decrease the human presence in dangerous or inaccessible areas. These areas are characterized by complete darkness and narrow tunnels, where the ground can often be rough and not traversible for mobile vehicles, thus the developments focus on Micro Aerial Vehicles (MAVs). MAVs are mechanically simple and agile platforms that can navigate through cluttered areas and have the potential to perform complex exploration tasks when equipped with proper onboard sensors. One of the key milestones in the development of autonomous robots is self-exploration. The definition of self-exploration according to [7] is "the act of moving through an unknown environment while building a map that can be used for subsequent navigation". By reaching this milestone, robots would be freed from the limitation of requiring already existing maps for navigation. In this thesis, a frontier-based exploration algorithm is established and evaluated to understand how such method could be used to reach the self-exploration milestone. By marking the border between what is known and unknown the method is able to determine the next desired position for the robot to expand the map. The resulting algorithm, together with a path planning method and 3-dimensional mapping framework, the method was tested and examined in simulated environments with different levels of complexity.

Graden av automation inom industrin ökar ständigt, inom fordonstillverkning fasas allt fler produktionsnära personal ut till förmån för automatiska system och robotar. På Scanias hyttfabrik i Oskarshamn är stora delar av produktionen automatiserad och det söks ständigt efter nya möjligheter till effektivisering via automation. Detta examensarbete syftade till att utvärdera om kvalitetsinspektionen efter applicering av grundfärg kan automatiseras. Genom att analysera processen genomfördes utvärderingar av två automatiska inspektionssystem som i dag används inom fordonstillverkning. Historisk data analyserades för att identifiera vad ett potentiellt automatiskt inspektionssystem bör klara av. I samband med detta samlades information in för att identifiera vad dessa inspektionssystem faktiskt klarar av. Detta ställdes emot varandra och föranledde en investeringskalkyl för att ta fram en återbetalningstid, samt för att upptäcka vilka monetära besparingar det går att erhålla via ett automatiskt inspektionssystem. Analys genomfördes där fördelar och nackdelar med både dagens arbetssätt samt ett eventuellt framtida arbetssätt med automatiska system vägdes samman. Dessutom analyserades det till vilken grad kvalitetinspektionsprocessen hos Scania kan automatiseras, och vilka kvalitativa vinningar det leder till. I tillägg återger examensarbetet generella slutsatser på hur automatiska inspektionssystem fungerar samt presenterar resultat på om tekniken i dagsläget är mogen, och till vilken utsträckning. Avslutningsvis presenteras rekommendationer till Scania utifrån möjligheter och potentiella besparingar. Rekommendationen är att inte investera i ett automatiskt inspektionssystem, eftersom examensarbetet kommit fram till att inget av de utvärderade systemen kan operera på en tillfredsställande nivå. Tekniken är det vill säga inte ännu mogen för att automatisera denna typ av process.

Cyberfysiska system, robotik och simulering är områden i stort fokus eftersom det har en stor del i industri 4.0. Syftet med projektet är att utveckla robotlaboratoriet på Högskolan i Gävle till att vara mer anpassat för införandet av CPS genom att skapa en virtuell robotcell och utveckla griparen hos robotarna. Den virtuella robotcellen kommer att öka säkerheten i laboratoriet och gör så fler personer kan arbeta med robotarna samtidigt. Robotarna och programvarorna som användes i projektet är utvecklade av företaget KUKA. Det användes tre programvaror till den virtuella robotcellen i projektet. De två som användes mest utförde simulering och programmering. Den sista programvaran hade möjligheten att sammankoppla och styra de andra programvarorna. Den virtuella robotcellen utvecklades i simuleringsprogramvaran och där skapades även program som utfördes av de fysiska robotarna i laboratoriet. De fysiska robotarna rörde sig inte på samma sätt som i den virtuella modellen. En felsökning utfördes och problemet fanns hos de fysiska robotarna. KUKA kommer därför utföra ett underhållsarbete på robotarna. Valideringen av den virtuella robotcellen utfördes därför i simuleringsprogrammet. När den fysiska robotcellen fungerar korrekt så kommer den också att kunna användas för att validera den virtuella robotcellen. En programvara som gav griparen fler funktionaliteter installerades på robotarna. Programvaran som användes var utvecklad av KUKA och fanns i laboratoriet. Funktionerna gör griparen lättare att använda vid manuell körning och programmering. Programvaran som utför sammankopplingen av simulering- och programmeringprogramvaran behöver undersökas vidare för att den virtuella robotcellen ska kunna utföra fler funktioner. Den virtuella robotcellen finns tillgänglig för alla på högskolan och kan användas för att få en förståelse för hur robotarna fungerar. Program kan optimeras i den virtuella miljön och utföras av de fysiska robotarna. / Cyber physical systems, robotics and simulation are areas of major focus as they play a big part in industry 4.0. The aim of the project is to develop the robotic laboratory at the University of Gävle to be more adapted to the introduction of CPS by creating a virtual robotic cell and developing the gripper of the robots. The virtual robotic cell are going to increase the safety in the laboratory and make it possible for more people to work with the robots at the same time. The robots and software used in the project were developed by the company KUKA. Three different software’s were used for the virtual robotic cell in the project. The two most commonly used were simulation and programming. The third software had the possibility to interconnect and control the other two software’s. The virtual robotic cell was developed in the simulation software and also programs were created that the physical robots in the laboratory performed. The physical robots did not move in the same way as the virtual model. Troubleshooting was performed and it showed that the problem was with the physical robots. Therefore KUKA will accomplish maintenance work on the robots. The validation of the virtual robotic cell were for that reason performed in the simulation program. When the physical robotic cell is operating correctly it will also be able to validate that the virtual robotic cell is working. A software that gave the gripper more functionality was installed on the robots. The software that was used was developed by KUKA and were available in the laboratory. The functions make the gripper easier to manage in manual driving and programming. The software that performs the interconnection of simulation and programming software is in need of further analyzation in order for the virtual robotic cell to be able to execute more functions. The virtual robotic cell is available for everyone at the university and can be used to understand how the robots work. Programs can be optimized in the virtual environment and performed by the physical robots.

Nowadays safety, along with efficiency, is one of the two strongest shaping forces of the automotive world, with advanced active safety applications being the major concentration of effort. Their development depends heavily on the quality of sensor data, a detailed measure of which is often up to the automotive manufacturers to derive, since the original equipment manufacturers (OEMs) may not disclose it on trade secrecy grounds. A model would not only provide a measure of the real-world performance of the sensor, but would also enable a higher degree of simulation accuracy which is vital to active safety function development. This is largely due to the high cost and risk involved in testing, a significant part of which is possible to be done in simulation alone. This thesis is an effort to derive a sensor model on behalf of Volvo Trucks of the performance of one of the most crucial sensors in current active safety - a lane detection camera.The work is focused on investigating approaches for modelling and simulation implementation of the lane estimation process within the black-box camera using reverse-engineering of the sensor's principles of operation. The main areas of analysis to define the factors that affect performance are the optics, image sensor, software and computer vision algorithms, and system interface. Each of them is considered separately and then methods for modelling are proposed, motivated, and derived accordingly. Finally, the finished model is evaluated to provide a measure of work success and a basis for further development. / Säkerhet är idag, tillsammans med effektivitet, en av de två starkaste förändringskrafterna i bilvärlden. Störst fokus ligger på avancerade aktiva säkerhetsfunktioner. Deras utveckling beror till stor del på kvaliteten på sensordata. En detaljerad modell för sensordata är ofta upp till fordonstillverkarna att härleda, eftersom tillverkare av sensorn ofta inte vill lämna ut sådan information. En modell ger inte bara ett mått på den verkliga prestandan hos sensorn, men ger också möjlighet till en högre grad av simuleringsnoggrannhet vilket är avgörande för utveckling av aktiva säkerhetsfunktioner. Tester är kostsamma och medför risker och en noggrann modell gör att tester kan utföras i simulering vilket minskar kostnader och risker. I denna avhandling härleds en sensormodell på uppdrag av Volvo Lastvagnar. Sensorn i fråga är en av de viktigaste sensorerna i det nuvarande aktiva säkerhetssystemet, kameran för att följa en körfil på vägen. Arbetet är fokuserat på undersökning av metoder för modellering och simulering av processen för filföljning baserat på sensorns funktionsprinciper. De viktigaste områdena för analys för att definiera de faktorer som påverkar prestanda är optik, bildsensorn, programvara, datorseendealgoritmer och systemets gränssnitt. Var och en av dessa behandlas separat och sedan föreslås och motiveras metoder för modellering. Slutligen utvärderas den färdiga modellen för att ge ett mått på hur framgångsrikt arbetet varit samt för att lägga en grund för ytterligare utveckling. / <p>The thesis work was carried out at Volvo Group Trucks Technology in Göteborg, Sweden. Supervisor for GTT: Mansour Keshavarz.</p>