Thesis (MScEng)--Stellenbosch University, 2012. / ENGLISH ABSTRACT: This thesis describes the process of developing a user interface for a seven degree of freedom (DOF), minimally invasive surgical robot. For the first two main stages of the overall project, completed by previous students, a primary slave manipulator (PSM) and a secondary slave manipulator (SSM) were developed. The stage in this thesis concentrates on creating a joystick that can control the combined movement of the PSM and SSM. Background information on the field of robotic surgery, with specific reference to current systems’ user interfaces, is given and the technical aspects of the PSM and SSM are determined. This is followed by the motivation and main objectives of the thesis. Objectives were divided into the main categories of mechanical design, electronic design, control system design and testing. The mechanical design of the joystick progresses through a concept development stage, before a final seven DOF articulated arm design is presented and evaluated based on engineering specifications. Aluminium is used as the construction material; electromagnetic brakes are specified for each joint, leading to the final assembly, which is a constructed joystick fulfilling all requirements. The electronic design implements magnetic rotary encoders for the joystick’s position and orientation tracking as well as designs of the necessary power and control circuitry to enable correct joystick functioning. The interfacing of the PSM and SSM had to enable successful communication capabilities between the master and the slave. Several necessary adjustments were therefore made to the slave system, after which the joystick and robot were electronically interfaced to provide a direct serial communication line. For control system design, the joystick and robot were modelled according to the Denavit-Hartenberg principle, which allows direct relation between the position and orientation of the respective end effectors on the joystick and robot sides. Forward kinematic equations were then applied to the joystick; the desired position and orientation of the robot end effector were determined, and inverse kinematic equations were applied to these data to establish the robot’s joint variables. This stage ended with the development of two operational modes: one where only the SSM motors are controlled in order for the slave to follow the master’s movements, and the other where the PSM’s motors are controlled separately. The simultaneous control of all robot motors could not be demonstrated due to fundamental mechanical flaws in the PSM and SSM designs. Finally, testing was undertaken to demonstrate movement control of the robot by the joystick. The intuitiveness of the product was also tested successfully. The study ends with the presentation of the conclusions, the main conclusions being the successful development and testing of a joystick that controls the movement of a surgical robot, as well as the achievement of all main thesis objectives. / AFRIKAANSE OPSOMMING: Hierdie tesis beskryf die proses vir die ontwikkeling van ’n gebruikerskoppelvlak vir ’n sewevryheidsgraad-, minimaal indringende chirurgiese robot. In die eerste twee hoofstadia van die algehele projek, voltooi deur ander studente, is ’n primêre slaafmanipuleerder (PSM) en ’n sekondêre slaafmanipuleerder (SSM) ontwikkel. Die stadium in hierdie tesis konsentreer op die skep van ’n stuurstok waarmee die gekombineerde beweging van die PSM en SSM beheer kan word. Agtergrondinligting oor die gebied van robotiese chirurgie word verskaf, met spesifieke verwysing na die gebruikerskoppelvlakke van huidige stelsels, en die spesifikasies van die PSM en SSM word vasgestel. Daarna volg die beweegrede sowel as die belangrikste oogmerke van die projek. Die oogmerke is in die hoofafdelings van meganiese ontwerp, elektroniese ontwerp, beheerstelselontwerp en toetsing verdeel. Die meganiese ontwerp van die stuurstok behels ’n konsepontwikkelingstadium, wat uitloop op ’n finale sewevryheidsgraad-ontwerp, wat dan op grond van ingenieurspesifikasies aangebied en beoordeel word. Aluminium word as boumateriaal gebruik; elektromagnetiese remme word vir elke koppeling gespesifiseer, en die finale samestel is ’n gekonstrueerde stuurstok wat aan alle vereistes voldoen. Die elektroniese ontwerp behels die gebruik van magnetiese draaikodeerders om die stuurstok se posisie en oriëntasie te bepaal, sowel as meganismes met die nodige krag- en beheerstroombaanwerk om die stuurstok reg te laat funksioneer. ’n Koppelvlak tussen die PSM en die SSM moes suksesvolle kommunikasie tussen die meester en die slaaf bewerkstellig. Verskeie nodige aanpassings is dus aan die slaafstelsel aangebring, waarna die stuurstok en robot elektronies gekoppel is om ’n direkte reekskommunikasielyn te skep. Vir beheerstelselontwerp is die stuurstok en robot volgens die Denavit- Hartenberg-beginsel gemodelleer, wat ’n direkte verhouding tussen die posisie en oriëntasie van die onderskeie eindpunt-effektors aan die stuurstok- en robotkant daarstel. Voorwaartse kinematiese vergelykings is daarna op die stuurstok toegepas; die gewenste posisie en oriëntasie van die robotiese eindpunt-effektor is bepaal, waarna terugwaartse kinematiese vergelykings op hierdie data toegepas is om die robot se koppelingveranderlikes te bepaal. Hierdie afdeling word afgesluit met die ontwikkeling van twee bedryfsmodusse: een waar slegs die SSM-motore beheer word sodat die slaaf die meester se bewegings kan navolg, en die ander waar die PSM se motore afsonderlik beheer word. Die gelyktydige beheer van al die robotmotore kon nie getoon word nie weens fundamentele meganiese tekortkominge in die PSM- en SSM-ontwerp. Laastens is ’n toets uitgevoer om die bewegingsbeheer van die robot deur die stuurstok te toon. Die intuïtiwiteit van die produk is ook suksesvol getoets. Die studie sluit af met die projekgevolgtrekkings, waarvan die belangrikste die suksesvolle ontwikkeling en toetsing van ’n stuurstok is wat daarin slaag om die beweging van ’n chirurgiese robot te beheer, sowel as die verwesenliking van alle hoofprojekoogmerke.

L’objectif de notre travail était, d’une part, de déterminer l’impact du délai de transmission des informations en téléchirurgie robotique, et d’autre part, de valider le rôle du simulateur robotique dans la formation en chirurgie robotique. Deux études ont été réalisées pour la partie concernant le délai. Il a été prouvé que la performance chirurgicale se détériore exponentiellement lorsque le délai augmente. Il est préférable d’envisager un geste de téléchirurgie associé à un délai inférieur à 200 ms, et des délais plus de 800 ms ne sont pas adaptés à la téléchirurgie. D’autre part, les chirurgiens ont démontré la capacité de s’adapter au délai par l’entraînement. Dans la deuxième partie, nous avons développé un curriculum sur un simulateur robotique pour une procédure chirurgicale robotique d’urologie. Une autre étude a été réalisée pour déterminer la validité d’un nouveau simulateur robotique (le XTT) comme un outil d’évaluation des techniques d’assistance en chirurgie robotique. / The objectives of our work were, on the one hand, to determine the impact of data-transmission latency in robotic telesurgey, and on the other hand, to validate the role of robotic simulator in robotic surgery training. Two studies were realized in the part concerning latency. It was proved that surgical performance deteriorates exponentially as latency increases. It is preferable to perform telesurgey with a delay less than 200 ms. The delay higher than 800 ms is not suitable for telesurgery. On the other hand, surgeons had demonstrated the capacity to adapt to delay through training. In the second part, we developed a curriculum on a robotic simulator for a urologic surgical procedure. Another study was realized to determine the validity of a new robotic simulator (the XTT) as an evaluation tool of robotic surgical assistance skills.

Serial manipulators, which have large work space with respect to their own volume and occupied floor space, are the most common industrial robots by far. However, in many environments the situation is unstructured and less predictable, such as aboard a space station, a nuclear waste retrieval site, or a lunar base construction site. It is almost impossible to design a single robotic system which can meet all the requirements for every task. In these circumstances, it is important to deploy a modular reconfigurable robotic system, which is suitable to various task requirements. Modular reconfigurable robots have a variety of attributes that are well suited to for these conditions, including: the ability to serve as many different tools at once (saving weight), packing into compressed forms (saving space) and having high levels of redundany(increasing robustness). By easy disassembly and reassembly features, this serial modular robotic system will bring advantages to small and medium enterprise to save costs in the long term. This thesis focuses on developing such a serial reconfigurable modular robotic system with remote control functionality. The robotic arms are assembled by PowerCube Modules with cubic outward appearance. The control and power electronics are fully integrated on the connector block inside of the modules. Those modules are connected in series by looping through, and can work completely independently. The communication between robotic arms and PC controller is connected by the Control Area Network bus. CAN protocol detects and corrects transmission errors caused by electromagnetic interference. The local PC can directly control the robotic arm via Visual Basic code, and it can also be treated as server controller. Client PCs can access and control the robotic arm remotely through Socket communication mechanism with certain IP address and port number. A Java3D model is created on the client PC synchronously for customers online monitoring and control. The forward and inverse kinematic analysis is solved by Vector Algebraic Method. The Neutral Network Method is also introduced to improve the kinematic analysis. Multiple-layer networks are capable of approximating any function with finite number of discontinuities. For learning the inverse kinematics neural network needs information about coordinates, joint angles and actuator positions. The desired Cartesian coordinates are given as input to the neural network that returns actuator positions as output. The robot position is simulated using these actuator positions as reference values for each actuator.

Metallic robotic arms, or manipulators, currently dominate automated industrial operations, but due to their intrinsic weight, have limited usefulness for large-scale applications in terms of precision, speed, and repeatability. This thesis focuses on exploring the feasibility of using polymeric composite materials for the construction of long-reach robotic arms. Different manipulator layouts were investigated and an ideal design was selected for a robotic arm that has a 5 [m] reach, 50 [kg] payload, and is intended to operate on large objects with complex curvature. The cross-sectional geometry of the links of the arm were analyzed for optimal stiffness- and strength-to-weight ratios that are capable of preserving high precision and repeatability under time-dependent external excitations. The results lead to a novel multi-segment link design and method of production. A proof-of-concept prototype of a two degrees-of-freedom (2-DOF) robotic arm with a reach of 1.75 [m] was developed. Both static and repeatability testing were performed for verification. The results indicated that the prototype robot main-arm constructed of carbon fiber-epoxy composite material provides good stiffness-to-weight and strength-to-weight ratios. Finite element analysis (FEA) was performed on a 3-D computer model of the arm. Successful verification led to the use of the 3-D model to define the dimensions of an industrial-sized robotic arm. The results obtained indicate high stiffness and minimal deflection while achieving a significant weight reduction when compared to commercial arms of the same size and capability.

Although much progress has been made in the field of robotic mapping, many challenges remain including: efficient semantic segmentation using RGB-D sensors, map representations that include complex features (structures and objects), and interfaces for interactive annotation of maps. This thesis addresses how prior knowledge of semi-structured human environments can be leveraged to improve segmentation, mapping, and semantic annotation of maps. We present an organized connected component approach for segmenting RGB-D data into planes and clusters. These segments serve as input to our mapping approach that utilizes them as planar landmarks and object landmarks for Simultaneous Localization and Mapping (SLAM), providing necessary information for service robot tasks and improving data association and loop closure. These features are meaningful to humans, enabling annotation of mapped features to establish common ground and simplifying tasking. A modular, open-source software framework, the OmniMapper, is also presented that allows a number of different sensors and features to be combined to generate a combined map representation, and enabling easy addition of new feature types.

Improving robotic manipulation is critical for robots to be actively useful in realworld factories and homes. While some success has been shown in simulation and controlled environments, robots are slow, clumsy, and not general or robust enough when interacting with their environment. By contrast, humans effortlessly manipulate objects. One possible reason for this discrepancy is that, starting from birth, humans have years of experience to collect data and develop good internal models of what happens when they manipulate objects. If robots could also learn models from a large amount of real data, perhaps they, too, could become more capable manipulators. In this thesis, we propose to improve robotic manipulation by solving two problems. First, we look at how robots can collect a large amount of manipulation data without human intervention. Second, we study how to build statistical models of robotic manipulation from the collected data. These data-driven models can then be used for planning more robust manipulation actions. To solve the first problem of enabling large data collection, we perform several different robotic manipulation experiments and use these as case studies. We study bin-picking, post-grasp manipulation, pushing, tray tilting, planar grasping, and regrasping. These case studies allow us to gain insights on how robots can collect a large amount of accurate data with minimal human intervention. To solve the second problem of statistically modeling manipulation actions, we propose models for different parts of various manipulation actions. First, we look at how to model post-grasp manipulation actions by modeling the probability distribution of where an object ends up in a robot’s hand, and how this affects its success rate at various tasks such as placing or insertion. Second, we model how robots can change the pose of an object in their hand with regrasp actions. Third, we improve on the place and pick regrasp action by modeling each separately with more data. These learned data-driven models can then be used for planning more robust and accurate manipulation actions.

The theme of this diploma paper is The technical aspects of robotic milking. This work has been written to provide an overview of the technologies used in robotic milking. In the first part there is a description of the general problems of cattle raising. In subsequent chapters of the work there is a technical description of the selected machine parts used during milking. The next part of this work is made of chapters which deal with the history of robotic milking, introduction of individual manufacturers and technologies used by these producers. Then, these technologies are evaluated in terms of the economic costs of their usage and their technical differences. In the conclusion, there are presented some novelties in the field of robotic milking , coming on the market in the next few years.

This thesis describes the design and construction of modular robotic system. Proposal of its management and simulation parameters given.

Robotic Process Automation (RPA) is an automation technology that has become increasingly common in recent years, as demands have increased on the use of new technologies to compete in the global economy. RPA means that a software robot is placed on top of existing systems where it performs tasks in a similar way as a human being previously did. No changes need to be made to the underlying systems, which means that RPA is rather quick and easy to implement at a relatively low cost. There are many advantages to the usage of RPA, but not all processes are suitable for the implementation of robots. The purpose of the study has been to help businesses select the parts of the organization that can be automated by identifying criteria which means that RPA can be applied to a process. In order to fulfill the purpose of the study, information has been collected through a literature study and semi-structured interviews with consultants working in the RPA area. The result shows that there are several criteria that increase the suitability of an RPA implementation on a process, but also that the importance of different criteria varies from situation to situation. The study has also identified criteria that must be met in order to be able to implement robots at all. Since the appearance of different companies and processes varies greatly, the criteria should be seen as a guideline where each situation is examined individually to identify what is important right there. Initially, a detailed analysis needs to be done of a candidate process in order to determine the degree of suitability and whether RPA is a good solution in that case. / Robotic Process Automation (RPA) är en automatiseringsteknik som blivit allt vanligare de senaste åren. Kraven har ökat på användandet av nya teknologier för att kunna konkurrera i den globala ekonomin. RPA innebär att en mjukvarurobot placeras ovanpå befintliga system där den utför uppgifter på ett likadant sätt som en människa tidigare gjorde. Inga ändringar behöver göras på de underliggande systemen vilket medför att RPA är förhållandevis snabbt och enkelt att införa med relativt låg kostnad. Det finns många fördelar med användningen av RPA, men alla processer är inte lämpliga att implementera robotar på. Studiens syfte har varit att hjälpa verksamheter att välja ut de delar i organisationen som kan automatiseras genom att identifiera kriterier som medför att RPA går att använda på en process. För att uppfylla studiens syfte har information samlats in via en litteraturöversikt och semistrukturerade intervjuer med konsulter som arbetar inom RPA-området. Resultatet visar att det finns ett flertal kriterier som ökar lämplighetsgraden för införandet av RPA på en process, men också att vikten av olika kriterier varierar från situation till situation. Studien har även identifierat kriterier som måste uppfyllas för att det överhuvudtaget ska gå att implementera robotar. Eftersom utseendet på olika företag och processer varierar kraftigt så bör kriterierna ses som en riktlinje där varje situation utreds individuellt för att identifiera vad som är viktigt just där. Initialt behöver en utförlig analys göras av en kandidatprocess för att kunna fastställa lämplighetsgraden och om RPA är en bra lösning i det fallet.

We still struggle to deliver autonomous robots that perform manipulation tasks as simple for a human as picking up items. A portion of the difficulty of this task lays on the fact that such operation requires a robot that can deal with uncertainty in an unstructured environment. We propose in this thesis the use of visual and tactile perception for providing solutions that can improve the robustness of a robotic manipulator in such environment. In this thesis, we approach robotic grasping using a single 3D point cloud with a partial view of the objects present in the scene. Moreover, the objects are unknown: they have not been previously recognised and we do not have a 3D model to compute candidate grasping points. In experimentation, we prove that our solution is fast and robust, taking in average 17 ms to find a grasp which is stable 85% of the time. Tactile sensors provide a rich source of information regarding the contact experienced by a robotic hand during the manipulation of an object. In this thesis, we exploit with deep learning this type of data for approaching the prediction of the stability of a grasp and the detection of the direction of slip of a contacted object. We prove that our solutions could correctly predict stability 76% of the time with a single tactile reading. We also demonstrate that learning temporal and spatial patterns leads to detections of the direction of slip which are correct up to 82% of the time and are only delayed 50 ms after the actual slip event begins. Despite the good results achieved on the previous two tactile tasks, this data modality has a serious flaw: it can only be registered during contact. In contrast, humans can estimate the feeling of grasping an object just by looking at it. Inspired by this, we present in this thesis our contributions for learning to generate tactile responses from vision. We propose a supervised solution based on training a deep neural network that models the behaviour of a tactile sensor, given 3D visual information of the target object and grasp data as an input. As a result, our system has to learn to link vision to touch. We prove in experimentation that our system learns to generate tactile responses on a set of 12 items, being off by only 0.06 relative error points. Furthermore, we also experiment with a semi-supervised solution for learning this task with a reduced need of labelled data. In experimentation, we show that it learns our tactile data generation task with 50% less data than the supervised solution, incrementing only 17% the error. Last, we introduce our work in the generation of candidate grasps which are improved through simulation of the tactile responses they would generate. This work unifies the contributions presented in this thesis, as it applies modules on calculating grasps, stability prediction and tactile data generation. In early experimentation, it finds grasps which are more stable than the original ones produced by our method based on 3D point clouds. / This doctoral thesis has been carried out with the support of the Spanish Ministry of Economy, Industry and Competitiveness through the grant BES-2016-078290.