This thesis presents the development of a model for analyzing the design of an automated live-bird transfer system (LBTS) developed at Georgia Tech. One of the most fundamental tasks in the automated transferring is to design and control a grasping system that is capable of accommodating a specified range of objects without causing damage. However, unlike grasping in robotic research that focuses on dexterous manipulation of a single object, repetitive transfer of live objects in a production line requires continuous grasping at high-speed. This thesis research investigates the use of rotating fingers (capable of undergoing large deflections) to cradle live birds on a moving conveyor for subsequent handling. As compared to fingers with multiple active joints, flexible fingers have many merits, for they are lightweight and have no relative individually moving parts. Their ability to accommodate a limited range of varying sizes, shapes, and the natural reactions of some objects makes rubber fingers an attractive candidate for use as graspers in a high-speed production setting. However, the advantages of flexible fingers are seldom exploited for grasping because of the complex analysis involved in the design. In order to reduce the number of birds and hardware/software design configurations to be tested, a good understanding of the object dynamics throughout the grasping process is necessary. In this thesis, a quasi-static model has been developed for predicting the contact force between a moving object and a rotating finger. The model has been validated with the experimentally measured data and the computed results using finite element (FE) methods. Finally, an illustrative application of the validated model has been demonstrated in the design of a rotating hand used in the automated LBTS. As illustrated in the simulation results, the computed contact forces can be used as a basis for predicting potential bruises on the bird that may be caused by the rotating fingers. The analytical model presented in this paper provides a rational basis for optimizing the design of the grasping system and developing a controller for a high-speed transfer system. It is expected that the analysis presented here can be readily extended to other dynamic systems involving the use of flexible beams.

abstract: The human hand is a complex biological system. Humans have evolved a unique ability to use the hand for a wide range of tasks, including activities of daily living such as successfully grasping and manipulating objects, i.e., lifting a cup of coffee without spilling. Despite the ubiquitous nature of hand use in everyday activities involving object manipulations, there is currently an incomplete understanding of the cortical sensorimotor mechanisms underlying this important behavior. One critical aspect of natural object grasping is the coordination of where the fingers make contact with an object and how much force is applied following contact. Such force-to-position modulation is critical for successful manipulation. However, the neural mechanisms underlying these motor processes remain less understood, as previous experiments have utilized protocols with fixed contact points which likely rely on different neural mechanisms from those involved in grasping at unconstrained contacts. To address this gap in the motor neuroscience field, transcranial magnetic stimulation (TMS) and electroencephalography (EEG) were used to investigate the role of primary motor cortex (M1), as well as other important cortical regions in the grasping network, during the planning and execution of object grasping and manipulation. The results of virtual lesions induced by TMS and EEG revealed grasp context-specific cortical mechanisms underlying digit force-to-position coordination, as well as the spatial and temporal dynamics of cortical activity during planning and execution. Together, the present findings provide the foundation for a novel framework accounting for how the central nervous system controls dexterous manipulation. This new knowledge can potentially benefit research in neuroprosthetics and improve the efficacy of neurorehabilitation techniques for patients affected by sensorimotor impairments. / Dissertation/Thesis / Doctoral Dissertation Neuroscience 2017

Le travail présenté s'intéresse à la problématique générale de la mise en oeuvre de mains robotiques à haut niveau de dextérité. Dans ce contexte, nous nous intéressons à la synthèse de prise d'objets en prenant en compte les contraintes propres à la tâche de manipulation visée. La manière dont l'objet est saisi a une importance capitale sur le bon déroulement d'une tâche.Le développement d'algorithmes capables de générer automatiquement des prises optimales implique avant tout la nécessité de définir la notion de prise optimale au regard de la tâche cible. Pour répondre à ce problème, la communauté scientifique propose dans la littérature de nombreux critères de qualité et continue à en développer de nouveaux. Dans cette thèse, nous présentons une extension des travaux proposés avec une étude approfondie de ces critères dans le cadre de la manipulation dextre. Ces critères sont évalués avec une main robotique entièrement actionnée à quatre doigts et seize articulations.Nous quantifions l'efficacité de ces critères dans le cadre de la réalisation de tâches de manipulation fine avec trois types d'objets spécifiques. Deux groupes de critères sont étudiés : d'une part des critères s'appuyant uniquement sur la position des points de contact, et, d'autre part, des critères prenant en compte la cinématique du préhenseur. Cette étude nous a permis de sélectionner un ensemble de critères pertinents pour résoudre le problème de synthèse de prise que nous avons mis en oeuvre dans un processus basé sur une approche évolutionnaire. Cette approche a été validée dans l'environnement de simulation OpenRAVE, puis expérimentalement avec la nouvelle main RoBioSS. / The work presented in this thesis concerns object grasping with dexterous robotic hands. In this work, we are going to focus on the grasp synthesis problem by taking into account the in-hand manipulation task. The initial grasp has a capital role for the successful completion of a given task.In order to develop algorithms which are able to generate automatically correct grasps for a manipulation task, we need to define suitable grasp quality metrics to assess the validity of a grasp. Throughout the years, a large variety of quality measures have been proposed in the literature and researchers keep on developing new ones. However those quality measures are generally developed for simple grippers and for grasping tasks. In this thesis, we will extend the study of selected interesting grasp quality measures for in-hand manipulation tasks. These quality measures will be evaluated on a four finger robotic hand with sixteen fully actuated degrees of freedom.We will assess the chosen quality measures for in-hand manipulation tasks with three different carefully selected type of objects. The quality metrics are classified in two groups, first one focuses exclusively on the location of contact points and the second one considers the kinematics of the robotic hand. The review of these quality measures led us to select the ones meaningful for solving the grasp synthesis problem for in-hand manipulation. The grasping pipeline implemented to generate the correct grasps is based on an evolutionary approach using a mix of the selected quality measures. The proposed approach was tested in the OpenRAVE robotic simulator and also validated experimentally with the new RoBioSS hand.

In near future, robots are envisioned to work alongside humans in professional anddomestic environments without significant restructuring of workspace. Roboticsystems in such setups must be adept at observation, analysis and rational de-cision making. To coexist in an environment, humans and robots will need tointeract and cooperate for multiple tasks. A fundamental such task is the manip-ulation of large objects in work environments which requires cooperation betweenmultiple manipulating agents for load sharing. Collaborative manipulation hasbeen studied in the literature with the focus on multi-agent planning and controlstrategies. However, for a collaborative manipulation task, grasp planning alsoplays a pivotal role in cooperation and task completion.In this work, a novel approach is proposed for collaborative grasping and manipu-lation of large unknown objects. The manipulation task was defined as a sequenceof poses and expected external wrench acting on the target object. In a two-agentmanipulation task, the proposed approach selects a grasp for the second agentafter observing the grasp location of the first agent. The solution is computed ina way that it minimizes the grasp wrenches by load sharing between both agents.To verify the proposed methodology, an online system for human-robot manipu-lation of unknown objects was developed. The system utilized depth informationfrom a fixed Kinect sensor for perception and decision making for a human-robotcollaborative lift-up. Experiments with multiple objects substantiated that theproposed method results in an optimal load sharing despite limited informationand partial observability.

Ingeniero Civil en Computación / Ingeniero Civil Eléctrico / Uno de los desafíos importantes para la Robótica, es la capacidad del robot de manipular objetos de su entorno, ya sea para transportarlos u operarlos de alguna manera. Si bien esta capacidad está prácticamente resuelta en ambientes controlados, es un problema abierto en el caso de robots autónomos y ambientes no controlados, dado que la forma de los objetos, sus características físicas y las cualidades del efector del robot no están acotadas. Como segundo requisito, se busca además que las soluciones sean robustas y funcionen en tiempo real para aumentar las aplicaciones reales de la robótica. Una de las partes centrales de un algoritmo que permita manipular objetos es la detección de puntos de agarre. Esto corresponde a calcular los puntos del objeto por donde un robot debe tomarlo para que este no se caiga. Existen varios algoritmos que intentan dar solución a esta problemática pero solo funcionan para ciertas familias de objetos y en muchos casos toma demasiado tiempo realizar el cálculo. En esta memoria se implementó un algoritmo de manipulación de objetos basado en un método del estado del arte. El algoritmo permite manipular objetos en tiempos razonables y no esta restringido a una familia específica de objetos, aunque los objetos manipulables requieren de cierta simetría axial. El algoritmo se implementó en C++ en un robot PR2, un robot especialmente diseñado para investigación, usando Robot Operating System (ROS) como framework de desarrollo, lo que permitirá que este algoritmo sea usado fácilmente por otros equipos de investigación y robots en diferentes partes del mundo. El algoritmo implementado consta de una etapa de filtrado y segmentación de una nube de puntos, la determinación de los puntos de agarre, muestreo de poses de agarre, descarte de éstas por diferentes criterios, la asignación de puntaje a los agarres y finalmente la ejecución del mejor agarre seleccionado. Los experimentos muestran que el algoritmo permite tomar objetos en simulación y en un robot PR2 real.

No description available.

We still struggle to deliver autonomous robots that perform manipulation tasks as simple for a human as picking up items. A portion of the difficulty of this task lays on the fact that such operation requires a robot that can deal with uncertainty in an unstructured environment. We propose in this thesis the use of visual and tactile perception for providing solutions that can improve the robustness of a robotic manipulator in such environment. In this thesis, we approach robotic grasping using a single 3D point cloud with a partial view of the objects present in the scene. Moreover, the objects are unknown: they have not been previously recognised and we do not have a 3D model to compute candidate grasping points. In experimentation, we prove that our solution is fast and robust, taking in average 17 ms to find a grasp which is stable 85% of the time. Tactile sensors provide a rich source of information regarding the contact experienced by a robotic hand during the manipulation of an object. In this thesis, we exploit with deep learning this type of data for approaching the prediction of the stability of a grasp and the detection of the direction of slip of a contacted object. We prove that our solutions could correctly predict stability 76% of the time with a single tactile reading. We also demonstrate that learning temporal and spatial patterns leads to detections of the direction of slip which are correct up to 82% of the time and are only delayed 50 ms after the actual slip event begins. Despite the good results achieved on the previous two tactile tasks, this data modality has a serious flaw: it can only be registered during contact. In contrast, humans can estimate the feeling of grasping an object just by looking at it. Inspired by this, we present in this thesis our contributions for learning to generate tactile responses from vision. We propose a supervised solution based on training a deep neural network that models the behaviour of a tactile sensor, given 3D visual information of the target object and grasp data as an input. As a result, our system has to learn to link vision to touch. We prove in experimentation that our system learns to generate tactile responses on a set of 12 items, being off by only 0.06 relative error points. Furthermore, we also experiment with a semi-supervised solution for learning this task with a reduced need of labelled data. In experimentation, we show that it learns our tactile data generation task with 50% less data than the supervised solution, incrementing only 17% the error. Last, we introduce our work in the generation of candidate grasps which are improved through simulation of the tactile responses they would generate. This work unifies the contributions presented in this thesis, as it applies modules on calculating grasps, stability prediction and tactile data generation. In early experimentation, it finds grasps which are more stable than the original ones produced by our method based on 3D point clouds. / This doctoral thesis has been carried out with the support of the Spanish Ministry of Economy, Industry and Competitiveness through the grant BES-2016-078290.

Dynamic grasping is the action of, using only contact force, manipulating the position of a moving object in space. Doing so with a robot is a quite complex task in itself, but is one with wide-ranging applications. Today, the work of automating most processes in society undergoes rapidly, and for many of these processes, the grasping of objects has a natural place. This work has explored using deep reinforcement learning techniques for dynamic grasping, in a simulated environment. Deep Deterministic Policy Gradient was chosen and evaluated for the task, both by itself and in combination with the Hindsight Experience Replay buffer. The reinforcement learning agent observed the initial state of the target object and the robot in the environment, simulated using AGX Dynamics, and then determined with what speed to move to which position. The agent's chosen action was relayed to ABB's virtual controller, which controlled the robot in the simulation. This meant that the agent was tasked with, in advance, parametrizing a predefined set of instructions to the robot, in such a way that the moving target object would be grasped and picked up. Doing it in this matter, as opposed to having the agent continuously control the robot, was a necessary challenge making it possible to utilize the intelligence already created for the virtual controller. It also means that transferring the things learned by an agent in a simulated environment to a real-world environment becomes easier. The accuracy of the target policy for the simpler agent was 99.07%, while the accuracy of the agent with the more advanced replay buffer came up to 99.30%. These results show promise for the future, both as we expect further fine-tuning to raise them even more, and as they indicate that deep reinforcement learning methods can be highly applicable to the robotics systems of today.

Grasping novel objects is challenging because of incomplete object data and because of uncertainties inherent in real world applications. To robustly perform grasps on previously unseen objects, feedback from touch is essential. In our research, we study how information from touch sensors can be used to improve grasping novel objects. Since it is not trivial to extract relevant object properties and deduce appropriate actions from touch sensing, we employ machine learning techniques to learn suitable behaviors. We have shown that grasp stability estimation based on touch can be improved by including an approximate notion of object shape. Further we have devised a method to guide local grasp adaptations based on our stability estimation method. Grasp corrections are found by simulating tactile data for grasps in the vicinity of the current grasp. We present several experiments to demonstrate the applicability of our methods. The thesis is concluded by discussing our results and suggesting potential topics for further research. / Att greppa nya föremål är utmanande, både eftersom roboten inte har fullständig information om objekten och på grund av den inneboende osäkerheten i verkliga tillämpningar. Återkoppling från känselsensorer är viktigt för att kunna greppa föremål som inte påträffats tidigare. I vår forskning så studerar vi hur information från känselsensorer kan användas för att förbättra greppandet av nya föremål. Eftersom det är svårt att extrahera relevanta egenskaper om föremål och härleda lämpliga åtgärder, baserat på känselsensorer, så har vi använt maskininlärning för att lära roboten lämpliga beteenden. Vi har visat att uppskattningar av stabiliteten av ett grepp baserat på känselsensorer kan förbättras genom att även använda en grov approximation av föremålets form. Vi har även konstruerat en metod som vägleder lokala justeringar av grepp, baserat på vår metod som uppskattar stabiliteten av ett grepp. Dess justeringar hittas genom att simulera känselsensordata för grepp i närheten av det nuvarande greppet. Vi presenterar flera experiment som demonstrerar tillämpbarheten av våra metoder. Avhandlingen avslutas med en diskussion om våra resultat och förslag på möjliga ämnen för fortsatt forskning. / <p>QC 20170510</p>

Greater than 50% of all spinal cord injuries (SCIs) in humans occur at the cervical level and the biggest desire of quadriplegic patients is recovery of hand and digit function. Several weeks after spinal cord injury, re-organization and re-modeling of spared endogenous pathways occurs and plasticity of both supraspinal and interneuronal networks are believed to mediate functional recovery. Propriospinal interneurons (PNs) are neurons found entirely in the spinal cord with axons projecting to different spinal segments. PNs function by modulating locomotion, integrating supraspinal motor pathways and peripheral sensory afferents. Recent studies have postulated that if PNs are spared following SCI, these neurons can contribute to functional recovery by establishing synaptic connections onto motor neurons. However, to what extent cervical PNs are involved in recovery of reaching behavior is not known. In our first study, we generated a lentiviral vector that permits highly efficient retrograde transport (HiRet) upon uptake at synaptic terminals in order to map supraspinal and interneuronal populations terminating near forelimb motoneurons (MNs) innervating the limb. With this vector, we found neurons labeled within the C3-C4 spinal cord and in the red nucleus, two major populations which are known to modulate forelimb reaching behavior. We also proceeded to use a novel two-viral vector method to specifically label ipsilateral C3-C4 PNs with tetracycline-inducible GFP. Histological analysis showed detailed labeling of somas, dendrites along with axon terminals. Based on this data, we proceeded to determine the contribution of C3-C4 PNs and rubrospinal neurons on forelimb reaching and grasping before and after cervical SCI. In our second study, we have examined a double-infection technique for shutdown of PNs and rubrospinal neurons (RSNs) in adult rats. Adult rats were microinjected with a lentiviral vector expressing tetracycline-inducible inhibitory DREADDs into C6-T1 spinal levels. Adeno-associated viral vectors (AAV2) expressing TetON mixed with GIRK2 were injected into the red nucleus and C3-C4 spinal levels respectively. Rats were tested for deficits in reaching behaviors upon application of doxycycline and clozapine-n-oxide (CNO) administration. No behavioral deficits were observed pre-injury. Rats then received a C5 spinal cord lesion to sever cortical input to forelimb motoneurons and were allowed four weeks to spontaneously recover. Upon re-administration of CNO to activate inhibitory DREADDs, deficits were observed in forelimb reaching. Histological analysis of the C3-C4 spinal cord and red nucleus showed DREADD+ neurons co-expressing GIRK2 in somas and dendrites of PNs and RSNs. PN terminals expressing DREADD were observed near C6-T1 motoneurons and in the brainstem. Control animals did not show substantial deficits with CNO administration. These results indicate both rubro- and propriospinal pathways are necessary for recovery of forelimb reaching. In a separate study, we sought to determine if promoting severed CST sprouting rostral to a C5 lesion near C3-C4 PNs could improve behavioral recovery post SCI. Past studies have examined sprouting and regeneration of corticospinal tract (CST) fibers post-cervical SCI through viral upregulation of key components of the PI3K/Akt/mTOR cascade. We examined the regenerative growth potential of CST fibers that are transduced with AAV2 expressing constituively active Akt3 or STAT3 both separately and in combination (Akt3 + STAT3). We have observed significant increases in CST axonal sprouting and regeneration in Akt3 and Akt3 + STAT3 transduced samples. However, no recovery was observed as animals transduced with viral constitutively active Akt3 displayed an epileptic phenotype. Further, epileptic animals with constitutively active Akt3 were found to have significant cortical neuron cell hypertrophy, activatived astrogliosis, increased dendritic arbors and hemimegencephalitis (HME). These results indicate a new model for examining mechanisms of HME and mTOR hyperactivity-induced epilepsy in adult rodents. / Biomedical Sciences