Robotic grasping and manipulation of three-dimensional deformable objects is a complex task that currently does not have robust and flexible solutions. Deformable objects include a wide variety of elastic and inelastic objects that change size and shape during manipulation. The development of adaptable methods for grasping and autonomously controlling the shape of three-dimensional deformable objects will benefit many commercial applications, including shaping parts for assembly in manufacturing, manipulating food for packaging and controlling tissues during robotic surgery. Controlling a deformable object to a desired shape requires first choosing contact points on the object's surface. Next, the robotic hand is positioned in the correct position and orientation to grasp and deform the object. After deformation, the object is assessed to evaluate the quality of the shape control procedure. In many cases, this process is completed without knowing the object's properties or behaviour before deformation. This work proposes and implements the framework for a robotic arm and hand system to control the shape of a previously unseen deformable object autonomously. Significant original contributions are made in developing an original algorithm to plan contact points on a three-dimensional object for grasping and shape control. This research uses a novel object representation to reduce the dimensionality of the deformable object manipulation problem. A path planning algorithm guides the robot arm to the optimal valid grasp pose to deform the object at the determined contact points. Additional contributions include developing a multi-view assessment strategy to determine the quality of the deformation towards the desired shape. The system completes the objectives using depth and colour images captured from a single point of view to locate and identify a previously unseen three-dimensional object within a robotic workspace. After estimating the unknown object's geometry, initial grasp contact points are planned to control the object to the desired shape. The grasp points are used to plan and execute a collision-free trajectory for the robot manipulator to place the robotic hand in the optimal position and orientation to grasp and deform the object. After the deformation is complete, the object is moved to a variety of assessment positions to determine the success of the shape control procedure. The system is validated experimentally on a variety of deformable three-dimensional objects.

Dexterous robotic manipulation of non-rigid objects is a challenging problem but necessary to explore as robots are increasingly interacting with more complex environments in which such objects are frequently present. In particular, common manipulation tasks such as molding clay to a target shape or picking fruits and vegetables for use in the kitchen, require a high-level understanding of the scene and objects. Commonly, the behavior of non-rigid objects is described by a model. Although, well-established modeling techniques are difficult to apply in robotic tasks since objects and their properties are unknown in such unstructured environments. This work proposes a sensing and modeling framework to measure the 3D shape deformation of non-rigid objects. Unlike traditional methods, this framework explores data-driven learning techniques focused on shape representation and deformation dynamics prediction using a graph-based approach. The proposal is validated experimentally, analyzing the performance of the representation model to capture the current state of the non-rigid object shape. In addition, the performance of the prediction model is analyzed in terms of its ability to produce future states of the non-rigid object shape due to the manipulation actions of the robotic system. The results suggest that the representation model is able to produce graphs that closely capture the deformation behavior of the non-rigid object. Whereas, the prediction model produces visually plausible graphs when short-term predictions are required.

Non-rigid shapes are generally known as objects whose three dimensional geometry may deform by internal and/or external forces. Deformable shapes are all around us, ranging from protein molecules, to natural objects such as the trees in the forest or the fruits in our gardens, and even human bodies. Two deformable shapes may be related by isometry, which means their intrinsic geometries are preserved, even though their extrinsic geometries are dissimilar. An important problem in the analysis of the deformable shapes is to identify the three-dimensional correspondence between two isometric shapes, given that the two shapes may be deviated from isometry by intrinsic distortions. A major challenge is that non-rigid shapes have large degrees of freedom on how to deform. Nevertheless, irrespective of how they are deformed, they may be aligned such that the geodesic distance between two arbitrary points on two shapes are nearly equal. Such alignment may be expressed by a permutation matrix (a matrix with binary entries) that corresponds to every paired geodesic distance in between the two shapes. The alignment involves searching the space over all possible mappings (that is all the permutations) to locate the one that minimizes the amount of deviation from isometry. A brute-force search to locate the correspondence is not computationally feasible. This thesis introduces a novel approach created to locate such correspondences, in spite of the large solution space that encompasses all possible mappings and the presence of intrinsic distortion. In order to find correspondences between two shapes, the first step is to create a suitable descriptor to accurately describe the deformable shapes. To this end, we developed deformation-invariant metric descriptors. A descriptor constitutes pair-wise geodesic distances among arbitrary number of discrete points that represent the topology of the non-rigid shape. Our descriptor provides isometric-invariant representation of the shape irrespective of its circumstantial deformation. Two isometric-invariant descriptors, representing two candidate deformable shapes, are the input parameters to our optimization algorithm. We then proceed to locate the permutation matrix that aligns the two descriptors, that minimizes the deviation from isometry. Once we have developed such a descriptor, we turn our attention to finding correspondences between non deformable shapes. In this study, we investigate the use of both classical and quantum particle swarm optimization (PSO) algorithms for this task. To explore the merits of variants of PSO, integer optimization involving test functions with large dimensions were performed, and the results and the analysis suggest that quantum PSO is more effective optimization method than its classical PSO counterpart. Further, a scheme is proposed to structure the solution space, composed of permutation matrices, in lexicographic ordering. The search in the solution space is accordingly simplified to integer optimization to find the integer rank of the targeted permutation matrix. Empirical results suggest that this scheme improves the scalability of quantum PSO across large solution spaces. Yet, quantum PSO's global search capability requires assistance in order to more effectively manoeuvre through the local extrema prevalent in the large solution spaces. A mutation based genetic algorithm (GA) is employed to augment the search diversity of quantum PSO when/if the swarm stagnates among the local extrema. The mutation based GA instantly disengages the optimization engine from the local extrema in order to reorient the optimization energy to the trajectories that steer to the global extrema, or the targeted permutation matrix. Our resultant optimization algorithm combines quantum Particle Swarm Optimization (PSO) and mutation based Genetic Algorithm (GA). Empirical results show that the optimization method presented is scalable and efficient on standard hardware across different solution space sizes. The performance of the optimization method, in simulations and on various near-isometric shapes, is discussed. In all cases investigated, the method could successfully identify the correspondence among the non-rigid deformable shapes that were related by isometry.

O crescente número de sistemas de animação que utilizam a cinemática para gerar movimento de objetos vem levando os pesquisadores a buscar outras alternativas para produzir resultados mais realistas. Com base nesta premissa, vários autores começaram a estudar a geração de movimento de objetos sintéticos através da aplicação da dinâmica. Assim surgiram os modelos baseados em leis físicas. Num primeiro momento foram abordados apenas objetos rígidos, passando-se mais tarde a considerar objetos articulados e, por fim, aqueles com características elásticas, também denominados de objetos flexíveis ou deformáveis. O objetivo principal do trabalho é a definição de um modelo para simulação de objetos deformáveis no espaço euclidiano. São abordados tanto o modelo geométrico utilizado como o modelo físico, sendo ressaltadas as forças aplicadas sobre o objeto e as restrições que podem ser impostas pelo mundo virtual no qual o mesmo está inserido. Dentre as forças descritas, pode-se destacar: força gravitacional, elasticidade, força de curvatura e torção, colisão e atrito. A fundamentação do trabalho desenvolvido é apresentada na forma de uma introdução aos sistemas de animação, enfatizando os sistemas baseados em leis físicas e de uma revisão bibliográfica dos métodos de deformação existentes. No que diz respeito à colisão de objetos elásticos, são descritos tanto os métodos estudados para a solução das mesmas, como as técnicas para detecção do choque. A simulação do movimento é descrita sob dois aspectos: o algoritmo utilizado para a geração do movimento e a integração numérica das equações diferenciais no tempo. É abordado ainda, em detalhe, o protótipo desenvolvido com o propósito de validar o modelo proposto, sendo descrita a linguagem criada a fim de permitir a especificação da animação e parâmetros diversos do modelo. Por fim, são apresentados e avaliados os resultados obtidos através do desenvolvimento do modelo proposto por intermédio do protótipo FLEX3D. É dedicada ainda especial atenção às perspectivas futuras deste trabalho. / The growing number of animation systems that use kinematics to generate the motion of objects have led to other alternatives to produce more realistic results. Some authors began to study the animation of synthetic objects through the application of the dynamic concepts, creating the modern physically based models. At first, only rigid objects were treated; later on articulated objects were considered. At last, those with elastic characteristics (called flexible or deformable objects) were taken into consideration. The main goal of this work is to define a simulation model for deformable objects in the euclidean space. Both the geometric and the physical models are presented, considering the forces applied to the object and the constraints defined by the virtual world. Described forces include gravity, elasticity, dumping force, collision and attrition. This work presents an introduction to animation systems focusing the physically based systems. After this, a bibliographic review of the existent deformation methods is made. Methods for detecting and solving the collision between two elastic objects are described. Two aspects of the motion simulation are described: the algorithm used to generate the motion and the numeric integration of the differential equations in time. A prototype named FLEX3D is presented to validate the proposed model. The language used for specifying the animation is described and results obtained through the use of FLEX3D are also presented. Special attention is given to the possible future works.

O crescente número de sistemas de animação que utilizam a cinemática para gerar movimento de objetos vem levando os pesquisadores a buscar outras alternativas para produzir resultados mais realistas. Com base nesta premissa, vários autores começaram a estudar a geração de movimento de objetos sintéticos através da aplicação da dinâmica. Assim surgiram os modelos baseados em leis físicas. Num primeiro momento foram abordados apenas objetos rígidos, passando-se mais tarde a considerar objetos articulados e, por fim, aqueles com características elásticas, também denominados de objetos flexíveis ou deformáveis. O objetivo principal do trabalho é a definição de um modelo para simulação de objetos deformáveis no espaço euclidiano. São abordados tanto o modelo geométrico utilizado como o modelo físico, sendo ressaltadas as forças aplicadas sobre o objeto e as restrições que podem ser impostas pelo mundo virtual no qual o mesmo está inserido. Dentre as forças descritas, pode-se destacar: força gravitacional, elasticidade, força de curvatura e torção, colisão e atrito. A fundamentação do trabalho desenvolvido é apresentada na forma de uma introdução aos sistemas de animação, enfatizando os sistemas baseados em leis físicas e de uma revisão bibliográfica dos métodos de deformação existentes. No que diz respeito à colisão de objetos elásticos, são descritos tanto os métodos estudados para a solução das mesmas, como as técnicas para detecção do choque. A simulação do movimento é descrita sob dois aspectos: o algoritmo utilizado para a geração do movimento e a integração numérica das equações diferenciais no tempo. É abordado ainda, em detalhe, o protótipo desenvolvido com o propósito de validar o modelo proposto, sendo descrita a linguagem criada a fim de permitir a especificação da animação e parâmetros diversos do modelo. Por fim, são apresentados e avaliados os resultados obtidos através do desenvolvimento do modelo proposto por intermédio do protótipo FLEX3D. É dedicada ainda especial atenção às perspectivas futuras deste trabalho. / The growing number of animation systems that use kinematics to generate the motion of objects have led to other alternatives to produce more realistic results. Some authors began to study the animation of synthetic objects through the application of the dynamic concepts, creating the modern physically based models. At first, only rigid objects were treated; later on articulated objects were considered. At last, those with elastic characteristics (called flexible or deformable objects) were taken into consideration. The main goal of this work is to define a simulation model for deformable objects in the euclidean space. Both the geometric and the physical models are presented, considering the forces applied to the object and the constraints defined by the virtual world. Described forces include gravity, elasticity, dumping force, collision and attrition. This work presents an introduction to animation systems focusing the physically based systems. After this, a bibliographic review of the existent deformation methods is made. Methods for detecting and solving the collision between two elastic objects are described. Two aspects of the motion simulation are described: the algorithm used to generate the motion and the numeric integration of the differential equations in time. A prototype named FLEX3D is presented to validate the proposed model. The language used for specifying the animation is described and results obtained through the use of FLEX3D are also presented. Special attention is given to the possible future works.

O crescente número de sistemas de animação que utilizam a cinemática para gerar movimento de objetos vem levando os pesquisadores a buscar outras alternativas para produzir resultados mais realistas. Com base nesta premissa, vários autores começaram a estudar a geração de movimento de objetos sintéticos através da aplicação da dinâmica. Assim surgiram os modelos baseados em leis físicas. Num primeiro momento foram abordados apenas objetos rígidos, passando-se mais tarde a considerar objetos articulados e, por fim, aqueles com características elásticas, também denominados de objetos flexíveis ou deformáveis. O objetivo principal do trabalho é a definição de um modelo para simulação de objetos deformáveis no espaço euclidiano. São abordados tanto o modelo geométrico utilizado como o modelo físico, sendo ressaltadas as forças aplicadas sobre o objeto e as restrições que podem ser impostas pelo mundo virtual no qual o mesmo está inserido. Dentre as forças descritas, pode-se destacar: força gravitacional, elasticidade, força de curvatura e torção, colisão e atrito. A fundamentação do trabalho desenvolvido é apresentada na forma de uma introdução aos sistemas de animação, enfatizando os sistemas baseados em leis físicas e de uma revisão bibliográfica dos métodos de deformação existentes. No que diz respeito à colisão de objetos elásticos, são descritos tanto os métodos estudados para a solução das mesmas, como as técnicas para detecção do choque. A simulação do movimento é descrita sob dois aspectos: o algoritmo utilizado para a geração do movimento e a integração numérica das equações diferenciais no tempo. É abordado ainda, em detalhe, o protótipo desenvolvido com o propósito de validar o modelo proposto, sendo descrita a linguagem criada a fim de permitir a especificação da animação e parâmetros diversos do modelo. Por fim, são apresentados e avaliados os resultados obtidos através do desenvolvimento do modelo proposto por intermédio do protótipo FLEX3D. É dedicada ainda especial atenção às perspectivas futuras deste trabalho. / The growing number of animation systems that use kinematics to generate the motion of objects have led to other alternatives to produce more realistic results. Some authors began to study the animation of synthetic objects through the application of the dynamic concepts, creating the modern physically based models. At first, only rigid objects were treated; later on articulated objects were considered. At last, those with elastic characteristics (called flexible or deformable objects) were taken into consideration. The main goal of this work is to define a simulation model for deformable objects in the euclidean space. Both the geometric and the physical models are presented, considering the forces applied to the object and the constraints defined by the virtual world. Described forces include gravity, elasticity, dumping force, collision and attrition. This work presents an introduction to animation systems focusing the physically based systems. After this, a bibliographic review of the existent deformation methods is made. Methods for detecting and solving the collision between two elastic objects are described. Two aspects of the motion simulation are described: the algorithm used to generate the motion and the numeric integration of the differential equations in time. A prototype named FLEX3D is presented to validate the proposed model. The language used for specifying the animation is described and results obtained through the use of FLEX3D are also presented. Special attention is given to the possible future works.

Non-rigid shapes are generally known as objects whose three dimensional geometry may deform by internal and/or external forces. Deformable shapes are all around us, ranging from protein molecules, to natural objects such as the trees in the forest or the fruits in our gardens, and even human bodies. Two deformable shapes may be related by isometry, which means their intrinsic geometries are preserved, even though their extrinsic geometries are dissimilar. An important problem in the analysis of the deformable shapes is to identify the three-dimensional correspondence between two isometric shapes, given that the two shapes may be deviated from isometry by intrinsic distortions. A major challenge is that non-rigid shapes have large degrees of freedom on how to deform. Nevertheless, irrespective of how they are deformed, they may be aligned such that the geodesic distance between two arbitrary points on two shapes are nearly equal. Such alignment may be expressed by a permutation matrix (a matrix with binary entries) that corresponds to every paired geodesic distance in between the two shapes. The alignment involves searching the space over all possible mappings (that is all the permutations) to locate the one that minimizes the amount of deviation from isometry. A brute-force search to locate the correspondence is not computationally feasible. This thesis introduces a novel approach created to locate such correspondences, in spite of the large solution space that encompasses all possible mappings and the presence of intrinsic distortion. In order to find correspondences between two shapes, the first step is to create a suitable descriptor to accurately describe the deformable shapes. To this end, we developed deformation-invariant metric descriptors. A descriptor constitutes pair-wise geodesic distances among arbitrary number of discrete points that represent the topology of the non-rigid shape. Our descriptor provides isometric-invariant representation of the shape irrespective of its circumstantial deformation. Two isometric-invariant descriptors, representing two candidate deformable shapes, are the input parameters to our optimization algorithm. We then proceed to locate the permutation matrix that aligns the two descriptors, that minimizes the deviation from isometry. Once we have developed such a descriptor, we turn our attention to finding correspondences between non deformable shapes. In this study, we investigate the use of both classical and quantum particle swarm optimization (PSO) algorithms for this task. To explore the merits of variants of PSO, integer optimization involving test functions with large dimensions were performed, and the results and the analysis suggest that quantum PSO is more effective optimization method than its classical PSO counterpart. Further, a scheme is proposed to structure the solution space, composed of permutation matrices, in lexicographic ordering. The search in the solution space is accordingly simplified to integer optimization to find the integer rank of the targeted permutation matrix. Empirical results suggest that this scheme improves the scalability of quantum PSO across large solution spaces. Yet, quantum PSO's global search capability requires assistance in order to more effectively manoeuvre through the local extrema prevalent in the large solution spaces. A mutation based genetic algorithm (GA) is employed to augment the search diversity of quantum PSO when/if the swarm stagnates among the local extrema. The mutation based GA instantly disengages the optimization engine from the local extrema in order to reorient the optimization energy to the trajectories that steer to the global extrema, or the targeted permutation matrix. Our resultant optimization algorithm combines quantum Particle Swarm Optimization (PSO) and mutation based Genetic Algorithm (GA). Empirical results show that the optimization method presented is scalable and efficient on standard hardware across different solution space sizes. The performance of the optimization method, in simulations and on various near-isometric shapes, is discussed. In all cases investigated, the method could successfully identify the correspondence among the non-rigid deformable shapes that were related by isometry.

"A number of outstanding problems in robotic motion and manipulation involve tasks where degrees of freedom (DoF), be they part of the robot, an object being manipulated, or the surrounding environment, cannot be accurately controlled by the actuators of the robot alone. Rather, they are also controlled by physical properties or interactions - contact, robot dynamics, actuator behavior - that are influenced by the actuators of the robot. In particular, we focus on two important areas of poorly controlled robotic manipulation: motion planning for deformable objects and in deformable environments; and manipulation with uncertainty. Many everyday tasks we wish robots to perform, such as cooking and cleaning, require the robot to manipulate deformable objects. The limitations of real robotic actuators and sensors result in uncertainty that we must address to reliably perform fine manipulation. Notably, both areas share a common principle: contact, which is usually prohibited in motion planners, is not only sometimes unavoidable, but often necessary to accurately complete the task at hand. We make four contributions that enable robot manipulation in these poorly controlled tasks: First, an efficient discretized representation of elastic deformable objects and cost function that assess a ``cost of deformation' for a specific configuration of a deformable object that enables deformable object manipulation tasks to be performed without physical simulation. Second, a method using active learning and inverse-optimal control to build these discretized representations from expert demonstrations. Third, a motion planner and policy-based execution approach to manipulation with uncertainty which incorporates contact with the environment and compliance of the robot to generate motion policies which are then adapted during execution to reflect actual robot behavior. Fourth, work towards the development of an efficient path quality metric for paths executed with actuation uncertainty that can be used inside a motion planner or trajectory optimizer."

Submitted by Silvana Teresinha Dornelles Studzinski (sstudzinski) on 2016-05-04T15:43:49Z No. of bitstreams: 1 Francisco de Assis da Silva Cavalcante Filho_.pdf: 3681280 bytes, checksum: 92b2f9dcc668fbbbd3c9feae15c9d569 (MD5) / Made available in DSpace on 2016-05-04T15:43:49Z (GMT). No. of bitstreams: 1 Francisco de Assis da Silva Cavalcante Filho_.pdf: 3681280 bytes, checksum: 92b2f9dcc668fbbbd3c9feae15c9d569 (MD5) Previous issue date: 2016-03-01 / IFRR - Instituto Federal de Educação Ciências e Tecnologia de Roraima / A Articulação Temporomandibular é uma das articulações mais complexas do corpo humano. Ela é composta de duas articulações, uma em cada lado da mandíbula, que trabalham juntas para realizar os movimentos de abertura e fechamento da boca, assim como os movimentos de mastigação. É uma das articulações que está constantemente sob pressão e a sobrecarga nessa articulação pode resultar em inúmeras condições médicas. Cerca de 30% da população apresenta algum sintoma de DTM, sendo o desalinhamento dos discos temporomandibulares o mais comum entre eles, atingindo 70% dos pacientes. Os trabalhos desenvolvidos até o momento tiveram como foco o estudo de características específicas da ATM, como a trajetória da mandíbula em determinados movimentos, os músculos que são ativados nestes movimentos ou a tensão sofrida pelos discos articulares. Estes modelos apresentam uma representação limitada da ATM e estruturas adjacentes ou utilizam técnicas que demandam muito poder computacional, restringindo sua utilização a pesquisas direcionadas. Este trabalho propõe um modelo para visualização dos movimentos da mandíbula de forma interativa com base nas restrições do disco articular e estruturas adjacentes. O modelo emprega técnicas de simulação física para obter maior realismo na visualização dos movimentos, permitindo sua aplicação em várias áreas da medicina. / The temporomandibular joint is one of the most complex joints in the human body. It is composed of two joints, one at each side of the jaw, that work together to perform the opening and closing movements of the mouth, as well as chewing movements. It is a joint that is constantly under pressure and the loading in this joint can result in numerous medical conditions. About 30% of the population have some symptom of TMD, and the temporomandibular discs displacement is the most common between them, reaching 70% of patients. The work carried out so far have focused on the study of specific features of ATM, as the trajectory of the jaw in certain movements, the muscles that are activated in these movements or stress suffered by the articular disc. These models have a limited representation of ATM and adjacent structures or use techniques that requires too much computational power, restricting its use to directed research. This paper presents a model for visualization of the jaw movements interactively based on disk restrictions and adjacent structures. The model uses physical simulation techniques to achieve more realism on movements visualization, allowing its application in various areas of medicine.

L'objectif fondamental de cette thèse est de développer et de mettre en oeuvre un outil interactif desimulation des techniques de micromanipulation biologiques de cellules. Au moyen de cet outil, l'opérateurpourra se former, s'entraîner et améliorer sa maîtrise en développant une gestuelle proche de celle exécutéeen réalité. La conception d'un tel environnement de simulation en temps-réel nécessite de trouver uncompromis entre le réalisme des modèles de comportement biomécanique utilisés, la précision et la stabilitédes algorithmes des méthodes de résolution et de rendu haptique utilisées ainsi que la vitesse de calcul. Lamodélisation mécanique retenue repose sur l'utilisation du modèle hyperélastique de St Venant-Kirchhoff etune formulation dynamique explicite éléments-finis du type masses-tenseurs. Le bien-fondé de cettemodélisation est vérifié sur des essais de microindentation par Microscopie à Force Atomique (AFM) decellules souches embryonnaires de souris et de microinjection d'ovocytes. Nous avons développé etimplémenté des modèles d'interaction en temps-réel qui s'articulent autour de la détection et la gestionrapide des collisions entre outil/cellule.La synthèse du rendu haptique fourni à l'opérateur est également proposée par l'intermédiaire d'un couplagevirtuel. Pour chaque application, nous avons justifié nos choix méthodologiques et Algorithmiques qui sontguidés par les contraintes de "réalisme+précision" "temps-réel". Les différents modèles proposés ont étéintégrés dans le simulateur SIMIC que nous avons développé pendant cette thèse. Ce dernier est dédié à lasimulation interactive pour l'aide à l'apprentissage du geste de microinjection et de nanoindentationcellulaire. / The fundamental objective of this thesis is to develop and implementing an interactive simulation techniquesfor micromanipulation biological cells. Using this tool, the operator can form, train and improve its control bydeveloping a gesture similar to that performed in reality. The design of such a simulation environment in realtime requires a compromise between the realism of biomechanical models used the accuracy and stability ofalgorithms and solution methods used haptic rendering and computational speed. Modeling Mechanicalrestraint involves the use of hyperelastic model of St Venant-Kirchhoff formulation and explicit dynamic finiteelement-type mass tensors. The validity of this model is tested on microindentation tests by Atomic ForceMicroscopy (AFM) of mouse embryonic stem cells and microinjection of oocytes. We have developed andimplemented models of real-time interaction that revolve around the detection and management of rapidcollisions between tool / cell.The synthesis of the haptic feedback provided to the operator is also available through a virtual coupling. Foreach application, we have justified our methodological choices and Algorithms that are guided by theconstraints of realism + precision "" real time ". The various proposed models have been integrated into thesimulator SIMIC that we developed during this thesis. This is dedicated to interactive simulation to supportlearning of gesture microinjection and cell nanoindentation.