abstract: Anticipatory planning of digit positions and forces is critical for successful dexterous object manipulation. Anticipatory (feedforward) planning bypasses the inherent delays in reflex responses and sensorimotor integration associated with reactive (feedback) control. It has been suggested that feedforward and feedback strategies can be distinguished based on the profile of grip and load force rates during the period between initial contact with the object and object lift. However, this has not been validated in tasks that do not constrain digit placement. The purposes of this thesis were (1) to validate the hypothesis that force rate profiles are indicative of the control strategy used for object manipulation and (2) to test this hypothesis by comparing manipulation tasks performed with and without digit placement constraints. The first objective comprised two studies. In the first study an additional light or heavy mass was added to the base of the object. In the second study a mass was added, altering the object's center of mass (CM) location. In each experiment digit force rates were calculated between the times of initial digit contact and object lift. Digit force rates were fit to a Gaussian bell curve and the goodness of fit was compared across predictable and unpredictable mass and CM conditions. For both experiments, a predictable object mass and CM elicited bell shaped force rate profiles, indicative of feedforward control. For the second objective, a comparison of performance between subjects who performed the grasp task with either constrained or unconstrained digit contact locations was conducted. When digit location was unconstrained and CM was predictable, force rates were well fit to a bell shaped curve. However, the goodness of fit of the force rate profiles to the bell shaped curve was weaker for the constrained than the unconstrained digit placement condition. These findings seem to indicate that brain can generate an appropriate feedforward control strategy even when digit placement is unconstrained and an infinite combination of digit placement and force solutions exists to lift the object successfully. Future work is needed that investigates the role digit positioning and tactile feedback has on anticipatory control of object manipulation. / Dissertation/Thesis / M.S. Bioengineering 2011

abstract: Object manipulation is a common sensorimotor task that humans perform to interact with the physical world. The first aim of this dissertation was to characterize and identify the role of feedback and feedforward mechanisms for force control in object manipulation by introducing a new feature based on force trajectories to quantify the interaction between feedback- and feedforward control. This feature was applied on two grasp contexts: grasping the object at either (1) predetermined or (2) self-selected grasp locations (“constrained” and “unconstrained”, respectively), where unconstrained grasping is thought to involve feedback-driven force corrections to a greater extent than constrained grasping. This proposition was confirmed by force feature analysis. The second aim of this dissertation was to quantify whether force control mechanisms differ between dominant and non-dominant hands. The force feature analysis demonstrated that manipulation by the dominant hand relies on feedforward control more than the non-dominant hand. The third aim was to quantify coordination mechanisms underlying physical interaction by dyads in object manipulation. The results revealed that only individuals with worse solo performance benefit from interpersonal coordination through physical couplings, whereas the better individuals do not. This work showed that naturally emerging leader-follower roles, whereby the leader in dyadic manipulation exhibits significant greater force changes than the follower. Furthermore, brain activity measured through electroencephalography (EEG) could discriminate leader and follower roles as indicated power modulation in the alpha frequency band over centro-parietal areas. Lastly, this dissertation suggested that the relation between force and motion (arm impedance) could be an important means for communicating intended movement direction between biological agents. / Dissertation/Thesis / Doctoral Dissertation Biomedical Engineering 2017

Le travail de thèse exposé dans ce manuscrit s'intéresse à la simulation d'un mouvement de manipulation d'objet, plus particulièrement à la saisie suivie de la rotation d'une sphère selon un axe fixe. Le but ici est, à partir d'une analyse de mouvements réels, d'être capable de proposer un algorithme de simulation reproduisant des mouvements comparables aux données expérimentales, en fournissant en entrée le moins d'informations possibles et en essayant d'introduire de la variabilité dans le mouvement simulé. 12 sujets volontaires ont participé à l'expérimentation. Nous leur avons demandé de saisir et de tourner une sphère de 60mm de diamètre avec des amplitudes allant de 45° à 360°, dans les deux sens. L'analyse de ces données expérimentales, complétée par des simulations de l'effet des limites articulaires sur le mouvement nous ont permis d'étudier plusieurs hypothèses sur le contrôle de mouvement telles que l'hypothèse de confort final (« end-state comfort ») et le principe de travail minimum. Une des conclusions majeures est que l'anticipation posturale au moment de saisie semble être expliquée davantage par le confort en fin de mouvement qu'au moment de saisie. Basé sur ces observations, nous proposons un algorithme de simulation avec pour nouveautés la prise en compte de l'espace de mouvements admissibles par les limites articulaires ainsi que l'introduction de la variabilité au choix de la posture de saisie. Les premiers résultats semblent être en accord en grande partie avec les observations expérimentales donnant une base de travail pour aller vers des outils de simulation se rapprochant de plus en plus vers un comportement "humain" / The thesis work presented in this manuscript focuses on the simulation of an handling motion, more specifically on the grasp followed by the rotation of a sphere along a fixed single axis. The aim here is, from the analysis of actual motions, to be able to propose an simulation algorithm reproducing motions comparable to experimental data, with the less input as possible and trying to introduce some variability into the simulated motion. 12 volunteers participated to the experiment. Subjects were asked to grasp and turn a sphere of 60mm of diameter. Amplitudes of rotations were ranged from 45° to 360°, in both directions. Experimental data analysis, completed with some simulations of the effect of joint limits on motion, allowed us to investigate several motion control hypothesis as the end-state comfort hypothesis or the minimum work principle. One of the main conclusions is that postural anticipation when grasping seems to be more explained by the comfort at the end of the motion than when grasping. Based on these observations, we proposed a simulation algorithm being original by the way of how it takes into account possible motions allowed by joint limits and by the introduction of variability into the simulated grasp posture. The first results seem to follow most part of the experimental observations giving a strong basis to go towards simulation tools that will come closer to a “human” behavior

Object manipulation is a complex task for robots. It often implies a compromise between the degrees-of-freedom of hand and its fingers have (dexterity) and its cost and complexity in terms of control. One strategy to increase the dexterity of robotic hands with low dexterity is called extrinsic manipulation and its principle is to exploit additional accelerations on the object caused by the effect of external forces. We propose a force control based method for performing extrinsic in-hand object manipulation, with force-torque feedback. For this purpose, we use a prehensile pushing action, which consists of pushing the object against an external surface, under quasistatic assumptions. By using a control strategy, we also achieve robustness to parameter uncertainty (such as friction) and perturbations, that are not completely captured by mathematical models of the system. The force control strategy is performed in two different ways: the contact force generated by the interaction between the object and the external surface is controlled using an admittance controller, while an additional control of gripping force applied by the gripper on the object is done through a PI controller. A Kalman filter is used for the estimation of the state of the object, based on force-torque measurements of a sensor at the wrist of the robot. We validate our approach by conducting experiments on a PR2 robot, available at the Robotics, Perception, and Learning lab at KTH Royal Institute of Technology. / Att greppa och manipulera objekt är en komplex uppgift för robotar. Det innebär ofta en kompromiss mellan hand och fingrars frihetsgrader (fingerfärdighet) mot reglersystemets kostnad och komplexitet. Extrinsic manipulation är en strategi för att öka fingerfärdigheten hos robothänder, och dess princip är att utnyttja accelerationer på objektet som orsakas av yttre krafter. Vi föreslår en metod baserad på att reglerakraft för hantering av objekt i handen, genom en återkoppling av kraftmomentet. För detta ändamål använder vi en prehensile pushing action, där objektet puttas mot en yta, under kvasistiska antaganden. Genom att använda en reglerstrategi får vi en robusthet mot parametrars osäkerhet (som friktion) och störningar, vilka inte beskrivs av systemets model. Kraftkontrollstrategin utförs på två olika sätt: kraften mellan objektet och den yttre ytan styrs med en admittance controller medan en ytterligare styrning av applicerad gripkraft på objektet görs med en PI-reglerare. Ett Kalman filter används för att estimera objektets tillstånd, baserat på mätningar av kraftmoment via en sensor vid robotens handled. Vi utvärderar vårt tillvägagångssätt genom att utföraexperiment på en PR2-robot vid KTHs Robotics, Perception och Learning Lab.

<p>In Virtual Reality (VR), users often need to explore a large virtual space within a limited physical space. However, as one of the most popular and commonly-used methods for such room-scale problems, teleport always relies on hand-based controllers. In applications that require consistent hand interaction, such teleport methods may conflict with the users' hand operation, and make them uncomfortable, thus affecting their experience. </p> <p>To alleviate these limitations, this research designs and implements a new interactive object-based VR locomotion method, ManiLoco, as an eye- and foot-based low-cost method. This research also evaluates ManiLoco and compares it with state-of-the-art Point & Teleport and Gaze Teleport methods in a within-subject experiment with 14 participants.</p> <p>The results confirm the viability of the method and its possibility in such applications. ManiLoco makes the users feel much more comfortable with their hands and focus more on the hand interaction in the application while maintaining efficiency and presence. Further, the users' trajectory maps indicate that ManiLoco, despite the introduction of walking, can be applicable to room-scale tracking space. Finally, as a locomotion method only relied on VR head-mounted display (HMD) and software detection, ManiLoco can be easily applied to any VR applications as a plugin.</p>

<p>Robotic platforms have several characteristics such as speed and precision that make them enticing for use in medical procedures. Companies such as Intuitive Medical and Titan Medical have taken advantage of these features to introduce surgical robots for minimally invasive procedures. Such robots aim to reduce procedure and patient recovery times. Current technology requires platforms to be master-slave manipulators controlled by a surgeon, effectively converting the robot into an expensive surgical tool. Research into the interaction between robotic platforms and deformable objects such as human tissue is necessary in the development of autonomous and semi-autonomous surgical systems. This thesis investigates a class of robust linear controllers based on a worst case performance measure known as the $H_{\infty}$ norm, for the purpose of performing so called Indirect Deformable Object Manipulation (IDOM). This task allows positional regulation of regions of interest in a deformable object without directly interacting with them, enabling tasks such as stabilization of tumors during biopsies or automatic suturing. A complete approach to generating linear $H_{\infty}$ based controllers is presented, from derivation of a plant model to the actual synthesis of the controller. The introduction of model uncertainty requires $\mu$ synthesis techniques, which extend $H_{\infty}$ designs to produce highly robust controller solutions. In addition to $H_{\infty}$ and $\mu$ synthesis designs, the thesis presents an approach to design an optimal PID controller with gains that minimize the $H_{\infty}$ norm of a weighted plant. The three control approaches are simulated performing set point regulation in $\text{MATLAB}^{TM}$'s $simulink$. Simulations included disturbance inputs and noises to test stability and robustness of the approaches. $H_{\infty}$ controllers had the best robust performance of the controllers simulated, although all controllers simulated were stable. The $H_{\infty}$ and PID controllers were validated in an experimental setting, with experiments performed on two different deformable synthetic materials. It was found that $H_{\infty}$ techniques were highly robust and provided good tracking performance for a material that behaved in a relatively elastic manner, but failed to track well when applied to a highly nonlinear rubber compound. PID based control was outperformed by $H_{\infty}$ control in experiments performed on the elastic material, but proved to be superior when faced with the nonlinear material. These experimental findings are discussed and a linear $H_{\infty}$ control design approach is proposed.</p> / Master of Applied Science (MASc)

Human sensorimotor control is remarkably adept at utilising contextual information to learn and recall systematic sensorimotor transformations. Here, we investigate the motor representations that underlie such learning, and examine how motor memories acquired based on different contextual information interact. Using a novel three-dimensional robotic manipulandum, the 3BOT, we examined the spatial transfer of learning across various movement directions in a 3D environment, while human subjects performed reaching movements under velocity-dependent force field. The obtained pattern of generalisation suggested that the representation of dynamic learning was most likely defined in a target-based, rather than an extrinsic, coordinate system. We further examined how motor memories interact when subjects adapt to force fields applied in orthogonal dimensions. We found that, unlike opposing fields, learning two spatially orthogonal force fields led to the formation of separate motor memories, which neither interfered with nor facilitated each other. Moreover, we demonstrated a novel, more general aspect of the spontaneous recovery phenomenon using a two-dimensional force field task: when subjects learned two orthogonal force fields consecutively, in the following phase of clamped error feedback, the expression of adaptation spontaneously rotated from the direction of the second force field, towards the direction of the first force field. Finally, we examined the interaction of sensorimotor memories formed based on separate contextual information. Subjects performed reciprocating reaching and object manipulation tasks under two alternating contexts (movement directions), while we manipulated the dynamics of the task in each context separately. The results suggested that separate motor memories were formed for the dynamics of the task in different contexts, and that these motor memories interacted by sharing error signals to enhance learning. Importantly, the extent of interaction was not fixed between the context-dependent motor memories, but adaptively changed according to the task dynamics to potentially improve overall performance. Together, our experimental and theoretical results add to the understanding of mechanisms that underlie sensorimotor learning, and the way these mechanisms interact under various tasks and different dynamics.

In daily activities, the brain regularly assigns different roles to the hands dependingon task and context. Yet, little is known about the underlying neural processes. Thiscertainly applies to how the brain, where each hemisphere primarily controls onehand, manages the between-hand coordination required in bimanual objectmanipulation. By using behavioral, neurophysiological and functional magneticresonance imaging techniques, the present thesis examines neural mechanisms thatsupport hand coordination during tasks where the two hands apply spatiotemporallycoupled but opposing forces for goal attainment, e.g., as when removing the cap froma bottle. Although the two hands seem to operate symmetrically in such tasks, Study Ishowed that one hand primarily acts while the other assists. Moreover, this roledifferentiation was found to be flexible with the brain appointing either hand asprime actor depending on the spatial congruency between hand forces and desiredmovement consequences. Accordingly, when we remove a cap from a bottle, the handthat grasps the cap, be it left or right depending on overall task constraints, isappointed as prime actor because the twist forces it generates are aligned with thegoal to remove the cap, while the other hand, holding the bottle, applies stabilizingforces in the opposite direction. Changes in hand assignments are caused by amidline shift of lateralized activity throughout the motor system, from distal handmuscles to corticospinal pathways and primary sensorimotor and cerebellar corticalareas (Study I). Although the bimanual actions examined involved both within- andbetween-hand coordination, Study II failed to reveal additional brain activity duringbimanual as compared to matching unimanual actions, except for the primarysensorimotor areas where subpopulations of neurons were preferentially engagedduring either bimanual or unimanual actions. Thus, dedicated neurons in the motorcortices might support critical bimanual coordinative operations. While imagingresults indicated that a mainly left-lateralized parietal-premotor network managedthe task irrespective of prime actor, premotor areas presumably established handassignment by allocating the lead either to the left or the right primary sensorimotorareas (Study I and II). Regarding the process of prime actor selection and hence thecontrol of these premotor networks, imaging results indicate a transitory involvementof prefrontal cortical areas (Study III). The detected areas belong to a networkconsidered critical for cognitive operations such as judgment and decision-making,and for evaluation of utility of actions, including conflict detection. The implicitselection of prime actor during bimanual tasks thus seems to be supported by corticalareas traditionally associated primarily with complex cognitive challenges.

Motion estimation and compensation plays an important role in video processing applications. Two-dimensional block-based and mesh-based models are widely used in this area. A 2-D mesh-based model provides a better representation of complex real world motion than a block-based model. Mesh-based motion estimation algorithms are employed in both frame-based and object-based video compression and coding. A hierarchical mesh-based algorithm is applied to improve the motion field generated by a single-layer algorithm. 2-D mesh-based models also enable the manipulation of video objects which is included in the MPEG-4 standard. A video object in a video clip can be replaced by another object by the use of a dynamic mesh structure. In this thesis, a comparative analysis of 2-D block-based and mesh-based motion estimation algorithms in both frame-based and object-based video representations is performed. The experimental results indicate that a mesh-based algorithm produces better motion compensation results than a block-based algorithm. Moreover, a two-layer mesh-based algorithm shows improvement over a one-layer mesh-based algorithm. The application of mesh-based motion estimation and compensation to video object replacement and animation is also performed.

Diss. (sammanfattning) Umeå : Umeå universitet, 2009. / Härtill 3 uppsatser. Även tryckt utgåva.