La présente thèse traite de l'utilisation de retours haptiques pour fournir des informations aux chirurgiens durant des opérations de chirurgie minimalement invasive dans le but de les aider à améliorer leurs gestes.De meilleurs résultats pour les patients on amené la chirurgie minimalement invasive à devenir le standard pour bon nombre d'interventions. Cependant, la perte de perception de profondeur, la coordination main-oeil compliquée ainsi que les distorsions de sensations haptiques compliquent largement la tâche pour le chirurgien. Nous explorons le potentiel de retours haptiques pour intuitivement assister les chirurgiens durant des gestes de chirurgie minimalement invasive. Les formes de retour évaluées sont principalement haptiques (tactiles et kinesthésiques), avec des comparaisons à des retours visuels et multi-modaux (combinaisons de retours visuels et haptiques).Nos expériences dans le domaine de la navigation d'outils de chirurgie montrent des résultats encourageants quand aux bénéfices obtenus par des retours haptiques en termes d'amélioration de la qualité du geste chirurgical. Les guides par "virtual fixtures" montrent une nette supériorité par rapport aux autres formes de retour étudiées, cependant les retours vibrotactiles permettent aussi des améliorations notables. Des travaux parallèles sur le retour d'informations au sujet des efforts d'intéraction en bout d'outils a mis en évidence des différences importantes en termes des exigences de conception pour le retour tactile. Ceci nous a permis d'effectuer une conception et validation préliminaire de retours tactiles spécifiques à des applications de maitrise d'efforts, en utilisant l'exemple de la suture. / The present thesis focuses on the use of haptic feedback technologies to provide information to surgeons during laparoscopic or minimal access surgery (MAS) with the aim of assisting them in improving their gestures.Better overall outcomes for patients have led MAS to become standard for many surgical interventions. However, loss of visual depth perception, difficult hand-eye coordination and distorted haptic sensation seriously complicate this task for the surgeon. We explore the potential of haptic cues for intuitively assisting surgeons during MAS gestures. Evaluated forms of feedback mainly focus on haptic (tactile and kinaesthetic) cues, but include comparisons to visual and multi-modal combined haptic and visual cues.Experiments on surgical tool navigation show encouraging results for the benefit of haptic cues in improving surgical gestures, with clear superiority of soft guidance virtual fixtures over other forms of feedback. However, promising results for the use of vibrotactile feedback are also obtained. These results are confirmed in preliminary experiments on tool navigation in preliminary experiments on tool navigation during a laparoscopic cutting training task.Parallel work on feeding back interaction forces highlighted significant differences in the usability and design requirements for tactile cues when compared to instrument navigation applications. This led us to design and perform preliminary testing on tactile cues appropriate force information in the case of intra-corporeal suture knot tying.

A generally accepted goal in orthopaedic surgery today is to maximize conservation of tissue and reduce tissue damage. Bone-conserving implants have bone-mating surfaces that reproduce the natural curvature of bone structures, requiring less bone removal. No small, reliable, inexpensive and universal bone sculpting technique currently exists, however, that can both create and accurately align such complex surfaces. The goal of this thesis was to develop a haptic hard surface emulation mechanism that could be applied to curvilinear bone sculpting using a surgical robot. A novel dynamic physical constraint concept was developed that is able to emulate realistic hard constraints, smooth surface following, and realistic surface rigidity, while allowing complete freedom of motion away from the constraints. The concept was verified through the construction of a two-link manipulator prototype. Tests were run on nine users that involved each user tracing out five different virtual surfaces on a drawing surface using the prototype. The primary purposes of prototype testing were to obtain subjective data on how effectively the dynamic physical constraint concept simulates simple surfaces, to assess how it reacts to typical user interactions and to identify any unexpected behaviour. Users were 100% satisfied with the prototype’s ability to emulate realistic and stiff hard surfaces and with its ease of manipulation. The amount of incursion into each of the virtual surfaces by all the users was measured to assess the precision of the system with the goal of deciding whether this new haptic concept should be further developed specifically for precision applications such as surgery. For curvilinear surfaces, 90% of the cumulative distribution of the measured data was less than 2mm, while for linear surfaces it was less than 6mm. Four behavioural effects were noticed: lateral deflection, reverse ‘stickiness’, hysteresis and instability in certain areas. These effects were studied in detail to determine how to either eliminate them or to minimize them through system design optimization. A computer simulation was also used to model the behaviour of the prototype and to gain further understanding of these effects. These analyses showed that the concept can be successfully used in curvilinear bone sculpting. / Applied Science, Faculty of / Mechanical Engineering, Department of / Graduate

Robots are required in many jobs. The jobs related to tele-operation may be very challenging and often require reaching a destination quickly and with minimum collisions. In order to succeed in these jobs, human operators are asked to tele-operate a robot manually through a user interface. The design of a user interface and of the information provided in it, become therefore critical elements for the successful completion of robot tele-operation tasks. Effective and timely robot tele-navigation mainly relies on the intuitiveness provided by the interface and on the richness and presentation of the feedback given. This project investigated the use of both haptic and visual feedbacks in a user interface for robot tele-navigation. The aim was to overcome some of the limitations observed in a state of the art works, turning what is sometimes described as contrasting into an added value to improve tele-navigation performance. The key issue is to combine different human sensory modalities in a coherent way and to benefit from 3-D vision too. The proposed new approach was inspired by how visually impaired people use walking sticks to navigate. Haptic feedback may provide helpful input to a user to comprehend distances to surrounding obstacles and information about the obstacle distribution. This was proposed to be achieved entirely relying on on-board range sensors, and by processing this input through a simple scheme that regulates magnitude and direction of the environmental force-feedback provided to the haptic device. A specific algorithm was also used to render the distribution of very close objects to provide appropriate touch sensations. Scene visualization was provided by the system and it was shown to a user coherently to haptic sensation. Different visualization configurations, from multi-viewpoint observation to 3-D visualization, were proposed and rigorously assessed through experimentations, to understand the advantages of the proposed approach and performance variations among different 3-D display technologies. Over twenty users were invited to participate in a usability study composed by two major experiments. The first experiment focused on a comparison between the proposed haptic-feedback strategy and a typical state of the art approach. It included testing with a multi-viewpoint visual observation. The second experiment investigated the performance of the proposed haptic-feedback strategy when combined with three different stereoscopic-3D visualization technologies. The results from the experiments were encouraging and showed good performance with the proposed approach and an improvement over literature approaches to haptic feedback in robot tele-operation. It was also demonstrated that 3-D visualization can be beneficial for robot tele-navigation and it will not contrast with haptic feedback if it is properly aligned to it. Performance may vary with different 3-D visualization technologies, which is also discussed in the presented work.

Personal identity refers to a set of attributes (e.g., name, social insurance number, etc.) that are associated with a person. Identity management is the process of creating, maintaining and destroying identities of individuals in a population. Biometric technologies are technologies developed to use statistical analysis of an individual’s biological or behavioral traits to determine his identity. Biometrics based authentication systems offer a reliable solution for identity management, because of their uniqueness, relative stability over time and security (among other reasons). Public acceptance of biometric systems will depend on their ability to ensure robustness, accuracy and security. Although robustness and accuracy of such systems are rapidly improving, there still remain some issues of security and balancing it with privacy. While the uniqueness of biometric traits offers a convenient and reliable means of identification, it also poses the risk of unauthorized cross-referencing among databases using the same biometric trait. There is also a high risk in case of a biometric database being compromised, since it’s not possible to revoke the biometric trait and re-issue a new one as is the case with passwords and smart keys. This unique attribute of biometric based authentication system poses a challenge that might slow down public acceptance and the use of biometrics for authentication purposes in large scale applications. In this research we investigate the vulnerabilities of biometric systems focusing on template security in iris-based biometric recognition systems. The iris has been well studied for authentication purposes and has been proven accurate in large scale applications in several airports and border crossings around the world. The most widely accepted iris recognition systems are based on Daugman’s model that creates a binary iris template. In this research we develop different systems using watermarking, bio-cryptography as well as feature transformation to achieve revocability and security of binary templates in iris based biometric authentication systems, while maintaining the performance that enables widespread application of these systems. All algorithms developed in this research are applicable on already existing biometric authentication systems and do not require redesign of these existing, well established iris-based authentication systems that use binary templates.

In order to leverage the use of various modalities such as audio-visual-touch in instilling learning behaviour, we present an intuitive approach of annotation based hapto-audio-visual interaction with the traditional digital learning materials such as eBooks. By integrating the traditional home entertainment system and respective media in the user's reading experience combined with haptic interfaces, we examine whether such augmentation of modalities influence the user's reading experience in terms of attention, entertainment and retention. The proposed Haptic E-Book (HE-Book) system leverages the haptic jacket, haptic arm band as well as haptic sofa interfaces to receive haptic emotive signals wirelessly in the form of patterned vibrations of the actuators and expresses the learning material by incorporating audio-video based augmentation in order to pave ways for intimate reading experience in the popular eBook platform. We have designed and developed desktop, mobile/tablet based HE-Book system as well as a semi-automated annotation authoring tool. Our system also supports multimedia based diverse quiz augmentations, which can help in learning tracking. We have conducted quantitative and qualitative tests using the developed prototype systems. We have adopted the indirect objective based performance analysis methodology, which is commonly used for multimedia based learning investigation. The user study shows that, there is a positive tendency of accepting multimodal interactions including haptics with traditional eBook reading experience. Though our limited number of laboratory tests reveal, that haptics can be an influencing media in eBook reading experience, but it requires large scale real life tests to provide a concluding remarks.

Technology such as television, computers, and video games are often in the line for reasons of why people lack physical activity and tend to gain weight and become obese. In the case of video games, with the advent of the so called “serious games initiative”, a new breed of video games have come into place. Such games are called “exergames” and they are intended to motivate the user to do physical activity. Although there is some evidence that some types of Exergames are more physically demanding than traditional sedentary games, there is also evidence that suggests that such games are not really providing the intensity of exert that is at the recommended levels for a daily exercise. Currently, most exergames have a passive approach. There is no real tracking of the players progress, there is no assessment of his/her level of exert, no contextual information, and there is no adaptability on the game itself to change the conditions of the game and prompt the desired physiological response on the player. In this thesis we present research work done towards the design and development of an architecture and related systems that support a shift in the exertion game paradigm. The contributions of this work are enablers in the design and development of exertion games with a strict serious game approach. Such games should have “exercising” as the primary goal, and a game engine that has been developed under this scheme should be aware of the exertion context of the player. The game should be aware of the level of exertion of the player and adapt the gaming context (in-game variables and exertion interface settings) so that the player can reach a predefined exertion rate as desired. To support such degree of adaptability in a multimedia, multimodal system, we have proposed a system architecture that lays down the general guidelines for the design and development of such systems.

With the advancement in multimedia systems and the increased interest in haptics to be used in interpersonal communication systems, where users can see, show, hear, tell, touch and be touched, mouse and keyboard are no longer dominant input devices. Touch, speech and vision will soon be the main methods of human computer interaction. Moreover, as interpersonal communication usage increases, the need for securing user authentication grows. In this research, we examine a user's identification and verification based on haptic information. We divide our research into three main steps. The first step is to examine a pre-defined task, namely a handwritten signature with haptic information. The user target in this task is to mimic the legitimate signature in order to be verified. As a second step, we consider the user's identification and verification based on user drawings. The user target is predefined, however there are no restrictions imposed on the order or on the level of details required for the drawing. Lastly, we examine the feasibility and possibility of distinguishing users based on their haptic interaction through an interpersonal communication system. In this third step, there are no restrictions on user movements, however a free movement to touch the remote party is expected. In order to achieve our goal, many classification and feature reduction techniques have been discovered and some new ones were proposed. Moreover, in this work we utilize evolutionary computing in user verification and identification. Analysis of haptic features and their significance on distinguishing users is hence examined. The results show a utilization of visual features by Genetic Programming (GP) towards identity verification, with a probability equal to 50% while the remaining haptic features were utilized with a probability of approximately 50%. Moreover, with a handwritten signature application, a verification success rate of 97.93% with False Acceptance Rate (FAR) of 1.28% and @11.54% False Rejection Rate (FRR) is achieved with the utilization of genetic programming enhanced with the random over sampled data set. In addition, with a totally free user movement in a haptic-enabled interpersonal communication system, an identification success rate of 83.3% is achieved when random forest classifier is utilized.

As a solution to mid-air haptic actuation with strong and continuous tactile force, Magnetic Rendering is presented as an intuitive haptic display method applying an electromagnet array to produce a magnetic field in mid-air where the force field can be felt as magnetic repulsive force exerted on the hand through the attached magnet discs. The magnetic field is generated by a specifically designed electromagnet array driven by direct current. By attaching small magnet discs on the hand, the tactile sensation can be perceived by the user. This method can provide a strong tactile force on multiple points covering user’s hand and avoid cumbersome attachments with wires, thus it is suitable for a co-located visual and haptic display. In my work, the detailed design of the electromagnet array for haptic rendering purposes is introduced, which is modelled and tested using Finite Element Method simulations. The model is characterized mathematically, and three methods for controlling the magnetic field are applied accordingly: direct control, system identification and adaptive control. The performance of the simulated model is evaluated in terms of magnetic field distribution, force strength, operation distance and force stiffness. The control algorithms are implemented and tested on a 3-by-3 and a 15-by-15 model, respectively. Simulations are performed on a 15-by-15 model to generate a haptic human face, which results in a smooth force field and accurate force exertion on the control points.

It is well known that home exercise is as good as rehab center. Unfortunately, passive devices such as dumbbells, elastic bands, stress balls and tubing that have been widely used for home-based arm rehabilitation do not provide therapists with the information needed to monitor the patient’s progress, identify any impairment, and suggest treatments. Moreover, the lack of interactivity of these devices turns the rehabilitation exercises into a boring, unpleasant task. In this thesis, we introduce a family of home-based post-stroke rehabilitation systems aimed at solving the aforementioned problems. We call such applications: “Shared Haptic Environment on the Cloud for Arm Rehabilitation Exercises (SHECARE)”. The systems combine recent rehabilitation approaches with efficient, yet affordable skeleton tracking input technologies, and multimodal interactive computer environment. In addition, the systems provide a real-time feedback to the stroke patients, summarize the feedback after each session, and predict the overall recovery progress. Moreover, these systems show a new style of home-based rehabilitation approach that motivate the patients by engaging the whole family and friends in the rehabilitation process and allow the therapists to remotely assess the progress of the patients and adjust the training strategy accordingly. Two mathematical models have been presented in this thesis. The first model is developed to find the relationship between upper extremity kinematics and the associated forces/strength. The second model is used to evaluate the medical condition of the stroke patients and predict their recovery progress depending on their performance history. The objective assessments, clinical tests, and the subjective assessments, usability studies have shown the feasibility of the proposed systems for rehabilitation in stroke patients with upper limb motor dysfunction.

Virtual reality is rapidly expanding its capabilities and accessibility to consumers. The application of virtual reality in training for precision tasks has been limited to specialized equipment such as a haptic glove or a haptic stylus, but not studied for handheld controllers in consumer-grade systems such as the HTC Vive. A straight-line precision steadiness task was adopted in virtual reality to emulate basic linear movements in industrial operations and disability rehabilitation. This study collected the total time and the error time for the straight-line task in both virtual reality and a physical control experiment for 48 participants. The task was performed at four different gap widths, 4mm, 5mm, 6mm, and 7mm, to see the effects of virtual reality at different levels of precision. Average error ratios were then calculated and analyzed for strong associations to various factors. The results indicated that a combination of Environment x Gap Width factors significantly affected average error ratios, with a p-value of 0.000. This human factors study also collected participants’ ratings of user experience dimensions, such as difficulty, comfort, strain, reliability, and effectiveness, for both physical and virtual environments in a questionnaire. The results indicate that the ratings for difficulty, reliability, and effectiveness were significantly different, with virtual reality rating consistently rating worse than the physical environment. An analysis of questionnaire responses indicates a significant association of overall environment preference (physical or virtual) with performance data, with a p-value of 0.027. In general, virtual reality yielded higher error among participants. As the difficulty of the task increased, the performance in virtual reality degraded significantly. Virtual reality has great potential for a variety of precision applications, but the technology in consumer-grade hardware must improve significantly to enable these applications. Virtual reality is difficult to implement without previous experience or specialized knowledge in programming, which makes the technology currently inaccessible for many people. Future work is needed to investigate a larger variety of precision tasks and movements to expand the body of knowledge of virtual reality applications for training purposes.