Dans un contexte de conduite automobile, afin de compenser la perte d’informations haptiques causée par la substitution d’interfaces mécaniques classiques par des écrans tactiles, nous avons envisagé d’étudier l’ajout d’un retour vibrotactile. Ces derniers seraient utilisés comme le témoin d’une rétroaction entre le système et l’utilisateur. Le premier objectif de ce travail a été de fournir une meilleure compréhension des facteurs et des paramètres physiques influençant la perception et l’évaluation de ces courts signaux vibrotactiles, afin d’en optimiser la création. Les données recueillies ont été à la fois subjectives (échelles de Likert) et objectives (temps de réaction, pourcentage d’erreurs). Un lien étroit entre les préférences des utilisateurs et les boutons mécaniques existants a été envisagé. Le second objectif a été d’étudié les signaux multimodaux (audio-haptique ou visuo-haptique) pour modifier la perception haptique. Pour cela, nous avons utilisé des paradigmes d’amorçage à court terme décomposés en deux phases : une phase d’apprentissage et une phase d’amorçage. La phase d’apprentissage se caractérise par l’association non explicite d’une composante haptique (une vibration de courte durée) et d’un autre composant sensoriel (soit un son, soit une image en mouvement ou statique). La phase d’amorçage se caractérise par la présentation du composant haptique seule (l’amorce) suivie d'un composant cible (soit sonore, soit visuelle). Nous avons ainsi observé que des composants perceptivement absents, mais préalablement associés, pouvaient modifier le traitement perceptif de composants en cours. Ce type de résultats nous a permis d’apporter de nouveaux arguments en faveur de mécanismes communs sous-jacents à la perception et à la mémoire à travers une modalité peu utilisée dans la littérature. Une meilleure compréhension de ces liens, nous a aidé à modifier la perception des utilisateurs, à partir de composants perceptivement absents. Une illusion a ainsi été créée à partir de signaux visuo-haptiques, permettant de se rapprocher des boutons mécaniques. / In the context of driving, in order to compensate the loss of haptic information caused by the substitution of conventional mechanical interfaces of touch screens, we proposed to study the addition of vibrotactile feedback. This would be used as a feedback control between the system and the user. The first objective of this work was to provide a better understanding of the factors and physical parameters influencing the perception and evaluation of these short vibrotactile signals in order to optimize their creation. The data collected were both subjective (Likert scales) and objective (reaction time, error rate). A link between the user’s preferences and existing mechanical buttons was considered. The second objective was to study multimodal signals (audio-haptic or visuo-haptic) in order to change haptic perception. For this, we used a short term priming paradigm divised in two phases : a learning phase and a priming phase. The priming phase is characterized by non-explicit association between a haptic component (a vibration) simultaneously with another sensorial component (either a sound, or a moving or static image). The priming phase is characterized by the presentation of one haptic component (prime) followed a target component (sound or visual). We observed that components perceptually absent, but previously associated, could change perceptual processing of target components. These types of results have allowed us to provide new arguments for common mechanisms underlying perception and memory through a modality rarely studied in scientific literature. A better understanding of these links, helped us to change the user’s perception from perceptually absent components. An illusion had been created with the visual-haptic signals allowing a closer approach to the mechanical buttons.

Os simuladores que empregam técnicas de Realidade Virtual são alternativas vantajosas às formas tradicionais de ensino e treinamento médico. Esses simuladores apresentam requisitos específicos, tais como: interação em tempo real e modelos realistas para representar órgãos e tecidos. Além disso, devem possuir comportamentos físicos suficientemente parecidos com os reais e gerar feedbacks dos procedimentos que estejam sendo simulados. Essas características exigem esforços de programação para o desenvolvimento de técnicas de interação e visualização 3D, além de estudos dos tecidos humanos, incluindo o comportamento físico dos órgãos e tecidos e o estudo das leis da Física envolvidas neste processo. O tema central desta pesquisa é a simulação de procedimentos de punção, sendo que nesse tipo de aplicações são necessários tanto o realismo visual como também o háptico, a fim de proporcionar ao usuário sensações parecidas com as encontradas nos procedimentos reais. Os métodos que utilizam parâmetros físicos são os mais utilizados alcançar o realismo exigido na interação háptica. No entanto, esses métodos deixam a desejar no que diz respeito à interação em tempo real. Dessa forma, o objetivo desta pesquisa foi desenvolver um novo método para simular a deformação de objetos tridimensionais que representam órgãos humanos. De forma que sejam alcançados o realismo visual, o realismo háptico e a interação em tempo real, com um custo computacional aceitável. O método desenvolvido consiste na divisão dos objetos tridimensionais em camadas, a fim de simular o volume e também a heterogeneidade dos órgãos humanos. O número de camadas e a atribuição de parâmetros físicos podem ser definidos de acordo com os diferentes tecidos que compõem o órgão humano e respectivos comportamentos que se pretenda simular. O método foi desenvolvido depois de conduzida uma Revisão Sistemática para levantamento dos métodos utilizados em aplicações para treinamento médico e respectivos níveis de realismo visual e háptico oferecidos. Para demonstrar e testar o funcionamento do método foi criado um simulador genérico de procedimentos de punção, no qual podem ser configurados o número de camadas, os parâmetros visco-elásticos, e assim permitir a avaliação do desempenho e o realismo das simulações. Como exemplo de aplicação o método foi aplicado em um simulador de punção de mama, cuja qualidade foi avaliada por médicos especialistas. Os protótipos foram criados no Laboratório de Tecnologias Interativas da Escola Politécnica da USP (Interlab), a partir de um Framework desenvolvido pelo Laboratório de Aplicações de Informática em Saúde da Escola de Artes Ciências e Humanidades da USP (LApIS). / Simulators that employ Virtual Reality techniques can prove to be an advantageous alternative to the traditional forms of medical learning and training. These simulators have specific requirements, such as real-time interaction and realistic models representing organs and tissues. Moreover, they should possess physical behavior similar enough to real life and generate feedback from procedures being simulated. These characteristics require programming efforts for the development of 3D visualization and interaction techniques, as well as studies of human tissue, including the physical behavior of organs and tissues and the study of the laws of Physics involved in this process. The main theme of this research is the simulation of puncture procedures. This type of application requires a realistic rendering of both visual and haptic traits in order to provide the user with sensations similar to those found in real procedures. Methods which employ physical parameters are more widely used to achieve the realism required in haptic interaction. However, these methods present shortcomings regarding real-time interaction. Thus, the aim of this research was to develop a new method to simulate the deformity of tridimensional objects that represent human organs and to achieve visual realism, haptic realism, and real-time interaction, with acceptable computational costs. The method developed in this study consists in dividing tridimensional objects into layers in order to simulate volume as well as heterogeneity of human organs. The number of layers and the attribution of physical parameters can be defined according to different tissues that compose the human organ and respective behaviors that one wishes to simulate. The method was developed after a systematic review to assess the methods employed in applications for medical training and their respective levels of visual and haptic realism. In order to demonstrate and to test how the method operates, we created a generic simulator of puncture procedures, which can be configured with any combination of layers of tissue and its viscoelastic parameters, allowing for the assessment of simulation performance and realism. As an example, the method was applied to a breast biopsy simulator whose quality was evaluated by specialist doctors. The prototypes were created in the Interactive Technology Laboratory (Interlab) of the Engineering School of the University of São Paulo, from a framework developed by the Laboratory of Computer Applications for Health Care (LApIS) of the School of Arts, Science and Humanities of the University of São Paulo.

O exame de palpação é um procedimento no qual um profissional da saúde pressiona uma região específica do corpo de um paciente com os dedos a fim de detectar a presença ou ausência de características e anormalidades sob a pele. Um simulador que forneça subsídios para o treinamento deste procedimento pode contribuir para o aprendizado da técnica e o aprimoramento de sua execução em pacientes, além de, dentre outras vantagens, eliminar a necessidade de exposição de pacientes aos riscos das situações reais, minimizar o uso de objetos físicos e tornar possível a simulação de diferentes casos e ações. O objetivo principal deste trabalho consiste em desenvolver um simulador de palpação mamária com retorno háptico realista. No contexto do exame de palpação, existe uma lacuna no que concerne a definição de técnicas e parâmetros de retorno de força que ofereçam uma experiência mais realista de interação para o usuário. Após uma revisão sistemática da literatura foi feito o mapeamento dos principais parâmetros para representar diferentes nódulos durante o treinamento virtual visando ao diagnóstico de anomalias mamárias. Tais parâmetros foram equacionados com a finalidade de obter uma representação realista em um dispositivo háptico. Além disso, interfaces foram implementadas para permitir a geração de casos com a variação desses parâmetros referentes aos nódulos (tamanho, consistência, localização e contorno). Os resultados obtidos a partir de uma avaliação com uma profissional da medicina mostraram que o retorno de força aplicado a cada uma das características foi realista para todos os parâmetros, com limitação na representação de nódulos espiculados, que deve ser melhorada. Além de contribuir com a área de Computação no sentido de ter definido técnicas e parâmetros de retorno de força importantes para a simulação de exames de palpação, o trabalho disponibiliza uma ferramenta que pode contribuir para a aquisição efetiva da habilidade necessária para a execução do procedimento em questão / Palpation exam is a procedure in which a health care professional presses a specific region of a patient\'s body with the fingers in order to detect the presence of features and abnormalities under the skin. A simulator that aids the training this procedure may contribute to the learning of the technique and the improvement of its implementation in real patients. The main objective of the Master\'s degree work proposed in this paper is to develop a breast palpation simulator with haptic feedback. The results of a Systematic Review conducted in order to assess the state-of-art of the simulation of the palpation procedure showed that despite being a new area, there are already many innovative works that received positive reviews. However, in the context of palpation examination, a gap exists regarding the definition of techniques and force feedback parameters to provide a more realistic experience to the user. After a Systematic Review from the literature the mapping of the main parameters was done to represent different nodules during the virtual training for the diagnosis of breast abnormalities. These parameters were transformed into equations with the goal of obtaining a realistic representation on a haptic device. Besides, user interfaces were implemented to allow the creation of cases with the variation of the parameters relative to the nodules (size, consistency, localization and contour). The results obtained from an assessment done with a medical professional showed that the force feedback applied to each one of the characteristics was realistic to all the parameters, with limitations on the representation of spiculated nodules, which must be improved. Aside from the contribution to the Computing area in the sense that force feedback techniques and parameters important to the simulation of palpation exams were defined, this work offers a tool that can contribute to the effective acquisition of the necessary skills to the execution of the procedure in question

Made available in DSpace on 2016-04-29T14:23:33Z (GMT). No. of bitstreams: 1 Nelson Baumgratz Ribeiro.pdf: 6073619 bytes, checksum: 2df093db1d33c78d97967095f32b28c0 (MD5) Previous issue date: 2014-10-27 / Within the AV industry, the critical challenge is the creation of skilled and qualified labor, which presumes, in addition to the theoretical learning, contact and interaction with technology and with the devices and equipment used in AV systems - complementing 'knowing' with 'doing'. Perfection, agility and safe operation of AV systems will be a result of long hours of hands-on practice exercises, of trial and error experience and of tutored learning. Due to the job dynamics in this industry, the adoption of the online learning modality seems imperative as a further enabling feature of this technological training. Based greatly on the author's experience and with the support of authors like Vygotsky (1998), Lévy (1993, 1999), Ausubel (1963, 1968, 1978, 1980, 1982), Gagné (1980), Goleman (1999, 2012), Mattar (2007, 2008) and Allegretti (2007), amongst others, this paper aims to propose immersive and/or' interactive experience simulation tools to provide effective professional audiovisual (AV) technology online learning. Although not completely replacing the physical experience necessary to such learning, the proposed online learning could effectively shorten the time of study and give the learner sufficient experience to safely master the technical and operational processes when faced with the real-world tasks that will eventually be required of him. It should also provide students with the best possible learning experience, taking into account the effect and interaction of the environment with the professional audiovisual systems and with the student himself. This work, through qualitative research and case study, will present the context of the AV Industry inc1uding its social and professional organization, market segments, products and services, and area of operation and implementation of professional audiovisual systems, as specified in market research sponsored by InfoComm International@ in 2014. It will also set out the characteristics of the subject learner and his/her required skills and abilities, plus demonstrate the teaching and learning object, by means of a Technological Higher Education project developed by the author or a higher education institution in the city of São Paulo. Finally, this paper will list the available technological resources or those still under development, that can be combined for use as effective tools of online Technological Education, designed to empower professionals capable of working in the AV industry / No âmbito da Indústria Audiovisual Profissional, o ponto nevrálgico é a formação de mão de obra especializada e altamente qualificada. Esta formação pressupõe, além do aprendizado teórico, o contato e o convívio estreito com a própria tecnologia audiovisual (AV), com os aparelhos e equipamentos utilizados nos sistemas AV, complementando assim o "saber" com o "fazer". A perfeição, agilidade e segurança da operação dos sistemas AV advirão de longas horas de prática hands-on, de exercícios de tentativa e erro, e de aprendizado tutelado; e a dinâmica do próprio mercado de trabalho desta indústria torna imperativa a adoção da modalidade de ensino online como mais um recurso viabilizador desta formação tecnológica. A partir da experiência do seu autor e amparado em autores como Vygotsky (1998), Lévy (1993, 1999), Ausubel (1963, 1968, 1978, 1980, 1982), Gagné (1980), Goleman (1999, 2012), Mattar (2007, 2008) e Allegretti (2007), entre outros, este trabalho tem como objetivo propor ferramentas eficazes de simulação e de experiência interativa e/ou imersiva que proporcionem um efetivo aprendizado online da tecnologia AV. Embora não substituindo completamente a experiência física necessária a tal aprendizado, essas ferramentas propostas deverão ser capazes de abreviar eficientemente o tempo de estudo e de dar ao aprendiz da tecnologia uma vivência suficientemente satisfatória para permitir-lhe a segurança e o domínio dos processos técnicos e operacionais ao se deparar com a experiência real que lhe será imposta por fim. Deverão também proporcionar ao estudante a melhor experiência de aprendizagem, levando em conta o efeito e a interação do ambiente com os sistemas audiovisuais profissionais e com o próprio estudante. Por meio de uma pesquisa aplicada de abordagem qualitativa, este trabalho apresenta, a partir do estudo de mercado promovido pela InfoComm International@ em 2014, o contexto da Indústria AV e sua organização social e profissional, com seus segmentos de mercado, seus produtos e serviços, sua área de atuação e a aplicação dos sistemas audiovisuais profissionais. Enunciam-se também as características do sujeito da aprendizagem e as competências e habilidades dele requeridas, e o respectivo objeto de ensino e aprendizagem, por meio de uma proposta de Projeto de Curso Superior Tecnológico desenvolvido pelo autor para uma instituição de ensino superior da cidade de São Paulo. Finalmente, serão elencados os recursos tecnológicos disponíveis ou ainda em desenvolvimento, que podem ser conjugados para que se utilizem como ferramentas eficazes de Ensino Tecnológico online, destinadas a capacitar profissionais aptos a trabalhar na Indústria AV

abstract: One type of assistive device for the blind has attempted to convert visual information into information that can be perceived through another sense, such as touch or hearing. A vibrotactile haptic display assistive device consists of an array of vibrating elements placed against the skin, allowing the blind individual to receive visual information through touch. However, these approaches have two significant technical challenges: large vibration element size and the number of microcontroller pins required for vibration control, both causing excessively low resolution of the device. Here, I propose and investigate a type of high-resolution vibrotactile haptic display which overcomes these challenges by utilizing a ‘microbeam’ as the vibrating element. These microbeams can then be actuated using only one microcontroller pin connected to a speaker or surface transducer. This approach could solve the low-resolution problem currently present in all haptic displays. In this paper, the results of an investigation into the manufacturability of such a device, simulation of the vibrational characteristics, and prototyping and experimental validation of the device concept are presented. The possible reasons of the frequency shift between the result of the forced or free response of beams and the frequency calculated based on a lumped mass approximation are investigated. It is found that one of the important reasons for the frequency shift is the size effect, the dependency of the elastic modulus on the size and kind of material. This size effect on A2 tool steel for Micro-Meso scale cantilever beams for the proposed system is investigated. / Dissertation/Thesis / Doctoral Dissertation Systems Engineering 2019

This Open Platform for Limit Protection guides the open design of maneuver limit protection systems in general, and manned, rotorcraft, aerospace applications in particular. The platform uses three stages of limit protection modules: limit cue creation, limit cue arbitration, and control system interface. A common set of limit cue modules provides commands that can include constraints, alerts, transfer functions, and friction. An arbitration module selects the best limit protection cues and distributes them to the most appropriate control path interface. This platform adopts a holistic approach to limit protection whereby it considers all potential interface points, including the pilots visual, aural, and tactile displays; and automatic command restraint shaping for autonomous limit protection. For each functional module, this thesis guides the control system designer through the design choices and information interfaces among the modules. Limit cue module design choices include type of prediction, prediction mechanism, method of critical control calculation, and type of limit cue. Special consideration is given to the nature of the limit, particularly the level of knowledge about it, and the ramifications for limit protection design, especially with respect to intelligent control methods such as fuzzy inference systems and neural networks. The Open Platform for Limit Protection reduces the effort required for initial limit protection design by defining a practical structure that still allows considerable design freedom. The platform reduces lifecycle effort through its open engineering systems approach of decoupled, modular design and standardized information interfaces. Using the Open Platform for Limit Protection, a carefree maneuver system is designed that addresses: main rotor blade stall as a steady-state limit; hub moment as a transient structural limit; and pilot induced oscillation as a controllability limit. The limit cue modules in this system make use of static neural networks, adaptive neural networks, and fuzzy inference systems to predict these limits. Visual (heads up display) and tactile (force-feedback) limit cues are employed. The carefree maneuver system is demonstrated in manned simulation using a General Helicopter (GENHEL) math model of the UH-60 Black Hawk, a projected, 53 degree field of view for the pilot, and a two-axis, active sidestick for cyclic control.

Digital Clay is a next generation human-machine communication interface based on a tangible haptic surface. This thesis embraces this revolutionary concept and seeks to give it a physical embodiment that will confirm its feasibility and enable experimentation relating to its utility and possible improvements. Per the approach adopted in work, Digital Clay could be described as a 3D monitor whose pixels can move perpendicularly to the screen to form a morphing surface. Users can view, touch and modify the shape of the working surface formed by these pixels. In reality, the pixels are the tips of micro hydraulic actuators or Hapcel (i.e. haptic cell, since the Digital Clay supports the haptic interface). The user can get a feel of the desired material properties when he/she touches the working surface. The potential applications of Digital Clay cover a wide range from computer aided engineering design to scientific research to medical diagnoses, 3D dynamic mapping and entertainment. One could predict a future in which, by using Digital Clay, not only could the user watch an actor in a movie, but also touch the face of the actor! This research starts from the review of the background of virtual reality. Then the concept and features of the proposed Digital Clay is provided. Research stages and a 5x5 cell array prototype are presented in this thesis on the structural design and control of Digital Clay. The first stage of the research focuses on the design and control of a single cell system of Digital Clay. Control issues of a single cell system constructed using conventional and off-the-shelf components are discussed first in detail followed by experimental results. Then practical designs of micro actuators and sensors are presented. The second stage of the research deals with the cell array system of Digital Clay. Practical structural design and control methods are discussed which are suitable for a 100x 100 (even 1000X 1000) cell array. Conceptual design and detailed implementations are presented. Finally, a 5 x 5 cell array prototype constructed using the discussed design solutions for testing is presented.

A formable surface is part of an effort to create a haptic device that allows for a three dimensional human-computer interface called digital clay. As with real clay, digital clay allows a user to physically manipulate the surface into some form or orientation that is sensed and directly represented in a computer model. Furthermore, digital clay will allow a user to change the computer model by manipulating the inputs that are directly represented in the physical model. The digital clay device being researched involves a computer-interfaced array of vertically displacing actuators that is bound by a formable surface. The surface is composed of an array of unit cells that are constructed of compliant spherical joints and translational joints. As part of this thesis, a series of unit cells were developed and planar surfaces were fabricated utilizing the additive manufacturing process of stereolithography. The process of computing the resultant shape of a manipulated surface was modeled mathematically through energy minimization algorithms that utilized least squares analysis to compute the positions of the unit cells of the surface. Simulation results were computed and analyzed against the movement of a fabricated planar surface. Once the mathematical models were validated against the manufactured surface, a method for attaching the surface to an array of actuators was recommended.

Human-machine interfaces influence both operator effectiveness and machine efficiency. Further immersion of the operator into the machine’s working environment gives the operator a better feel for the status of the machine and its working conditions. With this knowledge, operators can more efficiently control machines. The use of multi-modal HMIs involving haptics, sound, and visual feedback can immerse the operator into the machine’s environment and provide assistive clues about the state of the machine. This thesis develops a realistic excavator model that mimics a mini-excavator’s dynamics and soil interaction during digging tasks. A realistic graphical interface is written that exceeds the quality of current academic simulators. The graphical interface and new HMI are placed together with a model of the excavator’s mechanical and hydraulic dynamics into an operator workstation. Two coordinated control schemes are developed on an haptic display for a mini-excavator and preliminary tests are run to measure increases in operator effectiveness and machine efficiency.

In a large number of electronic devices, users interact with the system by navigating through various menus. Auditory menus can complement or even replace visual menus, so research on auditory menus has recently increased with mobile devices as well as desktop computers. Despite the potential importance of auditory displays on touch screen devices, little research has been attempted to enhance the effectiveness of auditory menus for those devices. In the present study, I investigated how advanced auditory cues enhance auditory menu navigation on a touch screen smartphone, especially for new input gestures such as tapping, wheeling, and flicking methods for navigating a one-dimensional menu. Moreover, I examined if advanced auditory cues improve user experience, not only for visuals-off situations, but also for visuals-on contexts. To this end, I used a novel auditory menu enhancement called a "spindex" (i.e., speech index), in which brief audio cues inform the users of where they are in a long menu. In this study, each item in a menu was preceded by a sound based on the item's initial letter. One hundred and twenty two undergraduates navigated through an alphabetized list of 150 song titles. The study was a split-plot design with manipulated auditory cue type (text-to-speech (TTS) alone vs. TTS plus spindex), visual mode (on vs. off), and input gesture style (tapping, wheeling, and flicking). Target search time and subjective workload for the TTS + spindex were lower than those of the TTS alone in all input gesture types regardless of visual type. Also, on subjective ratings scales, participants rated the TTS + spindex condition higher than the plain TTS on being 'effective' and 'functionally helpful'. The interaction between input methods and output modes (i.e., auditory cue types) and its effects on navigation behaviors was also analyzed based on the two-stage navigation strategy model used in auditory menus. Results were discussed in analogy with visual search theory and in terms of practical applications of spindex cues.