Approved for public release, distribution is unlimited / Once the military helicopter pilot deploys aboard a naval vessel he leaves behind all training platforms, short of the actual aircraft, that present enough fidelity for him to maintain the highest levels of readiness. To that end, this thesis takes a preliminary step in creating a trainer that places the pilot in an immersive and familiar environment to exercise myriad piloting tasks as faithfully and as rigorously as in actual flight. The focus of this thesis it to assess the viability of an chromakeyed augmented virtual environment (ChrAVE) trainer embedded into a helicopter for use in maintaining certain perishable skills. Specifically this thesis will address the task of helicopter low-level land navigation. The ChrAVE was developed to substantiate the viability of having embedded trainers in helicopters. The ChrAVE is comprised of commercial off the shelf (COTS) equipment on a transportable cart. In determining whether a system such as the ChrAVE is viable as a laboratory for continued training in virtual environment, the opinion of actual pilots that were tasked with realistic workloads was used. Additionally, empirical data was collected and evaluated according to the subject pool's thresholds for acceptable low-level navigation performance. / Captain, United States Marine Corps

Pessoas com tetraplegia possuem controle sobre poucas partes do corpo, por isso têm grande dificuldade de interação com o mundo ao seu redor. Interfaces humano-computador que utilizam a capacidade de controle muscular remanescente podem promover um acréscimo importante na autonomia do usuário com tetraplegia. Uma nova forma de interação com o computador é proposta: controle de um dispositivo de entrada por meio do lábio inferior. A ideia não foi partir da tecnologia e agregar recursos, mas sim, partir da pessoa e entender suas necessidades e potencialidades para pesquisar, propor, desenvolver e avaliar alternativas tecnológicas. Com o intuito de investigar se o lábio inferior seria capaz de controlar uma interface de maneira eficiente e precisa, foram desenvolvidos protótipos de um dispositivo de entrada vestível que pudesse ser acionado pelo lábio. Considerando o estado da arte, foi avaliada a capacidade do lábio inferior controlar esta interface proposta como um mouse de computador. Também foi realizada a comparação do controle pelo lábio inferior e pelo polegar utilizando o mesmo dispositivo sob as mesmas condições. O objetivo foi estabelecer referências sobre a capacidade do lábio inferior de controlar uma interface. Foram realizados e analisados os resultados de testes com sujeitos de pesquisa, mensurando e comprovando a capacidade tanto do lábio inferior, como parte do corpo capaz de controlar um dispositivo de entrada, quanto do próprio dispositivo. / People with tetraplegia control few body parts; therefore, they have great difficulty to interact with the environment. Human-computer interfaces that explore the residual muscular control capacity can provide the user with tetraplegia with an important autonomy increase. I propose a new form of interaction, a input device controlled by the lower lip. The idea was not starting from the technology and aggregate resources, but starting from the user needs to research, to propose, to develop and to evaluate technological alternatives. In order to investigate the lower lip ability to efficiently and precisely control an input device, prototypes of a wearable device were developed. I presented an evaluation of the lower lip potential to control a pointing input device, according to the state of the art. The lower lip throughput was compared with the thumb throughput using the same input device under the same conditions. The objective was to establish the baseline for future researches about the lower lip capacity to operate a computer pointing device. The results proved and measured the capacity of the interface as well as the lower lip as a body part able to control a input device.

Pessoas com tetraplegia possuem controle sobre poucas partes do corpo, por isso têm grande dificuldade de interação com o mundo ao seu redor. Interfaces humano-computador que utilizam a capacidade de controle muscular remanescente podem promover um acréscimo importante na autonomia do usuário com tetraplegia. Uma nova forma de interação com o computador é proposta: controle de um dispositivo de entrada por meio do lábio inferior. A ideia não foi partir da tecnologia e agregar recursos, mas sim, partir da pessoa e entender suas necessidades e potencialidades para pesquisar, propor, desenvolver e avaliar alternativas tecnológicas. Com o intuito de investigar se o lábio inferior seria capaz de controlar uma interface de maneira eficiente e precisa, foram desenvolvidos protótipos de um dispositivo de entrada vestível que pudesse ser acionado pelo lábio. Considerando o estado da arte, foi avaliada a capacidade do lábio inferior controlar esta interface proposta como um mouse de computador. Também foi realizada a comparação do controle pelo lábio inferior e pelo polegar utilizando o mesmo dispositivo sob as mesmas condições. O objetivo foi estabelecer referências sobre a capacidade do lábio inferior de controlar uma interface. Foram realizados e analisados os resultados de testes com sujeitos de pesquisa, mensurando e comprovando a capacidade tanto do lábio inferior, como parte do corpo capaz de controlar um dispositivo de entrada, quanto do próprio dispositivo. / People with tetraplegia control few body parts; therefore, they have great difficulty to interact with the environment. Human-computer interfaces that explore the residual muscular control capacity can provide the user with tetraplegia with an important autonomy increase. I propose a new form of interaction, a input device controlled by the lower lip. The idea was not starting from the technology and aggregate resources, but starting from the user needs to research, to propose, to develop and to evaluate technological alternatives. In order to investigate the lower lip ability to efficiently and precisely control an input device, prototypes of a wearable device were developed. I presented an evaluation of the lower lip potential to control a pointing input device, according to the state of the art. The lower lip throughput was compared with the thumb throughput using the same input device under the same conditions. The objective was to establish the baseline for future researches about the lower lip capacity to operate a computer pointing device. The results proved and measured the capacity of the interface as well as the lower lip as a body part able to control a input device.

Analisando-se os ambientes comerciais do setor imobiliário, pode-se verificar a presença frequente de maquetes que hoje fazem parte dos estandes de vendas dos edifícios. As maquetes são peças frágeis, de custo elevado, considerável prazo de execução e têm pequena vida útil, sendo usualmente descartadas no desmonte do estande de vendas. Nos estandes de vendas pode-se encontrar também o apartamento modelo decorado, com custos ainda mais elevados. É nesse ambiente que o MVE (Modelo Virtual Exploratório) se torna importante para os setores da construção e imobiliário, cujos mercados são cada vez mais competitivos, onde há a real necessidade da diminuição dos custos, da diminuição dos prazos de execução e do aumento da qualidade dos produtos. Por se tratar de um importante tema de pesquisa científica, este trabalho propôs o desenvolvimento do MVE para venda imobiliária residencial. O MVE foi modelado com um programa CAD (Computer-Aided Design) / BIM (Building Information Modeling) que se mostrou adequado às exigências levantadas na pesquisa. Avaliou-se o MVE através de estudos de caso em empreendimentos imobiliários reais, com corretores e clientes. Os resultados obtidos mostraram que o MVE pode ser desenvolvido com custo e prazo muito inferiores aos da maquete ou apartamento modelo e que tem grande aceitação tanto pelos profissionais de vendas quanto pelos clientes de empreendimentos imobiliários. / Examining the business environment of the real estate sector, it can be verified the large number of architectural mockups that today are part of real estate points-of-sale. Mockups are fragile, expensive, take considerable time to be made and have a short life span, as they are discarded when the sales booth is decommissioned. In the sales booths, it is also possible to find full size furnished mock-up flats, with even higher costs. In this environment, the MVE (Exploratory Virtual Model) becomes important for the construction and real estate industries, whose markets are increasingly competitive, where there is a real need to reduce costs, to decrease in execution times and to increase product quality. As it is an important topic of scientific research, this paper proposes the MVE development for selling residential real estate property. The MVE was modeled with a CAD (Computer-Aided Design) / BIM (Building Information Modeling) tool which proved adequate to meet requirements raised in the survey. The MVE was evaluated through case studies in real estate ventures with brokers and customers. Results show that MVE can be developed with much lower cost and in shorter time than the architectural mockup or the mock-up flat and that it has very well accepted both by real estate sales professionals and clients.

The main objective of the presented research is to design, fabricate, fully characterize, and assess the usability and functionality of a novel wireless tongue-operated assistive technology, called Tongue Drive System (TDS), that allows individuals with severe physical disabilities (such as quadriplegics) to effectively access computers, drive powered wheelchairs, and control environments using their voluntary tongue motion. The system can wirelessly detect users' tongue movements using an array of magnetic sensors, and a magnetic tracer secured on the tongue, and translate them into a set of user-defined commands in real time, which can then be used to communicate with target devices in users' environment. The principal advantage of the TDS is that a combination of magnetic sensors and a small permanent magnet can capture a large number of tongue movements, each of which can represent one specific command. A set of dedicated tongue movements can be configured as specific commands for each individual user based on his/her preferences, lifestyle, and remaining abilities. As a result, this technology can benefit a wide range of potential users with different types of disabilities. The work carried out in this dissertation is largely split into three portions: (1) Development, fabrication and characterization of external TDS (eTDS) prototypes to verify the concept of TDS that is detecting and extracting user's intention through their voluntary tongue motion, utilizing a combination of magnetic sensors and a small magnet, as well as the application of this idea in the context of assistive technology. This part of the work is presented in Chapters IV, V and VI. (2) Assessment of the TDS performance in medium term usage for both computer access and wheelchair control. The main purpose of this work was to gain valuable insight into the TDS learning process and its current limiting factors, which could lead the way in designing new generations of TDS with improved usability. This portion of the work is described in Chapter VII. (3) Development and performance assessment of a multimodal TDS (mTDS), that operates based on the information collected from two independent input channels: the tongue motion and speech. This multimodal system expands the access beyond one input channel and therefore improves the speed of access by increasing the information transfer bandwidth between users and computers. This part of the work is presented in Chapters VIII and IX. This dissertation has contributed to the innovation and advancement of the start-of-the-art assistive technology research by exploring, realizing and validating the use of tongue motion as a voluntary motor output to substitute some of the lost arm and hand functions in people with severe disabilities for computer access, wheelchair navigation, and environmental control.

With the improvement of multimedia technologies such as broadband-enabled HDTV, video on demand and internet TV, the computer and the TV are merging to become a single device. Moreover the previously cited technologies as well as DVD or Blu-ray can provide menu navigation and interactive content. The growing interest in video conferencing led to the integration of the webcam in different devices such as laptop, cell phones and even the TV set. Our approach is to directly use an embedded webcam to remotely control a TV set using hand gestures. Using specific gestures, a user is able to control the TV. A dedicated interface can then be used to select a TV channel, adjust volume or browse videos from an online streaming server. This approach leads to several challenges. The first is the use of a simple webcam which leads to a vision based system. From the single webcam, we need to recognize the hand and identify its gesture or trajectory. A TV set is usually installed in a living room which implies constraints such as a potentially moving background and luminance change. These issues will be further discussed as well as the methods developed to resolve them. Video browsing is one example of the use of gesture recognition. To illustrate another application, we developed a simple game controlled by hand gestures. The emergence of 3D TVs is allowing the development of 3D video conferencing. Therefore we also consider the use of a stereo camera to recognize hand gesture.

以肢體動作為基礎的人機介面一直被認為是未來家庭人機介面的表徵，然而，由於缺乏適合的應用環境和辨識技術，相關的應用尚未成熟。本研究嘗試提出一個互動式仿鏡顯示裝置作為肢體指令的平台，並提出相關的辨識技術，以設計一個可應用在智慧家庭環境中，符合人因工程的互動顯示裝置。 / Gesture-based user interfaces have long been associated with the image of future technology. However, due to the lack of proper environments and recognition technologies, practical applications of intelligent user interfaces are still rare in modern life. In this research, we propose an interactive mirror which can be controlled by gesture commands. We also provide several recognition techniques for this interactive display device. Practical applications are developed on this smart mirror, and user test is conducted to evaluate this novel user interface.

Traditional methods of input on mobile devices are cumbersome and difficult to use. Devices have become smaller, while their operating systems have become more complex, to the extent that they are approaching the level of functionality found on desktop computer operating systems. The buttons and toggle-sticks currently employed by mobile devices are a relatively poor replacement for the keyboard and mouse style user interfaces used on their desktop computer counterparts. For example, when looking at a screen image on a device, we should be able to move the device to the left to indicate we wish the image to be panned in the same direction. This research investigates a new input model based on the natural hand motions and reactions of users. The model developed by this work uses the generic embedded video cameras available on almost all current-generation mobile devices to determine how the device is being moved and maps this movement to an appropriate action. Surveys using mobile devices were undertaken to determine both the appropriateness and efficacy of such a model as well as to collect the foundational data with which to build the model. Direct mappings between motions and inputs were achieved by analysing users' motions and reactions in response to different tasks. Upon the framework being completed, a proof of concept was created upon the Windows Mobile Platform. This proof of concept leverages both DirectShow and Direct3D to track objects in the video stream, maps these objects to a three-dimensional plane, and determines device movements from this data. This input model holds the promise of being a simpler and more intuitive method for users to interact with their mobile devices, and has the added advantage that no hardware additions or modifications are required the existing mobile devices.

Analisando-se os ambientes comerciais do setor imobiliário, pode-se verificar a presença frequente de maquetes que hoje fazem parte dos estandes de vendas dos edifícios. As maquetes são peças frágeis, de custo elevado, considerável prazo de execução e têm pequena vida útil, sendo usualmente descartadas no desmonte do estande de vendas. Nos estandes de vendas pode-se encontrar também o apartamento modelo decorado, com custos ainda mais elevados. É nesse ambiente que o MVE (Modelo Virtual Exploratório) se torna importante para os setores da construção e imobiliário, cujos mercados são cada vez mais competitivos, onde há a real necessidade da diminuição dos custos, da diminuição dos prazos de execução e do aumento da qualidade dos produtos. Por se tratar de um importante tema de pesquisa científica, este trabalho propôs o desenvolvimento do MVE para venda imobiliária residencial. O MVE foi modelado com um programa CAD (Computer-Aided Design) / BIM (Building Information Modeling) que se mostrou adequado às exigências levantadas na pesquisa. Avaliou-se o MVE através de estudos de caso em empreendimentos imobiliários reais, com corretores e clientes. Os resultados obtidos mostraram que o MVE pode ser desenvolvido com custo e prazo muito inferiores aos da maquete ou apartamento modelo e que tem grande aceitação tanto pelos profissionais de vendas quanto pelos clientes de empreendimentos imobiliários. / Examining the business environment of the real estate sector, it can be verified the large number of architectural mockups that today are part of real estate points-of-sale. Mockups are fragile, expensive, take considerable time to be made and have a short life span, as they are discarded when the sales booth is decommissioned. In the sales booths, it is also possible to find full size furnished mock-up flats, with even higher costs. In this environment, the MVE (Exploratory Virtual Model) becomes important for the construction and real estate industries, whose markets are increasingly competitive, where there is a real need to reduce costs, to decrease in execution times and to increase product quality. As it is an important topic of scientific research, this paper proposes the MVE development for selling residential real estate property. The MVE was modeled with a CAD (Computer-Aided Design) / BIM (Building Information Modeling) tool which proved adequate to meet requirements raised in the survey. The MVE was evaluated through case studies in real estate ventures with brokers and customers. Results show that MVE can be developed with much lower cost and in shorter time than the architectural mockup or the mock-up flat and that it has very well accepted both by real estate sales professionals and clients.

Brain-computer interface (BCI) or Brain Machine Interface (BMI) provides direct communication channel between user’s brain and an external device without any requirement of user’s physical movement. Primarily BCI has been employed in medical sciences to facilitate the patients with severe motor, visual and aural impairments. More recently many BCI are also being used as a part of entertainment. BCI differs from Neuroprosthetics, a study within Neuroscience, in terms of its usage; former connects the brain with a computer or external device while the later connects the nervous system to an implanted device. A BCI receives the modulated input from user either invasively or non-invasively. The modulated input, concealed in the huge amount of noise, contains distinct brain patterns based on the type of activity user is performing at that point in time. Primary task of a typical BCI is to find out those distinct brain patterns and translates them to meaningful communication command set. Cursor controllers, Spellers, Wheel Chair and robot Controllers are classic examples of BCI applications. This study aims to investigate an Electroencephalography (EEG) based non-invasive BCI in general and its interaction with a web interface in particular. Different aspects related to BCI are covered in this work including feedback techniques, BCI frameworks, commercial BCI hardware, and different BCI applications. BCI paradigm Steady State Visually Evoked Potentials (SSVEP) is being focused during this study. A hybrid solution is developed during this study, employing a general purpose BCI framework OpenViBE, which comprised of a low-level stimulus management and control module and a web based Google Street View client application. This study shows that a BCI can not only provide a way of communication for the impaired subjects but it can also be a multipurpose tool for a healthy person. During this study, it is being established that the major hurdles that hamper the performance of a BCI system are training protocols, BCI hardware and signal processing techniques. It is also observed that a controlled environment and expert assistance is required to operate a BCI system.