Este trabalho tem como objetivo discutir o estado da arte e propor melhorias em uma área de grande potencial e de crescimento recente na computação: a análise do estado emocional do usuário de um sistema de informação e a aplicação dessa observação em cenários ricos de interação homem-máquina. Para foco em um escopo tratável, escolheu-se o estudo das técnicas atuais de captação afetiva através de três sensores diferentes (expressões faciais, expressões vocais e contexto semântico) e de como sua combinação poderia resultar em dados mais robustos sobre o estado emocional do usuário, em uma aplicação conhecida como Fusão Multimodal. Nesse sentido, o trabalho levanta não apenas a bibliografia recente sobre o estado da arte da aquisição de dados nas três áreas e em fusão, mas também aplica algumas das ferramentas existentes (tanto comerciais como abertas) para melhor compreensão do nível atual da tecnologia acessível ao público acadêmico. Dada a inexistência de uma aplicação facilmente disponível para análise de emoções no contexto semântico, desenvolveu-se uma ferramenta dedicada à pesquisa, nomeada emoCrawler, a qual utiliza redes sociais para avaliação do caráter emocional que palavras indiretamente podem acarretar em uma discussão. O emoCrawler busca avaliar as emoções evocadas em usuários de Twitter e as compara com um dicionário emocional conhecido, baseado em contribuições acadêmicas diversas, gerando um resultado quantitativo útil ao escopo do trabalho. Por fim, o trabalho lançou mão da ferramenta emoCrawler e das outras ferramentas selecionadas para análise de expressões faciais e vocais na criação de um sistema de fusão sensorial que aumente a confiabilidade global da emoção detectada. Para isso, inicialmente um corpus de áudio e vídeo disponível foi tratado e reduzido para o escopo da pesquisa. Em seguida, diferentes técnicas de fusão baseadas em algoritmos de classificação foram aplicadas (redes neurais, Naive Bayes e Support Vector Machine), com abordagens distintas de treinamento para busca de resultados melhores dos que os encontrados nas aplicações unimodais. Os resultados verificados de fato indicam para uma melhor taxa de detecção quando é aplicado o algoritmo de fusão. Nesse processo, a presença de dados fornecidos pelo emoCrawler parece ter contribuído positivamente. / This work aims at discussing the state of the art and proposing improvements in a field of great potential and recent growth in computing: the analysis of the emotional state of the user of an information system and the application of such observations in rich scenarios of human-machine interaction. In a tractable scope of research, this work evaluates the current techniques of affective observation applied on three different inputs (facial expressions, speech features and semantic context) and comprehends how their combination can result in more robust affirmations on the actual emotional status of the user an application called Multimodal Fusion. In this sense, the work presents not only the recent literature on the state of the art in data acquisition in the three aforementioned areas and in fusion, but also applies some of the existing tools (both commercial and open) for a better understanding of the current level of technology available to the academic audience. Given the lack of an application readily available for the analysis of semantic context in emotions, a dedicated piece of software to the research was developed, entitled emoCrawler, which uses social networks as a means to evaluate the emotional signature that words may leave in a discussion. emoCrawler aims at evaluating emotions evoked in Twitter users and comparing them with an emotion dictionary built around academic contributions, generating a quantitative result useful to this research. Finally, this work made use of emoCrawler and the other selected tools for both facial and speech expressions in the creation of a sensor fusion system to enhance the global reliability of the affective detection. To achieve this, an audiovisual database was initially processed and condensed. Then, different fusion techniques based on classification algorithms with distinct learning approaches (neural networks, naïve Bayes and Support Vector Machine) were applied in the pursuit of better outcomes than the unimodal applications. The final results indeed point to a better rate of detection when fusion algorithms are applied. In this process, the data provided by emoCrawler seems to have contributed positively.

Human operator models developed using optimal control theory are typically complicated and over-parameterized, even for simple controlled elements. Methods for generating less complicated operator models that preserve the most important characteristics of the full order model are developed so that the essential features of the operator dynamics are easier to determine. A new formulation of the Optimal Control Model (OCM) of the human operator is developed that allows order reduction techniques to be applied in a meaningful way. This formulation preserves the critical neuromotor dynamics and time delay characteristics of the human operator. The Optimal Projection (OP) synthesis technique is applied to a modified version of the OCM. Using OP synthesis allows one to determine operator models that minimize the quadratic performance index of the OCM with a constraint on model order. This technique allows analysts to formulate operator models of fixed order. Operator model reduction methods based on variations of balanced realization techniques are also developed since they reduce the computational complexity associated with OP synthesis yet maintain a reasonable level of accuracy. Computer algorithms are developed that insure that the reduced order models have noise to signal ratios that are consistent with OCM theory. The OP method generates operator models of fixed order that are consistent with OCM theory in all respects, i.e. optimality, neuromotor lag, time delay, and noise to signal ratios are all preserved. The other model reduction techniques preserve these features with the exception of optimality. Each technique is applied to a variety of controlled elements to illustrate how performance and frequency response fidelity degrade when the order of the operator model is reduced. / Ph. D.

Approved for public release; distribution is unlimited / This thesis investigates the problems in current naval aircraft cockpits as perceived by fleet naval aviators who are students at the U.S. Naval Aviation Safety School. A critical incident questionnaire provides data that examines the deficiencies of an individual aircraft. These individual aircraft deficiencies are then categorized into twelve major deficiency categories which are common to ore than one aircraft. Various recommendations are made concerning standards, specifications, cockpit research and cockpit design. / http://archive.org/details/pilotreportedhum00dani / Commander, United States Navy

Les travaux présentés dans ce manuscrit portent sur l'utilisation des humains virtuels et, plus globalement, des technologies du jeu vidéo pour améliorer le soin aux personnes âgées atteintes de troubles cognitifs. Nos travaux s'articulent autour de deux cas d'utilisation des humains virtuels : comme non-soi et comme soi. Plus spécifiquement, nous avons conçu, implémenté et évalué un agent conversationnel animé, appelé LOUISE (LOvely User Interface for Servicing Elders), ayant pour but de servir d'interface utilisateur accessible dans les dipositifs de compensation cognitives à destination des personnes âgées attenteintes de troubles cognitifs, et un dispositif de thérapie psychologique par la réalité virtuelle pour traiter les conséquences des chutes, appelé Promenade virtuelle. Ces deux projets ont été menés suivant des principes de conception participative en living lab, implicant plusieurs parties prenantes dans le processus de conception (patients, aidants et professionnels de santé). / The work presented in this manuscript investigates the use of virtual huamans, and, more broadly, of video game technologies to improve the care of older adults with cognitive impairment. Our work revolved around two use cases of virtual humans: as non-self and as self. More specifically, we have designed, implemented and evaluated an embodied conversational agent, called LOUISE (LOvely User Interface for Servicing Elders), meant to serve as an accessible user interface in cognitive compensation systems for older adults with cognitive impairment, and a system for virtual reality-based psychological therapy addressing the consequences of falls, called Virtual Promenade. These two projects have been conducted according to the principles of living lab participatory design, involving several stakeholders in the design process (patients, caregivers and healthcare professionals).

Novas interfaces homem-computador têm sido pesquisadas a fim de se tornarem mais naturais e flexíveis. O rastreamento labial, foco deste trabalho, é parte deste contexto, podendo ser utilizado na detecção de emoções, bem como no auxílio ao reconhecimento de voz. Pode assim tornar-se um módulo inicial para a leitura labial, na criação de interfaces voltadas a deficientes auditivos. Os algoritmos disponíveis na literatura foram analisados e comparados, mostrando os prós e contras de cada método. Finalmente foi escolhido desenvolver uma técnica baseada em Active Appearance Model (AAM). O AAM gera um modelo a partir de um conjunto de imagens de treinamento, que pode ser utilizado no rastreamento labial de novas imagens. A técnica proposta baseia-se no uso de algoritmos genéticos para o ajuste do modelo, diferente, portanto, da técnica proposta originalmente pelo AAM. A convergência da técnica proposta foi extensivamente analisada, com a variação de parâmetros, buscando a análise de erro residual da função custo e sua relação com o tempo de convergência e erro de posição. / New human-computer interfaces have been researched to make them more natural and flexible. The lip tracking, emphasis of this work, is part of this setting it can be used for detecting emotions and to aid the speech recognition. It may well become an initial module for lip-reading, in the creation of interfaces directed to hearing impaired people. Algorithms available on literature are analyzed and compared, explaining the upsides and downsides of each method. Finally was chose to develop a technique based on Active Appearance Model (AAM). The AAM generates a model from a set of training images, which can be used in tracking of new lip images. The proposed technique is based on the use of genetic algorithms for fitting the model, different therefore of the technique originally proposed by AAM. The convergence of the proposed technique has been extensively analyzed with the variation of parameters, searching the error analysis of the cost function residual error and its relation to the execution time and position error.

Made available in DSpace on 2016-04-04T18:27:47Z (GMT). No. of bitstreams: 1 Marcos de Toledo Benassi.pdf: 2077139 bytes, checksum: fb6293f043c90566fbb8dea6710e711c (MD5) Previous issue date: 1997-06-24 / Pontif?cia Universidade Cat?lica de Campinas / In our contemporary society the computer has been used in a wide range of activities, in which man can set up relationships with this machine. This man-machine interactions have been studied by several sciences, that contribute with the understanding of these interactions, aiming to build better computer software s interfaces. The hypertext is a kind of software that can organize the information in a non-linear way, using a particular interface to allow the access to its contents. The aim of this research was to understand how subjects interact with a hypertext system created for this work, in order to determine the style of interaction: linear, non-linear or a blend of both. In addition, another aim was to correlate these categories of interactions with answers gave by the subjects to some questions, in order to verify if some relationship can be established among the quality of the answers and the interaction style. Using the X2 test was possible to conclude that the group of subjects used the non-linear style of interaction. The Spearman s rs coefficient found no significant correlations among the quality of subjects answers and the interaction style. / O computador tem se introduzido na sociedade contempor?nea em uma grande variedade de atividades, nas quais o homem com ele estabelece rela??es, quer sejam de trabalho que de lazer. Estas intera??es homem-m?quina tem sido objeto de estudo de um conjunto de ci?ncias, que contribuem para a compreens?o de como estas intera??es se processam, visando aprimorar as interfaces oferecidas pelos programas de computador para sua utiliza??o. Um dos muitos tipos de programas dispon?veis ? o hipertexto, que permite uma modalidade n?o-linear de organiza??o de informa??es, provendo uma interface peculiar para o acesso ao seu conte?do. Este trabalho teve como objetivo a compreens?o das modalidades de intera??o estabelecidas por sujeitos com um hipertexto criado para esta pesquisa, visando determinar se esta seria linear, n?o-linear ou indiferenciada. Adicionalmente, foram tra?adas correla??es entre quest?es respondidas pelo sujeitos visando verificar a exist?ncia de rela??es entre sua qualidade e a modalidade predominante de intera??o do grupo de sujeitos com o produto. Atrav?s do registro automatizado das intera??es e posterior an?lise estat?stica utilizando-se a prova de X2, verificou-se que o grupo de sujeitos estabeleceu, no contexto experimental desta pesquisa, uma intera??o predominantemente n?o-linear, interagindo com o hipertexto explorando a peculiaridade da m?dia. Utilizando-se o coeficiente Rn de Spearman foi tra?ada a correla??o entre este padr?o predominante de intera??o e a qualidade da resposta, n?o se obtendo resultados estatisticamente significativos.

<p>This thesis is about the design and evaluation of a graphical display that aims at showing the behavior of a complex automated system in action. A problem with complex automation is that it sometimes surprises the user by performing actions that were not expected. The reason of this is poor communication of system activities. In order to study how to design for anticipative automation communication a prototype around automatic vehicle driving on highways was designed and evaluated. The design work focused on designing a display that contained anticipation as a use quality. This resulted in a display design that aimed at representing the working scene in such a way that the user would be able to recognize and compare the system view with the real situation. In order to accomplish this, three layers of information were merged together in the display, these layers were; the automaton’s image of how the world is seen, the way that image is perceived and interpreted in terms of system action necessity and out of that the actions that the system plans to perform in the near future. The evaluation of the design prototype showed that it was possible to anticipate system actions but that the information detail level was insufficient for the evaluators to completely trust system decisions. The evaluation also showed that trust can be created by letting the automaton represent the perception of the situation in such a way that the user is able to compare it to personal experience from performing the task manually. Anticipation can be created through showing what future the automaton is expecting and what actions that needs to be performed in order to reach or maintain the general system task or goal.</p>

This thesis is about the design and evaluation of a graphical display that aims at showing the behavior of a complex automated system in action. A problem with complex automation is that it sometimes surprises the user by performing actions that were not expected. The reason of this is poor communication of system activities. In order to study how to design for anticipative automation communication a prototype around automatic vehicle driving on highways was designed and evaluated. The design work focused on designing a display that contained anticipation as a use quality. This resulted in a display design that aimed at representing the working scene in such a way that the user would be able to recognize and compare the system view with the real situation. In order to accomplish this, three layers of information were merged together in the display, these layers were; the automaton’s image of how the world is seen, the way that image is perceived and interpreted in terms of system action necessity and out of that the actions that the system plans to perform in the near future. The evaluation of the design prototype showed that it was possible to anticipate system actions but that the information detail level was insufficient for the evaluators to completely trust system decisions. The evaluation also showed that trust can be created by letting the automaton represent the perception of the situation in such a way that the user is able to compare it to personal experience from performing the task manually. Anticipation can be created through showing what future the automaton is expecting and what actions that needs to be performed in order to reach or maintain the general system task or goal.

Bricoleur, digitala varelser och brus utgör tre viktiga noder, runt vilka mitt konstnärliga arbete rör sig. Parallellt med ett antal av mina verk beskrivs dessa begrepp kortfattat. / Bricoleur, digital beings and noise are three important nodes, making up cornerstones of my artistic work. These ideas are explained briefly parallel to a selection of my artworks.

<p>The purpose of this thesis is to explore how support systems enable human control within normal flight operations. The thesis focuses on the use of memory supports during flight, such as a handheld computing device, memory strategies and checklists. The support systems are studied from the theoretical perspective of Human Factors. In particular, decision making theories have contributed to the thesis. From previous research it is found that feedback to the operator in case of a human error is essential to keep him or her in a safe sequence of decisions and actions.</p><p>To facilitate the pilots’ tasks in cockpit, computing devices are out on the market. Several of the technical aids are computers installed in cockpit whereas others are smaller, portable devices with hardware not specifically designed for use in cockpit. Jump-seat observations have been performed at an airline company to explore the pilots’ work process in cockpit where a handheld computing device, with hardware not specifically designed for cockpit, is in use. Subsequent semi-structured interviews were conducted to receive the pilots’ experiences of findings from the observations and to receive descriptions of decisions and support systems.</p><p>The thesis includes a description of flight operations from a pilot perspective. The main focus is on operations in the preflight phase where the new computing device is used. Identified characteristics in flight operations are factors such as cooperation, communication, interruptions. Furthermore, identified factors in the decision making were such as routine, environmental constraints, discrete alternatives and dependency between decisions. Feedback points during the sequence of tasks performed with the handheld computing device were distinguished. These points are moments when feedback is possible. For example, when the pilots cross-check tasks they receive feedback from each other. It was found that the pilots did not use every opportunity to receive feedback on their performance. The reason of the non-used feedback point was that it was not required by the Standard Operating Procedures or by any functions or design of the device. Within flight operations in general, it was found that the most important techniques to detect a human error such as a memory lapse were by pilots’ earlier experiences, the use of checklists and by receiving feedback from the other pilot.</p>