Made available in DSpace on 2015-05-14T13:16:37Z (GMT). No. of bitstreams: 1 arquivototal.pdf: 1005482 bytes, checksum: 759ffb4376499fcf1bcd231746489fdb (MD5) Previous issue date: 2010-03-17 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / This research aimed to evaluate the role of the pupil diameter in attractiveness between individuals of opposite sex and to check the relationship between pupil diameter and some emotional expressions considered universal. Analyzed through both quantitative and qualitative methods, the research was divided into two independent studies. The stimuli for both of them were 10 volunteers faces, which the pupil diameters were manipulated to generate five different sizes (2, 3, 4, 5 and 6 mm). In study I, "Is there a relationship between facial attractiveness and pupil diameter?", 60 UFPB students, aged between 18 and 26 (M = 20.65, SD = 2.20), took part in the research, and the following instruments were used: visual acuity test, interview, questionnaire, answer sheet of the visual analogue scale (VAS) and two softwares. In addition, this study was divided in two procedures. In the first, the participant chose the face considered more attractive, among the five faces equal with different sizes of pupilla. From the analysis with Chi-square (X²), we found that there was significant effect for the factor pupilla in both group of men (X² = 21.93; p< 0.05), and of women (X² = 44.73; p< 0.05). In the second study, the participant indicated a score of attractiveness for each face, through the scale of VAS. The results were analyzed by repeated measures ANOVA and revealed a significant difference for the factor pupilla [F(1, 234) = 19.06; p < 0.05]. Through the post hoc test (Tukey HSD), we can say that the larger diameter pupils were considered more attractive and pupils of smaller diameter were considered less attractive. From the data obtained by interviews, it was conducted a Bardin analysis of content, with which we discovered a thematic class of attractiveness and subclasses of concept, incidence, attractive element and facial attractiveness. The results of experimental and content analyses support the idea that the pupil has a relevant role in attractiveness. In Study II, "Is there a relationship between emotional expressions and pupil diameter? , the participants, the stimuli and equipment were the same as in study I. The instruments in this study were: visual acuity test, interview, questionnaire and one specific software developed by LPNeC. In this experiment, participants must associate each displayed face with one of the six emotions (joy, sadness, fear, anger, surprise and contempt). The Chi-square (X²) showed significant differences in factor pupilla (p<0.05). The group of men associated the diameter of 4mm to emotion Joy (X² = 190.81; p<0.00) and the group of women Joined the diameter of 5 mm (X² = 29.61; p<0.00). Both the group of men (X² = 21.55; p < 0.05) of women (X² = 29.61; p < 0.05) were associated Smaller diameters (2 and 3 mm) with contempt. Only the men associated fear to pupilla of 6 mm (X²=30.21; p<0.05) and Surprise to pupilla of 5 mm (X²=31.84; p > 0.05). were not found significant differences between groups for the emotions of Sadness and Anger. In regard to the categorical analysis, it was created a thematic class of emotion, which was refered to the recognition of the specific emotions (sadness, joy and anger). These quantitative and qualitative results suggest that the pupil is related to the recognition of some specific emotions. / A presente pesquisa teve como objetivos gerais avaliar o papel do diâmetro da pupila na atratividade entre indivíduos de sexo oposto e verificar se há relação entre o diâmetro da pupila e algumas expressões emocionais consideradas universais. Analisada através do método quantitativo e qualitativo, a pesquisa foi dividida em dois estudos independentes. Os estímulos para ambos foram 10 faces de voluntários, cujos diâmetros das pupilas foram manipulados para gerar cinco dimensões diferentes (2, 3, 4, 5 e 6 mm). No estudo I, Existe relação entre atratividade facial e diâmetro de pupila? , participaram 60 alunos da Universidade Federal da Paraíba (UFPB), com idade entre 18 a 26 anos (M= 20,65; DP= 2,20), e utilizou os seguintes instrumentos: teste de acuidade visual, entrevista, questionário, folha de resposta da escala de VAS e dois softwares para rodar os experimentos (face mais atrativa e escala de VAS). Além disso, esse estudo subdividiu-se em dois procedimentos. No primeiro, o participante escolhia, dentre o conjunto de cinco faces iguais com diferentes tamanhos da pupila, aquela considerada mais atrativa. A partir das análises com Qui- Quadrado (X²), verificou-se que houve efeito significante para o fator pupila tanto no grupo dos homens (X² = 21,93; p< 0,05), como no das mulheres (X² = 44,73; p< 0,05). No segundo, o participante indicava, através da escala de VAS, um escore de atratividade para cada face. Os resultados foram analisados pela ANOVA para medidas repetidas e revelou diferença significante para o fator pupila [F(1, 234) = 19,06; p < 0,05]. Através do teste post hoc (Tukey HSD), pode-se afirmar que de forma geral as pupilas de diâmetro maior foram consideradas mais atrativas e as de menor diâmetro, menos atrativas. A partir dos dados obtidos pelas entrevistas, foi realizado a analise de conteúdo de Bardin, em que surgiu a classe temática de atratividade e subcategorias de conceito, ocorrência, elemento atrativo e atratividade facial. Os resultados das análises experimentais e de conteúdo apóiam a idéia de que a pupila tem um papel relevante na atratividade. No estudo II, Existe relação entre expressões emocionais e diâmetro de pupila? , os participantes, os estímulos e o equipamento seguiram o mesmo padrão do estudo I. Os instrumentos utilizados foram: teste de acuidade visual, entrevista, questionário e um software específico desenvolvido pelo LPNeC. Nesse experimento, os participantes tiveram que associar cada face visualizada a uma das seis emoções (alegria, tristeza, medo, raiva, surpresa e desprezo). O teste Qui-Quadrado (X²) mostrou diferenças significantes no fator pupila (p<0,05). Na emoção de Alegria, o grupo dos homens (X²=190,81; p<0,00) associaram ao diâmetro 4mm e as mulheres (X²=29,61; p<0,00) ao diâmetro de 5 mm. Já na emoção de Desprezo, tanto o grupo dos homens (X² = 21,55; p < 0,05) como o grupo das mulheres (X² = 29,61; p < 0,05) associaram aos diâmetros menores, 2 e 3 mm. Na emoção de Medo, foi revelada diferença significante apenas no grupo dos homens (X²=30,21; p<0,05), o qual associou a pupila de 6 mm, como também na emoção de Surpresa, o qual os homens (X²=31,84; p > 0,05) relacionaram ao diâmetro 5 mm. Nas emoções de Tristeza e Raiva não foram encontradas diferenças significantes entre ambos os grupos. Com relação à análise categorial, foi criada a classe temática de emoção, a qual se referia ao reconhecimento das emoções específicas (tristeza, alegria e raiva). Estes resultados quantitativos e qualitativos sugerem que a pupila está relacionada com o reconhecimento de algumas emoções específicas.

No description available.

No description available.

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

Physiological signals, which are controlled by the autonomic nervous system (ANS), could be used to detect the affective state of computer users and therefore find applications in medicine and engineering. The Pupil Diameter (PD) seems to provide a strong indication of the affective state, as found by previous research, but it has not been investigated fully yet. In this study, new approaches based on monitoring and processing the PD signal for off-line and on-line affective assessment (“relaxation” vs. “stress”) are proposed. Wavelet denoising and Kalman filtering methods are first used to remove abrupt changes in the raw Pupil Diameter (PD) signal. Then three features (PDmean, PDmax and PDWalsh) are extracted from the preprocessed PD signal for the affective state classification. In order to select more relevant and reliable physiological data for further analysis, two types of data selection methods are applied, which are based on the paired t-test and subject self-evaluation, respectively. In addition, five different kinds of the classifiers are implemented on the selected data, which achieve average accuracies up to 86.43% and 87.20%, respectively. Finally, the receiver operating characteristic (ROC) curve is utilized to investigate the discriminating potential of each individual feature by evaluation of the area under the ROC curve, which reaches values above 0.90. For the on-line affective assessment, a hard threshold is implemented first in order to remove the eye blinks from the PD signal and then a moving average window is utilized to obtain the representative value PDr for every one-second time interval of PD. There are three main steps for the on-line affective assessment algorithm, which are preparation, feature-based decision voting and affective determination. The final results show that the accuracies are 72.30% and 73.55% for the data subsets, which were respectively chosen using two types of data selection methods (paired t-test and subject self-evaluation). In order to further analyze the efficiency of affective recognition through the PD signal, the Galvanic Skin Response (GSR) was also monitored and processed. The highest affective assessment classification rate obtained from GSR processing is only 63.57% (based on the off-line processing algorithm). The overall results confirm that the PD signal should be considered as one of the most powerful physiological signals to involve in future automated real-time affective recognition systems, especially for detecting the “relaxation” vs. “stress” states.

La fatigue mentale est un état complexe qui résulte d'une activité cognitive prolongée. Les symptômes de la fatigue mentale inclus des changements d'humeur, de motivation et une détérioration temporaire de diverses fonctions cognitives. Plusieurs recherches approfondies ont été menées pour développer des méthodes de reconnaissance des signes physiologiques et psychophysiologiques de la fatigue mentale. Les signes psychophysiologiques concernent principalement signaux d'activité cérébrale et leur relation avec la psychologie et la cognition. Celles-ci ont permise le développement de nombreux modèles basés sur l'IA pour classer différents niveaux de fatigue, en utilisant des données extraites d'un appareil eye-tracking, d'un électroencéphalogramme (EEG) pour mesurer l’activité cérébrale ou d'un électrocardiogramme (ECG) pour mesurer l’activité cérébrale. Dans cette mémoire, nous présentons le protocole expérimental et développé par mes directeurs de recherche et moi-même, qui vise à la fois à générer et mesurer la fatigue mentale, et à proposer des stratégies efficaces de récupération via des séances de réalité virtuelle couplées à des dispositifs EEG et eye tracking. Réussir à générer de la fatigue mentale est nécessaire pour générer un ensemble de données suivant l’évolution de la fatigue et de la récupération au cours de l’expérience, et sera également utilisé pour classer différents niveaux de fatigue à l’aide de l’apprentissage automatique. Cette mémoire fournit d'abord un état de l'art complet des facteurs prédictifs de la fatigue mentale, des méthodes de mesure et des stratégies de récupération. Ensuite, l'article présente un protocole expérimental résultant de l'état de l'art pour (1) générer et mesurer la fatigue mentale et (2) évaluer l'efficacité de la thérapie virtuelle pour la récupération de la fatigue, (3) entrainer un algorithme d'apprentissage automatique sur les données EEG pour classer 3 niveaux de fatigue différents en utilisant un environnement simulé de réalité virtuelle (VR). La thérapie virtuelle est une technique favorisant la relaxation dans un environnement simulé virtuel et interactif qui vise à réduire le stress. Dans notre travail, nous avons réussi à générer de la fatigue mentale en accomplissant des tâches cognitives dans un environnement virtuel. Les participants ont montré une diminution significative du diamètre de la pupille et du score thêta/alpha au cours des différentes tâches cognitives. Le score alpha/thêta est un indice EEG qui suit les fluctuations de la charge cognitiveet de la fatigue mentale. Divers algorithmes d'apprentissage automatique ont été formés et testés sur des segments de données EEG afin de sélectionner le modèle qui s'ajuste le mieux à ces données en ce qui concerne la métrique d'évaluation "précision équilibrée" et "f1". Parmi les 8 différents classificateurs, le SVM RBF a montré les meilleures performances avec une précision équilibrée de 95 % et une valeur de mesure f de 0,82. La précision équilibrée fournit une mesure précise de la performance dans le cas de jeu de données déséquilibrées, en tenant compte de la sensibilité et de la spécificité, et le f-score est une mesure d'évaluation qui combine les scores de précision et de rappel. Finalement, nos résultats montrent que le temps alloué à la thérapie virtuelle n'a pas amélioré le diamètre pupillaire en période post-relaxation. D'autres recherches sur l'impact de la thérapie devraient consacrer un temps plus proche du temps de récupération standard de 60 min. / Mental fatigue is a complex state that results from prolonged cognitive activity. Symptoms of mental fatigue can include change in mood, motivation, and temporary deterioration of various cognitive functions involved in goal-directed behavior. Extensive research has been done to develop methods for recognizing physiological and psychophysiological signs of mental fatigue. Psychophysiological signs are mostly concern with patterns of brain activity and their relation to psychology and cognition. This has allowed the development of many AI-based models to classify different levels of fatigue, using data extracted from eye-tracking devices, electroencephalogram (EEG) measuring brain activity, or electrocardiogram (ECG) measuring cardiac activity. In this thesis, we present the experimental protocol developed by my research directors and I, which aims to both generate/measure mental fatigue and provide effective strategies for recuperation via VR sessions paired with EEG and eye-tracking devices. Successfully generating mental fatigue is crucial to generate a time-series dataset tracking the evolution of fatigue and recuperation during the experiment and will also be used to classify different levels of fatigue using machine learning. This thesis first provides a state-of-the-art of mental fatigue predictive factors, measurement methods, and recuperation strategies. The goal of this protocol is to (1) generate and measure mental fatigue, (2) evaluate the effectiveness of virtual therapy for fatigue recuperation, using a virtual reality (VR) simulated environment and (3) train a machine learning algorithm on EEG data to classify 3 different levels of fatigue. Virtual therapy is relaxation promoting technique in a virtual and interactive simulated environment which aims to reduce stress. In our work, we successfully generated mental fatigue through completion of cognitive tasks in a virtual simulated environment. Participants showed significant decline in pupil diameter and theta/alpha score during the various cognitive tasks. The alpha/theta score is an EEG index tracking fluctuations in cognitive load and mental fatigue. Various machine learning algorithm candidates were trained and tested on EEG data segments in order to select the classifier that best fits EEG data with respect to evaluation metric ‘balanced accuracy’ and 'f1-measures'. Among the 8 different classifier candidates, RBF SVM showed the best performance with 95% balanced accuracy 0.82 f-score value and on the validation set, and 92% accuracy and 0.90 f-score on test set. Balanced accuracy provides an accurate measure of performance in the case of imbalanced data, considering sensitivity and specificity and f-score is an evaluation metric which combines precision and recall scores. Finally, our results show that the time allocated for virtual therapy did not improve pupil diameter in the post-relaxation period. Further research on the impact of relaxation therapy should allocate time closer to the standard recovery time of 60 min.

Il lavoro di tesi si è focalizzato sullo studio dell’arousal come indice psicofisiologico di attivazione e sull’applicazione della metodica di stimolazione elettrica transcranica (tES) non invasiva con lo scopo di modulare tale indice. L’obiettivo è stato quello di indagare se, applicando la tES, fosse possibile migliorare la performance di soggetti giovani sani in compiti di tipo cognitivo. Il progetto di ricerca si è sviluppato in 2 studi principali per un totale di 4 esperimenti. Tali studi hanno previsto l’acquisizione e la successiva analisi sia di dati comportamentali (tempi di reazione, accuratezza), che di indici psicofisiologici (conduttanza cutanea, diametro pupillare). I risultati hanno evidenziato che è possibile modulare l’arousal con dei “bursts” di stimolazione elettrica transcranica, somministrati in concomitanza di stimoli salienti per il soggetto. Tale modulazione si è manifestata con una riduzione dei tempi di reazione ed un contemporaneo aumento della risposta di conduttanza cutanea. Questi dati supportano la possibilità di utilizzare questo protocollo in pazienti con difficoltà di attenzione o altri problemi cognitivi per aumentare l’efficacia di interventi di riabilitazione. / The thesis analyzed the role of the arousal as a psychophysiological index of activation, and the application of non-invasive transcranial electrical stimulation (tES) technique with the aim to modulate this index. In this work we investigated if the application of tES could increase the performance of healthy young subjects during cognitive tasks. The thesis is based on 2 main studies for a total of 4 experiments with the recording of behavioural (reaction times, accuracy) and psychophysiological (skin conductance, pupil diameter) indeces. The results showed that it is possible to modulate arousal with bursts of tES, administered during the presentation of salient stimuli for the subject. This modulation resulted in a reduction of reaction times and an increase of the skin conductance response. These data support the possibility to use this protocol of stimulation with patients with attentional and other cognitive deficits in a rehabilitative context.

The use of autonomous robots in our society is increasing every day and a robot is no longer seen as a tool but as a team member. The robots are now working side by side with us and provide assistance during dangerous operations where humans otherwise are at risk. This development has in turn increased the need of robots with more human-awareness. Therefore, this master thesis aims at contributing to the enhancement of human-aware robotics. Specifically, we are investigating the possibilities of equipping autonomous robots with the capability of assessing and detecting activities in human teams. This capability could, for instance, be used in the robot's reasoning and planning components to create better plans that ultimately would result in improved human-robot teamwork performance. we propose to improve existing teamwork activity recognizers by adding intangible features, such as stress, motivation and focus, originating from human behavior models. Hidden markov models have earlier been proven very efficient for activity recognition and have therefore been utilized in this work as a method for classification of behaviors. In order for a robot to provide effective assistance to a human team it must not only consider spatio-temporal parameters for team members but also the psychological.To assess psychological parameters this master thesis suggests to use the body signals of team members. Body signals such as heart rate and skin conductance. Combined with the body signals we investigate the possibility of using System Dynamics models to interpret the current psychological states of the human team members, thus enhancing the human-awareness of a robot. / Användningen av autonoma robotar i vårt samhälle ökar varje dag och en robot ses inte längre som ett verktyg utan som en gruppmedlem. Robotarna arbetar nu sida vid sida med oss och ger oss stöd under farliga arbeten där människor annars är utsatta för risker. Denna utveckling har i sin tur ökat behovet av robotar med mer människo-medvetenhet. Därför är målet med detta examensarbete att bidra till en stärkt människo-medvetenhet hos robotar. Specifikt undersöker vi möjligheterna att utrusta autonoma robotar med förmågan att bedöma och upptäcka olika beteenden hos mänskliga lag. Denna förmåga skulle till exempel kunna användas i robotens resonemang och planering för att ta beslut och i sin tur förbättra samarbetet mellan människa och robot. Vi föreslår att förbättra befintliga aktivitetsidentifierare genom att tillföra förmågan att tolka immateriella beteenden hos människan, såsom stress, motivation och fokus. Att kunna urskilja lagaktiviteter inom ett mänskligt lag är grundläggande för en robot som ska vara till stöd för laget. Dolda markovmodeller har tidigare visat sig vara mycket effektiva för just aktivitetsidentifiering och har därför använts i detta arbete. För att en robot ska kunna ha möjlighet att ge ett effektivt stöd till ett mänskligtlag måste den inte bara ta hänsyn till rumsliga parametrar hos lagmedlemmarna utan även de psykologiska. För att tyda psykologiska parametrar hos människor förespråkar denna masteravhandling utnyttjandet av mänskliga kroppssignaler. Signaler så som hjärtfrekvens och hudkonduktans. Kombinerat med kroppenssignalerar påvisar vi möjligheten att använda systemdynamiksmodeller för att tolka immateriella beteenden, vilket i sin tur kan stärka människo-medvetenheten hos en robot. / <p>The thesis work was conducted in Stockholm, Kista at the department of Informatics and Aero System at Swedish Defence Research Agency.</p>