User interface design is becoming more reliant on user emotional states to improve usability, adapt to the users state, and allow greater expressiveness. Historically, usability has relied on performance metrics for evaluation, but user experience, with an emphasis on aesthetics and emotions, has become recognized as important for improving user interfaces. Research is ongoing into systems that automatically adapt to users states such as expertise or physical impairments and emotions are the next frontier for adaptive user interfaces. Improving the emotional expressiveness of computers adds a missing element that exists in human face-to-face interactions. The first step of incorporating users emotions into usability evaluation, adaptive interfaces, and expressive interfaces is to sense and gather the users emotional responses. Affective computing research has used predictive modeling to determine user emotional states, but studies are usually performed in controlled laboratory settings and lack realism. Field studies can be conducted to improve realism, but there are a number of logistical challenges with field studies: user activity data is difficult to gather, emotional state ground truth is difficult to collect, and relating the two is difficult. In this thesis, we describe a software solution that addresses the logistical issues of conducting affective computing field studies and we also describe an evaluation of the software using a field study. Based on the results of our study, we found that a software solution can reduce the logistical issues of conducting an affective computing field study and we provide some suggestions for future affective computing field studies.

User interface design is becoming more reliant on user emotional states to improve usability, adapt to the users state, and allow greater expressiveness. Historically, usability has relied on performance metrics for evaluation, but user experience, with an emphasis on aesthetics and emotions, has become recognized as important for improving user interfaces. Research is ongoing into systems that automatically adapt to users states such as expertise or physical impairments and emotions are the next frontier for adaptive user interfaces. Improving the emotional expressiveness of computers adds a missing element that exists in human face-to-face interactions. The first step of incorporating users emotions into usability evaluation, adaptive interfaces, and expressive interfaces is to sense and gather the users emotional responses. Affective computing research has used predictive modeling to determine user emotional states, but studies are usually performed in controlled laboratory settings and lack realism. Field studies can be conducted to improve realism, but there are a number of logistical challenges with field studies: user activity data is difficult to gather, emotional state ground truth is difficult to collect, and relating the two is difficult. In this thesis, we describe a software solution that addresses the logistical issues of conducting affective computing field studies and we also describe an evaluation of the software using a field study. Based on the results of our study, we found that a software solution can reduce the logistical issues of conducting an affective computing field study and we provide some suggestions for future affective computing field studies.

This dissertation presents three studies consisting of seven experiments that investigate the relationship between human-computer interaction (HCI) behavior and changes in cognitive states by using keystroke dynamics (KD) and mouse dynamics (MD) as physiological indicators of cognitive change. The first two chapters discuss the importance of being able to detect changes in affect, cognitive load, and deception and provide a theoretical base for this research, pulling heavily from cognitive science, psychology and communication literature. We also discuss the current state of the art in keystroke and mouse dynamics and what makes the techniques presented here different. Chapters three and four present five experiments that explore the influence of affect and cognitive load on KD and MD. The results of these experiments suggest that many features of typing and mouse movement behavior including transition time, rollovers, duration, number of direction changes, and distance traveled are influenced by changes in affect and cognitive load. In chapter five we operationalize these findings in a credibility assessment context and describe two experiments in which participants behave deceptively in computer mediated interactions. In both experiments, we find significant differences in typing behavior, in line with the findings of the first two studies. Chapter six summarizes the results and provides a way forward for future research in human computer interaction. The work presented in this dissertation describes a novel approach to inferring cognitive changes using low cost, non-invasive, and transparent monitoring of HCI behavior with important implications for both research and practice.

Risk-based authentication is an increasingly popular component in the security architecture deployed by many organizations in mitigating online identity threat. Risk-based authentication uses contextual and historical information extracted from online communications to build a risk profile for the user that can be used to make accordingly authentication and authorization decisions. Existing risk-based authentication systems rely on basic web communication information such as the source IP address or the velocity of transactions performed by a specific account, or originating from a certain IP address. Such information can easily be spoofed and as such put in question the robustness and reliability of the proposed systems. In this thesis, we propose in this work an online risk-based authentication system which provides more robust user identity information by combining mouse dynamics, keystroke dynamics biometrics, and user site actions in a multimodal framework. We propose a Bayesian network model for analyzing free keystrokes and mouse movements involved in web sessions. Experimental evaluation of our proposed model with 24 participants yields an Equal Error Rate of 6.91%. This is encouraging considering that we are dealing with free text and mouse movements and the fact that many web sessions tend to be short. / Graduate

Verifying the identity of a user logged into a secure system is an important task in the field of information security. In addition to a password, it may be appropriate to include behavioral biometrics in the authentication process. The biometrics-based system monitors the user's behavior, compares it with his usual actions, and can thus point out suspicious inconsistencies. The goal of this thesis is to explore the possibility of creating a user identity verification model based on his behavior (usage of mouse and keyboard) in a web application. The work includes creation of a new keystroke and mouse dynamics dataset. The main part of the thesis provides the analysis of features (user characteristics) which can be extracted from the obtained data. Subsequently, we report the authentication accuracy rates achieved by basic machine learning models using the selected set of features. 1

User authentication is vital in a secure system. Authentication is achieved through something a genuine user knows, has, or is. The latter is called biometrics, commonly attributed with fingerprint and face modalities. It is also possible to identify a user based on their behavior, called behavioral biometrics. In this study, keyboard and mouse behavior were considered. Previous research indicate promise for this authentication method. The research however is scarce, old and often not comprehensive. This study focus on two available data sets, the CMU keystroke dynamics dataset and the ReMouse data set. The data was used together with a comprehensive set of multi-class supervised classification machine learning algorithms from the scikit-learn library for Python. By performing hyperparameter optimization, two optimal algorithms with modified hyperparameters were found that improved results compared with previous research. For keystroke dynamics a classifier based on a neural network, multi-layer perceptron, achieved an Equal Error Rate (EER) of 1.26%. For mouse dynamics, a decision tree classifier achieved an EER of 0.43%. The findings indicate that the produced biometric classifiers can be used in an authentication model and importantly to strengthen existing authentication models such as password based login as a safe alternative to traditional Multi-Factor Authentication (MFA). / Användarautentisering är vitalt i ett säkert system. Autentisering genomförs med hjälp av något en genuin användare vet, har eller är. Det senare kallas biometri, ofta ihopkopplat med fingeravtryck och ansiktigenkänning. Det är även möjligt att identifiera en användare baserat på deras beteende, så kallad beteendemässig biometri. I denna studie används tangentbords- och musanvändning. Tidigare forskning tyder på att denna autentiseringsmetod är lovande. Forskningen är dock knapp, äldre och svårbegriplig. Denna studie använder två publika dataset, CMU keystroke dynamics dataset och ReMouse data set. Datan används tillsammans med en utförlig mängd maskininlärningsalgoritmer från scitkit-learn biblioteket för programmeringsspråket Python. Genom att optimera algoritmernas hyper parametrar kunde två stycken optimala klassificerare tas fram som åstadkom förbättrade resultat mot tidigare forskning. För tangentbordsbeteende producerades en klassificerare baserat på neurala nätverk, så kallad multi-layer perceptron som åstadkom en EER på 1.26%. För musrörelser kunde en modell baserat på beslutsträd åstadkomma en EER på 0.43%. Resultatet av dessa upptäckter är att liknande klassificerare kan användas i en autentiseringsmodell men också för att förbättra säkerheten hos etablerade inloggningssätt som exempelvis lösenord och därmed utgöra ett säkert alternativ till traditionell MFA.