This dissertation presents three studies consisting of seven experiments that investigate the relationship between human-computer interaction (HCI) behavior and changes in cognitive states by using keystroke dynamics (KD) and mouse dynamics (MD) as physiological indicators of cognitive change. The first two chapters discuss the importance of being able to detect changes in affect, cognitive load, and deception and provide a theoretical base for this research, pulling heavily from cognitive science, psychology and communication literature. We also discuss the current state of the art in keystroke and mouse dynamics and what makes the techniques presented here different. Chapters three and four present five experiments that explore the influence of affect and cognitive load on KD and MD. The results of these experiments suggest that many features of typing and mouse movement behavior including transition time, rollovers, duration, number of direction changes, and distance traveled are influenced by changes in affect and cognitive load. In chapter five we operationalize these findings in a credibility assessment context and describe two experiments in which participants behave deceptively in computer mediated interactions. In both experiments, we find significant differences in typing behavior, in line with the findings of the first two studies. Chapter six summarizes the results and provides a way forward for future research in human computer interaction. The work presented in this dissertation describes a novel approach to inferring cognitive changes using low cost, non-invasive, and transparent monitoring of HCI behavior with important implications for both research and practice.

Risk-based authentication is an increasingly popular component in the security architecture deployed by many organizations in mitigating online identity threat. Risk-based authentication uses contextual and historical information extracted from online communications to build a risk profile for the user that can be used to make accordingly authentication and authorization decisions. Existing risk-based authentication systems rely on basic web communication information such as the source IP address or the velocity of transactions performed by a specific account, or originating from a certain IP address. Such information can easily be spoofed and as such put in question the robustness and reliability of the proposed systems. In this thesis, we propose in this work an online risk-based authentication system which provides more robust user identity information by combining mouse dynamics, keystroke dynamics biometrics, and user site actions in a multimodal framework. We propose a Bayesian network model for analyzing free keystrokes and mouse movements involved in web sessions. Experimental evaluation of our proposed model with 24 participants yields an Equal Error Rate of 6.91%. This is encouraging considering that we are dealing with free text and mouse movements and the fact that many web sessions tend to be short. / Graduate

Emotions play a significant role in many human mental activities, including decision-making, motivation, and cognition. Various intelligent and expert systems can be empowered with emotionally intelligent capabilities, especially systems that interact with humans and mimic human behaviour. However, most current methods in affect recognition studies use intrusive, lab-based, and expensive tools which are unsuitable for real-world situations. Inspired by studies on keystrokes dynamics, this thesis investigates the effectiveness of diagnosing users’ affect through their typing behaviour in an educational context. To collect users’ typing patterns, a field study was conducted in which subjects used a dialogue-based tutoring system built by the researcher. Eighteen dialogue features associated with subjective and objective ratings for users’ emotions were collected. Several classification techniques were assessed in diagnosing users’ affect, including discrimination analysis, Bayesian analysis, decision trees, and neural networks. An artificial neural network approach was ultimately chosen as it yielded the highest accuracy compared with the other methods. To lower the error rate, a hierarchical classification was implemented to first classify user emotions based on their valence (positive or negative) and then perform a finer classification step to determining which emotions the user experienced (delighted, neutral, confused, bored, and frustrated). The hierarchical classifier was successfully able to diagnose users' emotional valence, while it was moderately able to classify users’ emotional states. The overall accuracy obtained from the hierarchical classifier significantly outperformed previous dialogue-based approaches and in line with some affective computing methods.

Background. Stress can profoundly affect human behavior. Critical-infrastructure operators (e.g., at nuclear power plants) may make more errors when overstressed; malicious insiders may experience stress while engaging in rogue behavior; and chronic stress has deleterious effects on mental and physical health. If stress could be detected unobtrusively, without requiring special equipment, remedies to these situations could be undertaken. In this study a common computer keyboard and everyday typing are the primary instruments for detecting stress. Aim. The goal of this dissertation is to detect stress via keystroke dynamics – the analysis of a user’s typing rhythms – and to detect the changes to those rhythms concomitant with stress. Additionally, we pinpoint markers for stress (e.g., a 10% increase in typing speed), analogous to the antigens used as markers for blood type. We seek markers that are universal across all typists, as well as markers that apply only to groups or clusters of typists, or even only to individual typists. Data. Five types of data were collected from 116 subjects: (1) demographic data, which can reveal factors (e.g., gender) that influence subjects’ reactions to stress; (2) psychological data, which capture a subject’s general susceptibility to stress and anxiety, as well as his/her current stress state; (3) physiological data (e.g., heart-rate variability and blood pressure) that permit an objective and independent assessment of a subject’s stress level; (4) self-report data, consisting of subjective self-reports regarding the subject’s stress, anxiety, and workload levels; and (5) typing data from subjects, in both neutral and stressed states, measured in terms of keystroke timings – hold and latency times – and typographical errors. Differences in typing rhythms between neutral and stressed states were examined to seek specific markers for stress. Method. An ABA, single-subject design was used, in which subjects act as their own controls. Each subject provided 80 typing samples in each of three conditions: (A) baseline/neutral, (B) induced stress, and (A) post-stress return/recovery-to-baseline. Physiological measures were analyzed to ascertain the subject’s stress level when providing each sample. Typing data were analyzed, using a variety of statistical and machine learning techniques, to elucidate markers of stress. Clustering techniques (e.g., K-means) were also employed to detect groups of users whose responses to stress are similar. Results. Our stressor paradigm was effective for all 116 subjects, as confirmed through analysis of physiological and self-report data. We were able to identify markers for stress within each subject; i.e., we can discriminate between neutral and stressed typing when examining any subject individually. However, despite our best attempts, and the use of state-of-the-art machine learning techniques, we were not able to identify universal markers for stress, across subjects, nor were we able to identify clusters of subjects whose stress responses were similar. Subjects’ stress responses, in typing data, appear to be highly individualized. Consequently, effective deployment in a realworld environment may require an approach similar to that taken in personalized medicine.

In the age of a networking revolution, when the Internet has changed not only the way we see computing, but also the whole society, we constantly face new challenges in the area of user verification. It is often the case that the login-id password pair does not provide a sufficient level of security. Other, more sophisticated techniques are used: one-time passwords, smart cards or biometric identity verification. The biometric approach is considered to be one of the most secure ways of authentication. On the other hand, many biometric methods require additional hardware in order to sample the corresponding biometric feature, which increases the costs and the complexity of implementation. There is however one biometric technique which does not demand any additional hardware – user identification based on keyboard statistics. This thesis is focused on this way of authentication. The keyboard statistics approach is based on the user’s unique typing rhythm. Not only what the user types, but also how she/he types is important. This report describes the statistical analysis of typing samples which were collected from 20 volunteers, as well as the implementation and testing of the identity verification system, which uses the characteristics examined in the experimental stage.

One important issue faced by companies is to secure their information and resources from intrusions. For accessing a resource almost every system uses the approach of assigning a unique username and a password to all legitimate users. This approach has a major drawback. If an intruder gets the above information then he can become a big threat for the company and its resources. To strengthen the computer security there are several biometric methods for identity verification which are based on the human body’s unique characteristics and behavior including fingerprints, face recognition, retina scan and signatures. However most of these techniques are expensive and requires the installation of additional hardware. This thesis focuses on keystroke dynamics as an identity verifier, which are based on the user’s unique habitual typing rhythm. This technique is not just looking for what the user types but also how he types. This method does not require additional hardware to be installed and are therefore rather inexpensive to implement. This thesis will discuss how identity verification through keystroke characteristics can be made, what have been done in this area and give advantages and disadvantages of the technique.

Orientadores: João Baptista Tadanobu Yabu-uti, Lee Luan Ling / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de Computação / Made available in DSpace on 2018-08-06T06:55:05Z (GMT). No. of bitstreams: 1 Costa_CarlosRobertodoNascimento_M.pdf: 1033726 bytes, checksum: 1f87381d74a3e8cd3f4aec2d731d2044 (MD5) Previous issue date: 2006 / Resumo: Este trabalho apresenta uma nova abordagem para autenticação biométrica de usuários baseada em seu ritmo de digitação em teclados numéricos. A metodologia proposta é de baixo custo, nãointrusiva e pode ser aplicada tanto a um mecanismo de login em controle de acesso a áreas restritas como na melhoria do nível de segurança em transações bancárias. Inicialmente, o usuário indica a conta a ser acessada por meio de uma cadeia de caracteres digitada que é monitorada em tempo real pelo sistema. Simultaneamente, são capturados os tempos de pressionamento e soltura das teclas. Quatro características são extraídas do sinal: Código ASCII (American Standard Code for lnformation lnterchange) da tecla, duas latências e uma duração associada com a tecla. Alguns experimentos foram feitos usando amostras reais de usuários autênticos e impostores e um classificador de padrões baseado na estimação da máxima verossimilhança. Alguns aspectos experimentais foram analisados para verificar os seus impactos nos resultados. Estes aspectos são as características extraídas do sinal, a informação alvo, o conjunto de treinamento usado na obtenção dos modelos dos usuários, a precisão do tempo de captura das entradas, o mecanismo de adaptação do modelo e, finalmente, a técnica de obtenção do limiar ótimo para cada usuário. Esta nova abordagem traz melhorias ao processo de autenticação pois permite que a senha não seja mais segredo, assim como oferece uma opção para autenticação biométrica em dispositivos móveis, como celulares / Abstract: This work presents a new approach for biometric user authentication based on keystroke dynamics in numerical keyboards. The methodology proposed is low cost, unintrusive and could be applied in a login mechanism of access control to restricted area andJor to improve the security level in Automatic Teller Machines (ATM). Initially, the user indicates the account to be accessed by typing the target string that is monitored in real time by the system. Simultaneously, the times of key pressed and key released are captured. Four features are extracted from this input: The key ASCII code, two associated latencies and key durations, and some experiments using samples for genuines and impostors users were performed using a pattern classification technique based on the maximum likelihood estimation. Some experimental aspects had been analyzed to verify its impacts in the results. These aspects are the sets of features extracted from the signal, the set of training samples used to obtain the models, the time precisions where captures the inputs, the adaptation mechanism of the model and, finally, the technique to attainment of the excellent threshold for each user. This new approach brings improvements to the process of user authentication since it allows the password not to be a secret anymore, as well as it allows to include biometric authentication in mobile devices, such as cell phones / Mestrado / Telecomunicações e Telemática / Mestre em Engenharia Elétrica

A literature review was done to find that there are still issues with writing passwords. From the information gathered, it is stated that using keystroke characteristics could have the potential to add another layer of security to compromised user accounts. The world has become more and more connected and the amount of people who store personal information online or on their phones has steadily increased. In this thesis, a solution is proposed and evaluated to make authentication safer and less intrusive. Less intrusive in this case means that it does not require cooperation from the user, it just needs to capture data from the user in the background. As authentication methods such as fingerprint scanning and facial recognition are becoming more popular this work is investigating if there are any other biometric features for user authentication.Employing Artificial Intelligence, extra sensor metrics and Machine Learning models with the user's typing characteristics could be used to uniquely identify users. In this context the Neural Network and Support Vector Machine algorithms have been examined, alongside the gyroscope and the touchscreen sensors. To test the proposed method, an application has been built to capture typing characteristics for the models to train on. In this thesis, 10 test subjects were chosen to type a password multiple times so that they would generate the data. After the data was gathered and pre-processed an analysis was conducted and sent to train the Machine Learning models. This work's proposed solution and presented data serve as a proof of concept that there are additional sensors that could be used to authenticate users, namely the gyroscope. Capturing typing characteristics of users, our solution managed to achieve a 97.7% accuracy using Support Vector Machines in authenticating users.

Dynamika úhozů kláves je jednou z behaviorálních biometrických charakteristik, kterou je možné použít pro průběžnou autentizaci uživatelů. Vzhledem k tomu, že styl psaní na klávesnici se v čase mění, je potřeba rovněž upravovat biometrickou šablonu. Tímto problémem se dosud, alespoň pokud je autorovi známo, žádná studie nezabývala. Tato diplomová práce se pokouší tuto mezeru zaplnit. S pomocí dat o časování úhozů od 22 dobrovolníků bylo otestováno několik technik klasifikace, zda je možné je upravit na online klasifikátory, zdokonalující se bez učitele. Výrazné zlepšení v rozpoznání útočníka bylo zaznamenáno u jednotřídového statistického klasifikátoru založeného na normované Euklidovské vzdálenosti, v průměru o 23,7 % proti původní verzi bez adaptace, zlepšení však bylo pozorováno u všech testovacích sad. Změna míry rozpoznání správného uživatele se oproti tomu různila, avšak stále zůstávala na přijatelných hodnotách.

The ubiquity and speed of Internet access led over the past decade to an exponential increase in the use of thin clients and cloud computing, both taking advantage of the ability to remotely provide computing resources. The work investigates the role of remote display Protocol in behavioral biometric systems based on free-text keystroke dynamics. Authentication based on keystroke dynamics is easy in use, cheap, invisible for user and does not require any additional sensor.I n this project I will investigate how network characteristics affect the keystroke dynamics pattern in remote desktop scenario. Objectives: The aim of this project is to investigate the role of remote display Protocol in behavioral biometric system based on free-text keystroke dynamics, by measuring how network characteristics influence the computation of keystroke pattern in Virtual Desktop Infrastructure (VDI). Method: This thesis will answer all of its research question with the help of a Systematic Literature Review (SLR) and an Experiment. Literature review was conducted to gather information about the keystroke dynamics analysis, the applied algorithms and their performance; and to clarify the controlled changes of networking performance in VDI based scenario. Using the acquired knowledge, implemented keystroke dynamics pattern algorithm based on Euclidian distance statistical method, designed an experiment and performed a series of tests, in order to identify the influence of remote display protocol to keystroke pattern. Results: Through the SLR, keystroke dynamics analysis working structure is identified and illustrated, essential elements are summarized, and a statistical approach based on Euclidian distance is described; a technique to simulate and measure networklatency in VDI scenario is described including essential elements and parameters of VDI testbed. Keystroke analysis algorithm, dataset replication code and VDItestbed are implemented. The controlled experiment provided measurements of the metrics of the algorithm and network performance mentioned in objectives. Conclusions: During experimentation, I found that timing pattern in the keystroke dynamics data is affected by VDI in normal network conditions by 12% in average. Higher latency standard deviation, jitter, packet loss as well as remote display protocol overheads have a significant combined impact onto keystroke pattern. Moreover I found what maximum possible delay values does not affect keystroke pattern in a larger extent.