Back to search

Aristotle University Of Thessaloniki (AUTH)

Detection of psychomotor impairment from typing patterns on touchscreens using advanced machine learning techniques

Abstract

dc:description

In this thesis, we contributed methods for digital biomarkers extraction based on commercial devices used in an ecologically valid framework (in-the-wild) based on machine learning methods to detect fine-motor impairment (FMI) from keystroke variables from routine typing activity from the natural user-interaction with smartphone’s touchscreens. We initially explored how the symptoms of Parkinson’s Disease (PD) related with FMI can influence the keystroke variables that resembles the time-sequences of typing; key hold and key flight time. This was initiated from an in-the-clinic data collection experiment which reflected promising results on detecting PD from typing pattern analysis, achieving promising diagnostic properties with features plausibly correlating with clinical scores of relevant PD motor symptoms. As a next step we expanded the validity of the latter findings in-the-wild scenarios as a regression problem to interpret the footprint of specific underlying symptoms such as brady-/hypokinesia and rigidity to keystroke dynamics. Data harvested in-the-wild for a longitudinal period of time validated the method’s transferability hypothesis. The third stride was to investigate whether a hybrid mechanism that combines the two different environments (in-the-clinic and in-the-wild) via the use of deep learning for representation learning, can be more effective in remote symptoms’ severity estimation and PD detection. Apparently, the proposed approaches validates in a wide cohort the conceptualised approaches towards unobtrusive remote screening and detection of specific early PD signs from mobile-based human-computer interaction and contributes to the continuous improvement of deployed tools and technologies in-the-wild. The ability to effectively infer FMI in-the-wild allows us to expand the research focus on the detection capabilities of other psychomotor-related disorders. In this vein, the first expansion proposes a machine learning-based method for discriminating between subjects with depressive tendency and healthy controls based on typing patterns captured in-the-wild. The best-performing pipeline achieved high classification scores, with the outputted scores to significantly correlate with the respective severity, indicating that symptoms have similar representation-space on typing patterns. The second expansion aims to use deep networks developed for PD symptoms detection to detect whether patients with mild cognitive impairment face similar subtle fine-motor impairment, which can be detected in-the-wild via natural typing on smartphone touchscreens. The acquired results have shown that FMI estimates plausibly correlate with clinical scales, indicating the potentiality of new digital biomarkers to capture early stages of cognitive decline in-the-wild via a novel pathway. Finally, based on the same principles we researched remote PD screening via passively captured running speech-based multilingual data by investigating a privacy-aware method for classifying patients and HC, on the grounds of speech impairment present in PD was proposed using running speech data analysis, applied on passively-captured voice data during phone calls made in-the-wild. Language-aware training/evaluation of multiple- and single-instance learning classifiers was employed to fuse and predict on voice and demographic data from a multilingual cohort. The approach outperformed other methods proposed for language-aware PD detection considering the ecological validity of the voice data. In conclusion, the thesis comes along to build towards a privacy-aware and long-term adherence perspective of low-cost clinical monitoring tools via commercial devices.

Degree

thesis:*
Grantor dc:publisher
Aristotle University Of Thessaloniki (AUTH)
Year dc:date
2021

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Iakovakis, Dimitrios
  • Ιακωβάκης, Δημήτριος

Subjects

dc:subject × 18

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/49741

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Iakovakis, Dimitrios; Ιακωβάκης, Δημήτριος. Detection of psychomotor impairment from typing patterns on touchscreens using advanced machine learning techniques. Aristotle University Of Thessaloniki (AUTH), 2021. http://hdl.handle.net/10442/hedi/49741