Georgia Institute of Technology
Multimodal assessment of neuropsychiatric disorders using audiovisual recordings
Abstract
dc:description.abstractOver one billion people worldwide live with a neuropsychiatric disorder, yet most do not have access to adequate diagnosis and care. Accurate, fast, and accessible detection of those disorders is critical to early and effective interventions. Over the last decade, digitally administered assessments have emerged as one of the most promising approaches. Moreover, the increasing use of telemedicine in psychiatry and neurology in recent years presented an unprecedented opportunity to use audiovisual data for accessible neuropsychiatric assessments without the limitation of geographical location and specialized hardware. This dissertation describes the use of low-cost audiovisual data collected from in-lab and remote mobile devices to assess neuropsychiatric conditions by extracting and combining various behavioral and physiological indicators. First, we showed that facial and speech emotions can be effectively estimated from audiovisual data collected in interviews and used for major depressive disorder evaluation. Then, we presented the automated assessment of cognitive impairment using facial emotions and viewing behaviors recognized from videos passively collected from a mobile device. Lastly, we further improved the scalability and accessibility by extracting facial, vocal, linguistic, and cardiovascular features from audiovisual data collected remotely from heterogeneous mobile devices and validating them in both clinician-rated and self-rated mental health condition evaluation.
Degree
thesis:*- Level thesis:degree_level
- Doctoral
- Department dc:contributor.department
- Biomedical Engineering (Joint GT/Emory Department)
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2023
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Jiang, Zifan
- Advisor dc:contributor.advisor
-
- Clifford, Gari D.
- Committee members dc:contributor.committeemember
-
- Mitchell, Cassie S.
- Dyer, Eva L.
- Haider, Bilal
- Bahrami Rad, Ali
Subjects
dc:subject × 10Rights
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/76786