Back to search

University of Glasgow

Contactless AI-enabled hybrid sensing for cognitive impairment

Abstract

dc:description.abstract

This thesis covers various aspects of Multi Model (MM) hearing impairments including human speech, sign language, behavior analysis, and facial expressions, which facilitate the deaf communities. Recent research using wearable, audio, and visual technologies for monitoring cognitive impairments in deaf individuals has its benefits but also presents certain limitations. For instance, while wearable devices provide body-mounted monitoring, their constant use can be uncomfortable, and there is a risk that deaf individuals might forget to wear them. Moreover, these devices need regular removal for recharging. In audio noise, even individuals with regular hearing may struggle to clearly hear someone’s voice. Camera-based visual information raises privacy concerns, and legal implications might restrict its broad usage in public and private areas due to issues like filming without consent, which is illegal in many countries. This thesis explores the use of Radio Frequency (RF) signals to sense human speech, sign recognition, behavior identification, and facial expressions using Wi-Fi, radar, and Radio Frequency Identification (RFID) signals. RF sensing provides an exciting opportunity for next-generation MM hearing aid devices. The RF-based hearing aid just requires Tx and Rx on a single chip. Additionally, RF signal in the form of Wi-Fi is currently present in many homes. People move around Wi-Fi signals, signal propagation is affected. Channel State Information (CSI) in Wi-Fi describes how a signal propagates from the transmitter to the receiver. In this thesis, the data collected in the form of CSI, micro-doppler, and Received Signal Strength Indicator (RSSI) signals are fed into Machine Learning (ML) and Deep Learning (DL) techniques for classification purposes. The proposed techniques successfully differentiate various activities, such as speech recognition, sign language recognition, and behavior analysis by using head movements, and facial expressions to understand the expressions of individuals when communicating with deaf people. These techniques utilise RF signals to individually differentiate each activity and achieve over 90% test accuracy. This thesis serves as a proof of concept for contactless MM hearing aid systems that can assist deaf people with different perspectives to live independently without the need to wear monitoring devices, audio, and visual devices.

Degree

thesis:*
Level dc:type.qualificationlevel
PhD
Grantor dc:publisher.institution
University of Glasgow
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Hameed, Hira

Subjects

dc:subject × 3

Rights

Language dc:language
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
University of Glasgow
Base URL
theses.gla.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Hameed, Hira. Contactless AI-enabled hybrid sensing for cognitive impairment. PhD thesis, University of Glasgow, 2024. https://doi.org/10.5525/gla.thesis.84179