Back to search

Rochester Institute of Technology

Making Social Interactions Accessible for the Blind and Low-Vision People by Detecting Emotions through Vocal Tone, Facial Expressions, and Body Language

Abstract

dc:description.abstract

Despite the thriving research in the field of accessibility, there are several niche area where there is a need for specific solutions. One such area is social interaction. Blind and Low-Vision people (BLV) are limited in identifying emotions through verbal communication alone, which is often biased and unreliable in certain environments. Prior research illustrates the importance of body language in expressions and articulates the mechanism behind human emotions. Drawing inspiratoin from such research and various studies that were focused on emotion recognition systems, our research focused on identifying a portable accessible solution for BLV that classified emotion based on multi-modal input such as facial expressions, body language and tone, with an audio feedback. 2 studies were conducted to understand the effectiveness of using an external aid in identifying emotions with 10 participants who identified as Low-Vision between the ages 31-45. Study one targeted at measuring the success of unassisted identification and Study 2 tested the effectiveness of prototype. BLV participants resonated with the purpose of the prototype and identified certain aspects that could be rendered in a different way. Although BLV were able to successfully detect certain emotions without assistance, most emotions, accompanied with background noise were harder to detect only based on tonality. Overall, the function of the prototype was identified to have a positive impact with minor limitations on the interface and feedback mechanics. As a part of our research we also proposed design considerations that can cater to the utility of such devices in multiple environments and thrive in conditions with multiple speakers. Further, we also proposed a co-design structure that can bestow ownership and enable active participation from BLV.

Degree

thesis:*
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Information, School of

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Krishna, Gautham
Contributors dc:contributor
  • Tae Oh

Subjects

dc:subject × 1

Identifiers

dc:identifier.*
Repository record dc:identifier
https://repository.rit.edu/theses/11936

Chain of custody

source
Harvested from
Rochester Institute of Technology
Base URL
repository.rit.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Krishna, Gautham. Making Social Interactions Accessible for the Blind and Low-Vision People by Detecting Emotions through Vocal Tone, Facial Expressions, and Body Language. Thesis thesis, https://repository.rit.edu/theses/11936