Back to search

Rochester Institute of Technology

Learning to see and hear in 3D: Virtual reality as a platform for multisensory perceptual learning

Abstract

dc:description.abstract

Virtual reality (VR) is an emerging technology which allows for the presentation of immersive and realistic yet tightly controlled audiovisual scenes. In comparison to conventional displays, the VR system can include depth, 3D audio, fully integrated eye, head, and hand tracking, all over a much larger field of view than a desktop monitor provides. These properties demonstrate great potential for use in vision science experiments, especially those that can benefit from more naturalistic stimuli, particularly in the case of visual rehabilitation. Prior work using conventional displays has demonstrated that that visual loss due to stroke can be partially rehabilitated through laboratory-based tasks designed to promote long-lasting changes to visual sensitivity. In this work, I will explore how VR can provide a platform for new, more complex training paradigms which leverage multisensory stimuli. In this dissertation, I will (I) provide context to motivate the use of multisensory perceptual training in the context of visual rehabilitation, (II) demonstrate best practices for the appropriate use of VR in a controlled psychophysics setting, (III) describe a prototype integrated hardware system for improved eye tracking in VR, and (IV, V) discuss results from two audiovisual perceptual training studies, one using multisensory stimuli and the other with cross-modal audiovisual stimuli. This dissertation provides the foundation for future work in rehabilitating visual deficits, by both improving the hardware and software systems used to present the training paradigm as well as validating new techniques which use multisensory training not previously accessible with conventional desktop displays.

Degree

thesis:*
Name thesis:degree_name
Imaging Science (Ph.D.)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Chester F. Carlson Center for Imaging Science (COS)

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Fromm, Catherine A
Contributors dc:contributor
  • Gabriel J. Diaz

Subjects

dc:subject × 4

Identifiers

dc:identifier.*
Repository record dc:identifier
https://repository.rit.edu/theses/11325

Chain of custody

source
Harvested from
Rochester Institute of Technology
Base URL
repository.rit.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Fromm, Catherine A. Learning to see and hear in 3D: Virtual reality as a platform for multisensory perceptual learning. Dissertation thesis, https://repository.rit.edu/theses/11325