Back to search

Georgia Institute of Technology

Supporting human interpretation and analysis of activity captured through overhead video

Abstract

dc:description.abstract

Many disciplines spend considerable resources studying behavior. Tools range from pen-and-paper observation to biometric sensing. A tool's appropriateness depends on the goal and justification of the study, the observable context and feature set of target behaviors, the observers' resources, and the subjects' tolerance to intrusiveness. We present two systems: Viz-A-Vis and Tableau Machine. Viz-A-Vis is an analytical tool appropriate for onsite, continuous, wide-coverage and long-term capture, and for objective, contextual, and detailed analysis of the physical actions of subjects who consent to overhead video observation. Tableau Machine is a creative artifact for the home. It is a long-lasting, continuous, interactive, and abstract Art installation that captures overhead video and visualizes activity to open opportunities for creative interpretation. We focus on overhead video observation because it affords a near one-to-one correspondence between pixels and floor plan locations, naturally framing the activity in its spatial context. Viz-A-Vis is an information visualization interface that renders and manipulates computer vision abstractions. It visualizes the hidden structure of behavior in its spatiotemporal context. We demonstrate the practicality of this approach through two user studies. In the first user study, we show an important search performance boost when compared against standard video playback and against the video cube. Furthermore, we determine a unanimous user choice for overviewing and searching with Viz-A-Vis. In the second study, a domain expert evaluation, we validate a number of real discoveries of insightful environmental behavior patterns by a group of senior architects using Viz-A-Vis. Furthermore, we determine clear influences of Viz-A-Vis over the resulting architectural designs in the study. Tableau Machine is a sensing, interpreting, and painting artificial intelligence. It is an Art installation with a model of perception and personality that continuously and enduringly engages its co-occupants in the home, creating an aura of presence. It perceives the environment through overhead cameras, interprets its perceptions with computational models of behavior, maps its interpretations to generative abstract visual compositions, and renders its compositions through paintings. We validate the goal of opening a space for creative interpretation through a study that included three long-term deployments in real family homes.

Degree

thesis:*
Department dc:contributor.department
Computing
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2009

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Romero, Mario
Advisor dc:contributor.advisor
  • Abowd, Gregory D.
Committee members dc:contributor.committeemember
  • Do, Ellen Yi-Luen
  • Foley, James
  • Peponis, John
  • Stasko, John

Subjects

dc:subject × 7

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/1853/29771

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Romero, Mario. Supporting human interpretation and analysis of activity captured through overhead video. Georgia Institute of Technology, 2009. http://hdl.handle.net/1853/29771