Back to search

Cornell University

Toward Perception Models Beyond Internet Applications

Abstract

dc:description.abstract

For the past decades, we have observed tremendous success in developing perception models — computational models that could perceive our world through images, videos, LiDAR point clouds, and so on. Currently, we have perception models that can recognize thousands of concepts commonly seen on the Internet. The ability of these models to recognize concepts is undeniably impressive, but their successes are only limited to concepts or data modalities (e.g., images) commonly seen on the Internet. Beyond applications in the Internet domain such as remote sensing or medical imagery, perception models have yet to show their prowess. The key challenge in building perception models beyond Internet applications is the requirement of extensive expert involvement. Training performant perception models in these domains often requires non-trivial involvement from experts, especially during the data collection process. In this dissertation, we investigate how we could reduce experts’ burden when developing perception models. Specifically, we will focus on the angle of label efficiency, i.e., developing perception models that could be trained with fewer annotations. We will present two broad categories of approaches. The first category relies on minimal assumptions and could be applied to various problem domains; along this vein, we will examine how we could leverage pre-trained models, unlabeled data, and coarsely-labeled data to enhance label efficiency. The second category leverages domain knowledge to enhance label efficiency. For this category of approaches, we will look at two specific domains: autonomous driving and remote sensing. We will investigate how repeated traversals of the same location could be used to improve perception models for self-driving vehicles and how ground images could be used to train vision-language models for remote sensing without any textual annotations. We will end this dissertation with a brief discussion of how we could further reduce experts’ burden when developing perception models, enabling broader success of perception models beyond Internet applications.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Computer Science
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Computer Science
Grantor
Cornell University
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Phoo, Cheng
Chair dc:contributor.chair
  • Hariharan, Bharath
Committee members dc:contributor.committeemember
  • Sridharan, Karthik
  • Damle, Anil

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
16575525
Dc Identifier Other
ProQuest Submission ID: 14300
ProQuest Publication ID: 31243387

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Phoo, Cheng. Toward Perception Models Beyond Internet Applications. Doctor of Philosophy thesis, Cornell University, 2024. https://hdl.handle.net/1813/115981