Rochester Institute of Technology
Toward a Unified Framework for Open World Visual Learning
Abstract
dc:description.abstractArtificial intelligence systems have achieved remarkable performance across a wide range of visual tasks. However, most existing models operate under the unrealistic closed-world assumption, where training and test data are drawn from the same distribution. In real-world applications such as anomaly detection, autonomous driving, and medical diagnosis, learning systems frequently encounter novel or out-of-distribution scenarios. These settings require models that can recognize unknown inputs, adapt to new information over time, and maintain reliable performance under evolving conditions. This dissertation studies the problem of Open World Visual Learning, a paradigm that enables visual learning systems to operate robustly in dynamic and unpredictable environments. The research addresses three fundamental challenges: unknown awareness, efficient adaptation, and robust generalization. First, to improve unknown awareness, we develop a weakly supervised framework for open-set video anomaly detection that identifies previously unseen abnormal events while reducing reliance on expensive frame-level annotations. Second, to enable efficient adaptation, we address both label efficiency and resource efficiency. We introduce an active learning framework based on a mixture of diverse experts that selectively acquires informative samples for annotation, significantly improving data efficiency. In addition, we develop a continual learning approach that allows machine learning models to incrementally acquire new knowledge over time without requiring full retraining. Third, to further support robust generalization in continual learning settings, we propose training strategies that encourage models to generalize beyond the training distribution, including robustness to unseen subpopulations and the ability to reason over novel compositional visual question answering (VQA) tasks. Together, these contributions advance the development of adaptive visual intelligence capable of recognizing unknowns, learning efficiently, and generalizing robustly in open-world environments.
Degree
thesis:*- Name thesis:degree_name
- Computing and Information Sciences (Ph.D.)
- Level thesis:degree_level
- Dissertation
- Discipline thesis:degree_discipline
- Computing and Information Sciences Ph.D, Department of
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Zhu, Yuansheng
- Contributors dc:contributor
-
- Xumin Liu
Subjects
dc:subject × 5Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.rit.edu/theses/12676