Georgia Institute of Technology
Enabling semantically richer queries over unstructured data
Abstract
dc:description.abstractQuerying unstructured data such as video, audio, and text is critical for domains ranging from traffic surveillance to healthcare and finance. Modern AI models (e.g., vision and language models) unlock significant potential for extracting fine-grained information from such data. However, current data systems that leverage these models primarily focus on efficiently executing semantically simple queries. This thesis argues that enabling semantically rich queries over unstructured data requires rethinking both the query execution strategies and the query interfaces. To this end, we develop four systems that can efficiently and accurately process semantically rich queries over unstructured data. First, we present Zeus, a video analytics system that efficiently localizes complex actions in videos using a reinforcement learning (RL)-based query executor. By using accuracy-based rewards during query planning, Zeus substantially improves efficiency while meeting user-specified accuracy targets. Next, we propose Tracer, an adaptive query processing framework for multi-camera re-identification queries. Tracer uses a recurrent network with a probabilistic search model to optimally select camera feeds to process at each time step. Tracer significantly reduces the cost of re-identification queries on synthetically generated and real-world datasets. We then introduce SketchQL, a visual query interface that allows users to sketch complex video moments. SketchQL maps these sketches to fine-grained video moments using a transformer model trained on synthetically generated data. SketchQL greatly enhances the usability and accuracy of fine-grained video moment retrieval. Finally, we present Halo, a long-context question answering (QA) framework designed for domain-augmented queries. Halo incorporates domain knowledge into the QA pipeline via a Domain Hints interface, allowing users to specify structured suggestions that augment the original query. A three-stage execution pipeline applies these hints automatically and optimally, improving both the efficiency and accuracy of long-context QA.
Degree
thesis:*- Level thesis:degree_level
- Doctoral
- Department dc:contributor.department
- Computer Science
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Chunduri, Pramod
- Advisor dc:contributor.advisor
-
- Arulraj, Joy
- Committee members dc:contributor.committeemember
-
- Rong, Kexin
- Chu, Xu
- Navathe, Shamkant B.
- Payani, Ali
Subjects
dc:subject × 6Rights
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/78732