Back to search

University of British Columbia

Improving commonsense reasoning in multimodal models

Abstract

dc:description

Humans navigate the world by filling in observational gaps through commonsense knowledge and reasoning. Although large-scale foundation models have achieved strong performance on commonsense benchmarks, they struggle to reason beyond surface-level patterns, particularly in novel and ambiguous settings. This thesis develops methods to evaluate and improve commonsense reasoning in foundation models across language, vision, and video modalities. We first introduce frameworks that generate and leverage temporal inferences, improving performance on downstream language tasks requiring event reasoning. We then extend this idea to the visual domain with an architecture that incorporates commonsense inferences, improving reasoning abilities for knowledge-intensive Visual Question Answering (VQA). Next, we turn to diagnosing the limits of current models, designing evaluations that reveal systematic weaknesses in their spatio-temporal and causal reasoning. Targeting these gaps, we introduce a human-inspired framework for detecting surprising events in video, improving performance across five video understanding tasks. Overall, our results highlight that explicitly modeling the implicit dimensions of human reasoning, such as commonsense, causes, effects, and beliefs, can significantly enhance the robustness and reasoning abilities of modern AI systems. Looking ahead, our findings motivate future work on agents that reason adaptively, maintain robust internal world models, and integrate richer spatio-temporal understanding for real-world deployment.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy - PhD
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ravi, Sahithya
Contributors dc:contributor
  • Shwartz, Vered
  • Ng, Raymond Tak-yan, 1963-

Rights

dc:rights
Statement dc:rights
  • Attribution-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/95460

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Ravi, Sahithya. Improving commonsense reasoning in multimodal models. doctoral thesis, University of British Columbia, 2026. http://hdl.handle.net/2429/95460