Back to search

University of North Texas

Modeling Synergistic Relationships Between Words and Images

Abstract

dc:description

Texts and images provide alternative, yet orthogonal views of the same underlying cognitive concept. By uncovering synergistic, semantic relationships that exist between words and images, I am working to develop novel techniques that can help improve tasks in natural language processing, as well as effective models for text-to-image synthesis, image retrieval, and automatic image annotation. Specifically, in my dissertation, I will explore the interoperability of features between language and vision tasks. In the first part, I will show how it is possible to apply features generated using evidence gathered from text corpora to solve the image annotation problem in computer vision, without the use of any visual information. In the second part, I will address research in the reverse direction, and show how visual cues can be used to improve tasks in natural language processing. Importantly, I propose a novel metric to estimate the similarity of words by comparing the visual similarity of concepts invoked by these words, and show that it can be used further to advance the state-of-the-art methods that employ corpus-based and knowledge-based semantic similarity measures. Finally, I attempt to construct a joint semantic space connecting words with images, and synthesize an evaluation framework to quantify cross-modal semantic relationships that exist between arbitrary pairs of words and images. I study the effectiveness of unsupervised, corpus-based approaches to automatically derive the semantic relatedness between words and images, and perform empirical evaluations by measuring its correlation with human annotators.

Degree

thesis:*
Grantor dc:publisher
University of North Texas
Year dc:date
2012

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Leong, Chee Wee
Contributors dc:contributor
  • Mihalcea, Rada, 1974-
  • Tarau, Paul
  • Ruiz, Miguel E.
  • Pedersen, Ted

Subjects

dc:subject × 5

Rights

dc:rights
Statement dc:rights
  • Public
  • Leong, Chee Wee
  • Copyright
  • Copyright is held by the author, unless otherwise noted. All rights Reserved.
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
https://digital.library.unt.edu/ark:/67531/metadc177223/
ark: ark:/67531/metadc177223

Chain of custody

source
Harvested from
University of North Texas
Base URL
digital.library.unt.edu/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Leong, Chee Wee. Modeling Synergistic Relationships Between Words and Images. University of North Texas, 2012. https://doi.org/10.12794/metadc177223