Back to search

Michigan State University

Using semantic structure of the data and knowledge in question answering systems

Abstract

dc:description

Understanding and reasoning over natural language is one of the most crucial and long-standing challenges in Artificial Intelligence (AI). Question answering (QA) is the task of automatically answering questions posed by humans in a natural language form. It is an important criterion to evaluate the language understanding and reasoning capabilities of AI systems. Though machine learning systems on Question Answering (QA) have shown tremendous success in language understanding, they still suffer from a lack of interpretability and generalizability, in particular, when complex reasoning is required to answer the questions. In this dissertation, we aim to build novel QA architectures that answer complex questions using the explicit relational structure of the raw data, that is, text and image, and exploiting external knowledge. We investigate a variety of problems, including answering natural language questions when the answer can be found in multiple modalities, including 1) Textual documents (Document-level QA), 2) Images (Cross-Modality QA), 3) Knowledge graphs (Commonsense QA) and, 4) Combination of text and knowledge graphs. First, for Document-level QA, we develop a new technique, Semantic Role Labeling Graph Reasoning Network (SRLGRN), via which the explicit semantic structure of multiple textual documents is used. In particular, based on semantic role labeling, we form a multi-relational graph that jointly learns to find cross-paragraph reasoning paths and answers multi-hop reasoning questions. Second, for the type of QA that requires causal reasoning over textual documents, we propose a new technique, Relational Gating Network (RGN), that jointly learns to extract the entities and their relations to help highlight the important entity chains and find how those affect each other. Third, for the type of questions that require complex reasoning over language and vision modalities (Cross-Modality QA), we propose a new technique, Cross-Modality Relevance (CMR). This technique considers the relevance between textual tokens and visual objects by aligning the two modalities. Fourth, for answering questions based on given Knowledge Graphs (KG), we propose a new technique, Dynamic Relevance Graph Network (DRGN). This technique is based on a graph neural network and re-scales the importance of the neighbor nodes in the graph dynamically by training a relevance matrix. The new neighborhoods trained by relevance help fill in the knowledge gaps in the KG for more effective knowledge-based reasoning. Fifth, for answering questions using a combination of textual documents and an external knowledge graph, we propose a new technique, Multi-hop Reasoning Network over Relevant Commonsense Subgraphs (MRRG). MRRG technique extracts the most relevant KG subgraph for each question and document and uses that subgraph combined with the textual content and question representations for answering complex questions. We improve the performance, interpretability, and generalizability of various challenging QA benchmarks based on different modalities. Our ideas have proven to be effective in multi-hop reasoning, causal reasoning, cross-modality reasoning, and knowledge-based reasoning for question answering.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Zheng, Chen (Research scientist)
Contributors dc:contributor
  • Kordjamshidi, Parisa
  • Zhou, Jiayu
  • Johnson, Kristen
  • Peng, Taiquan

Subjects

dc:subject × 5

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-ShareAlike 4.0 International
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
etd:50870
isbn:9798358489134
oclc:on1373601702
oclc:1373601702
umi:29998766
local:Zheng_grad.msu_0128D_19546

Chain of custody

source
Harvested from
Michigan State University
Base URL
d.lib.msu.edu/oai
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Zheng, Chen (Research scientist). Using semantic structure of the data and knowledge in question answering systems. 2022. https://doi.org/doi:10.25335/edrm-cd14