Back to search

University of British Columbia

Computational experiment comprehension using provenance summarization

Abstract

dc:description

Scientists often use complex multistep workflows to computationally analyze data. These workflows might include downloading datasets, installing packages, data processing, model training and evaluating the results. It is difficult to effectively manage and track these computational workflows. The fast-paced and iterative nature of research programming leads these workflows to include unused code, multiple versions of the same script, and untracked dependencies. These issues cause difficulties when researchers try to reproduce code that someone else has written, or even code that they have written themselves. Research programmers can address these problems by collecting data provenance: a record of what happened during an experiment, including files touched, execution order, and software dependencies. Provenance provides a record of experiment execution, but provenance graphs are often large and complicated, and quickly become incomprehensible. We propose a new method for summarizing provenance graphs using recent advances in prompting large language models. We use large language model prompting to develop textual summaries of provenance graphs. We perform a user study to compare textual summaries to traditional node-link diagrams for experiment reproduction tasks. Our results show that textual summaries are a promising approach to summarizing provenance for experiment reproduction. We use qualitative results from the user study to motivate future designs for reproducibility tools.

Degree

thesis:*
Name thesis:degree_name
Master of Science - MSc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Boufford, Nichole
Contributors dc:contributor
  • Pasquier, Thomas

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/87659

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Boufford, Nichole. Computational experiment comprehension using provenance summarization. master's thesis, University of British Columbia, 2024. http://hdl.handle.net/2429/87659