Baylor University.
Program synthesis for distributed computing : translating sequential programs to distributed programs with automated test generation.
Abstract
dc:description.abstractWith the growing demand for large-scale data processing, distributed computing has become an essential paradigm, allowing complex tasks to be handled across multiple computational nodes. However, transitioning from sequential to distributed programming presents significant challenges, such as the requirement for specialized knowledge and the complexity of distributed environments. This study is centered around three main objectives to tackle this issue: automating the conversion of Python code into distributed PySpark code, enhancing translation accuracy by leveraging data-driven methods, and guaranteeing functional equivalence through verification. First, this research automates the translation of sequential code into distributed formats by leveraging program synthesis techniques. Second, the integration of a fine-tuned BERT model enhances translation accuracy, particularly when predicting complex PySpark API calls and their combination. Finally, property-based testing, using the Hypothesis framework, ensures functional equivalence between the original Python code and its distributed PySpark counterpart. This dissertation demonstrates how effective these methods are in simplifying distributed programming while maintaining accuracy and performance when compared to their sequential counterparts. The contributions of this work make distributed computing more accessible and efficient, with future efforts aimed at expanding frameworks, optimizing performance, and scaling to larger datasets.
Degree
thesis:*- Name thesis:degree_name
- Ph.D.
- Level thesis:degree_level
- Doctoral
- Department thesis:degree_department
- Dept. of Computer Science.
- Grantor
- Baylor University.
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Sanjel, Arun, 1995-
- Advisor dc:contributor.advisor
-
- Speegle, Gregory David.
Subjects
dc:subject × 5Rights
dc:rights.accessrights- Statement dc:rights.accessrights
-
- No access - Contact librarywebmaster@baylor.edu
- Baylor University works are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. Contact libraryquestions@baylor.edu for inquiries about permission.
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/2104/13803