Back to search

University of Saskatchewan

Large Language Models for Code Generation and Program Comprehension: Exploring Capabilities, Context, and Developer Adaptation

Abstract

dc:description.abstract

Program comprehension and code generation are central to modern software engineering, influencing tasks such as problem solving, documentation, and long-term maintenance. With the rapid advancement of Large Language Models (LLMs), understanding their capabilities across both code generation and comprehensionoriented activities has become a core research priority. This thesis examines the role of LLMs in supporting software development through three interconnected empirical studies, spanning code generation, contextual understanding, and developer-adaptive summarization. The first study evaluates the code generation, and problem-solving abilities of advanced LLMs across competitive programming platforms, including LeetCode, Codeforces, and HackerRank. Using 224 problems spanning 15 categories, we compare three conversational LLMs, ChatGPT, Gemini, and Meta AI, against human programmers. Results show that LLMs achieved a 71.43% success rate on structured LeetCode problems and performed competitively in reasoning-based tasks, demonstrating human-comparable comprehension of programming logic, control flow, and problem intent. While their performance declines on more complicated, time-constrained challenges, the findings confirm that LLMs possess foundational semantic understanding essential for higher-level comprehension tasks such as summarization and documentation generation. The second study introduces HelpCOM, a dependency-aware method-level comment generation technique designed to address the contextual limitations of existing models. Analyzing 647K Java methods from ten popular GitHub repositories, we found that dependent methods constitute 69.25% of all methods and are more change-prone than independent ones. Experimental results show that HelpCOM outperforms state-ofthe-art baselines (e.g., CodeT5+, CodeBERT, and ASAP) by 5.6%–50.4% across syntactic, semantic, and LLM-based evaluation metrics. A practitioner survey involving 156 developers further confirmed HelpCOM’s effectiveness in producing more complete and comprehensible documentation for dependent methods. The third study extends this work toward Developer-Adaptive Code Summarization, where summaries are personalized to the reader’s expertise level. Adaptive prompts were integrated into GitHub Copilot Chat, enabling real-time generation of novice- and expert-oriented summaries. Semantic similarity analysis showed strong alignment between Copilot-based and API-based outputs (69% for novice and 77% for expert summaries). Additionally, interviews with five experienced developers validated the approach’s practicality and usefulness, noting that adaptive summaries enhanced clarity, reduced cognitive effort, and improved integration into daily workflows. Collectively, these studies offer a comprehensive perspective on how LLMs can support both code generation and program comprehension, from solving complex programming tasks to generating context-aware and expertise-adaptive documentation. The results highlight the promise and current limitations of LLMs, paving the way for future research on building intelligent, context-sensitive, and human-aligned tools for software engineering.

Degree

thesis:*
Name thesis:degree_name
Master of Science (M.Sc.)
Level thesis:degree_level
Masters
Discipline thesis:degree_discipline
Computer Science
Department thesis:degree_department
Computer Science
Grantor
University of Saskatchewan
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Billah, Md Mustakim
Advisor dc:contributor.advisor
  • Roy, Banani
Committee members dc:contributor.committeemember
  • Yeh, Yen-Ting (Allen)
  • Eager, Derek
ORCID person:identifier.orcid

Subjects

dc:subject × 2

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/10388/17566

Chain of custody

source
Harvested from
University of Saskatchewan
Base URL
harvest.usask.ca/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Billah, Md Mustakim. Large Language Models for Code Generation and Program Comprehension: Exploring Capabilities, Context, and Developer Adaptation. Masters thesis, University of Saskatchewan, 2025. https://hdl.handle.net/10388/17566