Back to search

The Open University

Automated Processing of Referring Acts in Pair-Programming Dialogue

Abstract

dc:description.abstract

Pair programming is a collaborative technique where two people work together (in person or remotely) on the same piece of code. Numerous studies have demonstrated its benefits in industrial and educational settings, though they also highlight the challenges for its implementation. AI tools have been suggested as a way to overcome some of these obstacles, for instance by having a conversational agent act as a partner where no suitable human peer is available. With the advent of Large Language Models (LLMs), developing sophisticated conversational agents has become a simpler task. However, pair-programming dialogue presents some unique challenges, particularly due to its being a form of spoken dialogue and containing programming domain terminology. In this work, we looked at how LLMs understand pair-programming dialogue by looking at reference processing, as reference processing tasks represent a key structure in discourse, the attentional structure. We carried out experiments in mention detection, coreference resolution, and reference generation. In order to carry out our experiments, we first collected a dataset of remote pair-programming dialogues between students completing simple Python tasks. We collected speech data, keylog records, code files, and screen captures. We then transcribed the dialogues and combined the different data sources. Next, we developed an annotation scheme to annotate referring acts in the dialogues and trained a team of seven annotators on how to annotate the different layers of our scheme. Our experiments showed that recent LLMs could process references comparably to humans to some extent, with the characteristics of pair-programming dialogue sometimes hindering performance. We also observed that different LLMs can present completely different attentional structures, some more human-like and more suitable for the collaborative talk required in pair programming. With this work, we release a public dataset of pair-programming dialogues with rich contextual information and reference annotations, as well as the annotation scheme used to develop our dataset. We also contribute some empirical evidence about how LLMs process references in a challenging context, and which discourse features need to be taken into account when dealing with complex text genres. We also illuminate some links with how LLMs process references and their suitability for educational pair programming.

Degree

thesis:*
Name dc:type.qualificationname
phd
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
The Open University
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Domingo, Cecilia

Rights

Language dc:language
en

Chain of custody

source
Harvested from
The Open University
Base URL
oro.open.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Domingo, Cecilia. Automated Processing of Referring Acts in Pair-Programming Dialogue. doctoral thesis, The Open University, 2026.