Athens University Economics and Business (AUEB)
Data analysis applications in software engineering
Abstract
dc:descriptionSoftware engineering research has evolved significantly over the past five decades, producing a substantial body of knowledge, tools, and methodologies that underpin the multi-trillion dollar software industry. However, as the field has grown in complexity and scope, critical questions have emerged about how effectively this research translates into practical impact, what role emerging technologies like machine learning (ML) play in advancing software engineering practices, and how research artifacts contribute meaningfully to the broader research community. This dissertation addresses these fundamental questions through four interconnected research strands that systematically examine different aspects of software engineering research impact, methodology, and practice. The first contribution investigates the usage and impact of data papers published in the Mining Software Repositories (MSR) conference. This study reveals that while data papers have been widely adopted by the research community, there are opportunities to improve their documentation and accessibility. Data papers constitute valuable research artifacts, with each paper being strongly cited on average 5.4 times, indicating their utility for conducting empirical studies. The MSR data showcase track has successfully catalyzed the publication of data papers, with data papers constituting more than 15% of MSR publications in 2019. However, product-oriented data papers outnumber process-oriented ones, while certain SWEBOK knowledge areas, such as software requirements, economics, and engineering models and methods, are underrepresented. The second contribution examines the practical impact of software engineering research through a novel combination of patent analysis and author surveys. This work demonstrates that software engineering researchers have successfully equipped practitioners with various tools, processes, and methods, and improved many existing products. Practitioner-oriented publication venues appear more impactful than researcher-oriented ones, while insufficient funding and unfavorable cost-benefit trade-offs are the main obstacles to research adoption. The patent analysis reveals that while only a limited number of patents cite software engineering research papers, almost half of the patents that do cite software engineering papers have been renewed, indicating their commercial value. Cross-disciplinary software engineering areas, particularly those related to programming languages, compilers, and management, have the most significant practical impact. The third contribution provides a comprehensive tertiary study of ML applications in software engineering, revealing that while ML techniques have been extensively applied across various software engineering tasks, significant gaps remain in their empirical validation and industrial adoption. ML is primarily employed for automating and optimizing testing, predicting software faults and quality metrics, and estimating project costs and effort. However, research opportunities exist in software construction, configuration management, and models and methods, while there is a critical need for more empirical and industrial studies evaluating ML techniques in software engineering contexts. The findings demonstrate that the lack of empirical validation studies, comparative analyses, and industrial trials represents a significant barrier to the practical adoption of ML techniques in software engineering. The fourth contribution explores the application of Large Language Models (LLMs) to code completion tasks, with a particular focus on understanding model confidence through perplexity analysis. This work provides the first large-scale, language-specific analysis of code perplexity measured through LLMs, offering insights into how model confidence varies across programming languages and how this relates to code quality and correctness. The study reveals that code perplexity varies significantly across programming languages, with strongly-typed languages generally exhibiting lower perplexity than scripting languages. High-level programming languages show lower perplexity than low-level ones, while the language distribution in training data does not appear to significantly affect perplexity measurements. Furthermore, code perplexity depends on the employed LLM, with models from the same family tending to have similar rankings, but is independent of the specific code dataset used for its evaluation. Altogether, these contributions provide a comprehensive understanding of how software engineering research translates into practical applications and how emerging technologies can enhance software development practices. The findings demonstrate that software engineering research has provided significant contributions to practice, though opportunities exist for enhancing empirical validation, improving research artifact design, and expanding cross-disciplinary collaboration. The systematic approaches employed throughout this dissertation ensure that the findings are robust and can be replicated or extended by other researchers. This research establishes a foundation for future research directions and provides valuable insights for researchers, practitioners, funding agencies, and the broader software engineering community to ensure that academic efforts continue to contribute meaningfully to industry needs and societal challenges.
Degree
thesis:*- Grantor dc:publisher
- Athens University Economics and Business (AUEB)
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kotti, Zoi
- Κώττη, Ζωή
Subjects
dc:subject × 12- Τεχνολογία λογισμικού
- Τεχνητή νοημοσύνη
- Ανάλυση δεδομένων
- Software engineering
- Artificial intelligence
- Data analysis
- Άλλες Επιστήμες Μηχανικού και Τεχνολογίες
- Επιστήμες Μηχανικού και Τεχνολογία
- Μηχανική και Τεχνολογίες, άλλοι τομείς
- Other Engineering and Technologies
- Engineering and Technology
- Engineering and Technologies, miscellaneous
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/60524