Back to search

Institutional Repository of Vilnius University

Developing educational content using artificial intelligence /

Abstract

dc:description

Creating programming exercises is time-consuming for educators, as each exercise needs a correct solution, valid test cases, and a clear problem description. This study evaluates three large language models, Google's Gemini 2.5 Flash, Anthropic's Claude Haiku 4.5, and OpenAI's GPT-5.4-mini, for automatically generating C programming exercises, comparing their technical correctness and educational quality. A software system was developed that sends identical prompts to each model and validates exercises through a unified pipeline covering parsing, compilation, and test case execution. A dataset of 240 exercises was generated across four topics of increasing complexity: variables and operators, conditionals and loops, arrays and strings, and pointers and memory allocation. Of these, 212 (88.3%) passed automated validation, with Gemini achieving the highest pass rate (93.75%), followed by Claude (86.25%) and GPT (85%). The dominant cause of failure was incorrect expected values in test cases rather than logically wrong solution code, indicating that models can write correct C code but struggle to trace through their own solutions. Human evaluation by 15 evaluators with C programming experience assessed exercises for clarity, concept targeting, difficulty, and usability. Gemini scored highest on clarity and usability (93.3% usable without modification), GPT produced clear but consistently too easy exercises, and Claude generated more demanding exercises with less clear descriptions (70.8% usable as is). Clarity and usability declined across all models on the most advanced topic. The results show that current budget-tier large language models can generate C programming exercises that are suitable for classroom use, but human review is still necessary, particularly for advanced topics, difficulty calibration, and test case accuracy. A concrete improvement would be executing generated solutions and using actual output as expected test case values rather than relying on models to predict them.

Degree

thesis:*
Grantor dc:publisher
Institutional Repository of Vilnius University
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Jemeljanovas, Šarūnas,

Subjects

dc:subject × 1

Rights

dc:rights
Statement dc:rights
  • info:eu-repo/semantics/openAccess
Language dc:language
eng

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Vilnius University
Base URL
epublications.vu.lt/oai
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Jemeljanovas, Šarūnas,. Developing educational content using artificial intelligence /. Institutional Repository of Vilnius University, 2026. https://repository.vu.lt/VU:ELABAETD308337086&prefLang=en_US