{"id":{"repo_id":"vilnius","oai_identifier":"oai:vu.lt:elaba:308337086"},"canonical_url":"https://search.ndltd.org/etd/vilnius/oai:vu.lt:elaba:308337086","repository":{"repo_id":"vilnius","name":"Vilnius University","base_url":"https://epublications.vu.lt/oai"},"display":{"title":"Developing educational content using artificial intelligence /","abstract":"Creating programming exercises is time-consuming for educators, as each exercise needs a correct solution, valid test cases, and a clear problem description. This study evaluates three large language models, Google's Gemini 2.5 Flash, Anthropic's Claude Haiku 4.5, and OpenAI's GPT-5.4-mini, for automatically generating C programming exercises, comparing their technical correctness and educational quality. A software system was developed that sends identical prompts to each model and validates exercises through a unified pipeline covering parsing, compilation, and test case execution. A dataset of 240 exercises was generated across four topics of increasing complexity: variables and operators, conditionals and loops, arrays and strings, and pointers and memory allocation. Of these, 212 (88.3%) passed automated validation, with Gemini achieving the highest pass rate (93.75%), followed by Claude (86.25%) and GPT (85%). The dominant cause of failure was incorrect expected values in test cases rather than logically wrong solution code, indicating that models can write correct C code but struggle to trace through their own solutions. Human evaluation by 15 evaluators with C programming experience assessed exercises for clarity, concept targeting, difficulty, and usability. Gemini scored highest on clarity and usability (93.3% usable without modification), GPT produced clear but consistently too easy exercises, and Claude generated more demanding exercises with less clear descriptions (70.8% usable as is). Clarity and usability declined across all models on the most advanced topic. The results show that current budget-tier large language models can generate C programming exercises that are suitable for classroom use, but human review is still necessary, particularly for advanced topics, difficulty calibration, and test case accuracy. A concrete improvement would be executing generated solutions and using actual output as expected test case values rather than relying on models to predict them.","abstract_html":"Creating programming exercises is time-consuming for educators, as each exercise needs a correct solution, valid test cases, and a clear problem description. This study evaluates three large language models, Google&#x27;s Gemini 2.5 Flash, Anthropic&#x27;s Claude Haiku 4.5, and OpenAI&#x27;s GPT-5.4-mini, for automatically generating C programming exercises, comparing their technical correctness and educational quality. A software system was developed that sends identical prompts to each model and validates exercises through a unified pipeline covering parsing, compilation, and test case execution. A dataset of 240 exercises was generated across four topics of increasing complexity: variables and operators, conditionals and loops, arrays and strings, and pointers and memory allocation. Of these, 212 (88.3%) passed automated validation, with Gemini achieving the highest pass rate (93.75%), followed by Claude (86.25%) and GPT (85%). The dominant cause of failure was incorrect expected values in test cases rather than logically wrong solution code, indicating that models can write correct C code but struggle to trace through their own solutions. Human evaluation by 15 evaluators with C programming experience assessed exercises for clarity, concept targeting, difficulty, and usability. Gemini scored highest on clarity and usability (93.3% usable without modification), GPT produced clear but consistently too easy exercises, and Claude generated more demanding exercises with less clear descriptions (70.8% usable as is). Clarity and usability declined across all models on the most advanced topic. The results show that current budget-tier large language models can generate C programming exercises that are suitable for classroom use, but human review is still necessary, particularly for advanced topics, difficulty calibration, and test case accuracy. A concrete improvement would be executing generated solutions and using actual output as expected test case values rather than relying on models to predict them.","abstract_has_math":false,"creators":["Jemeljanovas, Šarūnas,"],"institution":"Institutional Repository of Vilnius University","degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2026,"date_issued":"2026","date_published":"2026","updated_at":"2026-11-19T10:12:59Z","subjects":["programming education, programming exercises, c programming language, large language models, artificial intelligence"],"languages":["eng"],"rights":["info:eu-repo/semantics/openAccess"],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://repository.vu.lt/VU:ELABAETD308337086&prefLang=en_US","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://repository.vu.lt/VU:ELABAETD308337086&prefLang=en_US","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":"open","embargo_until":null,"access_note":null},"source_record":{"url":"https://epublications.vu.lt/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Avu.lt%3Aelaba%3A308337086","prefix":"oai_dc"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:creator","label":"Author","values":["Jemeljanovas, Šarūnas,"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2026"]},{"key":"dc:publisher","label":"Institution","values":["Institutional Repository of Vilnius University"]},{"key":"dc:relation","label":"Dc Relation","values":["https://epublications.vu.lt/object/elaba:308337086/308337086.pdf"]},{"key":"dc:type","label":"Dc Type","values":["info:eu-repo/semantics/bachelorThesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["programming education, programming exercises, c programming language, large language models, artificial intelligence"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["info:eu-repo/semantics/openAccess"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://repository.vu.lt/VU:ELABAETD308337086&prefLang=en_US"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Creating programming exercises is time-consuming for educators, as each exercise needs a correct solution, valid test cases, and a clear problem description. This study evaluates three large language models, Google's Gemini 2.5 Flash, Anthropic's Claude Haiku 4.5, and OpenAI's GPT-5.4-mini, for automatically generating C programming exercises, comparing their technical correctness and educational quality. A software system was developed that sends identical prompts to each model and validates exercises through a unified pipeline covering parsing, compilation, and test case execution. A dataset of 240 exercises was generated across four topics of increasing complexity: variables and operators, conditionals and loops, arrays and strings, and pointers and memory allocation. Of these, 212 (88.3%) passed automated validation, with Gemini achieving the highest pass rate (93.75%), followed by Claude (86.25%) and GPT (85%). The dominant cause of failure was incorrect expected values in test cases rather than logically wrong solution code, indicating that models can write correct C code but struggle to trace through their own solutions. Human evaluation by 15 evaluators with C programming experience assessed exercises for clarity, concept targeting, difficulty, and usability. Gemini scored highest on clarity and usability (93.3% usable without modification), GPT produced clear but consistently too easy exercises, and Claude generated more demanding exercises with less clear descriptions (70.8% usable as is). Clarity and usability declined across all models on the most advanced topic. The results show that current budget-tier large language models can generate C programming exercises that are suitable for classroom use, but human review is still necessary, particularly for advanced topics, difficulty calibration, and test case accuracy. A concrete improvement would be executing generated solutions and using actual output as expected test case values rather than relying on models to predict them."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Developing educational content using artificial intelligence /","Edukacinio turinio kūrimas naudojant dirbtinį intelektą."]}]}],"canonical_facts":{"dc:creator":["Jemeljanovas, Šarūnas,"],"dc:date":["2026"],"dc:description":["Creating programming exercises is time-consuming for educators, as each exercise needs a correct solution, valid test cases, and a clear problem description. This study evaluates three large language models, Google's Gemini 2.5 Flash, Anthropic's Claude Haiku 4.5, and OpenAI's GPT-5.4-mini, for automatically generating C programming exercises, comparing their technical correctness and educational quality. A software system was developed that sends identical prompts to each model and validates exercises through a unified pipeline covering parsing, compilation, and test case execution. A dataset of 240 exercises was generated across four topics of increasing complexity: variables and operators, conditionals and loops, arrays and strings, and pointers and memory allocation. Of these, 212 (88.3%) passed automated validation, with Gemini achieving the highest pass rate (93.75%), followed by Claude (86.25%) and GPT (85%). The dominant cause of failure was incorrect expected values in test cases rather than logically wrong solution code, indicating that models can write correct C code but struggle to trace through their own solutions. Human evaluation by 15 evaluators with C programming experience assessed exercises for clarity, concept targeting, difficulty, and usability. Gemini scored highest on clarity and usability (93.3% usable without modification), GPT produced clear but consistently too easy exercises, and Claude generated more demanding exercises with less clear descriptions (70.8% usable as is). Clarity and usability declined across all models on the most advanced topic. The results show that current budget-tier large language models can generate C programming exercises that are suitable for classroom use, but human review is still necessary, particularly for advanced topics, difficulty calibration, and test case accuracy. A concrete improvement would be executing generated solutions and using actual output as expected test case values rather than relying on models to predict them."],"dc:format":["application/pdf"],"dc:identifier":["https://repository.vu.lt/VU:ELABAETD308337086&prefLang=en_US"],"dc:language":["eng"],"dc:publisher":["Institutional Repository of Vilnius University"],"dc:relation":["https://epublications.vu.lt/object/elaba:308337086/308337086.pdf"],"dc:rights":["info:eu-repo/semantics/openAccess"],"dc:subject":["programming education, programming exercises, c programming language, large language models, artificial intelligence"],"dc:title":["Developing educational content using artificial intelligence /","Edukacinio turinio kūrimas naudojant dirbtinį intelektą."],"dc:type":["info:eu-repo/semantics/bachelorThesis"]},"updated_at":"2026-11-19T10:12:59Z"}