Institutional Repository of Vilnius University
Edukacinio turinio generavimas naudojant DKM /
Abstract
dc:descriptionThe aim of this thesis is to develop automated tools for the generation and evaluation of computer science questions, to investigate the ability of modern large language models (referred to as LLMs) to generate questions that correspond to the cognitive complexity levels of Bloom’s taxonomy and satisfy quality requirements, and also to evaluate their ability to detect question defects. The thesis consists of four parts. The first part analyzes scientific literature on LLMs, prompt engineering strategies, and the application of LLMs in the field of education. It also reviews LLM-based evaluation systems, discusses evaluation criteria for educational questions, and presents Bloom’s taxonomy for the classification of tasks according to cognitive complexity levels. The second part justifies the choice of Moodle learning management system, defines the functional and non-functional requirements of the developed plugins, and describes the implementation details. The remaining parts present the methodologies of the conducted studies and analyze the obtained results. GPT 5.5 was used in the study of generated questions quality. The high proportion of questions meeting the quality criteria suggests that modern LLMs are capable of generating high-quality computer science questions, although certain challenges still remain. The automatic question quality evaluation study employed Deepseek v4 pro. Classical evaluation metrics were applied in the study, including precision, recall, and F1-score. The obtained results indicate that automatic question quality evaluation is a promising area; however, the ability of modern models to detect certain types of question defects remains limited.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Furs, Renat,
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD308337195&prefLang=en_US