School teachers spend approximately 30 percent of their time grading exams and other assessments. With an increasingly digitized education, a research field have been initiated that aims to reduce the time spent on grading by automating it. This is an easy task for multiple-choice questions but much harder for open-ended questions requiring free-text answers, where the latter have shown to be superior for knowledge assessment and learning consolidation. While results in previous work have presented promising results of up to 90 percent grading accuracy, it is still problematic using a system that gives the wrong grade in 10 percent of the cases. This has given rise to a research field focusing on assisting teachers in the grading process, instead of fully replacing them. Cluster analysis has been the most popular tool for this, grouping similar answers together and letting teachers process groups of answers at once, instead of evaluating each question one-at-a-time. This approach has shown evidence to decrease the time spent on grading substantially, however, the methods for performing the clustering vary widely between studies, leaving no apparent methodology choice for real-use implementation. Using several techniques for pre-processing, text representation and choice of clustering algorithm, this work compared various methods for clustering free-text answers by evaluating them on a dataset containing almost 400 000 student answers. The results showed that using all of the tested pre-processing techniques led to the best performance, although the difference to using minimum pre-processing were small. Sentence embeddings were the text representation approach that performed the best, however, it remains to be answered how it should be used when spelling and grammar is part of the assessment, as it lacks the ability to identify such errors. A suitable choice of clustering algorithm is one where the number of clusters can be specified, as determining this automatically proved to be difficult. Teachers can then easily adjust the number of clusters based on their judgement. / Skollärare spenderar ungefär 30 procent av sin tid på rättning av prov och andra bedömningar. I takt med att mer utbildning digitaliseras, försöker forskare hitta sätt att automatisera rättning för att minska den administrativa bördan för lärare. Flervalsfrågor har fördelen att de enkelt kan rättas automatiskt, medan öppet ställda frågor som kräver ett fritt formulerat svar har visat sig vara ett bättre verktyg för att mäta elevers förståelse. Dessa typer av frågor är däremot betydligt svårare att rätta automatiskt, vilket lett till forskning inom automatisk rättning av dessa. Även om tidigare forskning har lyckats uppnå resultat med upp till 90 procents träffsäkerhet, är det fortfarande problematiskt att det blir fel i de resterande 10 procenten av fallen. Detta har lett till forskning som fokuserar på underlätta för lärare i rättningen, istället för att ersätta dem. Klusteranalys har varit det mest populära tillvägagångssättet för att åstadkomma detta, där liknande svar grupperas tillsammans, vilket möjliggör rättning av flera svar samtidigt. Denna metod har visat sig minska rättningstiden signifikant, däremot har metoderna för att göra klusteranalysen varierat brett, vilket gör det svårt att veta hur en implementering i ett verkligt scenario bör se ut. Genom att använda olika tekniker för textbearbetning, textrepresentation och val av klusteralgoritm, jämför detta arbete olika metoder för att klustra fritext-svar, genom att utvärdera dessa på nästan 400 000 riktiga elevsvar. Resultatet visar att mer textbearbetning generellt är bättre, även om skillnaderna är små. Användning av så kallade sentence embeddings ledde till bäst resultat när olika tekniker för textrepresentation jämfördes. Däremot har denna teknik svårare att identifiera grammatik- och stavningsfel, hur detta ska hanteras är en fråga för framtida forskning. Ett lämpligt val av klustringsalgoritm är en där antalet kluster kan bestämmas av användaren, då det visat sig svårt att bestämma det automatiskt. Lärare kan då justera antalet kluster ifall det skulle vara för få eller för många.

The Technische Universität Dresden has multiple e-learning projects in use. The Chair of Software Technology uses Inloop to teach students object-oriented programming through automatic feedback. In the last years, interest has grown in giving students automated feedback on modeling tasks. This is why there was an extension developed by Hamann to automate the assessment of modeling tasks in 2020. The TU Dresden currently has plans to replace Inloop with Artemis, a comparable system. Artemis currently supports the semi-automatic assessment of modeling exercises. In contrast, the system proposed by Hamann, called Inloom, is based on a rule-based approach and provides instant feedback. A rule-based system has certain advantages over a similarity-based system. One advantage is the mostly better feedback that these systems generate. To give instructors more flexibility and choice, this work tries to identify possible ways of extending Artemis with the rule-based approach Inloom. In the second step, this thesis will provide a proof of concept implementation. Furthermore, a comparison between different systems is developed to help instructors choose the best suitable system for their usecase.:Introduction, Background, Related Work, Analysis, System Design, Implementation, Evaluation, Conclusion and Future Work, Bibliography, Appendix

Aim Develop an IT artifact that provides formative feedback for students based on their programming assignments. Background One of the best methods to learn programming is by practice. Providing feedback to students is an important and a valuable factor for improving learning, which plays a vital part in the student’s possibility to enhance and improve its solutions. Software development courses have several assignments and each course instructs about 100 students. To assess and provide feedback for all the students and each assignment demands considerable resources. In a survey conducted by TCO (2013) half of the respondents’ state that feedback is rarely or never given in reasonable time. Method Action Design Research (ADR) was used to intervene an organizational problem in parallel with building and evaluating an IT artifact. Conclusion The results from the study were four generated design principles and a proposed solution on how to use existing static code analysis tools for provide formative feedback to students.