The thesis investigates the automatic and semi-automatic evaluation of programming assignments in teaching of programming, exams and competitions. Manual evaluation of programming assignments is still dominant in programming teaching in Lithuania. In some contexts (i.e. programming competitions, basics of programming and algorithms course) there is widely applied automatic and semi-automatic evaluation of programming tasks based on black-box testing approach. However, automatic evaluation based on black-box testing is criticized for its inability to demonstrate the strengths and the weaknesses of the program being evaluated. Automatic evaluation is not capable of identifying the level of achievement in incomplete programs. The thesis addresses the problem of evaluation of practical programming assignments in Information Technology maturity exam (IT VBE). This problem involves requirements to evaluate reliably thousands of programming assignment solutions in a reasonable amount of time. The thesis presents analysis of systems for automatic evaluation of programming tasks and appropriate types of assignments. The analysis led to the improvement of the evaluation method. The most important result of the research is the proposed way to improve the semi-automatic evaluation and testing method by increasing its interactivity. A semi-automatic assessment system is expanded by adding the components responsible for the modification of submitted programs, monitoring changes in the... [to full text] / Disertacijoje nagrinėjamas automatinis ir pusiau automatinis programavimo užduočių vertinimas programavimo mokyme, egzaminuose ir varžybose. Šiuo metu informatikos mokyme Lietuvoje vis dar dominuoja rankinis programavimo užduočių vertinimas. Atskirose švietimo srityse (programavimo varžybos, algoritmavimo kursai) išplito automatinis ir pusiau automatinis programavimo užduočių vertinimas, paremtas juodosios dėžės testavimo principu. Tačiau automatinio vertinimo pagrindu gauti įverčiai kritikuojami kaip neatskleidžiantys visų pateiktos programos privalumų ir trūkumų. Ypatingai dideli vertinimo netikslumai gaunami vertinant nebaigtas programas. Disertacijoje sprendžiama Informacinių technologijų valstybinio brandos egzamino (IT VBE) praktinių programavimo užduočių vertinimo problema – siekiama gauti patikimus įvertinimus dideliam pateiktų programavimo užduočių sprendimų kiekiui per priimtiną laiką. Išanalizuotos automatinės programavimo užduočių vertinimo sistemos bei joms tinkami užduočių tipai atvedė prie idėjos tobulinti vertinimo metodą. Svarbiausias disertacijos rezultatas – pasiūlytas būdas tobulinti pusiau automatinį vertinimo ir testavimo metodą didinant jo interaktyvumą. Tam pusiau automatinė vertinimo sistema turi būti papildoma komponentais, leidžiančiais modifikuoti pateiktą programą, stebėti modifikuotos programos funkcionalumo pokyčius. Pasiūlytas patobulintas vertinimo metodas išbandytas praktiškai, sukurta IT VBE praktinių užduočių vertinimo sistema. Taikant šį... [toliau žr. visą tekstą]

Disertacijoje nagrinėjamas automatinis ir pusiau automatinis programavimo užduočių vertinimas programavimo mokyme, egzaminuose ir varžybose. Šiuo metu informatikos mokyme Lietuvoje vis dar dominuoja rankinis programavimo užduočių vertinimas. Atskirose švietimo srityse (programavimo varžybos, algoritmavimo kursai) išplito automatinis ir pusiau automatinis programavimo užduočių vertinimas, paremtas juodosios dėžės testavimo principu. Tačiau automatinio vertinimo pagrindu gauti įverčiai kritikuojami kaip neatskleidžiantys visų pateiktos programos privalumų ir trūkumų. Ypatingai dideli vertinimo netikslumai gaunami vertinant nebaigtas programas. Disertacijoje sprendžiama Informacinių technologijų valstybinio brandos egzamino (IT VBE) praktinių programavimo užduočių vertinimo problema – siekiama gauti patikimus įvertinimus dideliam pateiktų programavimo užduočių sprendimų kiekiui per priimtiną laiką. Išanalizuotos automatinės programavimo užduočių vertinimo sistemos bei joms tinkami užduočių tipai atvedė prie idėjos tobulinti vertinimo metodą. Svarbiausias disertacijos rezultatas – pasiūlytas būdas tobulinti pusiau automatinį vertinimo ir testavimo metodą didinant jo interaktyvumą. Tam pusiau automatinė vertinimo sistema turi būti papildoma komponentais, leidžiančiais modifikuoti pateiktą programą, stebėti modifikuotos programos funkcionalumo pokyčius. Pasiūlytas patobulintas vertinimo metodas išbandytas praktiškai, sukurta IT VBE praktinių užduočių vertinimo sistema. Taikant šį... [toliau žr. visą tekstą] / The thesis investigates the automatic and semi-automatic evaluation of programming assignments in teaching of programming, exams and competitions. Manual evaluation of programming assignments is still dominant in programming teaching in Lithuania. In some contexts (i.e. programming competitions, basics of programming and algorithms course) there is widely applied automatic and semi-automatic evaluation of programming tasks based on black-box testing approach. However, automatic evaluation based on black-box testing is criticized for its inability to demonstrate the strengths and the weaknesses of the program being evaluated. Automatic evaluation is not capable of identifying the level of achievement in incomplete programs. The thesis addresses the problem of evaluation of practical programming assignments in Information Technology maturity exam (IT VBE). This problem involves requirements to evaluate reliably thousands of programming assignment solutions in a reasonable amount of time. The thesis presents analysis of systems for automatic evaluation of programming tasks and appropriate types of assignments. The analysis led to the improvement of the evaluation method. The most important result of the research is the proposed way to improve the semi-automatic evaluation and testing method by increasing its interactivity. A semi-automatic assessment system is expanded by adding the components responsible for the modification of submitted programs, monitoring changes in the... [to full text]

Sedan läsåret 2018/2019 ska matematiklärare använda programmering som en problemlösningsmetod inom vissa kurser i ämnet matematik på gymnasiet samt i grundskolans senare år eftersom det finns i det centrala innehållet i kursplanerna. I grundskolans senare år är det i ämnena teknik och matematik där programmering ska ingå.Studiens syfte är att sammanställa på vilket sätt och inom vilka ämnesområden i matematiken som matematiklärare skapar programmeringsuppgifter för elever som går i grundskolans senare år eller gymnasieskolan. Studien har två frågeställningar. En av frågeställningarna är ”Inom vilka matematiska ämnesområden skapar matematiklärare uppgifter, är elever ska använda programmering som en lösningsmetod”. Den andra frågeställningar är ”Hur utformar matematiklärare uppgifter, där eleverna ska använda programmering som en problemlösningsmetod”.Metoden är en tematisk analys som är en kvalitativ metod, där tjugosju programmeringsuppgifter har undersökts utifrån frågeställningarna. Urvalet består av ett bekvämlighetsurval. Resultatet och analysen tyder på att lärarna skapar uppgifter där eleverna får möjlighet att på något sätt arbeta med en programkod eller det matematiska innehållet i programmeringsuppgiften i syfte att utveckla sin problemlösningsförmåga. Resultatet tyder på att ämneslärare skapar programmeringsuppgifter inom olika ämnesområden, men två matematiska ämnesområden var mer representerade än de andra. De två ämnesområdena är geometri och algebra. Slutsatsen är att alla lärare, förutom två, är förtrogna med att skapa programmeringsuppgifter där eleverna får möjlighet att utveckla deras problemlösningsförmåga. / Since the school year of 2018/2019 the teacher of mathematics is supposed to use programming as a problem-solving method in some of the courses in mathematics in upper secondary school and in the elementary school´s later years. In the elementary school´s later years, the subjects are technology and mathematics, which include programming.The purpose of the study is to compile how and in which disciplines of mathematics that mathematics teachers create programming-assignments to students of the later part of elementary school or in upper secondary school. The research has two questions. One of the questions are “In which disciplines of mathematics are math teachers creating assignments, where the students should use programming as a solving method”. The other question is “How do math teachers create assignments, where students are supposed to use programming as a problem-solving method”.The method is a thematic analysis that is a qualitative method, where twenty-seven assignments of programming has been analyzed based on the questions. The sample consists of a convenience selection.The result and the analysis indicates that the students are given the opportunity by working in some way with a program code or the mathematical content of the programming-assignments to develop their ability of problem solving. Math teachers create assignments in different disciplines of mathematics, but two disciplines were more represented than the others. The two disciplines are geometry and algebra. The conclusion is that all teachers, except two, is confidential to create programming assignments, where students can develop their ability of problem-solving.

A avaliação de trabalhos práticos de programação é uma tarefa dispendiosa. Diversas ferramentas têm sido propostas e desenvolvidas a fim de automatizar as tarefas repetitivas realizadas pelos professores na avaliação de trabalhos práticos de programação e fornecer um feedback mais rápido e adequado aos alunos. No entanto, a adição de uma nova ferramenta aumenta a sobrecarga de novas informações e ambientes que os alunos têm que lidar. De forma análoga, a ferramenta de avaliação é mais um recurso que o professor tem de configurar, manter e ensinar os alunos a utilizar. Por este motivo, vários trabalhos vêm sendo conduzidos a fim de integrar ferramentas de avaliação para trabalhos de programação em sistemas de gestão da aprendizagem (Learning Management Systems LMSs). A integração de ferramentas em LMSs promove sua adoção em disciplinas de computação, uma vez que estarão em concordância com o LMS já familiar aos alunos e professores, dispensando a necessidade de adotar, aprender e gerenciar a submissão e correção de trabalhos em um outro ambiente. No entanto, tais trabalhos consistem apenas em modelos conceituais abstratos, não propõem uma integração com vários LMSs ou não tratam certas dificuldades específicas encontradas quando integrando ferramentas de avaliação. Nessa perspectiva, este trabalho tem como objetivo o desenvolvimento da arquitetura IMPACTLE, uma solução que permite a integração de diferentes ferramentas de avaliação para trabalhos de programação em vários LMSs. A ideia é que os alunos e professores possam acessar as funcionalidades das ferramentas por meio dos LMSs em que já estão habituados, sem a necessidade de aprender a utilizar e adotar uma nova ferramenta. Protótipos da arquitetura IMPACTLE foram instanciados e experimentos foram realizados envolvendo a utilização de diferentes ferramentas de avaliação por meio de LMSs. De modo geral, observou-se que a utilização da IMPACTLE possibilita que professores e alunos realizem as tarefas relacionadas às atividades de programação de forma mais eficiente, eficaz e efetiva por meio dos LMSs. / The assessment of programming assignments is a costly task. Several tools have been proposed and developed in order to automate the repetitive tasks performed by instructors in the assessment of programming assignments and provide a faster and more adequate feedback to students. However, adding a new tool increases the overload of new information and environments that students have to deal with. Similarly, the assessment tool is one more resource that the instructor has to configure, maintain and teach students to use, spending time and effort that could be used in other pedagogical activities. For this reason, several works have been conducted to integrate assessment tools for programming assignments in learning management systems (LMSs). The integration of assessment tools into LMSs promotes their adoption in computing courses, since they will be in agreement with the LMS already familiar to students and instructors, without the need to adopt, learn and manage the submission and correction of assignments in another environment. However, these works consist only of abstract conceptual models, do not propose integration with several LMSs, or do not address certain specific difficulties faced when integrating assessment tools. In this perspective, this work aims at the proposition of the IMPACTLE architecture, a solution that allows the integration of different assessment tools for programming assignments in LMSs. The idea is that students and teachers can access the features of the tools through the LMSs they are already used to, without the need to learn how to use and adopt a new tool. Architecture prototypes were instantiated and experiments were carried out involving the use of different assessment tools through LMSs. In general, we noticed that the use of IMPACTLE enables instructors and students perform tasks related to programming activities in a more efficient, efficacious and effective way through the LMSs.

A avaliação de trabalhos práticos de programação é uma tarefa dispendiosa. Diversas ferramentas têm sido propostas e desenvolvidas a fim de automatizar as tarefas repetitivas realizadas pelos professores na avaliação de trabalhos práticos de programação e fornecer um feedback mais rápido e adequado aos alunos. No entanto, a adição de uma nova ferramenta aumenta a sobrecarga de novas informações e ambientes que os alunos têm que lidar. De forma análoga, a ferramenta de avaliação é mais um recurso que o professor tem de configurar, manter e ensinar os alunos a utilizar. Por este motivo, vários trabalhos vêm sendo conduzidos a fim de integrar ferramentas de avaliação para trabalhos de programação em sistemas de gestão da aprendizagem (Learning Management Systems LMSs). A integração de ferramentas em LMSs promove sua adoção em disciplinas de computação, uma vez que estarão em concordância com o LMS já familiar aos alunos e professores, dispensando a necessidade de adotar, aprender e gerenciar a submissão e correção de trabalhos em um outro ambiente. No entanto, tais trabalhos consistem apenas em modelos conceituais abstratos, não propõem uma integração com vários LMSs ou não tratam certas dificuldades específicas encontradas quando integrando ferramentas de avaliação. Nessa perspectiva, este trabalho tem como objetivo o desenvolvimento da arquitetura IMPACTLE, uma solução que permite a integração de diferentes ferramentas de avaliação para trabalhos de programação em vários LMSs. A ideia é que os alunos e professores possam acessar as funcionalidades das ferramentas por meio dos LMSs em que já estão habituados, sem a necessidade de aprender a utilizar e adotar uma nova ferramenta. Protótipos da arquitetura IMPACTLE foram instanciados e experimentos foram realizados envolvendo a utilização de diferentes ferramentas de avaliação por meio de LMSs. De modo geral, observou-se que a utilização da IMPACTLE possibilita que professores e alunos realizem as tarefas relacionadas às atividades de programação de forma mais eficiente, eficaz e efetiva por meio dos LMSs. / The assessment of programming assignments is a costly task. Several tools have been proposed and developed in order to automate the repetitive tasks performed by instructors in the assessment of programming assignments and provide a faster and more adequate feedback to students. However, adding a new tool increases the overload of new information and environments that students have to deal with. Similarly, the assessment tool is one more resource that the instructor has to configure, maintain and teach students to use, spending time and effort that could be used in other pedagogical activities. For this reason, several works have been conducted to integrate assessment tools for programming assignments in learning management systems (LMSs). The integration of assessment tools into LMSs promotes their adoption in computing courses, since they will be in agreement with the LMS already familiar to students and instructors, without the need to adopt, learn and manage the submission and correction of assignments in another environment. However, these works consist only of abstract conceptual models, do not propose integration with several LMSs, or do not address certain specific difficulties faced when integrating assessment tools. In this perspective, this work aims at the proposition of the IMPACTLE architecture, a solution that allows the integration of different assessment tools for programming assignments in LMSs. The idea is that students and teachers can access the features of the tools through the LMSs they are already used to, without the need to learn how to use and adopt a new tool. Architecture prototypes were instantiated and experiments were carried out involving the use of different assessment tools through LMSs. In general, we noticed that the use of IMPACTLE enables instructors and students perform tasks related to programming activities in a more efficient, efficacious and effective way through the LMSs.

Aim Develop an IT artifact that provides formative feedback for students based on their programming assignments. Background One of the best methods to learn programming is by practice. Providing feedback to students is an important and a valuable factor for improving learning, which plays a vital part in the student’s possibility to enhance and improve its solutions. Software development courses have several assignments and each course instructs about 100 students. To assess and provide feedback for all the students and each assignment demands considerable resources. In a survey conducted by TCO (2013) half of the respondents’ state that feedback is rarely or never given in reasonable time. Method Action Design Research (ADR) was used to intervene an organizational problem in parallel with building and evaluating an IT artifact. Conclusion The results from the study were four generated design principles and a proposed solution on how to use existing static code analysis tools for provide formative feedback to students.