OpenDSA is a system for creating rich eTextbooks that combine quality text with visualizations and interactive, auto-graded exercises. As OpenDSA gains recognition, its use increases each year. This mandates a scalable, reliable, and sustainable infrastructure to accommodate the fast-growing demand for OpenDSA access. We implemented OpenDSA-LTI, an interoperable infrastructure which transforms OpenDSA from a standalone, self-contained eTextbook to an integrated learning tool communicating with a Learning Management System (LMS) through the Learning Tool Interoperability (LTI) protocol. OpenDSA-LTI delivers OpenDSA content and interactive materials to students through a reliable and secure LMS interface. LTI integration encourages OpenDSA adoption by providing easy, intuitive tools that help instructors to build and generate OpenDSA eTextbooks in their LMS courses. OpenDSA-LTI allows OpenDSA content developers to take advantage of various tools already provided by the LMS instead of reproducing these through their own proprietary services. The OpenDSA-LTI extendable design allows for adding new LTI-compliant exercises to OpenDSA books. This changes OpenDSA developers' efforts to searching for learning tools instead of reimplementing them. As an example, instead of maintaining the original OpenDSA programming evaluation engine, we could easily replace it with the Code Workout online drill-and-practice system. Since its launch in August 2016 until June 2017, OpenDSA-LTI has hosted 36 active courses offered by 25 different universities in 6 countries, 41 instructors have used OpenDSA-LTI to host their courses on the Canvas LMS, and the system has 2,729 registered students. / Master of Science

In recent years there has been considerable adoption of the IMS Learning Tools Interoperability (LTI) standard among both Learning Management Systems (LMS), and learning applications. The LTI standard defines a way to securely connect learning applications and tools with platforms like LMS, enabling content from external learning tools to appear as if it were a native part of the LMS, and enabling these learning tools to send users' scores directly to the gradebook in the LMS. An example of such a learning tool is the OpenDSA eTextbook system which provides materials that cover a variety of Computer Science-related topics, incorporating hundreds of interactive visualizations and auto-graded exercises. Previous work turned OpenDSA into an LTI tool provider, allowing for OpenDSA eTextbooks to be integrated with the Canvas LMS. In this thesis, we further explore the problem of connecting educational systems while documenting challenges, issues, and design rationales. We expand upon the existing OpenDSA LTI infrastructure by turning OpenDSA into an LTI tool consumer, thus enabling OpenDSA to better integrate content from other LTI tool providers. We also describe how we expanded OpenDSA's LTI tool provider functionality to increase the level of granularity at which OpenDSA content can be served, and how we implemented support for several LMS, including challenges faced and remaining issues. Finally, we discuss the problem of sharing analytics data among educational systems, and outline an architecture that could be used for this purpose. / Master of Science / In recent years there has been considerable adoption of the IMS Learning Tools Interoperability (LTI) standard among Learning Management Systems (LMS) like Blackboard and Canvas, and among learning tools. The LTI standard allows for learning tools to be securely connected with platforms like LMS, enabling content from external learning tools to appear as if it were built into the LMS, and enabling these learning tools to send users’ scores directly to the gradebook in the LMS. An example of such a learning tool is the OpenDSA online textbook system which provides materials that cover a variety of Computer Science-related topics, incorporating hundreds of interactive visualizations and auto-graded exercises. Previous work enabled OpenDSA textbooks to be connected with the Canvas LMS using LTI. In this thesis, we further explore the problem of connecting educational systems while documenting challenges, issues, and design rationales. We expand the existing OpenDSA system to allow OpenDSA to better integrate content from other learning tools. We also describe how we expanded OpenDSA’s features to increase number of ways that OpenDSA content can be consumed, and how we implemented support for adding OpenDSA content to several LMS, including challenges faced and remaining issues. Finally, we discuss the problem of sharing analytics data among educational systems, and outline a potential way to connect educational systems for this purpose.

Die vorliegende Bachelorarbeit befasst sich mit dem Learning Tools Interoperability Standard, kurz LTI Standard. Ziel ist es hierbei nicht nur den LTI Standard an sich in seiner Anwendung zu evaluieren sondern ebenso eine eigene Implementierung dieser Schnittstellendefinition zu erarbeiten. Um eine solche Implementierung zu erreichen wird sich Eingangs intensiv mit dem Standard in seinen verschiedenen Versionen auseinandergesetzt und anschließend der praktische Einsatz dieser Schnittstelle durch die Evaluation bestehender Kopplungsmöglichkeiten erprobt. Zur Evaluierung wird eine Auswahl von Lehrumgebungen und Lernanwendungen, bei welchen im Anschluss unter verschiedenen Gesichtspunkten die Kopplungen betrachtet und diskutiert werden.

Until recently, most educational tools have worked in silos. If a teacher wanted her students to complete small programming exercises, record videos, and collaborate through discussion boards, three disconnected tools were probably needed. Learning Tools Interoperability (LTI) is a communication protocol that enables different learning tools to talk to each other and share scores with a Learning Management System (LMS). While most commercial LMS now support LTI, most educational software developed by small research efforts do not. This is often because of the lack of resources needed to understand the working of LTI and the process of using LTI in their applications. Our aim is to encourage the use of LTI within the CS Education community. We have developed tutorials that include example applications. We also provide a use case of how LTI is implemented in the OpenDSA eTextbook system. As another use case, we have enabled auto-grading of Jupyter Notebook assignments by providing immediate feedback to students and updating scores to the Canvas gradebook. We provide a Jupyter plugin to upload notebook files to the Web-CAT auto-grading system. We integrate Aalto University's ACOS content into OpenDSA as a third use case. / Master of Science / Until recently, most educational tools have worked in silos. If a teacher wanted her students to complete small programming exercises, record videos, and collaborate through discussion boards, three disconnected tools were probably needed. These disconnected tools did not integrate with the Learning Management Systems (LMS), such as Canvas and Moodle. Instructors had to manually manage these separate tools and enter scores into the LMS. There are standards such as Learning Tools Interoperability (LTI) that these learning tools can implement to enable them to talk to each other and to share scores with an LMS. However, most educational software developed by small research efforts do not support LTI. This is often because of the lack of resources needed to understand the working of LTI and the process of using LTI in their applications. We aim to encourage the use of LTI within the CS Education community. We have developed tutorials that include example applications. We also provide a use case of how LTI is implemented in OpenDSA, an eTextbook system developed at Virginia Tech. As another use case, we have enabled auto-grading of Jupyter Notebook (documents that run in a browser and can contain equations, visualizations, live code, and text) assignments by providing immediate feedback to students and updating scores to the Canvas gradebook. We provide a plugin to upload notebook files to the WebCAT auto-grading system directly from the browser. We integrate Aalto University’s ACOS content (Python and Java exercises) into OpenDSA as a third use case.

Learning Management Systems (LMSs) are widely used in higher education to improve the learning, teaching, and administrative tasks for both students and instructors. Such systems enrich the educational experience by integrating a wide range of services, such as on-demand course material and training, thus empowering students to achieve their learning outcomes at their own pace. Courses in various sub-fields of Computer Science that seek to provide rich electronic learning (e-learning) experience depend on exercise material being offered in the forms of quizzes, programming exercises, laboratories, simulations, etc. Providing hands on experience in courses such as Internetworking could be facilitated by providing laboratory exercises based on virtual machine environments where the student studies the performance of different internet protocols under different conditions (such as different throughput bounds, error rates, and patterns of changes in these conditions). Unfortunately, the integration of such exercises and their tailored virtual environments is not yet very popular in LMSs. This thesis project investigates the generation of on-demand virtual exercise environments using cloud infrastructures and integration with an LMS to provide a rich e-learning in an Internetworking course. The software deliverable of this project enables instructors to dynamically instantiate virtual laboratories without incurring the overhead of running and maintaining their own physical infrastructure. This sets the foundations for a virtual classroom that can scale in response to higher system utilization during specific periods of the academic calendar. / Lärplattformar (eng. Learning Management Systems (LMS)) används i stor utsträckning för högre utbildning för att förbättra lärande, undervisning och administrativa uppgifter för både studenter och instruktörer. Sådana system berikar den pedagogiska erfarenheten genom att integrera ett brett utbud av tjänster, såsom on-demand kursmaterial och träning, vilket ger studenterna möjlighet att uppnå sina lärandemål i egen takt. Kurser inom olika delområden av datavetenskap som syftar till att ge en bred erfarenhet av elektroniskt lärande (e-learning) har träningsmaterial i form av frågesporter, programmeringsövningar, laboratorier, simuleringar etc. Praktiskt erfarenhet i kurser som Internetworking kan underlättas genom att tillhandahålla laboratorieövningar baserade på virtuella maskinmiljöer där studenten studerar prestanda för olika internetprotokoll under olika förhållanden (t.ex. olika gränsvärden, felfrekvenser och förändringsmönster under dessa förhållanden). Tyvärr är integrationen av sådana övningar och deras skräddarsydda virtuella miljöer ännu inte populär i LMSs. Detta examensarbete undersöker generering av virtuella träningsmiljöer på begäran med hjälp av molninfrastruktur och integration med en LMS för att ge ett rikt e-lärande i en Internetworking-kurs. Programvaran som levereras av detta projekt gör det möjligt för instruktörer att dynamiskt instansera virtuella laboratorier utan att behöva hantera sin egen fysiska infrastruktur. Detta sätter grunden för ett virtuellt klassrum som kan skala med högre systemutnyttjande under specifika perioder av den akademiska kalendern.

Whenever a student submits their final version of their thesis, a series of processes is triggered to finalize and archive the report. These processes are often handled in a less than efficient way which results in excessive manual labor and costs that can be prevent if automated. This report describes a solution that automates the series of processes that occur following a final thesis report submission. By utilizing the available information in a Canvas course and the content in the submitted thesis much of the manual cut-and-paste effort is avoided. Entering this data into DiVA is done by automated interaction via a browser, as DiVA does not have an application programming interface that could be used. The conclusion is that it is possible to automate this process through a headless browser. However, the automated parsing of the PDF version of the thesis proven to be inconsistent which results in the extracted data being inconsistent. With some improvements to the parsing module, the entire process could be fully automated. / Varje gång en student skickar in sin slutgiltiga version av sitt examensarbete, utlöses en serie av processer för att slutföra och arkivera examensarbetet. Dessa processer hanteras ofta på ett mindre än effektivt sätt vilket resulterar i extra mycket manuellt arbete och kostnader som kan förhindras ifall de automatiseras. Denna uppsats beskriver en lösning som automatiserar serien av processer som inträffar efter att ett slutgiltigt examensarbete har godkänts. Genom att använda tillgänglig information i en Canvas-kurs och innehållet i det inlämnade examensarbetet undviks mycket av den manuella ”klipp-och-klistra”-insatsen. Inmatning av den relevanta data från examensarbetet måste göras via automatiserad interaktion via en webbläsare i DiVA, eftersom DiVA inte hade ett API som kunde användas. Slutsatsen är att det är möjligt att automatisera detta genom en huvudlös webbläsare, även om modulen som behandlar PDF har visat sig vara inkonsekvent vilket i sin tur har resulterat i att den automatiska interaktionen är inkonsekvent. Med några optimeringar i analysmodulen kan hela processen automatiseras.

At KTH, students who want to start their degree project must complete a paper form called “UT-EXAR: Ansökan om examensarbete/application for degree project”. The form is used to determine students’ eligibility to start a degree project, as well as potential examiners for the project. After the form is filled in and signed by multiple parties, a student can initiate his or her degree project. However, due to the excessively time-consuming process of completing the form, an alternative solution was proposed: a survey in the Canvas Learning Management System (LMS) that replace s the UT-EXAR form. Although the survey reduces the time required by students to provide information and find examiners, it is by no means the most efficient solution. The survey suffers from multiple flaws, such as asking students to answer unnecessary questions, and for certain questions, presenting students with more alternatives than necessary. The survey also fails to automatically organize the data collected from the students’ answers; hence administrators must manually enter the data into a spreadsheet or other record. This thesis proposes an optimized solution to the problem by introducing a dynamic survey. Moreover, this dynamic survey uses the Canvas Representational State Transfer (REST) API to access students’ program-specific data. Additionally, this survey can use data provided by students when answering the survey questions to dynamically construct questions for each individual student as well as using information from other KTH systems to dynamically construct customized alternatives for each individual student. This solution effectively prevents the survey from presenting students with questions and choices that are irrelevant to their individual case. Furthermore, the proposed solution directly inserts the data collected from the students into a Canvas Gradebook. In order to implement and test the proposed solution, a version of the Canvas LMS was created by virtualizing each Canvas-based microservice inside of a Docker container and allowing the containers to communicate over a network. Furthermore, the survey itself used the Learning Tools Interoperability (LTI) standard. When testing the solution, it was seen that the survey has not only successfully managed to filter the questions and alternative answers based on the user’s data, but also showed great potential to be more efficient than a survey with statically-presented data. The survey effectively automates the insertion of the data into the gradebook. / På KTH, studenter som skall påbörja sitt examensarbete måste fylla i en blankett som kallas “UT-EXAR: Ansökan om examensarbete/application for degree project”. Blanketten används för att bestämma studenters behörighet för att göra examensarbete, samt potentiella examinator för projektet. Efter att blanketten är fylld och undertecknad av flera parter kan en student påbörja sitt examensarbete. Emellertid, på grund av den alltför tidskrävande processen med att fylla blanketten, var en alternativ lösning föreslås: en särskild undersökning i Canvas Lärplattform (eng. Learning Management System(LMS)) som fungerar som ersättare för UT-EXAR-formulär. Trots att undersökningen har lyckats minska den tid som krävs av studetenter för att ge information och hitta examinator, det är inte den mest effektiva lösningen. Undersökningen lider av flera brister, såsom att få studenterna att svara på fler frågor än vad som behövs, och för vissa frågor, presenterar studenter med fler svarsalternativ än nödvändigt. Undersökningen inte heller automatiskt med att organisera data som samlats in från studenters svar. Som ett resultat skulle en administratör behöva organisera data manuellt i ett kalkylblad. Detta examensarbete föreslår en mer optimerad lösning på problemet: omskrivning av undersökningens funktionaliteter för att använda Representational State Transfer(REST) API för att komma åt studenters programspecifika data i back-end, såväl att använda speciella haschar för att hålla referenser till uppgifter som lämnas av studenterna när de svarar på frågorna i undersökningen, så att undersökningen inte bara kan använda dessa data för att dynamiskt konstruera frågor för varje enskild student, men också dynamiskt konstruera svarsalternativ för varje enskild student. Denna lösning förhindrar effektivt undersökningen från att presentera studenter med frågor och valbara svarsalternativ som är helt irrelevanta för var och en av deras individuella fall. Med den föreslagna lösningen kommer undersökningen dessutom att kunna organisera de data som samlats in från Studenterna till ett speciellt Canvas-baserat kalkyllblad, kallas som Betygsbok. För att genomföra och testa den förslagna lösningen skapades en testbar version av Canvas LMS genom att virtualisera varje Canvas-baserad mikroservice inuti en dockercontainer och tillåter containers att kommunicera över ett nätverk. Dessutom var undersökningen själv konfigurerad för att använda Lärverktyg Interoperability (LTI) standard. Vid testning av lösningen, det visade sig att undersökningen på ett sätt effektivt har lyckats använda vissa uppgifter från en testanvändare att bara endast svara på de relevanta frågorna, men också presentera användaren med en mer kondenserad lista svarsalternativ över baserat på data.<p>

Indiana University-Purdue University Indianapolis (IUPUI) / The emergence of information exchange and act of offering features through external interfaces is a vast but immensely valuable challenge, and essential elements of learning environments cannot be excluded. Nowadays, there are a lot of different service providers working in the learning systems market and each of them has their own advantages. On that premise, in today's world even large learning management systems are trying to cooperate with each other in order to be best. For instance, Instructure is a substantial company and can easily employ a dedicated team tasked with the development of a video conferencing functionality, but it chooses to use an open source alternative instead: The BigBlueButton. Unfortunately, different learning system manufacturers are using different technologies for various reasons, making integration that much harder. Standards in learning environments have come to resolve problems regarding exchanging information, providing and consuming functionalities externally and simultaneously minimizing the amount of effort needed to integrate systems. In addition to defining and simplifying these standards, careful consideration is essential when designing new, comprehensive and useful systems, as well as adding interoperability to existing systems, all which subsequently took part in this research. In this research I have reviewed most of the standards and protocols for integration in learning environments and proposed a revised approach for app stores in learning environments. Finally, as a case study, a learning tool has been developed to avail essential functionalities of a social educational learning management system integrated with other learning management systems. This tool supports the dominant and most popular standards for interoperability and can be added to learning management systems within seconds.