In der vorliegenden Synopse werden drei veröffentliche Publikationen gebündelt. Die Experimente tragen zur aktuellen wissenschaftlichen Diskussion über die positiven oder negativen Auswirkungen sozialer Entitäten und sozialer Hinweisreize in digitalen Lernmedien bei. Zunächst werden die theoretischen Grundlagen zusammengefasst. Weiterhin werden die Methoden, die zentralen Ergebnisse und auch die Implikationen und Limitationen diskutiert. Die bisherige und vor allem aktuelle Literatur zu diesem Forschungsschwerpunkt hat inkonsistente Ergebnisse hervorgebracht (z.B., Dinçer & Doğanay, 2017; Mayer & DaPra, 2012; Stull et al., 2018; Wang et al., 2018; Wilson, 2018). Klassische kognitive Lerntheorien wie die Cognitive Load Theory (CLT; Sweller, 1994; Kalyuga, & Singh, 2016) und die Cognitive Theory of Multimedia Learning (Mayer, 2014a) postulieren einen negativen Einfluss zusätzlich implementierter sozialer Entitäten auf Lernprozesse. Zusätzliches visuelles Material muss zwangsläufig zusätzlich zu dem eigentlichen Lernmaterial verarbeitet werden, wodurch eine kognitive Überlastung resultieren kann (Sweller et al., 2011). Soziale Entitäten wirken somit tendenziell ablenkend und sollten nicht implementiert werden. Andere Theorien, wie die Social Agency Theory (Mayer et al., 2003) oder die Cognitive-Affective-Social Theory of Learning with Media (CASTLM; Schneider et al., 2018b) postulieren im Gegensatz positive Einflüsse sozialer Entitäten und Hinweiszeichen auf Lernprozesse. Verfügbare kognitive Ressourcen werden nicht automatisch zur Schemakonstruktion genutzt (Huk & Ludwigs, 2009). Stattdessen aktivieren erst affektive, motivationale und auch soziale Prozesse diese Ressourcen. Soziale Entitäten können somit lernförderlich implementiert werden, da durch Prozesse der sozialen Identifikation, parasozialen Kommunikation sowie der affektiven und motivationalen Stimulation, Lernprozesse positiv beeinflusst werden können. Trotz dieser inkonsistenten Befundlage sind soziale Entitäten in zahlreichen Lernmedien integriert. Sichtbare Dozierende befinden sich in vielen Massive Open Online Courses (MOOCs; z.B., Hernandez et al., 2013). Zusätzlich zu Lehrvideos sind soziale Entitäten, wie beispielsweise pädagogische Agenten in interaktiven Lernmedien enthalten (Lin et al., 2013). Neben vollständigen Figuren, sind simplere soziale Hinweiszeichen, wie beispielsweise sichtbare Hände vor allem in Demonstrationsvideos integriert (Fiorella et al., 2017). Neben den empirischen Inkonsistenten besteht somit allerdings eine große praktische Relevanz. Mit experimentellen Untersuchungen muss der Forschungsschwerpunkt aufgegriffen und weiter ausdifferenziert werden. Die inkludierten quantitativ-experimentellen Studien werden in Tabelle 1 zusammengefasst. Im Weiteren wird die Methodik der Untersuchungen kurz zusammengefasst. Hierzu werden Probanden, Designs der Studien, Materialen und Messverfahren der abhängigen Variablen skizziert. Wie auch bei den anschließenden Ergebnissen ist die Beschreibung stark komprimiert, um Redundanzen mit den enthaltenen Artikeln zu vermeiden. So werden in der Synopse jene relevanten Kernergebnisse präsentiert, welche später im Fokus der Diskussion stehen. Neben Lernergebnissen werden hierbei auch kognitive, soziale und affektive Variablen untersucht, um einen detaillierten Einblick in den Lernprozess zu erlangen. Daraus leiten sich die folgenden Implikationen, Limitation und zukünftigen Forschungsprozesse ab. Es konnte experimentell festgestellt werden, dass durch gezielte Veränderungen des Designs, der Präsentation oder des Verhaltens von sozialen Entitäten Lernprozesse gefördert werden können. Dies spiegelt sich nicht nur in den Lernergebnissen wieder, auch Prozessvariablen wie bestimmte Facetten der parasozialen Interaktion konnten gezielt beeinflusst werden. Somit konnten Grundannahmen und Implikationen der Social Agency Theory und CASTLM experimentell gestützt werden. Die Wirkmechanismen sozialer Hinweisreize in Instruktionsvideos konnten herausgearbeitet werden und somit wurde durch theoriegeleitete experimentelle Methodik ein essentieller Beitrag zur gegenwärtigen Diskussion bezüglich sozialer Entitäten in multimedialen Lernumgebungen erbracht. Darüber hinaus können aus den Untersuchungen unmittelbare praktische Implikationen zur Gestaltung sozialer Entitäten abgeleitet werden. Die vorliegende Synopse zielte darauf ab zu untersuchen, wie soziale Entitäten möglichst effektiv in Lernumgebungen integriert werden können. Sie untersucht allerdings nicht ob eine grundlegende Integration sinnvoll ist, da keine Kontrollgruppen ohne Entitäten untersucht wurden. Diese bisherige Einschränkung ist eine Forschungsherausforderung gegenwärtiger und zukünftiger Projekte des Autors. Weiterhin sind die einbezogenen Studien der Synopse Grundlage weiterer Forschungsprojekte zu Effekten von Perspektive, Stimme und Verhaltensweisen von Personen und pädagogischen Agenten in Lehr-/Lernszenarios.

For teachers and students to be able to make informed decisions about how to best improve learning, it is important to compare learning strategies that are known to be effective. Both multimedia learning, based on the notion that individuals learn better from words and pictures presented together than from words alone, and retrieval practice, based on the idea that retrieving knowledge from the memory is an active process that has a beneficial impact on learning, have been found robust learning strategies in earlier research. However, the two strategies remain to be investigated in combination. The combination of the two seemingly robust strategies was investigated in Study I and results showed a modest effect of retrieval practice in terms of decreased forgetting and a strong effect of multimedia learning. Retrieval practice did not improve memory performance beyond the beneficial effect of using a visual illustration. Study II investigated the beneficial effects of the use of visual illustrations in more detail in terms of preferred learning style (visual, verbal or mixed), a notion that has reached wide popular ac- ceptance within the educational field. Support was not found for the learning styles hypothesis. Rather, results showed that the positive effects of learning with the aid of a visual illustration holds independently of preferred learning style, which renders strong support for multimedia learning in terms of its generalizability. Most interestingly, students with mixed or visual learning styles performed generally better on the learning test than students with a verbal learning style, which may imply that it is worthwhile to help students develop a preference for visual or multimodal aspects of information pro- cessing in order to further improve learning. The findings presented in this thesis provide new knowledge regarding the combination of learning strate- gies and contribute with important insights into the relation between learning style and the use of visual illustrations in psychology teaching. The findings also pose challenges for students and teachers, as well as people designing learning materials, concerning how to approach the use of visual illustrations and retrieval practice in teaching and learning.

No description available.

There is an increasing interest in using video games as a means to deliver training to individuals learning new skills or tasks. However, current research lacks a clear method of developing effective instructional material when these games are used as training tools and explaining how gameplay may affect learning. The literature contains multiple approaches to training and GBT but generally lacks a foundational-level and theoretically relevant approach to how people learn specifically from video games and how to design instructional guidance within these gaming environments. This study investigated instructional delivery within GBT. Video games are a form of multimedia, consisting of both imagery and sounds. The Cognitive Theory of Multimedia Learning (CTML; Mayer 2005) explicitly describes how people learn from multimedia information, consisting of a combination of narration (words) and animation (pictures). This study empirically examined the effects of the modality and temporal contiguity principles on learning in a game-based virtual environment. Based on these principles, it was hypothesized that receiving either voice or embedded training would result in better performance on learning measures. Additionally, receiving a combination of voice and embedded training would lead to better performance on learning measures than all other instructional conditions. A total of 128 participants received training on the role and procedures related to the combat lifesaver - a non-medical soldier who receives additional training on combat-relevant lifesaving medical procedures. Training sessions involved an instructional presentation manipulated along the modality (voice or text) and temporal contiguity (embedded in the game or presented before gameplay) principles. Instructional delivery was manipulated in a 2x2 between-subjects design with four instructional conditions: Upfront-Voice, Upfront-Text, Embedded-Voice, and Embedded-Text. Results indicated that: (1) upfront instruction led to significantly better retention performance than embedded instructional regardless of delivery modality; (2) receiving voice-based instruction led to better transfer performance than text-based instruction regardless of presentation timing; (3) no differences in performance were observed on the simple application test between any instructional conditions; and (4) a significant interaction of modality-by-temporal contiguity was obtained. Simple effects analysis indicated differing effects along modality within the embedded instruction group, with voice recipients performing better than text (p = .012). Individual group comparisons revealed that the upfront-voice group performed better on retention than both embedded groups (p = .006), the embedded-voice group performed better on transfer than the upfront text group (p = .002), and the embedded-voice group performed better on the complex application test than the embedded-text group (p =.012). Findings indicated partial support for the application of the modality and temporal contiguity principles of CTML in interactive GBT. Combining gameplay (i.e., practice) with instructional presentation both helps and hinders working memory's ability to process information. Findings also explain how expanding CTML into game-based training may fundamentally change how a person processes information as a function of the specific type of knowledge being taught. Results will drive future systematic research to test and determine the most effective means of designing instruction for interactive GBT. Further theoretical and practical implications will be discussed.

No description available.

Multimedia theory has generated a number of principles and guidelines to support computer-based training (CBT) design. However, the cognitive processes responsible for learning, from which these principles and guidelines stem from, are only indirectly derived by focusing on cognitive learning outcome differences. Unfortunately, the effects that cognitive processes have on learning are based on the assumption that cognitive learning outcomes are indicative of certain cognitive processes. Such circular reasoning is what prompted this dissertation. Specifically, this dissertation looked at the notion of referential connections, which is a prevalent cognitive process that is thought to support knowledge acquisition in a multimedia CBT environment. Referential connections, and the related cognitive mechanisms supporting them, are responsible for creating associations between verbal and visual information; as a result, their impact on multimedia learning is theorized to be far reaching. Therefore, one of the main goals of this dissertation was to address the issue of indirectly assessing cognitive processes by directly measuring referential connections to (a) verify the presence of referential connections, and (b) to measure the extent to which referential connections affect cognitive learning outcomes. To achieve this goal, a complete review of the prevalent multimedia theories was brought fourth. The most important factors thought to be influencing referential connections were extracted and cataloged into variables that were manipulated, fixed, covaried, or randomized to empirically examine the link between referential connections and learning. Specifically, this dissertation manipulated referential connections by varying the temporal presentation of modalities and the color coding of instructional material. Manipulating the temporal presentation of modalities was achieved by either presenting modalities simultaneously or sequentially. Color coding manipulations capitalized on pre-attentive highlighting and pairing of elements (i.e., pairing text with corresponding visuals). As such, the computer-based training varied color coding on three levels: absence of color coding, color coding without pairing text and corresponding visual aids, and color coding that also paired text and corresponding visual aids. The modalities employed in the experiment were written text and static visual aids, and the computer-based training taught the principles of flight to naive participants. Furthermore, verbal and spatial aptitudes were used as covariates, as they consistently showed to affect learning. Overall, the manipulations were hypothesized to differentially affect referential connections and cognitive learning outcomes, thereby altering cognitive learning outcomes. Specifically, training with simultaneously presented modalities was hypothesized to be superior, in terms of referential connections and learning performance, to a successive presentation, and color coding modalities with pairing of verbal and visual correspondents was hypothesized to be superior to other forms of color coding. Finally, it was also hypothesized that referential connections would positively correlate with cognitive learning outcomes and, indeed, mediate the effects of temporal contiguity and color coding on learning. A total of 96 were randomly assigned to one of the six experimental groups, and were trained on the principles of flight. The key construct of referential connections was successfully measured with three methods. Cognitive learning outcomes were captured by a traditional declarative test and by two integrative (i.e., knowledge application) tests. Results showed that the two multimedia manipulation impacted cognitive learning outcomes and did so through corresponding changes of related referential connections (i.e., through mediation). Specifically, as predicted, referential connections mediated the impact of both temporal contiguity and color coding on lower- and higher-level cognitive learning outcomes. Theoretical and practical implications of the results are discussed in relation to computer-based training design principles and guidelines. Specifically, theoretical implications focus on the contribution that referential connections have on multimedia learning theory, and practical implications are brought forth in terms of instructional design issues. Future research considerations are described as they relate to further exploring the role of referential connections within multimedia CBT paradigms.

No description available.

Complementing the theoretical concepts taught in the classroom with practice has been known to enhance students' contextual understanding of the subject matter. Exposing students to practical knowledge is crucial as employers are expressing discontent with the skills of newly hired graduates. In construction education, site visits have been identified as one of the most effective tools to support theory with practice. While site visits allow students to observe construction projects and engage with field personnel, numerous barriers limit its use as an effective educational tool. For instance, there are safety, cost, schedule, and weather constraints, in addition to the logistics of accommodating large class sizes. As a result, instructors employ videos of construction projects as an alternative to physical site visits. However, videos alone are insufficient to draw students' attention to essential practice concepts. Annotations can be used to attract students' attention to practical knowledge while reducing distractions and assumptions. Leveraging on the recent progress in computer vision techniques, this study presents an AI-annotated video learning tool that instructors can utilize to equip students with practice knowledge when there is limited access to physical construction sites. First, this study investigated the construction practice concepts that industry practitioners would want students to know when engaging them in site visits. Afterward, the design and development of the AI-annotated learning tool were guided by the identified practice concepts, cognitive theory of multimedia learning, and dual coding theory. To determine if the learning tool can call students' attention to annotated practice concepts in videos, a usability evaluation was conducted. Finally, this research investigated the influence of individual differences that could contribute to how learners notice practice concepts in videos. This study contributes to the body of knowledge by identifying what construction professionals notice about their work and what they would like students to notice about construction practice. This study reveals that annotations of learning contents in construction videos can direct students' focus to the annotated contents, thereby contributing to the cognitive theory of multimedia learning and dual coding theory. By leveraging machine learning classification algorithms, this research identified the extent to which individual differences such as gender, academic program, and cognitive load can be detected from the ways students notice information in construction videos. Results from this research provide opportunities for researchers to further advance the potential of annotated videos in the construction domain and other fields that employ video as a learning tool. / Doctor of Philosophy / Instructors often support classroom teaching with practical experiences to enhance students' understanding. This is especially important as employers are expressing discontent with the skills of fresh graduates. In construction engineering education, taking students to construction sites to observe the processes and operations is one of the common ways of providing students with these practical experiences. However, barriers such as safety concerns, cost, schedule, weather constraints, and the logistics of accommodating large class sizes make it challenging to engage students in construction site visits. Owing to these barriers, instructors utilize construction site videos instead of physical site visit experiences. Despite the benefits of using videos to teach, research has shown that presenting videos only to students might not be sufficient for learning as relevant and irrelevant information are usually present in videos. Therefore, calling out relevant information in videos would enable students to focus on them, enhancing their learning. To this end, this study presented a video-based learning tool that instructors can utilize to provide students with site visit experiences. In the environment, important information are called out using boundary boxes and texts. To achieve this, first, the study identified the practical knowledge that industry experts would want students to know about construction sites. Then, the identified information was annotated in construction videos via the guidance of learning theories such as the cognitive theory of multimedia learning and dual coding theory. A usability evaluation was conducted to test if students could notice the annotated contents in the video. Afterward, individual differences such as gender, academic program, and mental workload that could contribute to how students would notice annotated information in construction videos were investigated. The study contributes to the practical concepts learners need to acquire to prepare them for the workforce. Additionally, this study proved that annotating important information in videos can direct student attention to those contents. Furthermore, to make learning environments flexible for different learners, this study identified the extent to which individual differences such as gender, academic programs, and cognitive loads can be recognized from the way learners notice annotated contents of videos. Finally, the outcomes of this study would make it possible for other researchers to further advance the potentiality of teaching with annotated videos in the construction domain and other related fields.

National dialogue and scholarly research illustrate the need for engaging science, math, technology, and engineering (STEM) innovations in K-12 environments, most importantly in low-income communities (President's Council of Advisors on Science and Technology, 2012). According to Educating the Engineer of 2020, "current curricular material does not portray STEM in ways that seem likely to excite the interest of students from a variety of ethnic and cultural backgrounds" (Phase, 2005). The National Educational Technology Plan of 2010 believes that one of the most powerful ways to transform and improve K-12 STEM education it to instill a culture of innovation by leveraging cutting edge technology (Polly et al., 2010). Augmented reality (AR) is an emerging and promising educational intervention that has the potential to engage students and transform their learning of STEM concepts. AR blends the real and virtual worlds by overlaying computer-generated content such as images, animations, and 3D models directly onto the student's view of the real world. Visual representations of STEM concepts using AR produce new educational learning opportunities, for example, allowing students to visualize abstract concepts and make them concrete (Radu, 2014). Although evidence suggests that learning can be enhanced by implementing AR in the classroom, it is important to take into account how students are processing AR content. Therefore, this research aims to examine the unique benefits and challenges of utilizing augmented reality (AR) as a supplemental learning technique to reinforce mathematical concepts while concurrently responding to students' cognitive demands. To examine and understand how cognitive demands affect students' information processing and creation of new knowledge, Mayer's Cognitive Theory of Multimedia Learning (CTML) is leveraged as a theoretical framework to ground the AR application and supporting research. Also, to enhance students' engagement, gamification was used to incorporate game elements (e.g. rewards and leaderboards) into the AR applications. This research applies gamification and CTML principles to tablet-based gamified learning AR (GLAR) applications as a supplemental tool to address three research objectives: (1) understanding the role of prior knowledge on cognitive performance, (2) examining if adherence to CTML principles applies to GLAR, and, (3) investigating the impact of cognitive style on cognitive performance. Each objective investigates how the inclusion of CTML in gamifying an AR experience influences students' perception of cognitive effects and how GLAR affects or enhances their ability to create new knowledge. Significant results from objective one suggest, (1) there were no differences between novice and experienced students' cognitive load, and, (2) novice students' content-based learning gains can be improved through interaction with GLAR. Objective two found that high adherence to CTML's principles was effective at (1) lowering students' cognitive load, and, (2) improving GLAR performance. The key findings of objective three are (1) there was no difference in FID students' cognitive load when voice and coherence were manipulated, and, (2) both FID and FD students had content-based learning gains after engagement with GLAR. The results of this research adds to the existing knowledge base for researchers, designers and practitioners to consider when creating gamified AR applications. Specifically, this research provides contributions to the field that include empirical evidence to suggest to what degree CTML is effective as an AR-based supplemental pedagogical tool for underrepresented students in southwest Virginia. And moreover, offers empirical data on the relationship between underrepresented students' perceived benefits of GLAR and it is impact on students' cognitive load. This research further offers recommendations as well as design considerations regarding the applicability of CTML when developing GLAR applications. / PHD / The purpose of this research is to examine the unique benefits and challenges of using augmented reality (AR) to reinforce underrepresented students’ math concepts while observing how their process information. Gamification and Mayer’s Cognitive Theory of Multimedia Learning (CTML) principles are applied to create tablet-based gamified learning AR (GLAR) applications to address three research objectives: (1) understanding the role of prior knowledge on cognitive performance, (2) examining if adherence to CTML principles applies to GLAR, and, (3) investigating the impact of cognitive style on cognitive performance. Each objective investigates how the inclusion of CTML in gamifying an AR experience influences students’ perception of cognitive effects and how GLAR affects or enhances their ability to create new knowledge. This research offers recommendations as well as design considerations regarding the applicability of CTML when developing GLAR applications for underrepresented students in southwest Virginia.

Submitted by Priscilla Araujo (priscilla@ibict.br) on 2017-06-20T18:23:12Z No. of bitstreams: 2 license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Dissertacao-KarlaCorvino-2017.pdf: 2233201 bytes, checksum: 3361f03e8e81044bf0068b50c3e6c6ca (MD5) / Made available in DSpace on 2017-06-20T18:23:12Z (GMT). No. of bitstreams: 2 license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Dissertacao-KarlaCorvino-2017.pdf: 2233201 bytes, checksum: 3361f03e8e81044bf0068b50c3e6c6ca (MD5) Previous issue date: 2017-03-06 / A tecnologia permite elaborar conteúdos instrucionais em diferentes suportes e formatos com o uso de áudios, textos, vídeos e animações, os quais influenciam distintamente na maneira como o ser humano interage com a informação e aprende. A compreensão dessas influências é importante para adaptar a tecnologia multimídia para melhorar a aprendizagem humana. A Teoria Cognitiva de Aprendizagem Multimídia de Richard Mayer apresenta princípios básicos para a elaboração de material instrucional multimídia visando a facilitar a aprendizagem, com base em como as pessoas aprendem. Estudos empíricos nas áreas de psicologia, educação, tecnologia e outras, têm buscado validar os princípios da teoria e testar suas implicações no processo de aprendizagem. O objetivo desta pesquisa é identificar critérios informacionais, com base na análise sistemática de estudos empíricos sobre a consistência dos princípios preconizados pela Teoria Cognitiva de Aprendizagem Multimídia, que possam ser utilizados na elaboração e na adaptação de material instrucional multimídia na web, de modo a maximizar a aprendizagem e a transferência cognitiva de conhecimento técnico especializado. A partir desses achados, material instrucional multimídia do programa Dia de Campo na TV é analisado e adaptações são sugeridas, para facilitar o processo de transferência de tecnologia e de conhecimento por meio da web. Por fim, uma lista de recomendações para subsidiar a criação de novos conteúdos instrucionais é oferecida, a fim de ampliar a efetividade do uso de recursos multimídia na promoção da aprendizagem e na difusão coletiva de conhecimento especializado. / Technology allows the development of instructional content in different supports and formats, including audio, text, video and animations, which affect the ways humans interact with information and learn. It is important to understand these influences in order to adapt multimedia technology to enhance human learning. Richard Mayer's Cognitive Theory of Multimedia Learning (CTML) presents principles to which multimedia instructional material should conform to facilitate learning, built upon an understanding of how people learn. Empirical studies in the areas of psychology, education, technology and others have sought to validate these principles and test their implications in the learning process. The aim of this research is to identify informational criteria, based upon a systematic review of the literature about the consistency of principles advocated by CTML, to be used in the creation and adaptation of multimedia instructional material on the Internet, in order to maximize learning experience and specialized knowledge cognitive transfer. From these findings, the multimedia instructional material of Field Day on TV program is analyzed and adaptations are suggested to improve the knowledge and technology transfer process throgh the web. Finally, a list of recommendations to support the creation of new instructional contents is organized to increase the effectiveness of the use of multimedia resources to enhance learning experience and specialized knowledge diffusion.