This dissertation involves a holistic and interconnected examination of the validity, reliability, and fairness of the Online Research and Comprehension Assessment (ORCA). The ORCA is a large-scale assessment of New Literacies that challenges Grade 7 students to solve research problems (e.g., Does playing video games harm your eyes?) by locating, critically evaluating, and synthesizing online information in order to communicate their results in online genres such as email and wiki. My goal in this study was to understand how ORCA test score data should be used and interpreted, in what context, and for what purposes. A secondary goal was to examine the cognitive and metacognitive processes required to support research writing in online contexts. The study involved three interconnected phases. The first was a systematic, mixed methods literature review of 101 peer-reviewed texts from the last 50 years in order to articulate the construct underlying the ORCA. Finding no construct in the literature that considered the important ways in which the Internet has changed the construct of writing, I opted to conceptualize one of my own. This construct also serves as the theoretical framework for the rest of the dissertation. In the next phase of the study, I again explored the construct underlying the ORCA, but this time via a mixed methods investigation of the response processes—both cognitive and metacognitive—elicited by the ORCA. By observing both expert and novice participants’ response processes, I analyzed the extent to which the tasks and types of responses elicited by the ORCA fit the intended construct. Further, by observing response processes, I was also able to analyze construct underrepresentation and construct-irrelevant variance, which are fundamental to the ORCA’s appraisal. The results suggested that there are complex and sophisticated cognitive and metacognitive processes underlying the ORCA and online research writing more generally, many of which are unique to online contexts. Further, both quantitative and qualitative results suggest significant differences between novice and expert groups. The third phase of this research concludes with an integrated consideration of the ORCA’s validity, reliability, and fairness. Here, I analyzed data collected from the previous two phases; previous validation work done on the ORCA by my colleagues; and new forms of validation evidence collected for this study. I did so in order to build a comprehensive validity argument to demonstrate the ways in which ORCA test scores should be used and interpreted, and the consequences which follow. I used cued retrospective reporting, semi-structured interviews, Venn diagrams, surveys, and writing artefacts to investigate the response processes elicited by the ORCA and to compare and contrast those to the writing practices that participants used in their school, work, and/or personal lives. I also completed an extensive analysis of the sample of observations permitted by the ORCA juxtaposing those with the target domain. Results of this study indicate that the ORCA provides an important form of assessment data regarding 21st century literacies previously neglected on traditional assessments. Limitations of the ORCA such as construct-irrelevant variance and construct underrepresentation are also explored. The results of the study suggest how the ORCA could be re-designed to improve the validity of inferences made.

This portfolio contains three separate research essays and a reflection that are related through a common theme, that of ethos. The research papers include: (1) an essay on the pedagogical value of Wikipedia when teaching students about developing authorial ethos; (2) a research essay on the value of writing centers employing tutors whose writing is not yet proficient in the standards set for academic discourse; and (3) a research essay on the use of blogs in first-year composition courses as a means of fostering agency, ownership of ideas, audience awareness, and metadiscourse use. These papers represent the variety of research I undertook during the M.A. program as well as my research interests moving toward the future.

Syftet med denna kvalitativa forskningsstudie är att undersöka om och hur lärare/speciallärare inom grundskolans högstadium samt gymnasium upplever att de arbetar med att stärka elevens metakognitiva förmåga i den specialpedagogiska matematikundervisningen. Vi har i studien valt frågeställningarna: 1. Hur ser lärare/speciallärare på begreppet metakognition och inom vilka områden i matematiken blir denne varse elevens metakognitiva förmåga? 2. Hur upplever läraren/specialläraren att denne arbetar med elevens metakognitiva förmåga och upplevs denna förmåga olika hos elever med olika funktionsvariationer? 3. Finns det några framgångsfaktorer i det pedagogiska arbetet med elevernas metakognitiva förmåga inom matematiken? Denna studie omfattar åtta intervjuer med speciallärare/lärare, verksamma på högstadiet och gymnasiet, vilka ger särskilt stöd inom matematik. Vi vill undersöka hur de i sitt arbete stärker elevens metakognitiva förmåga (tankar om tänkande). Intervjuerna var cirka 45 minuter långa och materialet transkriberades samt analyserades för att kunna urskilja passager och kategorier. Studien har en kvalitativ ansats med induktiv analysmetod och teorin grundar sig på det socialkonstruktivistiska perspektivet - som menar att kunskapstillväxten är individuell men sker i ett socialt sammanhang. Resultatet visar att informanterna ser metakognition som en inre utveckling gentemot självständighet och en medvetenhet om sin egna lärprocess, en förmåga att använda ändamålsenliga strategier för att föra tanken framåt. Informanterna uttrycker att de blir varse elevens metakognition i matematisk problemlösning samt i situationer där eleverna skall prata matematik. Elever anses generellt ha svårt med metakognitiva strategier och för elever inom olika funktionsvariationer upplevs det vara en stor spridning av den metakognitiva förmågan. Elever med arbetsminnesproblematik kan ha extra svårt med de metakognitiva strategierna. Däremot upplever informanter att elever med ADHD kan påvisa en mer utvecklad metakognitiv förmåga, än elever med AST, då elever med ADHD har en självinsikt i sin diagnos samt vad den innebär. Några av de framgångsfaktorer som studien påvisar är ett medvetet arbete med synliga mål, möjlighet till reflektion, arbete med metakognitiva strategier, motiverande samtal samt matematiska samtal. Informanterna beskriver att de arbetar med den metakognitiva förmågan dagligen, men uttryckte däremot att de inte alltid arbetar med det medvetet.

Collaboration is an important yet difficult skill for learners in the 21st century. Recent research has examined how supports, such as group scripts and roles, can help learners collaborate effectively. However, little is known about the perceptions learners have of these supports when provided to them. The purpose of this study was to examine (a) learner’s choices and reasonings for role choices and (b) the impact of group roles. Participants included 111 undergraduate students in a learning strategies for university success course. During the course, students completed two required collaborative tasks, including pre-task planning and a post-task reflection. Students made choices regarding roles in individual and group planning sessions and explained their reasoning for making those choices. Students frequently chose roles relating to strategic task enactment, motivation, and concept/domain knowledge in their individual and group planning sessions with their primary reasons being focused on “self” knowledge, or information about themselves, such as strengths or weaknesses. These findings suggest learners may believe these are important roles for collaboration. The reliance on “self” knowledge for making decisions may be attributed to the fact these groups were collaborating for the first and second times and may not have a plethora of group information to utilize. Contributions of this study to theory, research, and practice will be discussed. / Graduate

The purposes of this study are to determine the level of growth by reading improvement students when metacognitive skills are taught and to determine which instructional approach is the most effective in maximizing reading comprehension.

Doctor of Philosophy / Curriculum and Instruction Programs / F. Todd Goodson / The purpose for this study is to look at the effects of a specific comprehension strategy—Reciprocal Teaching (RT; Palincsar & Brown, 1983,1984) on the performance of intermediate elementary students in the intermediate elementary grades on a standardized literacy assessment. The study was carried out in two implementation stages: a pilot with a small sample of fourth graders and a full-scale study with third, fourth, and fifth graders in a suburban public school district in the Midwest. The pilot study was implemented in two fourth grade classrooms, a total of 39 students, nine of which met the criterial for the experimental group who scored below the 25th percentile on an AIMSweb Maze 2016 fall benchmark. The full-scale study consisted of 269 students in grades three, four, and five across the same district, except for the fourth graders who previously were involved in the pilot. There were 71 students who met the criteria for the experimental group of scores below the 25th percentile on an AIMSweb Maze 2016 winter benchmark; however, after signed consent forms were returned, 59 participated. The students in the experimental group participated in intervention utilizing RT for approximately four weeks in Tier II small groups, instructed by classroom teachers at each elementary school. Regression discontinuity was used to determine the effect of RT on the students’ comprehension as indicated by their spring benchmark scores. The results of the pilot study showed a significant main effect in the scores along the regression line, indicating a positive effect for the intervention. Regression discontinuity analysis between groups indicated that a significant main effect exists for the intervention group. The full-scale study examined the effects per grade level for all students and with a set caliper for students whose scores were four points above and below the cut-off point, or winter 2016 benchmark, for the grade level. The results of the full-scale study focusing on all scores showed a significant main effect in the scores along the regression line, indicating a positive effect for the intervention for all grades; the effect was significant for all grades, but mostly for fourth and fifth graders. Regression discontinuity analysis between groups indicated that a significant main effect exists for the intervention group. When looking at scores within the caliper of four points above or below the benchmark, all scores showed a significant main effect in the scores along the regression line, indicating a positive effect for the intervention for all grades. Regression discontinuity analysis between groups indicated that a significant main effect exists for the intervention group within the caliper at each grade level.

This study investigated the conditions for developing science teachers’ Technological Pedagogical Content Knowledge (TPACK). It also explored the opportunities offered by two strategies to enhance science teachers’ ability to design technology-based inquiry activities for science learning: Experiencing Inquiry Model (EIM) and Metacognitive Scaffolding (MS). These strategies were adopted to support the processing necessary for developing teachers’ knowledge and for negotiating the integration of computer technology in science instruction. Situated Cognition Theory was used as a theoretical framework for learning, and TPACK was used as a conceptual framework for technology integration. 33 science teachers from four intermediate and high schools participated in the study. 17 and 16 teachers were conveniently assigned to EIM and MS, respectively. The study employed a mixed method of quantitative and qualitative evidence. As per the quantitative method, a quasi-experimental design that employed the 2 Teaching Strategy (EIM or MS) × 2 Time (pre- and post-intervention) of learning split-plot factorial design was applied in the study. Concurrently with the quantitative data collection, the qualitative evidence was collected from the researcher’s logbook, participants’ written documents, and interviews. The findings suggested that there were no significant differences between EIM and MS for developing the knowledge components embodied in TPACK. Nevertheless, the participants who learned through the MS strategy outperformed their counterparts in designing technology-based inquiry activities for science learning. The latter result suggested that teachers who received metacognitive scaffolding were more equipped to connect curriculum goals with technology and instruction.

Thesis advisor: Elida V. Laski / Strategy selection is an essential aspect of problem-solving, particularly within the domain of mathematics. This dissertation examines the mechanisms that guide children’s arithmetic strategy selection in order to advance theoretical understanding of this essential component of cognitive development. Better understanding of arithmetic strategy selection is important because individual differences in children’s arithmetic strategies are predictive of arithmetic accuracy and later math achievement. The current study builds upon prior research that has identified cognitive processes associated with strategy selection by considering the role of metacognitive judgments. The study investigated the direct and indirect effects of cognitive and metacognitive factors on strategy selection in a group of first and second grade students (n = 126). The majority of students generated metacognitive judgments of their competence using decomposition (an advanced strategy at this age) that were consistent with their actual ability. In these cases, their judgments of competence were related to the frequency with which they used decomposition strategies. Additionally, children’s metacognitive judgments of the anticipated amount effort required to execute decomposition mediated the association between children’s cognitive processes/pre-requisite knowledge (working memory and fact fluency) and the frequency with which they used it. Theoretical and practical implications of these findings are discussed. / Thesis (PhD) — Boston College, 2019. / Submitted to: Boston College. Lynch School of Education. / Discipline: Counseling, Developmental and Educational Psychology.

Indiana University-Purdue University Indianapolis (IUPUI) / Background: People with schizophrenia experience significant deficits in empathic skills, which are important for effective interpersonal relationships. Researchers have speculated about the roles of personal distress, emotion regulation, and metacognition in empathic interaction, but the impact of these constructs on empathy has yet to be empirically investigated. This study examines the relationships among these constructs in a sample of people with schizophrenia receiving community-based treatment (N = 58). It was hypothesized that better emotion regulation and metacognition, as well as reduced personal distress, would predict empathy. Further, emotion regulation was expected to mediate the relationship between personal distress and empathy, and metacognition was expected to moderate the relationship between personal distress and empathy. Method: Participants with schizophrenia or schizoaffective disorder completed self-report questionnaires of emotion regulation and personal distress, a performance-based measure of empathy, and an observer-rated interview to assess metacognition. Results: Metacognition, but not emotion regulation or personal distress, significantly predicted cognitive empathy performance, with a trend-level association for affective empathy performance. Mediation analyses revealed that emotion regulation mediates the relationship between personal distress and affective empathy performance, and moderation analyses revealed that metacognition moderates the same relationship. Moderation results suggest the relationship between personal distress and affective empathy performance is significant for those with low metacognition, but that the relationship is the opposite of hypotheses – increased personal distress is associated with better performance. Conclusions: This study is the first of its kind to examine performance-based empathy with personal distress, emotion regulation, and metacognition. Results suggest interventions targeted to improve metacognition may be useful in enhancing empathic skills. Future work is needed to improve existing measures of empathy and personal distress, and to parse apart the intricacies of the relationships among personal distress, emotion regulation, and empathy.

Students in Saudi Arabia need opportunities to develop higher-order thinking and problem-solving skills, but teachers typically rely upon knowledge transmission in face-to-face teaching. Flipped learning (FL), which integrates the features of online and face-to-face learning, has demonstrated potential for increasing higher-level learning. However, learners may not be able to manage FL activities on their own, such as by watching short videos outside the classroom, and thus might not be well prepared for in-class activities. Therefore, the self-regulated flipped learning (SRFL) approach seeks to address this challenge by integrating self-regulated learning (SRL) into the FL environment.