Global ETD Search

1	Mathematical literacy assessment design : a dimensionality analysis of Programme for International Student Assessment (PISA) mathematics framework Ekmekci, Adem 26 September 2013 (has links) The National Research Council (NRC) outlines an assessment design framework in Knowing What Students Know. This framework proposes the integration of three components in assessment design that can be represented by a triangle, with each corner representing: cognition, or model of student learning in the domain; observation, or evidence of competencies; and interpretation, or making sense of this evidence. This triangle representation signifies the idea of a need for interconnectedness, consistency, and integrated development of the three elements, as opposed to having them as isolated from each other. Based on the recommendations for research outlined in the NRC's assessment report, this dissertation aims to conduct a dimensionality analysis of Programme for International Student Assessment (PISA) mathematics items. PISA assesses 15-year olds' skills and competencies in reading, math, and science literacy, implementing an assessment every three years since 2000. PISA's mathematics assessment framework, as proposed by the Organisation for Economic Co-operation and Development (OECD), has a multidimensional structure: content, processes, and context, each having three to four sub-dimensions. The goal of this dissertation is to show how and to what extent this complex multidimensional nature of assessment framework is reflected on the actual tests by investigating the dimensional structure of the PISA 2003, 2006, and 2009 mathematics items through the student responses from all participating OECD countries, and analyzing the correspondence between the mathematics framework and the actual items change over time through these three implementation cycles. Focusing on the cognition and interpretation components of the assessment triangle and the relationship between the two, the results provide evidence addressing construct validity of PISA mathematics assessment. Confirmatory factor analysis (CFA) and structural equation modeling (SEM) were used for a dimensionality analysis of the PISA mathematics items in three different cycles: 2003, 2006, and 2009. Seven CFA models including a unidimensional model, three correlated factor (1-level) models, and three higher order factor (2-level) models were applied to the PISA mathematics items for each cycle. Although the results did not contradict the multidimensionality, stronger evidence was found to support the unidimensionality of the PISA mathematics items. The findings also showed that the dimensional structure of the PISA mathematics items were very stable across different cycles. / text Mathematics Mathematical literacy Assessment Dimensionality Construct validity Confirmatory factor analysis Assessment triangle
2	Evaluation of a Novel Biochemistry Course-Based Undergraduate Research Experience (CURE) Stefan M Irby (6326255) 15 May 2019 (has links) <p>Course-based Undergraduate Research Experiences (CUREs) have been described in a range of educational contexts. Although various learning objectives, termed anticipated learning outcomes (ALOs) in this project, have been proposed, processes for identifying them may not be rigorous or well-documented, which can lead to inappropriate assessment and speculation about what students actually learn from CUREs. Additionally, evaluation of CUREs has primarily relied on student and instructor perception data rather than more reliable measures of learning.This dissertation investigated a novel biochemistry laboratory curriculum for a Course-based Undergraduate Research Experience (CURE) known as the Biochemistry Authentic Scientific Inquiry Lab (BASIL). Students participating in this CURE use a combination of computational and biochemical wet-lab techniques to elucidate the function of proteins of known structure but unknown function. The goal of the project was to evaluate the efficacy of the BASIL CURE curriculum for developing students’ research abilities across implementations. Towards achieving this goal, we addressed the following four research questions (RQs): <b>RQ1</b>) How can ALOs be rigorously identified for the BASIL CURE; <b>RQ2</b>) How can the identified ALOs be used to develop a matrix that characterizes the BASIL CURE; <b>RQ3</b>) What are students’ perceptions of their knowledge, confidence and competence regarding their abilities to perform the top-rated ALOs for this CURE; <b>RQ4</b>) What are appropriate assessments for student achievement of the identified ALOs and what is the nature of student learning, and related difficulties, developed by students during the BASIL CURE? To address these RQs, this project focused on the development and use of qualitative and quantitative methods guided by constructivism and situated cognition theoretical frameworks. Data was collected using a range of instruments including, content analysis, Qualtrics surveys, open-ended questions and interviews, in order to identify ALOs and to determine student learning for the BASIL CURE. Analysis of the qualitative data was through inductive coding guided by the concept-reasoning-mode (CRM) model and the assessment triangle, while analysis of quantitative data was done by using standard statistical techniques (e.g. conducting a parried t-test and effect size). The results led to the development of a novel method for identifying ALOs, namely a process for identifying course-based undergraduate research abilities (PICURA; RQ1; Irby, Pelaez, & Anderson 2018b). Application of PICURA to the BASIL CURE resulted in the identification and rating by instructors of a wide range of ALOs, termed course-based undergraduate research abilities (CURAs), which were formulated into a matrix (RQs 2; Irby, Pelaez, & Anderson, 2018a,). The matrix was, in turn, used to characterize the BASIL CURE and to inform the design of student assessments aimed at evaluating student development of the identified CURAs (RQs 4; Irby, Pelaez, & Anderson, 2018a). Preliminary findings from implementation of the open-ended assessments in a small case study of students, revealed a range of student competencies for selected top-rated CURAs as well as evidence for student difficulties (RQ4). In this way we were able to confirm that students are developing some of the ALOs as actual learning outcomes which we term VLOs or verified learning outcomes. In addition, a participant perception indicator (PPI) survey was used to gauge students’ perceptions of their gains in knowledge, experience, and confidence during the BASIL CURE and, therefore, to inform which CURAs should be specifically targeted for assessment in specific BASIL implementations (RQ3;). These results indicate that, across implementations of the CURE, students perceived significant gains with large effect sizes in their knowledge, experience, and confidence for items on the PPI survey (RQ3;). In our view, the results of this dissertation will make important contributions to the CURE literature, as well as to the biochemistry education and assessment literature in general. More specifically, it will significantly improve understanding of the nature of student learning from CUREs and how to identify ALOs and design assessments that reveal what students actually learn from such CUREs - an area where there has been a dearth of available knowledge in the past. The outcomes of this dissertation could also help instructors and administrators identify and align assessments with the actual features of a CURE (or courses in general), use the identified CURAs to ensure the material fits departmental or university needs, and evaluate the benefits of students participating in these innovative curricula. Future research will focus on expanding the development and validation of assessments so that practitioners can better evaluate the efficacy of their CUREs for developing the research competencies of their undergraduate students and continue to render improvements to their curricula.</p> Biochemistry Education Course-based Undergraduate Research Course-based Research Experiences CURE anticipated learning outcomes ALOs ALO CURAs research abilities PICURA ALO matrix BASIL weighted-relevance WR verified learning outcomes VLOs CUREs biochemistry computational biochemistry bioinformatics biochemistry education chemistry education chemistry teaching laboratory upper-division ACE-Bio experimental competencies Authentic science inquiry chemical education assessment assessment design Student understanding student difficulties biochemistry laboratory experiments biochemistry teaching laboratory chemistry teaching laboratory chemistry teaching laboratories predicting protein function protein inquiry-based learning inquiry-based teaching content analysis open-ended survey interviews semi-structured interviews PPI participant perception indicator assessment triangle CRM conceptual reasoning mode conceptual-reasoning-mode model curriculum evaluation experimental abilities discovery skills learning objectives learning outcomes LO LOs collaborations collaborative course implementation implementation student difficuluties experimental learning experimental literacy data-representation ACE-Bio Network Likert-survey

Search results

Mathematical literacy assessment design : a dimensionality analysis of Programme for International Student Assessment (PISA) mathematics framework

Evaluation of a Novel Biochemistry Course-Based Undergraduate Research Experience (CURE)