<p>LIU, FU, Ph.D. Comparisons of Subscoring Methods in Computerized Adaptive Testing: A Simulation Study. (2015) Directed by Dr. Terry A. Ackerman. 187 pp. Given the increasing demands of subscore reports, various subscoring methods and augmentation techniques have been developed aiming to improve the subscore estimates, but few studies have been conducted to systematically compare these methods under the framework of computerized adaptive tests (CAT). This research conducts a simulation study, for the purpose of comparing five subscoring methods on score estimation under variable simulated CAT conditions. Among the five subscoring methods, the IND-UCAT scoring ignores the correlations among subtests, whereas the other four correlation-based scoring methods (SEQ-CAT, PC-MCAT, reSEQ-CAT, and AUG-CAT) capitalize on the correlation information in the scoring procedure. By manipulating the sublengths, the correlation structures, and the item selection algorithms, more comparable, pragmatic, and systematic testing scenarios are created for comparison purposes. Also, to make the best of the sources underlying the assessments, the study proposes a successive scoring procedure according to the structure of the higher-order IRT model, in which the test total score of individual examinees can be calculated after the subscore estimation procedure is conducted. Through the successive scoring procedure, the subscores and the total score of an examinee can be sequentially derived from one test. The results of the study indicate that in the low correlation structure, the original IND-CAT is suggested for subscore estimation considering the ease of implementation in practice, while the suggested total score estimation procedure is not recommended given the large divergences from the true total scores. For the mixed correlation structure with two moderate correlations and one strong correlation, the original SEQ-CAT or the combination of the SEQ-CAT item selection and the PC-MCAT scoring should be considered not only for subscore estimation but also for total score estimation. If the post-hoc estimation procedure is allowed, the original SEQ-CAT and the reSEQ-CAT scoring could be jointly conducted for the best score estimates. In the high correlation structure, the original PC-MCAT and the combination of the PC-MCAT scoring and the SEQ-CAT item selection are suggested for both the subscore estimation and the total score estimation. In terms of the post-hoc score estimation, the reSEQ-CAT scoring in conjunction with the original SEQ-CAT is strongly recommended. If the complexity of the implementation is an issue in practice, the reSEQ-CAT scoring jointly conducted with the original IND-UCAT could be considered for reasonable score estimates. Additionally, to compensate for the constrained use of item pools in PC-MCAT, the PC-MCAT with adaptively sequencing subtests (SEQ-MCAT) is proposed for future investigations. The simplifications of item and/or subtest selection criteria in a simple-structure MCAT, PC-MCAT, and SEQ-MCAT are also pointed out for the convenience of their applications in practice. Last, the limitations of the study are discussed and the directions for future studies are also provided.

<p> The general areas of research for this study are personality and second language acquisition. The three goals of this study are to (a) develop a personality instrument (the Questionnaire of English Environment Personality [QuEEP]) that accounts for second language influences on personality, and more effectively captures personality than an established personality instrument (the International Personality Item Pool Big Five Factor Markers [IPIP BFFM]), (b) determine if personality changes after studying abroad for a month, and (c) determine if certain personality types are likely to improve oral proficiency when studying abroad.</p><p> In relation to the study&rsquo;s first goal, 262-items, using a five-point Likert scale, were created and administered to 287 Japanese university students to measure five personality factors based on the extraversion, emotional stability, openness, agreeableness, and conscientiousness factors of the Big Five model of personality (McCrae &amp; Costa, 1987). These items were then culled to 50 items by examining their suitability through factor analysis and Rasch analysis. Two 50-item versions of the QuEEP were drawn from the same 262-items, one based on three factor analyses, and the other based on Rasch analysis. Both versions of the QuEEP included 10 items for each of the five personality factors in the Big Five. Both versions of the QuEEP outperformed the IPIP BFFM on four measures of validity, including content validity, structural validity, external validity, and generalizability, while the IPIP BFFM outperformed both versions of the QuEEP on the substantive aspect of construct validity. As a result, it was concluded that the QuEEP, specifically the version derived from the Rasch analysis, was more effective at capturing personality that was influenced by a second language than the IPIP BFFM.</p><p> In relation to the study&rsquo;s second goal, the personality for 38 study-abroad students was assessed, through a pre-departure and post-return administration of the QuEEP and IPIP BFFM, to determine if the participants&rsquo; personality changed after one month abroad. The results indicated that the personality measures of extraversion and emotional stability increased significantly after one month abroad, as measured by the QuEEP. The IPIP BFFM did not indicate any significant personality changes.</p><p> In relation to the study&rsquo;s third goal, the 38 study-abroad students also completed a pre-departure and post-return interview test to determine if certain personality types benefitted more from studying abroad in terms of oral proficiency, which included eight measures of fluency, complexity, and accuracy. The results indicated that when the participants were divided into high and low groups for each personality dimension (e.g., a high extraversion and a low extraversion group), the only significant differences between the groups in measures of oral proficiency involved the pauses fluency variable (low QuEEP emotional stability group), the words per second fluency variable (high IPIP BFFM extraversion group), the pauses fluency variable (high IPIP BFFM extraversion group), and the accuracy variable (low IPIP BFFM openness group). After Bonferroni adjustments were conducted, these findings were rendered not significant. However, when analyzed cross-sectionally rather than longitudinally, there were several significant correlations involving the QuEEP pretest and pre-interview test data, most notably between oral proficiency and extraversion and emotional stability. The IPIP BFFM posttest also indicated significant correlations between oral proficiency and agreeableness and openness. The QuEEP posttest and post-interview test data, and the IPIP BFFM pretest and pre-interview test data indicated fewer significant correlations with oral proficiency.</p>

<p> The purpose of this study was to determine if there was a correlation between the implementation of standards-based grading (SBG) and a rise in student achievement as indicated by the evaluation of end-of-course exam data from the four core subject areas in secondary schools&mdash;English, Algebra, Government, and Biology. This mixed methods study focused on the collected data of 6,000 test scores, split evenly from tests taken prior to the launch of SBG and after the launch of SBG, as well as focusing on varying perceptions of SBG from both teachers and members of the community as a whole. Quantitative data consisted of test scores aggregated by the Missouri Department of Elementary and Secondary Education (MODESE) and distributed to school districts. Qualitative data were secondary in nature and taken from two separate surveys administered by the Cooperating School District to teachers and parents concerning how they felt about the implementation of SBG. These data were aggregated and analyzed by using coding techniques for qualitative data to determine the study outcomes. The quantitative data indicated that there was no statistically significant increase in test scores over the five years researched for this study. The qualitative data indicated that perceptions of SBG were frequently negative in nature, although there were varying degrees of negativity. This result came from both the teacher and parent responses. According to these qualitative data, parents and teachers alike viewed SBG as diminishing the foundation of the education the students were receiving, while at the same time inadequately preparing them for the post-secondary world in that too many chances were given for them to succeed under SBG.</p><p>

<p> The purpose of this study was to examine and describe the nature of the research to practice gap in learning analytics applications in K12 educational settings. It was also the purpose of this study to characterize how learning analytics are currently implemented and understood. A secondary objective of this research was to advance a preliminary learning analytics implementation framework for practitioners. To achieve these purposes, this study applied quantitative content analysis using automated text analysis techniques to assess the quality of information provided on analytics-based product websites against learning analytics research. Because learning analytics implementations require adoption of analytical tools, characterizing content on analytics-based product websites provides insight into data practices in K12 schools and how learning analytics are practiced and understood. A major finding of this study was that learning analytics do not appear to be applied in ways that will improve learning outcomes for students as described by the research. A second finding was that policy influence expressed in the study corpus suggest competing interests within the current policy structure for K12 education settings. Keywords: quantitative content analysis, automated text analysis, learning analytics, big data, frameworks, educational technology, website content analysis </p><p>

abstract: Institutions of higher education often tout that they are developing students to become lifelong learners. Evaluative efforts in this area have been presumably hindered by the lack of a uniform conceptualization of lifelong learning. Lifelong learning has been defined from institutional, economic, socio-cultural, and pedagogical perspectives, among others. This study presents the existing operational definitions and theories of lifelong learning in the context of higher education and synthesizes them to propose a unified model of college students' orientation toward lifelong learning. The model theorizes that orientation toward lifelong learning is a latent construct which manifests as students' likelihood to engage in four types of learning activities: formal work-related activities, informal work-related activities, formal personal interest activities, and informal personal interest activities. The Postsecondary Orientation toward Lifelong Learning scale (POLL) was developed and the validity of the resulting score interpretations was examined. The instrument was used to compare potential differences in orientation toward lifelong learning between freshmen and seniors. Exploratory factor analyses of the responses of 138 undergraduate college students in the pilot study data provided tentative support for the factor structure within each type of learning activity. Guttman's <&lambda;>&#955;2 estimates of the learning activity subscales ranged from .78 to .85. Follow-up confirmatory factor analysis using structural equation modeling did not corroborate support for the hypothesized four-factor model using the main student sample data of 405 undergraduate students. Several alternative reflective factor structures were explored. A two-factor model representing factors for Instructing/Presenting and Reading learning activities produced marginal model-data fit and warrants further investigation. The summed POLL total scores had a relatively strong positive correlation with global interest in learning (.58), moderate positive correlations with civic engagement and participation (.38) and life satisfaction (.29), and a small positive correlation with social desirability (.15). The results of the main study do not provide support for the malleability of postsecondary students' orientation toward lifelong learning, as measured by the summed POLL scores. The difference between freshmen and seniors' average total POLL scores was not statistically significant and was negligible in size. / Dissertation/Thesis / M.A. Educational Psychology 2011

<p> My dissertation explores how elicited gestures can support 6th grade students&rsquo; understanding of nonlinear dynamics of complex systems. My hypothesis is that elicited gestures make these patterns salient to the student, who learns about these quantitative dynamics via embodied mechanisms. To elicit these gestures, I designed a computer simulation called the Embodied Simulation of Population Dynamics (ESPD). The ESPD elicits bimanual gestures to represent nonlinear changes between two quantities. I explore how the learning from elicited gestures can take place at three different granularity levels. First, I compare the ESPD versus a non-embodied instructional intervention. This comparison tests whether physical movement has an effect on learning. Second, having compared individual learning, I examine the effect elicited gestures have in the way a group of students build situated meaning of graphical representations. Third, I explore a measurement model of students&rsquo; enacted movements using the ESPD log data. With the log data, I measure the student ability to enact the elicited gestures. Results show (1) statistically significant higher learning gains for students in the ESPD condition; (2) students in the ESPD condition spontaneously used more complex, action-laden gestures to convey their understanding of the quantitative dynamics; and (3) the ability to enact the elicited movement predicts learning gains. Findings from this dissertation will be useful to researchers, teachers, and designers who want to include elicited gestures as part of their instructional approach with early middle school students.</p><p>

<p> The adoption of the Next Generation Science Standards (NGSS) has brought with it a need for classroom assessments that measure students&rsquo; ability to make sense of, explain, and use science, with many of the performance expectations asking students to apply scientific principles and evidence to produce an explanation. But creating coherent written scientific explanations that demonstrate understanding of scientific principles is not an easy task for students and selecting or creating classroom assessments to properly evaluate this skill is not an easy task for teachers. This study serves to incorporate both of these problems by exploring what happens when scaffolds are included in the assessment process for the students of a group of science teachers at an urban high school in Northern California who were receiving coaching in scaffolding assessments. Presented as three distinct but related chapters I examine how the scientific explanations of students changed when a detailed graphic organizer/rubric scaffold, called SET4CER, was provided; I explore coaching as a conveyor of professional development for high school science teachers to incorporate scaffolding techniques into their classroom assessments; and I document how students in a kinesiology class interact with teacher designed checklist scaffolds to support a summative quiz taken under test conditions. The findings illuminate the difficulties facing teachers in both assessment and scaffold design and indicate that well-designed scaffolds add value to assessments by supporting students to give their best performance and ensuring that assessments evaluate students&rsquo; conceptual understanding rather than their memorization or organizational skills. Analysis of the coaching cycles revealed that although the coaching process is not always straightforward, it has potential as a method of professional development delivery that facilitates adoption of the techniques being introduced.</p><p>

<p> This study analyzed the results of undergraduate and graduate teacher candidate scores on the Elementary Missouri Content Assessment at a private Midwestern University. The intent was to examine the possibility of a difference between pre-service teachers who satisfied coursework requirements through an undergraduate program or through a graduate program. This study focused on the content knowledge domain of teacher knowledge that had been the focus of teacher competency studies since Lee Shulman&rsquo;s research in 1986. This study used pre-service teacher Missouri Content Assessment scores, subtest scores, number of attempts to earn a passing score, number of years since a college math or science class was taken, cumulative grade point average, and grade point average in each content area in order to determine if any correlation existed between academic performance and performance on the Elementary Missouri Content Assessment. </p><p> The researcher anticipated that pre-service teachers with higher GPA would receive higher scores on the Missouri Content Assessment. The 12 hypotheses in this study tested the relationship of both undergraduate and graduate pre-service teacher academic performance and performance on the Missouri Content Assessment. The site of the study was a medium-sized Private Midwestern University. A random sample of 50 undergraduate and 35 graduate pre-service teachers were selected from a total population of 205 graduates, which included 169 undergraduate candidates and 36 graduate candidates. The study was quantitative utilizing the Pearson Product-Moment Correlation Coefficient and a <i>z</i>-test for difference in means. The analysis of the 12 hypotheses revealed no statistically significant correlation between academic performance and the Missouri Content Assessment scores. The lack of a significant correlation suggests that further studies are conducted to determine what factors could be indicators that both the Researched University and future pre-service teachers enrolled at the Researched University use as predictors of readiness and success on the MoCA. Because the MoCA assessment has a limited implementation of four years, it may be beneficial to examine what changes the research university has made in course and program design to meet the requirements and then examine the MoCA scores after the changes.</p><p>

<p> This qualitative, phenomenological study explored the lived experiences of four state licensed kindergarten teachers with regard to classroom assessment. The purpose of the inquiry was to gain understanding of classroom assessment and its intersection with teacher's practices. The study question was, "What are the lived experiences of state licensed kindergarten teachers with regard to classroom assessment?" A critical theory theoretical framework structured and guided the study. </p><p> &bull; Using the reduction methods as suggested by Van Manen (2014) allowed the researcher to enter a space of openness and understand the teachers' lived experiences with classroom assessment practices. The use of this phenomenological process resulted in the unveiling of four major themes <i>teacher agency, the absence of developmentally appropriate practice, teachers feelings/knowing of assessment and the damage from early childhood assessment</i>.</p><p> Research on early childhood education assessment and teacher practice is a timely endeavor as more children enter schools, daycares, or other means of childcare. Policy makers just like early childhood "creators" of assessment tools need to be mindful of the children first; and acknowledge and understand the place that children and teachers hold when they are making decisions of such importance.</p><p>

<p> This quantitative, quasi-experimental, <i>ex post facto</i> analysis examined the relationship between the English proficiency level (ELP) attained by English Learners in the state of Georgia in 2016 and 2017 and their performance on standardized core content assessments administered entirely in English. From the theoretical perspective of validity theory, the researcher investigated the alternative hypothesis that the results of standardized content assessments administered in English to English Learner students yield little meaningful data that can be interpreted and used with validity in and of themselves. To investigate this interpretation, the researcher analyzed 176,941 individual 2016 and 2017 Georgia Milestones Assessment System (GMAS) assessment records (N = 102,312 for 2016; 74,649 for 2017) matched to the corresponding ACCESS for ELLs scores (N = 34,420 for 2016; 35,805 for 2017), employing linear regression to quantify the degree to which ELP influenced English Learner students&rsquo; performance on standardized assessments in English Language Arts, Mathematics, Science, and Social Studies and to define the threshold proficiency level at which the preponderance of English Learners were able to score at the Developing and Proficient levels on the content assessments. </p><p>