Background: Providing examinee performance data is a common procedural modification in modified-Angoff standard setting exercises. Results of research examining the impact of the practice are mixed, and despite the frequency with which this procedural modification is done, the overall impact on the outcomes is not well understood. This research presents a large-scale evaluation of the impact of providing performance data in the context of operational standard setting exercises for a medical licensing examination program. Methods: Data from a total of 18 independent standard setting panels across three different examinations were analyzed to evaluate whether and how the provision of performance information impacted the resulting cut scores. Results: Significant cut-score changes were found between initial and final judgments for all panels. A decrease in standard deviation between initial and final judgments for all panels was indicative of the convergence of judgments following the provision of data, and a lack of systematic changes in final cut scores suggested that judges were not simply changing their content-based judgments to align with the normative data. Both findings provide important procedural validity evidence for the standard setting process. Implications: The current data set (replications of the procedure within a single cycle, across cycles for an individual examination, and across multiple examinations) allowed for the most comprehensive evaluation of this topic that has been described to date and provides valuable insight into an issue that is relevant to many testing contexts. / Educational Psychology

The performance of three popular procedures for detecting differential item functioning (DIF), the Mantel-Haenszel (MH), the Simultaneous Item Bias (SIB), and the Logistic Regression (LR) procedures were investigated and compared in three different studies. The first study compares the MH and the SIB procedures with respect to their Type I error rates and power to detect uniform DIF. Data for the study were simulated to reflect a variety of conditions. The results revealed that both the MH and the SIB procedures were equally powerful in detecting uniform DIF under most of the studied conditions. The SIB procedure showed higher detection rates than the MH procedure as the ability distribution differences increased. The second study investigated the distributions of the SIB and two variations (with and without the continuity correction in the MH statistic), to determine whether or not their distributional assumptions held. The results showed that the SIB statistic generally had the expected distributions when the sample size of the reference and the focal groups exceeded 200. The distributions assumptions of the MH statistic without the continuity correction were more readily met than those of the MH statistic with the continuity correction for all the studied conditions. The third study investigated the MH, the SIB, and the LR procedures with respect to their Type I error rates and power to detect non-uniform DIF. Data for the study were simulated under a variety of conditions. The results revealed that both the SIB and LR procedures were equally powerful in detecting non-uniform DIF under most conditions. The MH procedure was not very effective in identifying non-uniform DIF items that showed disordinal interactions. The investigation of the Type I error rates in all the three studies showed that they were within the expected limits for the MH procedure, higher than expected for the SIB and LR procedures with the SIB results showing an overall increase of about 1% over the LR results. With respect to power, the results show that the MH statistic was very effective in detecting only uniform DIF; both the SIB and LR procedures were very effective in detecting uniform as well as non-uniform DIF.

<p> The ultimate goal of elementary and secondary education is to produce citizens who can contribute to and succeed in today&rsquo;s global economy (Alliance for Excellent Education, 2011; Biancarosa &amp; Snow, 2006). The most significant skill required in reaching that level of success and contribution is a person&rsquo;s ability to read. &ldquo;Where the press is free, and every man able to read, all is safe&rdquo; (Jefferson, 1816, para. 5). Approximately 200 years later, where are America&rsquo;s literacy rates? Sixty-eight percent of eighth grade students read below level (NAEP, 2009). The overarching goal of this research study was to develop best practices for RTI implementation at the high school level that include teacher professional learning, effective universal components, and methods that contribute to increased student performance, specifically in the areas of literacy development and reading comprehension. The purpose of this study was to (1) examine the programs and processes teachers are using to identify, intervene, and remediate reading deficits with ninth grade students; (2) review and analyze data from ninth grade below-benchmark; (3) review and analyze progress data of students receiving interventions; and (4) explore teachers&rsquo; perceptions regarding resources, professional learning, and student performance. For this research study, both quantitative and qualitative methods were used. The study reported descriptive statistics of student performance on the universal screening measure at five high schools. The qualitative component of this study was designed to collect data using focus group interviews with 33 teacher participants. The information gained from the focus group interviews with teachers was crucial to determining what impact the components of RTI have on student performance. Findings indicated a need for additional remediation resources, increased professional development, technical assistance and support with the scheduling and staffing components of RTI, as well early identification of students needing to participate in the RTI process.</p>

<p>The importance of school readiness to both the future of an individual child and society as a whole has given rise to several state-specific indexes designed to measure county-level risk for starting school unprepared to learn. One such index is the Oklahoma School Readiness Risk Index (OK SRRI), comprised of indicators known to be associated with poor school readiness. Among several indicators considered for the index, the final set was determined on the basis of multivariate statistical methods. Selected indicators related to race/ethnicity, family structure and economics, and child maltreatment. No health-related indicators were included. </p><p> A limitation of indexes is that there are no agreed-upon best methods or established theoretical framework of measurement for their construction, which makes indexes subject to specification error. Many indexes are developed using reflective measurement models, which assume indicators reflect a unidimensional latent construct. Indexes, however, follow formative measurement models in which indicators define a multidimensional construct. The use of an erroneous measurement model has considerable implications for policy and resource allocation decisions. </p><p> This study examined the sensitivity of the OK SRRI to changes to the indicator set. An alternate index was created that reduced the number of racial/ethnic indicators and included those related to health, such as low birth weight. Indicator selection was guided by a theoretical framework based on transactional/ecological and cumulative risk models of child development, as well as assumptions of formative measurement models. Nearly one-third of Oklahoma's counties experienced considerable shifts in rank from the original to the alternate index. Most increases occurred for counties with high rates on at least one health indicator, while many decreases were among counties with high rates on multiple racial/ethnic-related indicators. </p><p> This study demonstrated that changes to the indicator set can change the meaning of a construct, which underscores the significance of the indicator selection process. Given the political nature of indexes, it is imperative that those with a stake in the outcomes be included in these processes. As most indexes related to social constructs are intended to inform policy and resource decision-making, this study has important implications for the field of index construction.</p>

<p> This mixed methods study examined the responses of four administrators and three English learner (EL) teachers in a rural Indiana school corporation to the mandates of the state's accountability system. In addition, the study investigated the performance of ELs from the participating school corporation on the state's standardized test, the Indiana Statewide Testing for Educational Progress Plus (ISTEP+). The constant comparative method was used to analyze interview data, and descriptive statistics were generated to analyze four years of student test scores. Findings indicate that school corporation personnel have implemented a number of programmatic and instructional changes in response to the state's accountability system. Results from both the qualitative and quantitative analyses elucidate the level of English proficiency required for ELs to pass the ISTEP+. This study provides a starting point for reassessing Indiana's EL achievement objectives.</p>

<p> The proposed study involves the use of mixed methodological research of a conceptual model of Response to Intervention (RTI) as an instructional process aimed at increasing student achievement and thus reducing the achievement gap. The study aims to examine the relationship between implementation of a district-wide systematic process of Response to Intervention (RTI) in reading and student achievement in middle school. Embedded in this model is the possible impact of staff perception on fidelity of implementation. The construct of RTI employed in this study is the <i>hybrid approach,</i> which was developed from the modern standard protocol and problem-solving approaches, both which were born from the historical models of RTI developed by Bergan and Deno (Batsche et al., 2006). A conceptual framework was developed to conceptualize the required components of a systematic process of RTI with an academic focus, staff perceptions of the implementation process, and the potential impact on student achievement.</p>

<p> Research has suggested the <i>No Child Left Behind Act</i> passed in 2001 has resulted in narrowed curriculum and a failed attempt at its primary objective to close the achievement gap. Understanding changes that have occurred in student achievement from 2011 to 2014 of the NCLB requirements is the focus of this quantitative ex-post facto study. Whether change occurred or not in student achievement during this time was analyzed by comparing aggregate average mathematics scaled scores for grades three, five and eight within a southwestern suburban county in the state of Arizona. Additionally, as part of NCLB requirements, Arizona has assigned performance based school labels to the general public called Arizona A-F Letter Grades. This study includes an examination of changes in the Arizona A-F Letter Grades assigned to schools by the Arizona Department of Education from 2011 to 2014. </p><p> The findings for the research can be summarized under six major themes. The first theme was that student average scaled scores for all students did not show significant change from 2011 to 2014 for third, fifth, or eighth grades. When compared relative to all groups, there were no overall statistically significant changes in average mathematics scaled scores from 2011 to 2014. </p><p> The second theme was that no statistically significant changes overall were found in mathematics average scaled scores among ethnic subgroups from 2011 to 2014 in 3rd, 5th, or 8th grade in the southwestern suburban county studied.</p><p> The third theme was that no statistically significant changes overall were found in mathematics average scaled scores among socioeconomic subgroups from 2011 to 2014 in 3rd, 5th, or 8th grade in the southwestern suburban county studied.</p><p> The fourth and fifth theme, identified through additional analysis, were that when achievement scores were compared together from 2011-2014, substantial achievement gaps remained between all ethnic groups as well as between high-SES and low-SES groups.</p><p> The sixth theme was that Arizona A-F Letter Grades assigned to schools showed statistically significant growth from 2011 to 2014. Although the results showed small movement, they showed that schools in this southwestern suburban county are making some improvement in overall student achievement during this time.</p><p> Future research suggestions and implications for current practice are included as a result of this research study.</p>

<p> This program evaluation examined students that entered kindergarten not ready on the Brigance screener and received research based reading interventions to determine if an elementary school was successful in closing the achievement gap so that they would have the literacy skills needed to be successful in first grade. Evidence of success was based on progress monitoring through Brigance and MAP testing. The researcher concluded that while 50% of the students were ready at the end of kindergarten only 7% returned to first grade ready. Other results indicated that the achievement gap was closing for students in special education but not low income students as well as gains for students receiving intervention and those not receiving intervention were equivalent. Implications for practice include the importance of preschool and full-day kindergarten as well as ensuring that summer learning is considered between kindergarten and first grade.</p>

<p> Personalized learning is not a new concept and can be traced back to the 1700s, but it has become increasingly popular in today&rsquo;s schools (Bray &amp; McClaskey, 2015). With the high rate of technology introduction into classrooms, the implementation of personalized learning has become easier as instruction shifts from teacher-led to student-centered (Grant &amp; Bayse, 2014). The purpose of this study was to identify the role of technology in personalized learning and to determine if the implementation of the pedagogy had an effect on student achievement scores. Elementary classroom teachers and elementary principals served as the participants for this study, which was conducted in a southwest Missouri school district. Teachers rated their personal level of personalized learning implementation on a survey. The rating was compared to mean student scale score gains in reading and mathematics, as measured on the i-Ready Diagnostic Assessment. Teachers and administrators also identified how technology impacted personalized learning in their classrooms or buildings. After an analysis of the data, it was determined there was no statistically significant correlation between the degree of implementation of personalized learning and student achievement. According to survey responses, 89% of educators agreed technology eased the implementation of personalized learning, and the teachers and administrators indicated using technology for research and accessing resources outside of the classroom as the most frequent use for devices.</p>

<p> Over the last two decades, online education has become a popular concept in universities as well as K-12 education. This generation of students has grown up using technology and has shown interest in incorporating technology into their learning. The idea of using technology in the classroom to enhance student learning and create higher achievement has become necessary for administrators, teachers, and policymakers. Although online education is a popular topic, there has been minimal research on the effectiveness of online and blended learning strategies compared to the student learning in a traditional K-12 classroom setting. </p><p> The purpose of this study was to investigate differences in standardized test scores from the Biology End of Course exam when at-risk students completed the course using three different educational models: online format, blended learning, and traditional face-to-face learning. Data was collected from over 1,000 students over a five year time period. Correlation analyzed data from standardized tests scores of eighth grade students was used to define students as &ldquo;at-risk&rdquo; for failing high school courses. </p><p> The results indicated a high correlation between eighth grade standardized test scores and Biology End of Course exam scores. These students were deemed &ldquo;at-risk&rdquo; for failing high school courses. Standardized test scores were measured for the at-risk students when those students completed Biology in the different models of learning. Results indicated significant differences existed among the learning models. Students had the highest test scores when completing Biology in the traditional face-to-face model. Further evaluation of subgroup populations indicated statistical differences in learning models for African-American populations, female students, and for male students. </p>