Post-Apartheid South Africa has undergone an educational language policy shift from only Afrikaans and English in education to the representation of all 11 official languages: Afrikaans, English, isiZulu, isiXhosa, isiNdebele, siSwati, Sesotho, Setswana, Tshivenda and Xitsonga. The national languages policy included the Language in Education Policy (LiEP), which stipulates that learners from grades 1- 3 in all ways possible should be provided the opportunity to be taught in their home language (HL). With this change, there has been a need to increase access to African languages in education. The 2007 Status of LoLT report released by the Department of Education (DoE) revealed that since 1996 up to 65% of learners in the foundation phase are being taught in their home language. In other ways, the LiEP has been successful in bridging the gap of access to African languages in the basic education system. With that said, there has been rapid growth of interest in early childhood crosscultural literacy assessment across the globe. Internationally South Africa has participated in the Southern and Eastern Africa Consortium for Monitoring Education Quality as well as the Progress in International Reading Literacy Study studies. The design of these particular international studies meant participation in the same assessment but in different languages, calling into question the equivalence of assessments across languages. Assessing across languages should aim to encourage linguistic equivalence, functioning equivalence, cultural equivalence as well as metric equivalence. South Africa has taken part in three cycles of the Progress in International Reading Literacy (PIRLS) study. The purposes of the current study is to present secondary analysis of the prePIRLS 2011 data, to investigate any differential item functioning (DIF) of the achievement scores between English and isiXhosa. The Organisation for Economic Co-operation and Development (OECD) developed a framework of input, process and output for curriculum process. The framework shows the multiple facets that needs to be considered when implementing a curriculum in a country. The curriculum process framework was used as the theoretical framework for this study. The framework views curriculum success as a process of measuring how the intended curriculum (input) was implemented (process) and should be reflected in the attained curriculum (output). The adapted framework is LiEP as the attained curriculum, as learners in the prePIRLS 2011 are tested in the LoLT in Grades 1-3. Followed by the prePIRLS 2011 assessment, as the implemented curriculum testing the learners’ comprehension skills requires by grade 4 in their HL. Lastly, the attained curriculum refers the learners’ achievement scores in the prePIRLS 2011 study. A sample of 819 Grade 4 learners (539 English L1 speaking learners and 279 isiXhosa L1 speakign learners) that participated in the prePIRLS 2011 study were included in this study. These learners wrote a literary passage called The Lonely Giraffe, accompanied by 15 items. The study made use of the Rasch model to investigate any evidence of Differential Item Functioning (DIF) on the reading achievement of the learners. The findings showed that the items did not reflect an equal distribution. In addition, an item by item DIF analysis revealed discrimination on one subgroup over the other. A further investigation showed that these discriminations could be explained by means of inaccurate linguistic equivalence. The linguistic equivalence could be explained by means of mistranslation and/or dialectal differences. Subsequently, the complexities of dialects in African languages are presented by providing isiXhosa alternative translations to the items. The significance of the current study is in its potential contribution in further understanding language complexities in large-scale assessments. Additionally, in attempts to provide valid, reliable and fair assessment data across sub-groups. / Dissertation (MEd)--University of Pretoria, 2017. / Science, Mathematics and Technology Education / Centre for Evaluation & Assessment (CEA) / MEd / Unrestricted

Thesis advisor: Larry H. Ludlow / Preparing and supporting teachers to enact teaching practice that responds to diversity, challenges educational inequities, and promotes social justice is a pressing yet daunting and complex task. More research is needed to understand how and to what extent teacher education programs prepare and support teacher candidates to enhance the achievement of all learners while challenging systematic inequity (Cochran-Smith, Ell, Ludlow, Grudnoff, & Aitken, 2014). One piece of empirical evidence needed is a measure that captures the extent to which teachers enact teaching practice for equity. This study developed an instrument – the Teaching Equity Enactment Scenario Scale (TEES) - to measure the extent of equity-centered teaching practice by applying Rasch measurement theory (Rasch, 1960) and Guttman’s facet theory (Borg & Shye, 1995). The research question addressed whether the TEES scale can measure teachers’ self-reported enactment of practice for equity in a reliable, valid, and authentic manner. This study employed a three-phase design, comprising an extensive process of item development, a pilot study and a final full-scale administration. Fifteen scenario-style items were developed to capture the enactment levels of six interconnected principles of teaching practice for equity. Using the Rasch rating scale model the outcome was a 15-item TEES scale that reliably and validly measures increasing levels of teaching practice for equity progressing through low, moderate, and high levels of enactment. The distribution of the scenarios confirmed their hypothesized order and the instrument development principles of Rasch measurement - unidimensionality, variation and a hierarchical order of the items, as well as a uniform continuum defining the construct. The scale also provides meaningful interpretations of what a raw score means regarding one’s equity-centered teaching practice. The overall findings suggest that the novel approach of combining Rasch measurement and facet theory can be successful in developing a scenario-style scale that measures a complex construct. Moreover, the scale can provide the evidence needed in research on preparing and supporting teachers to teach with a commitment to equity and social justice.

Background: Among the instruments commonly used by occupational therapists working in the area of rehabilitation of persons with neurological disorders are evaluations of both occupation, such as activities of daily living (ADL), and body functions. While persons with neurological diagnoses typically have symptoms that represent diminished neurobehavioral functions, the resulting pattern of neurobehavioral impairments affecting ADL performance often differs among diagnostic groups. Usually, neurobehavioral impairments are evaluated in a context that is separate from and not natural for ADL task performance. The A-ONE is a unique instrument that can be used to evaluate both ADL performance (ADL scale) and, in the natural context of the ADL task performance, the underlying neurobehaviors that cause diminished ADL task performance among persons with neurological disorders (Neurobehavioral scale). The scales of the instrument are of ordinal type, and in their existing form, do not have measurement properties. Measurement properties are a requirement of evidence-based and quality assured rehabilitation services. The overall aim of this doctoral study was to further develop and validate the A-ONE. This included (a) internal validation to explore the potential for converting the ordinal scales of the instrument to interval scales, (b) examination of which of the neurobehavioral items would be most beneficial and clinically useful for constructing a new Neurobehavioral Impact (NBI) scale for evaluating persons with different neurological diagnoses, and (c) exploration of whether persons with right and left cerebrovascular accidents (RCVA, LCVA) differ in mean NBI measures.  Methods: This thesis is comprised of four studies which all contribute in different ways to the validation of the scales of the A-ONE. In the first three studies, Rasch analyses, a widely accepted modern test theory methodology, was used to examine internal validity of the scales and the reliability of the A-ONE measures. In the fourth study, ANCOVA was used to explore between group differences, and Pearson correlation coefficients were used to explore relations between person measures from the different A-ONE scales. Results: The first study of 209 persons diagnosed with CVA and dementia provided support for converting the ordinal ADL scale to an interval scale that has potential to be used to measure change in ADL performance over time. The second and third studies, including 206 and 422 persons respectively, indicated that it is possible to construct several unidimensional versions of a new NBI scale from the neurobehavioral items of the instrument, each with different item content and hierarchical item structure. Further, some of these NBI scales could be used across different diagnostic groups. When exploring differences between 215 persons with RCVA and LCVA on the NBI scale developed for CVA, results of the ANCOVA (with ADL ability as a covariate) indicated that there is no significant difference between groups in their mean NBI measures, despite known differences in patterns of neurobehavioral impairments. Conclusions: The results of this thesis indicate that the A-ONE, although developed by traditional psychometric methods for the purpose of providing useful information for intervention planning, now also has the potential to be used to measure change and compare diagnostic groups. This additional feature will likely enhance both clinical and research potential of the instrument. In order to make the results of the study accessible for clinicians, conversion tables need to be developed.

This study aims to develop assessment which measures learning progressions for important scientific concepts such as oxidation-reduction (redox) and to identify students¡¦ zone of proximal development (ZPD) through teaching practice incorporating assessment feedback. The assessment items of redox were developed based on the framework of the BEAR (Berkeley Evaluation and Assessment Research) Assessment System. Six experts from chemistry, science education, and educational assessment, and three high school chemistry teachers with fruitful instructional experiences were recruited into the assessment team. Through 24 panel discussions, 28 ordered multiple-choice items were developed. Two samples of Taiwanese middle-school students participated in the test development: one for item revision and the other for validation. Sample 1 and 2 consisted of 626 middle school students (304 8th graders and 322 9th graders) and 903 9th graders, respectively. The materials for instruction integrated assessment feedback were designed by the researcher and two middle-school science teachers through seven group meetings. A teaching experiment was implemented to examine the effect of assessment feedback on students¡¦ understandings of redox and to identify their ZPD. The teaching experiment employed a quasi-experiment with a non-equivalent-group pretest-posttest design. Participants were 196 eighth graders (101 boys and 95 girls) from three middle schools. The findings showed that (a) the BEAR assessment system and Rasch measurement approaches provided a feasible framework for developing validated tools to assess learning progressions; (b) the empirical data supported students¡¦ learning of redox concept usually progressed ¡§from uni-structure to multi-structure¡¨ and ¡§from discrete sub-concepts to integrated concepts¡¨; (c) the teaching practice integrated assessment feedback effectively facilitated students¡¦ understanding of scientific concepts; (d) the assessment of learning progressions provided a mechanism for identifying students¡¦ ZPD and helped realize the abtract idea of ZPD in teaching practices. The main contributions of the study included (a) demostrating how to carry out the idea of ZPD into teaching practices through linking learning progressions and ZPD; (b) presenting how to apply BEAR assessment system and Rasch techniques to develop tools for assessing learning progressions; (c) developing a set of items for assessing learning progressions of redox and a series of materials for teaching practices integrated assessment feedback.

Of the many potential and espoused outcomes of higher education, it was satisfaction that rose to prominence for Alexander Astin, stating, “it is difficult to argue that student satisfaction can be legitimately subordinated to any other education outcome” (1993, p. 273). This high endorsement of the construct of satisfaction is backed by a plethora of arguments of its importance for college and university decision makers. A thorough and accurate rendering of student satisfaction measurement is requisite. To calculate student satisfaction as the magnitude of item endorsement leaves a measure that is sample specific. The goal of a universal and unidimensional measure is only advanced by determining which items do or do not contribute to a model of linearity and unidimensionality. This research utilizes the Rasch model to advance exploration of the variable of student satisfaction. Using data collected from the Noel-Levitz Student Satisfaction Inventory, analysis was conducted to determine if reported ascribed importance and experienced satisfaction adhered to the assumption of the Rasch model. Results suggest that student satisfaction and ascribed importance do adhere to these assumptions of measurement, but only after ordinal rankings of dissatisfaction are collapsed into a single entity. The determined separation of satisfaction and dissatisfaction likens Herzberg’s Motivation-Hygiene Theory. Additional discussion and implications focus on contrasting analysis when applying the Rasch analysis relative to classical test theory, recommendations of modified instrument scaling to better capture the construct, implications for higher education, and heightened understanding of student satisfaction as a whole.

Diss. (sammanfattning) Umeå : Umeå universitet, 2010.

Thesis advisor: Larry H. Ludlow / In the field of teacher education, it is well documented that the most influential part is the clinical component, often referred to as student teaching or the practicum experience (Cochran-Smith, 1991; Darling-Hammond, 2014; Evertson, 1990). During the practicum, there exists a triad of three individuals: a university supervisor, a K-12 classroom teacher (often referred to as the cooperating teacher), and the teacher candidate. While much research has been conducted on teacher candidates and the cooperating teacher role, there has been a lack of research on the role of the university supervisor. The lack of measurement instruments to assess the quality of the university supervisor puts teacher education programs at a disadvantage, both from a programmatic improvement standpoint, as well as meeting accreditation requirements. This dissertation provides an answer to this issue, creating a new instrument that assesses the quality of the university supervisor, from the perspective of teacher candidates. This instrument is constructed under the Rasch-Guttman Scenario (RGS) framework, as most clearly defined by Ludlow, Baez-Cruz, et al. (2020). The RGS framework derives its influence from the works of George Rasch (1960/80) as well as Louis Guttman’s facet theory (Guttman 1954; Guttman 1957). The result is a new scale, entitled the University Supervisor Quality (USQ) scale, consisting of nine scenario items. All nine items include four facets that comprise the construct of university supervisor quality: resourcefulness, constructive feedback, mentorship, and collaboration. The results of this dissertation suggest that the utilization of the RGS framework is successful for developing a scale about university supervisor quality. In addition, the use of cognitive interviews provide valuable insight into the development of scales using the RGS framework. This scale has the potential for use in teacher education programs for evaluating the quality of their supervisors, and to utilize as evidence for accreditation purposes. / Thesis (PhD) — Boston College, 2022. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.

Background: The use of outcome measures to evaluate upper extremity function after stroke is highly recommended in clinical practice and research. The Chedoke Arm and Hand Activity Inventory (CAHAI) is a recommended measure as it has strong psychometric properties and clinical utility. However, the measure has not been validated in Asia and there are also gaps in the knowledge about the psychometric properties of the CAHAI. Aim & Objectives: This thesis is dedicated to the continued evaluation of the CAHAI with two main objectives: (1) to develop a Singapore version of the CAHAI, and (2) to re-evaluate the original CAHAI using modern test theories. Method: We conducted a study to cross-culturally adapt the CAHAI and evaluated the psychometric properties in a stroke sample in Singapore. Two studies were conducted to re-evaluate the original CAHAI using modern test theories. In the first study, item response theory and Rasch measurement theory were used to evaluate the psychometric properties of the measure. Following which, both measurement theories were used to revise the CAHAI in the second study. Results: Two test items were modified for the Singapore version of the CAHAI, and the measure had good inter-rater reliability (intra-class correlation coefficient = 0.95 – 0.97) and construct validity. The evaluation of the original CAHAI using modern test theories identified three main problems: (1) the scoring scale was not working as intended, (2) local dependency, and (3) the measure was not unidimensional. Revisions to the CAHAI included collapsing the 7-category scale to four categories, deleting two test items, and developing two new shortened versions. Conclusion: The Singapore version of the CAHAI is a valid and culturally relevant outcome measure that can be used to evaluate post-stroke upper extremity function. The original CAHAI was refined into a new 11- and 5-item versions with a 4-category scale which clinicians may find easier to use. / Thesis / Doctor of Philosophy (PhD)

This dissertation illustrates how to detect the rater centrality effect in a simulation study that approximates data collected in large scale performance assessment settings. It addresses three research questions that: (1) which of several centrality-detection indices are most sensitive to the difference between effect raters and non-effect raters; (2) how accurate (and inaccurate), in terms of Type I error rate and statistical power, each centrality-detection index is in flagging effect raters; and (3) how the features of the data collection design (i.e., the independent variables including the level of centrality strength, the double-scoring rate, and the number of raters and ratees) influence the accuracy of rater classifications by these centrality-detection indices. The results reveal that the measure-residual correlation, the expected-residual correlation, and the standardized deviation of assigned scores perform better than the point-measure correlation. The mean-square fit statistics, traditionally viewed as potential indicators of rater centrality, perform poorly in terms of differentiating central raters from normal raters. Along with the rater slope index, the mean-square fit statistics did not appear to be sensitive to the rater centrality effect. All of these indices provided reasonable protection against Type I errors when all responses were double scored, and that higher statistical power was achieved when responses were 100% double scored in comparison to only 10% being double scored. With a consideration on balancing both Type I error and statistical power, I recommend the measure-residual correlation and the expected-residual correlation for detecting the centrality effect. I suggest using the point-measure correlation only when responses are 100% double scored. The four parameters evaluated in the experimental simulations had different impact on the accuracy of rater classification. The results show that improving the classification accuracy for non-effect raters may come at a cost of reducing the classification accuracy for effect raters. Some simple guidelines for the expected impact of classification accuracy when a higher-order interaction exists summarized from the analyses offer a glimpse of the "pros" and "cons" in adjusting the magnitude of the parameters when we evaluate the impact of the four experimental parameters on the outcomes of rater classification. / Ph. D.

L’étude cherche à provoquer la convergence des regards sur des enjeux méthodologiques fondamentaux, soit les enjeux de mesure, de décision et d’impact inhérents à toute démarche de sélection académique. À cet effet, elle explorera la capacité de prédiction de certaines variables non cognitives envers la compétence de professionnalisme observée chez les étudiants du doctorat professionnel de premier cycle en pharmacie. La sélection des candidats au sein des programmes académiques en santé repose en grande partie sur une évaluation de la capacité cognitive des étudiants. Tenant compte du virage compétence pris par la majorité des programmes en santé, la pertinence et la validité des critères traditionnels de sélection sont remises en question. La présente étude propose de valider l’utilisation des échelles de mesure de la personnalité, des valeurs et de l’autodétermination pour guider l’optimalité et l’équité des décisions de sélection. Les enjeux de mesure de ces variables seront abordés principalement par la modélisation dichotomique et polytomique de Rasch. L’application de la méthode des strates permettra, par la suite, de répondre aux enjeux de décision en procédant à une différenciation et un classement des étudiants. Puis, les enjeux d’impact seront, à leur tour, explorés par le modèle de régression par classes latentes. L’étude démontre notamment que le recours à la modélisation a permis une différenciation précise des étudiants. Cependant, la violation de certaines conditions d’application des modèles et la faible différenciation établie entre les étudiants sur la base des critères de professionnalisme, rendent l’évaluation de la capacité de prédiction de la personnalité, des valeurs et de l’autodétermination hasardeuse. À cet effet, les modèles identifiés par les analyses de régression par classes latentes s’avèrent peu concluants. Les classes latentes ainsi identifiées ne présentent pas de distinctions marquées et utiles à la sélection. Bien que les diverses procédures de modélisation proposées présentent des avantages intéressants pour une utilisation en contexte de sélection académique, des recherches additionnelles sur la qualité des critères de professionnalisme et sur la qualité des échelles de mesure des variables non cognitives demeurent nécessaires. / The study seeks to draw attention on fundamental issues of any academic admission procedures, specifically on measurement, decision and impact issues. Therefore, it will explore the predictive ability of three non-cognitive variables toward the observed professionalism of PharmD students. Admitting student in the health sciences often rely solely on the student cognitive ability. Taking into account that health sciences program have moved toward competency oriented curriculum, the relevance and validity of traditional admission criteria are challenged. This study proposes to validate the use of personality , values and self-determination measurement scales to pursue effective and equitable admission decisions. The challenges of measuring these variables will be addressed primarily by the dichotomous and polytomous Rasch models. The Strata method will, thereafter, provide methodological basis for decision issues by enabling for student ranking and classification. Then, impact issues will, in turn, be explored by latent class regression analysis. The study demonstrates that the use of such modeling procedures has allowed accurate differentiation, ranking and classification of students. However, the violation of certain basic model requirements and the lack of differentiation made between students on the basis of the criteria of professionalism, made the assessment of the predictive power of personality, values and self-determination hazardous. In fact, the latent class models identified were non conclusive. They showed no marked distinctions useful to guide admission procedures. Although, the Rasch modeling, the latent class regression model and the Strata procedure shows interesting benefits for admission procedures, additional research on the quality of professionalism criteria and non-cognitive variables measurement scales are required.