Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 25 for “"MFRM"”.
-
Rubric Rating with MFRM vs. Randomly Distributed Comparative Judgment: A Comparison of Two Approaches to Second-Language Writing Assessment
… assessment and many-facet Rasch modeling (MFRM) to comparative judgment (CJ), the new approach, which shows promising results in terms of reliability and validity. We employed two groups of raters”novice and experienced”and used essays that had been previously double-rated, analyzed with …
-
An Investigation of University General Education Assessment Practice
Many-facets Rasch modeling (MFRM) is an effective way to compensate for rater effects (e.g., rater leniency/severity) in the ratings assigned during rater-mediated assessment. While a fully crossed MFRM rating design, in which all raters rate every rating object, is the ideal from a psychometric …
-
Validating the Rating Process of an English as a Second Language Writing Portfolio Exam
… an analysis with Many-Facet Rasch Model (MFRM) software, called FACETS, which looked for anomalies in levels, classes, examinees, raters, writing criteria, and the rating scale categories. Qualitative methods involved a rater survey, rater Think Aloud Protocols (TAPs), and rater …
-
The Effects of Incomplete Rating Designs on Results from Many-Facets-Rasch Model Analyses
… rater severity is the many-facets Rasch model (MFRM). However, unless the incomplete rating design used to gather the ratings is appropriately linked, the results of the MFRM analysis may not be on the same scale and therefore may not be fairly compared. Given non-trivial numbers of raters …
-
Investigating the Impact of Rater Modeling Approaches on Rater Effect Indicators in Performance Assessments
… models, such as the many-facet Rasch model (MFRM) and item response theory. The review showed that reporting practices varied across studies and often lacked sufficient detail in areas important for interpreting rating quality. The findings also suggested a need to make advanced …
-
Investigating the effects of Rater's Second Language Learning Background and Familiarity with Test-Taker's First Language on Speaking Test Scores
… analyzed using Many-Facet Rasch Measurement (MFRM). The results suggest that extensive rater training can be quite effective: there was no significant effect of either raters' familiarity with examinees' L1, or raters' teaching experience on the scores. However, even after training, the raters …
-
Case Specificity and Intervening Variables Affecting Diagnostic Accuracy within Two Similar Clinical Presentations
… similar CPs. Also, Multi-Facet Rasch Modelling (MFRM) was employed to determine degree of impact on diagnostic performance by the independent variables. This study answered three research questions related to diagnostic generalizability and intervening variables that contribute to diagnostic …
-
The effect of task complexity on rater severity in an adaptive performance-based second language oral communication test
… t-test, and many-facet Rasch measurement (MFRM). To further investigate the effect of task complexity on rater severity, nine newly trained raters were trained to judge 80 speech samples of 40 test takers in the OECT. A partial credit model of MFRM was used to analyze raters’ use of the …
-
The effect of genre-based instruction on academic speech
… employed: (a) multi-faceted Rasch measurement (MFRM) was used to assess the extent to which the participants’ performance improved quantitatively, (b) descriptive analyses were used to investigate frequency changes in the use of target lexis, and (c) genre analysis was used to analyze how the …
-
Investigating test delivery modes within video-conferenced English speaking proficiency assessment
… analysis (CFA) and Multifaceted Rasch model (MFRM) analysis. For dataset 1, I qualitatively analyzed the raters' verbal reports.Dataset 1 had an asynchronous format; all test-takers' performances were recorded for rating. Eight trained raters gave scores to the audio-recorded and video 1 mode …
-
Investigating reliability and construct validity of a source-based academic writing test for placement purposes
… analyzed using Many-Facets Rasch Measurement (MFRM) and Generalizability theory to address the research questions related to the rubric’s functionality, raters’ performance, and score reliability. In Phase 2, 630 experimental ratings, 90 stimulated recalls collected with assistance from records …
-
Task Parallelness : Investigating the Difficulty of Two Spoken Narrative Tasks.
… (1960) in terms of the ratings calculated by MFRM analysis, the linguistic performances of 65 Japanese candidates and 11 native speakers of English, expert judgements by Japanese teachers of English, and perceptions of the Japanese candidates and native speakers of English. The validity of the …
-
An investigation into the rater cognition of novice raters and the impact of cognitive attributes when assessing speaking
… a series of Many-Facets Rasch Measurement (MFRM) analyses. Next, relationships between individual attributes and measures of rater quality and behaviour were explored in a series of correlational analyses. Finally, the handwritten notes and self-report data from four selected raters were …
-
The relationship between vocabulary size, depth, and reading comprehension
… analyzed using many-facet Rasch measurement (MFRM; Linacre, 1989) to examine their reliability and validity. The dimensionality of the vocabulary depth tests was investigated with MFRM, followed by factor analysis. The difficulty of the vocabulary depth tests was examined with the Wright Map …
-
Assessing listening comprehension skills in paired oral communication test: An evidence-centered design
… skills category. Many-Facet Rasch Measurement (MFRM) was used to assess rater reliability and the psychometric functioning of the new descriptor. Results indicated that raters consistently noticed comprehension-related behaviors such as evidence of understanding and relevant follow-up questions …
-
Examining physics teachers’ formative assessment knowledge: a many-facet Rasch model approach
… a southeastern state. Many-facet Rasch modeling (MFRM) was used to determine the psychometric properties of the instrument and assuming acceptable psychometric properties, determine other teacher characteristics which affected formative assessment knowledge (years of teaching experience, gender, …
-
Social Support Among Undergraduate Students: Measure Development and Validation
… χ2 difference test (χ2 (4) = 2223.74, p < .001). MFRM results indicated that there was not a significant difference between the groups in logit position for the family factor (χ2 (2) = 4.8, p = .09, SD = .07, Separation = 1.23, Strata = 1.97, Reliability = .60) or the friend/classmate factor (χ2 …
-
Advancing Language and Literacy Assessments through Technological Integrations
… integration of psychometric approaches like MFRM with machine learning model performance metrics to identify and address specific biases, noting that conventional metrics like F1 scores might not fully reveal nuanced biases. The final study employed cognitive diagnostic modelling, dynamic …
-
The Effect of Raters and Rating Conditions on the Reliability of the Missionary Teaching Assessment
… (G theory) and many-facets Rasch measurement (MFRM). Results indicated that, in general, the reliability of the ratings obtained from the 2 rating conditions were not statistically significantly different from each other. The implications of these findings are addressed.
-
Effects of rating criteria order on the halo effect in second language writing assessment
… model within a many-facet Rasch measurement (MFRM) framework was fitted to estimate the magnitude of the halo effect. Think-aloud verbal protocol analysis was conducted to examine how rating criteria order affects the rating process. The overall results of this study showed that the similar …
Page 1 of 2