No description available.

No description available.

No description available.

No description available.

No description available.

The use of rating scales in the evaluation of secondary teacher performance has been called into question and widely criticized. Of particular concern has been the use of student ratings of teacher performance. A review of instruments and practices used in the rating process reveals serious design flaws that account for the criticisms leveled against the use of rating scales. This study sought to address the limitations evident in previous rating efforts by utilizing a combination of design methodologies and measurement models including elements of Classical Test Theory (CTT), factor analysis, and Item Response Theory (IRT). The IRT model employed was the one-parameter logistic model also known as the Rasch model. Twelve scales were developed consisting of a total of ninety-two items. These scales were developed to facilitate student ratings of secondary level teachers of religion in the Church Educational System (CES) of the Church of Jesus Christ of Latter-day Saints (LDS). In addition to exploring rating scale design methodology and scale performance, this study also examined a potential threat to the validity of decisions based on ratings referred to as halo effect. Using a variety of approaches to operationally define and estimate halo error, the extent to which male and female students exhibit differing degrees of halo in their ratings of teachers was examined. The results of the study revealed that of the twelve teacher traits hypothesized in the design of the rating scales, only three met defensible criteria based on CTT and Rasch model standards: the Student-Teacher Rapport Scale (STRS), the Scripture Mastery Expectation Scale (SMES), and the Spiritual Learning Environment Scale (SLES). Secondary students were unable to meaningfully discriminate between all twelve traits. Traditional approaches to halo effect estimation suggest that males exhibited halo to a greater degree than females, whereas Rasch model approaches to halo effect estimation were less consistent. Considered together, however, the evidence suggests differential halo error by gender, with males exhibiting halo to a greater degree than females. The implications of these findings for teacher evaluation, instructional design, and future research efforts are also addressed.

The importance of valid and reliable couple assessment has been increasing with growth in research on couple and family relationships as well as in therapeutic and educational interventions for couples and families. However, self-report instruments–the most popular type of couple assessment–have been criticized at least partly due to limitations in Classical Test Theory (CTT) which has been used solely in developing and evaluating couple assessments for decades. In an effort to address the limitations in the sole use of CTT in developing self-report couple assessments, the present study integrated a modern test theory called Item Response Theory (IRT) and evaluated the properties of subscales in the RELATionship Evaluation (RELATE) using the existing data from 4,784 participants. Using the Rasch rating scale or partial credit model which is one of the IRT models, the author demonstrated that some of the RELATE subscales had items and response categories that functioned less optimally or in an unexpected way. The results suggested that some items misfit the model or overlapped with other items, many scales did not cover the entire range of the measured construct, and response categories for many items malfunctioned. The author made recommendations on possible remedies that could be adopted to improve the function of individual scales and items.

A majority of research investigating aggression and its development in children has relied upon the use of rating scales such as the Child Behavior Checklist (CBCL). These scales are typically developed using a conventional factor analytic approach for the selection and retention of scale items, but may not contain sufficient numbers of items to adequately assess the unidimensional construct or developmental trajectory of aggressive behavior in youths. The present study evaluates specific psychometric properties of CBCL Aggressive and Delinquency Problems clinical syndrome scale items to determine the degree to which they reflect the breadth and established developmental trajectory of aggressive behavior in youth.

The purpose of this quasi-experimental, mixed-methods study was to address the gap in understanding regarding second language (L2) comprehension of stories by using a taxonomical approach based on Anderson and Krathwohl’s (2001) revised taxonomy that includes multiple levels of cognitive processing to provide insight into L2 learners’ depth of comprehension. In addition, this study investigated the impact of reading, reading-while-listening, and listening input modalities as it applies to story comprehension. L2 comprehension studies where all three of these input methods are compared are scarce. Finally, the effect of input type on learner affect—task enjoyment and perceived task difficulty—was explored. No previous mixed methods L2 comprehension studies have accounted for all the above variables. The participants (N = 134, 85 male and 49 female students) of the quantitative aspect of this study were Japanese university students who were streamed into the university’s reading and writing or listening and speaking classes. At the time of the study, they were first- and second-year, non-English majors taking English courses as a general university requirement at a private university in western Japan. Participants from six intact classes were tasked with reading two chapters, reading-while-listening to two chapters, and listening to two chapters of a six-chapter graded reader. Using a Latin squares design, each class received a different input method at the beginning, middle, and end of each story. Three 250-headword, CEFR level 1 short stories of similar lengths from the Oxford Dominos series were used for each participant group. The participants received short, bilingual vocabulary lists for vocabulary that fell outside of the first 1,000 BNC/COCA high-frequency words of English prior to receiving each story. After finishing two chapters using one of the above three input modalities, the texts were returned to me and the participants answered bilingual remember questions of the factual recall subtype, understand questions of the inferencing subtype, and evaluate questions of the judging or critique subtypes based on Anderson and Krathwohl’s (2001) revised taxonomy. They then rated task difficulty and task enjoyment. Both the remember and understand questions were four-option multiple-choice questions, while the evaluate questions required written responses in Japanese. To ensure task time equivalency, the participants received two repeated listenings at approximately 138–157 words per minute (WPM) for the reading-while-listening and listening tasks. The participants were given approximately 18 minutes to complete each of the three chapters and 10 minutes to answer questions. Using data from Rasch person measures, a series of mixed model analyses were used to assess the participants’ performances on remember, understand, and evaluate comprehension questions as mediated by input modality. Input modality—reading, reading-while-listening, and listening—was the independent variable, while the remember, understand, and evaluate comprehension questions made up the dependent variables. The New Vocabulary Levels Test (NVLT) scores were used as the covariate. The results indicated that the participants scored higher overall on the remember questions (M = 54.06) than on the understand questions (M = 52.62) or the evaluate questions (M = 49.31). Regarding task-type findings, the reading and reading-while-listening tasks resulted in significantly better comprehension than listening tasks but were not significantly different from one another for remember and understand comprehension questions. For evaluate comprehension questions, all three inputs resulted in significantly different comprehension with reading resulting in the highest comprehension, followed by reading-while-listening, and then listening. The NVLT was a significant predictor of comprehension at all levels, but it had small R2 values. Listening tasks resulted in significantly lower scores than both the reading and reading-while-listening input conditions for all three comprehension levels. In addition, the participants rated that they perceived the reading-while-listening and listening tasks to be more difficult than reading tasks. Reading and reading-while-listening tasks were rated as more enjoyable than listening tasks. After the quantitative data were gathered, qualitative interviews were conducted to better investigate the research hypotheses and the quantitative findings. Six participants were from the same university as the quantitative sample (N = 6, 1 male and 5 female) and four participants were from another nearby university (N = 4, 3 male and 1 female). These participants completed the second text, The Bottle Imp (Stevenson, 2008), by reading Chapters 1–2, reading-while-listening to Chapters 3–4, and listening to Chapters 5–6 in the same manner as the quantitative groups except they did not follow a Latin squares design. After the completion of each two-chapter section, the participants provided verbal recalls to ensure that they correctly performed the task and to gain insight into what was understood from the text. Next, they answered comprehension questions and gave task enjoyment and task difficulty ratings. Finally, the participants answered interview questions designed to clarify their thoughts about the tasks. Verbal recalls and interviews were audio-recorded, and the interviews were transcribed and coded based on Saldaña (2016). The mixed-methods results identified a gap between Japanese L2 learners’ ability to comprehend listening to texts in English compared to the reading and reading-while-listening conditions. This gap persisted regardless of the levels of critical thinking required. The participants also performed more poorly on tasks as the critical thinking levels increased. Receptive vocabulary knowledge as measured by the NVLT was found to predict comprehension; however, because the graded readers were typically lexically appropriate for the participants, it had small R2 values. More lexically complex texts might have resulted in the NVLT correlating more strongly with comprehension. Finally, listening tasks were perceived to be more difficult and less enjoyable than reading. These findings suggest that input method has a significant influence on L2 learners’ ability to complete comprehension tasks at three levels of Anderson and Krathwohl’s (2001) revised taxonomy. The bulk of comprehension test items are from the bottom two levels of the taxonomy, so educators and researchers should consider the role that input method plays. Additionally, L2 learners might benefit from instruction that builds listening comprehension and listening fluency skills. As critical thinking demands increased, comprehension scores decreased regardless of the input method. This finding aligns with Anderson and Krathwohl’s Revised taxonomy, which posits that a critical thinking hierarchy exists and that a degree of proficiency with lower levels of the hierarchy is necessary for the successful completion of higher-level tasks. Educators should consider how critical thinking contributes to task difficulty and language learners should be provided with language tasks that work to improve critical thinking skills. / Applied Linguistics

No description available.