This paper examined the inter-rater reliability of psychological assessments in practical field with 42 inmates’ PCL-R scores. As results, this study showed similar ICC and SEM values to those from PCL-manual. Concerning PCL-R structure, factor 2 showed higher ICC value than factor 1, and facet 4 showed higher ICC value than facet 1, 2, or 3. Especially, facet 2 showed low ICC value. Those are consistent with previous studies. However, ICC yielded by factor 2 only and both factor 1 and 2 showed similar ICC values. Considering theoretical and clinical aspects, it was recommendable to use PCL-R total score as risk assessment, though interpreting facet 2 requires cautions. Concerning to rater’s characteristics, the most influential factor to keep the PCL-R reliability was conducting it on regular basis, rather than licensed status. It was difficult to examine whether or not singed-off contribute to maintain sufficient reliability due to small sample size. In regression model, all rater related variables were not significantly correlated to PCL-R score change between two assessment occasions. PCL-R scores at Time 1 was moderately and negatively correlated to PCL-R score change. This indicated natural regression toward the mean. It is desirable to conduct additional study after obtaining more sample and rater related information, such as clinical experience. Additionally, it requires a consideration to apply findings in this study to female psychopathic subjects. As a policy implication, it is recommendable for personnel division to have psychologists to remain in their psychological work.

No description available.

Objective To identify inter-rater reliability and accuracy of pharmacy faculty members' classification of exam questions based on Bloom's Taxonomy. Methods Faculty at a college of pharmacy was given six example exam questions to assign to the appropriate Bloom's level. Results Inter-rater reliability and accuracy were both low at 0.25 and 46.0%, respectively. Accuracy increased to 81.8% when the six Bloom's levels collapsed to three. Conclusions Both inter-rater reliability and accuracy were low. Faculty members' misclassifications suggested a three-tier combination of the Bloom's levels that would optimally improve accuracy: Knowledge, Comprehension/Application, and Analysis/Synthesis/Evaluation. Faculty development should also be considered in improving accuracy and reliability.

In spite of growing scholarship on environmentally induced non-migration research in Africa, comprehensive empirical evidence of non-migration drivers is extremely difficult to find. We review 77 rigorously selected empirical articles on the drivers of environmental non-migration. A variety of relevant keywords was applied to search, identify, and select key publications from ScienceDirect, Web of Knowledge, Google Scholar, and the Climig databases. Content analysis and inter-rater reliability (IRR) analysis were used to summarize the literature and identify key drivers of environmental non-migration decisions across all retained articles. The study structure was informed by the Foresight (2011a) conceptual framework. A growth in the non-migration literature across the time period was observed. Social factors, particularly place-based attachment and family/cultural obligations, was identified as the most important driver of non-migration (IRR score = 0.67). Environmental factors were ranked second, particularly the ability of the affected to develop coping capacity through experiential learning even in contexts marred by resource scarcity and widespread poverty. Given the limited literature on environmental nonmigration decisions, we recommend increased non-migration research across Africa to better inform policy decisions. This is particularly important as climate-related disasters surge. Frequent reviews on diverse aspects of non-migration studies are recommended to redefine future research and non-migration policy considerations in Africa.

The purpose of this study was to determine the interrater reliability of the Texas Teacher Appraisal System instrument. The performance indicators, criteria, domains, and total instrument were analyzed for inter rater reliability. Five videotaped teaching episodes were viewed and scored by 557 to 881 school administrators trained to utilize the Texas Teacher Appraisal System. The fifty-five performance indicators were analyzed for simple percentage of agreement. The ten criteria, four performance domains, and) the whole instrument were analyzed utilizing Ruder-Richardson Formula 20. Indicators were judged reliable if there was 75 percent or greater agreement on four of the five videotaped exercises. Criteria, domains, and the whole instrument were judged reliable if they yielded a -Ruder-Richardson Formula 20 score of .75 or greater on four of the Based on the findings of this study, the following conclusions v/ere drawn: 1. Forty-eight of the fifty-five performance indicators were reliable in evaluation teacher performance. 2. Seven of the performance indicators were unreliable in evaluating teacher performance. 3. None of the ten performance criteria appeared to be reliable in evaluating teacher performance. 4. None of the four performance domains appeared to be reliable in evaluating teacher performance. 5. The whole instrument was reliable in evaluating teacher performance. 6. Reliability problems with the criteria and domains appeared to be an underestimate of reliability of the Kuder-Richardson Formula 20.

Data from rating scale assessments have rank-invariant properties only, which means that the data represent an ordering, but lack of standardized magnitude, inter-categorical distances, and linearity. Even though the judgments often are coded by natural numbers they are not really metric. The aim of this thesis is to further develop the nonparametric rank-based Svensson methods for paired ordinal data that are based on the rank-invariant properties only. The thesis consists of five papers. In Paper I the asymptotic properties of the measure of systematic disagreement in paired ordinal data, the Relative Position (RP), and the difference in RP between groups were studied. Based on the findings of asymptotic normality, two tests for analyses of change within group and between groups were proposed. In Paper II the asymptotic properties of rank-based measures, e.g. the Svensson’s measures of systematic disagreement and of additional individual variability were discussed, and a numerical method for approximation was suggested. In Paper III the asymptotic properties of the measures for paired ordinal data, discussed in Paper II, were verified by simulations. Furthermore, the Spearman rank-order correlation coefficient (rs) and the Svensson’s augmented rank-order agreement coefficient (ra) were compared. By demonstrating how they differ and why they differ, it is emphasized that they measure different things. In Paper IV the proposed test in Paper I for comparing two groups of systematic changes in paired ordinal data was compared with other nonparametric tests for group changes, both regarding different approaches of categorising changes. The simulation reveals that the proposed test works better for small and unbalanced samples. Paper V demonstrates that rank invariant approaches can also be used in analysis of ordinal data from multi-item scales, which is an appealing and appropriate alternative to calculating sum scores.

<p>Syftet med denna studie var att undersöka reliabiliteten i en metod för att observera affektuttryck, Stålforsmetoden. Stålforsmetoden fokuserar primärt på affektuttryck i ansiktet, och mer specifikt den första affekten som en patient uttrycker under en psykoterapisession (”överföringsaffekt”). Den teoretiska grunden är affektteori som utvecklats av Silvan Tomkins och Paul Ekman. Data har samlats in med strukturerad observation och analyseras kvantitativt. Interbedömarreliabilitet beräknades med Cohen’s Kappa och uppgick till K = 0,03, vilket innebär att det inte finns någon statistiskt säker överensstämmelse mellan bedömarna. Möjliga orsaker till avsaknaden av interbedömarreliabilitet såsom utbildningens utformning och omfattning samt operationaliseringen av observationsvariabeln diskuteras och förslag på framtida forskning lämnas.</p> / <p>The purpose of this study was to explore the reliability of a method for observing expressions of affect, “Stålforsmetoden”. Stålforsmetoden focuses primarily on facial expression of affect, and more precisely the first expression presented by a patient in a psychotherapy session (referred to as transference affect). The theoretical basis is affect theory as developed in the works of Silvan Tomkins and Paul Ekman respectively. Data has been collected through structured observation and analyzed quantitatively. Inter-rater reliability was calculated using Cohen’s Kappa and amounted to K = 0.03, which means that there was no significant agreement between raters. This result implies that the reliability of Stålforsmetoden in its present form is insufficient and that further development of the method is needed. Possible reasons for the absence of inter-rater reliability such as the adequacy of education and the operationalization of transference affect are discussed and suggestions for future research are presented.</p>

Syftet med denna studie var att undersöka reliabiliteten i en metod för att observera affektuttryck, Stålforsmetoden. Stålforsmetoden fokuserar primärt på affektuttryck i ansiktet, och mer specifikt den första affekten som en patient uttrycker under en psykoterapisession (”överföringsaffekt”). Den teoretiska grunden är affektteori som utvecklats av Silvan Tomkins och Paul Ekman. Data har samlats in med strukturerad observation och analyseras kvantitativt. Interbedömarreliabilitet beräknades med Cohen’s Kappa och uppgick till K = 0,03, vilket innebär att det inte finns någon statistiskt säker överensstämmelse mellan bedömarna. Möjliga orsaker till avsaknaden av interbedömarreliabilitet såsom utbildningens utformning och omfattning samt operationaliseringen av observationsvariabeln diskuteras och förslag på framtida forskning lämnas. / The purpose of this study was to explore the reliability of a method for observing expressions of affect, “Stålforsmetoden”. Stålforsmetoden focuses primarily on facial expression of affect, and more precisely the first expression presented by a patient in a psychotherapy session (referred to as transference affect). The theoretical basis is affect theory as developed in the works of Silvan Tomkins and Paul Ekman respectively. Data has been collected through structured observation and analyzed quantitatively. Inter-rater reliability was calculated using Cohen’s Kappa and amounted to K = 0.03, which means that there was no significant agreement between raters. This result implies that the reliability of Stålforsmetoden in its present form is insufficient and that further development of the method is needed. Possible reasons for the absence of inter-rater reliability such as the adequacy of education and the operationalization of transference affect are discussed and suggestions for future research are presented.

As ultrasound imaging gains popularity in speech research, an important question to address is the reliability of the measures taken from these images. This study examines the reliability of hand measures of ultrasound data collected by graduate student researchers in the University of South Florida's speech science lab. Speech production data from Ultrasound analysis of velar fronting (Wodzinski, 2004) and Ultrasound study of errors in speech production (Frisch, 2003) were used to obtain inter-rater reliability measures. This study compares the raters choice of video frame depicting alveolar or velar closure image, anterior and posterior points of closure, tongue blade and velar angle measurements, as well as a measurement of the tongue dorsum distance from the ultrasound probe.

Unga med normbrytande beteende löper en relativt hög risk för en långvarig negativ utveckling. För att förhindra detta krävs tidiga effektiva insatser som i sin tur kräver tillförlitliga bedömningsinstrument som identifierar risker och behov hos unga med, eller i riskzonen för normbrytande beteende. Just detta är syftet med ESTER-bedömning. Föreliggande studies syfte var att undersöka interbedömarreliabiliteten av ESTER-bedömning inklusive en ny kandidatskala för riskfaktorerna . Två oberoende bedömare genomförde ESTER-bedömningar på journalmaterial tillhörande 30 tvångsomhändertagna flickor, 15-20 år. Resultaten visar en spridning mellan bristfällig till mycket bra interbedömarreliabilitet på de 19 risk- och skyddsfaktorerna i ESTER-bedömning, med få fall av total oenighet mellan bedömarna. En jämförelse mellan den befintliga skalan och kandidatskalan visade marginella skillnader. Vidare forskning av interbedömarreliabilitet för ESTER-bedömning bör testa skalorna var för sig och inkludera intervjuer som informationskälla. / Youths with normbreaking behavior is at higher risk for a negative development. To prevent this, there is a need for reliable assessments that can identify risk and need for youths with, or at risk for normbreaking behavior. This is the purpose of ESTER-assessment. This study evaluated the inter-rater reliability of two different scales in ESTER-assessment. Two independent judges conducted ESTER-assessment on case files of 30 institutionalized girls, aged 15-20 years. The results revealed poor to excellent agreement and few cases of total disagreements. The two different scales showed a minimal difference. In further research of the inter-rater reliability of ESTER-assessment there is a need for testing the two scales separately and to include interviews as a source of information.