There is a dearth of validated neonatal feeding assessment instruments available for use in clinical practice in resource-constrained developing contexts. The Neonatal Feeding Assessment Scale (NFAS) was developed to identify and diagnose oropharyngeal dysphagia (OPD) in neonates. The main aim of the study was to develop and test the psychometric performance of a clinical assessment scale for the early identification and diagnosis of OPD in the high-risk neonatal population in South Africa. To meet the main aim, the research project was divided into three separate studies. The research design across the three studies was an exploratory sequential mixedmethod design. The NFAS was developed using the Delphi method in the first study. Two international and three South African speech-language therapists (SLTs) formed the expert panel that participated in two rounds of electronic questioning to develop the instrument. For the second and third studies, a comparative crosssectional within-subject design was used. In the second study the participants were 20 neonates with a median age of 35.0 weeks gestational age (GA) in a 29-bed neonatal intensive care unit (NICU). In the third study 48 participants with a median age of 35.5 weeks GA were included. During the second study the preliminary psychometric performance of the NFAS was determined and in the third study, the final psychometric properties of the NFAS were determined to describe the validity and reliability of the NFAS. The NFAS was developed and approved, using expert collaboration through the Delphi method in the first study. All participants agreed on the need for the development of a valid clinical feeding assessment instrument to use with the highrisk neonatal population. The initial NFAS consisted of 240 items across 8 sections; after the Delphi process was implemented, the final format was reduced to 211 items across 6 sections. The final format of the NFAS is scored using a binary scoring system guiding the clinician to identify the presence or absence of OPD. All members agreed on the format, the scoring system and the feeding constructs addressed in the final format of the NFAS. The second study showed that 9 out of 20 participants presented with OPD on the NFAS. Comparison of NFAS results with modified barium swallow studies (MBSS). indicated that all participants with OPD were correctly identified (100% specificity). The sensitivity was 78.6%, indicating that three participants were falsely identified with OPD on the NFAS. The instrument took approximately 30 minutes to complete during observation of a habitual feeding session with the mother. Inter-rater reliability was determined on 50% (n=10) of the study sample. Substantial agreement (80%) was obtained between two raters in five of the six sections of the NFAS and on the diagnostic outcome of the scale. The preliminary performance of the NFAS appeared to be promising. The formal validation process of the NFAS then followed on a larger sample in the third study. Results of the third study indicated that 15 of the 48 participants were identified with OPD on the NFAS, whereas 14 of these 15 infants were diagnosed on MBSS. A sensitivity score of 78.6% was obtained, with specificity determined to be 88.2% for the newly developed NFAS. The subsequent accuracy of the NFAS to identify OPD correctly was 85.4% when compared with the MBSS outcome. Inter-rater reliability was determined using 35% of the sample. The agreement on overall instrument outcome between the two raters was considered substantial beyond chance, with Cohen's Kappa at 0.598, with an asymptotic standard error of 0.211. The scale may be of use to SLTs working without MBSS equipment and to reach underserved preterm neonates. Inexperienced SLTs may benefit from observational prompts provided by the NFAS. The NFAS may be suitable for use in South Africa and similar developing contexts to identify and diagnose high-risk neonates with OPD. / Thesis (DPhil)--University of Pretoria, 2016. / Speech-Language Pathology and Audiology / DPhil / Unrestricted

Alla elever i svensk grundskola har rätt till utbildning på lika villkor och det innebär att deras prestationer bör bli bedömda på ett likvärdigt sätt oavsett sociala, ekonomiska eller geografiska förhållanden. Ett försök att uppnå likvärdiga, rättvisa och jämförbara betyg är att använda nationella prov och det överordnade syftet i föreliggande uppsats är bidra med kunskap om det nationella provet i svenska för årskurs 6 ur ett reliabilitetsperspektiv. Studien genomförs med en kombination av kvantitativ och kvalitativ metod. I den kvantitativa delen undersöks samstämmigheten i 15 svensklärares bedömningar av 30 elevtexter från ett tidigare nationellt prov i form av statistiska samstämmighetsmätningar. I den kvalitativa delen av studien analyseras tillämpningen av anvisad bedömningsmatris i sambedömningssamtal. Materialet som används i uppsatsen är skapat inom ramen för projektet Equa, som är en studie där effekten av en specifik bedömarträningsmodell, Rater Identity Development (RID), på reliabiliteten vid bedömning av elevers språkförmåga undersöks. Studiens resultat visar att nivåerna av samstämmighet i bedömargruppen är låga, vilket innebär en variation i bedömningarna med stora konsekvenser för elevernas resultat. Resultaten indikerar vidare att lärarna är skickligare på att rangordna elevtexterna på ett likvärdigt sätt än att vara helt överens om betygssättningen. Analyserna av sambedömningssamtalen visar att bedömningsmatrisen visserligen ger lärarna viss stöttning, men i stor utsträckning skapar densamma även utmaningar i bedömningsprocessen. / Every student in the Swedish school system have the right to equal education which means that their performances should be assessed on an equal basis regardless of social, economic or geographical conditions. As one way to obtain equivalent, fair and comparable grading Sweden has for some time had a system of national tests as a guide for teachers setting final overall grades and the main purpose of this essay is to provide further knowledge about the national test in Swedish for year six in a reliability perspective. The study was carried out using a combination of quantitative and qualitative method. In the quantitative part of the study the reliability of the assessment of 30 student texts from previous national tests done by 15 Swedish teachers is measured. The qualitative part consists of the analysis of the application of the designated rating scale in the recorded collaborative assessment talks. The material in the study is created within the Equa project, which is a study where the effect of a specific rater training model, Rater Identity Development (RID), on the reliability of the assessment of students language performances is studied. The results of the study show low levels of inter-rater reliability, which means that there is a variation in the ratings with large consequences for the students´ results. The findings also indicate that the teachers are more skilled in ranking the students texts in a similar way than to exactly agree on the grading. The analysis of the collaborative assessment talks indicate that although the rating scale may be of some help, it also poses many challenges in the assessment process.

No description available.

The Performance Assessment and Evaluation System (PAES) is used by all major universities in the state of Utah to measure the effective teaching skills of preservice candidates as they progress through their teaching preparation program. The resulting ratings are used to make high-stakes decisions relating to course completion as well as recommendation for licensure. This study analyzes the factor structure and tests for measurement invariance of PAES ratings assigned to 663 elementary education candidates at Brigham Young University across two measurement occasions. The candidates were rated by 30 clinical faculty associates. This study also examines the degree to which differential rater effects impact the PAES ratings of these candidates. A bifactor model, with a general factor measuring effective teaching skills measured through observation, and a specific factor measuring effective teaching skills evaluated through conversation best fit the model. Evidence of measurement invariance was found between evaluations completed for Practicum 1 and Practicum 2 candidates. This study also found that differential rater effects impact the PAES ratings of individual candidates, indicating that a candidate's rating may depend on which rater completed the evaluation. Similar research studies should be conducted to analyze the quality of PAES ratings of teacher candidates in the various secondary education programs at BYU. In addition, since the PAES is used at other teacher preparation colleges and universities in Utah, similar research studies should be conducted to examine the quality of PAES ratings of teacher candidates at these other institutions.

Bakgrund En handhållen dynamometer (HHD) är ett kliniskt användbart mätverktyg för att mäta muskelstyrka. På Akademiska sjukhuset i Uppsala finns en HHD av märket MicroFET2® (Hoggan Scientific). Det finns ett fåtal studier gjorda som undersöker intrabedömarreliabiliteten vid mätningar av isometrisk muskelstyrka av knäledens muskelgrupper med denna dynamometer. Syfte Syftet med studien var att undersöka den absoluta och relativa reliabiliteten vid mätningar av maximal muskelstyrka i knäledens muskelgrupper med en HHD. Metod Psykometrisk design med test-retest förfarande. Totalt undersöktes 24 friska vuxna vid två mättillfällen med 6–8 dagars mellanrum. Den relativa reliabiliteten presenterades med Intraclass correlation coefficient (ICC 2,1). Den absoluta reliabiliteten presenteras i standard error of measurement (SEM), SEM% samt smallest detectable change (SDC). Resultat Resultaten visade mycket hög relativ reliabilitet för knäledens extensioner (ICC = 0,950,96) och för knäledens flexorer (ICC = 0,94–0,96). SEM varierade mellan 33,58N-34,81N för knäledens extensorer och 14,16N-17,16N för knäledens flexorer. SDC varierade mellan 93,08N–96,49N för knäledens extensorer och 39,25N-47,57N för knäledens flexorer. Konklusion Resultaten indikerar att HHD har mycket hög relativ reliabilitet för bedömning av styrka i knäledens stora muskelgrupper hos ett bekvämlighetsurval med friska vuxna människor.  Den relativa och absoluta reliabiliteten bedömdes överlag vara i enlighet med tidigare studier. För implementering i klinik behöver framtida studier utvärdera absolut och relativ reliabilitet på specifika patientpopulationer. / Background Hand held dynamometry (HHD) is a practical device for testing muscle strength. There is currently a HHD (MicroFET2® - Hoggan Scientific) available at Uppsala University hospital. Few reliability studies excist where the objective is to examine the intra-rater reliability of isometric maximal knee-strength using this device. Objective To examine the absolute and relative reliability of isometric maximal knee-strength using a MicroFET2. Method Psychometric design through test-retest with 6–8 days between measures on 24 healthy adults. The relative reliability expressed as Intra-class correlation coefficient (ICC 2,1) and the absolute reliability as standard error of measurement (SEM), SEM%, and smallest detecatble change (SDC). Results The results of this study showed very good reliability for both knee extensors (ICC = 0,950,96) and knee flexors (ICC = 0,94–0,96). Regarding the absolute reliability SEM ranged from 33,58N-34,81N for knee extensors and 14,16N-17,16N for knee flexors. SDC ranged from 93,08N-96,49N for knee extensors and 39,25N-47,57N for knee flexors. Conclusion The results indicate that HHD has a very good relative reliability for assessing strength in the large muscle groups of the knee joint when examined on a group of healty adults. The relative and absolute reliability was generally in accordance when compared with previous studies. Before being implemented in praxis, future studies need to evaluate absolute and relative reliability on specific patient populations.

The focus of this research was to examine the initial year of implementation of the Marzano Teacher Evaluation Model and iObservation®tool (Learning Sciences International, 2012) as it related to student achievement in the School District of Osceola County, Florida and to determine if the Marzano model improved the ability to determine teacher effectiveness with more accuracy than previous models of teacher evaluation used in the school district. Twelve research questions guided this study concerning the relationship and predictability between the variables of teacher instructional practice scores, number of observations reported in the iObservation®tool, and student achievement in Grades 3-5 using reading and mathematics FCAT 2.0 DSS scores. Linear Regression analysis suggested that for Grade 3 reading and mathematics the instructional practice mean had statistical significance in predicting performance and was a strong predictor of Grade 3 FCAT reading and mathematics performance. Linear Regression analysis suggested that for Grade 3 reading and mathematics the instructional practice mean had statistical significance in predicting performance and was a strong predictor of Grade 3 FCAT reading and mathematics performance. Linear Regression analysis further suggested no statistical significance or predictability for Grades 4, 5 for instructional practice mean and Grades 3,4,5 for observation mean related to FCAT reading and mathematics performance. Caution should be used when attempting to interpret these findings, as this study was based solely on intitial year implemention data. Implications for practice are also discussed in this study.

The likelihood of conducting safe operations increases when operators ave effectively integrated their knowledge of the operation into meaningful relationships, referred to as knowledge structures (KSs). Unlike knowing isolated facts about an operation, well integrated KSs reflect a deeper understanding. It is, however, only the isolated facts that are often evaluated in training environments. To know whether an operator has formed well integrated KSs, KS evaluation methods must be employed. Many of these methods, however, require subjective, human-rated evaluations. These ratings are often prone to the negative influence of a rater's limitations such as rater biases and cognitive limitations; therefore, the extent to which KS evaluations are beneficial is dependent on the degree to which the rater's limitations can be mitigated. The main objective of this study was to identify factors that will mitigate rater limitations and test their influence on the reliability and validity of KS evaluations. These factors were identified through the delineation of a framework that represents how a rater's limitations will influence the cognitive processes that occur during the evaluation process. From this framework, one factor (i.e., operation knowledge), and three mitigation techniques (i.e., frame-of-reference training, reducing the complexity of the KSs, and providing referent material) were identified. Ninety-two participants rated the accuracy of eight KSs over a period of two days. Results indicated that reliability was higher after training. Furthermore, several interactions indicated that the benefits of domain knowledge, referent material, and reduced complexity existed within subsets of the participants. For example, reduced complexity only increased reliability among evaluators with less knowledge of the operation. Also, referent material increased reliability only for those who scored less complex KSs. Both the practical and theoretical implications of these results are provided.

No description available.

Objective: The objective for this study was to evaluate the psychometric properties of Psychodynamic Therapy Scale (PTS), including inter-rater reliability, internal consistency and predictive validity. Method: 20 video-taped sessions from 16 different therapists were evaluated by two independent raters using the PTS. The therapy conducted was Life-Stress Interview which is a single-session treatment that stems from psychodynamic theory. Inter-rater reliability was calculated using Intraclass Correlation Coefficient (ICC). Internal consistency was measured using Cronbach's α. Predictive validity was calculated using a linear regression model on the change between baseline and six week follow-up on two outcome measures (DERS-16 & PHQ-15). Results: The ICC for the complete 12-item scale was .77, suggesting a good inter-rater reliability for the PTS. The internal consistency was α = .82 for the entire scale, indicating a good internal consistency. The regression analysis did not show any significant regression on either outcome measure, despite DERS-16 coming close. Discussion: The results indicate that the PTS has a good overall inter-rater reliability and internal consistency. However, further research is needed to determine if these findings still holds true when evaluating more traditional psychodynamic therapy methods and with a higher degree of independence between the raters. Furthermore, future studies could explore if there are potential alterations that can be made to improve the scale.

Fragestellung und Zielsetzung: Ausgehend von dem Ziel, kommunikative Fähigkeiten von Studierenden der Humanmedizin objektiv zu messen, sollte in dieser Studie untersucht werden, ob eine ins Deutsche übersetzte Kurzversion des „Calgary-Cambridge Observation Guide for the Medical Interview“ (CCOG) von Kurtz und Silverman (1996) valide und reliabel ist und sich damit die Kommunikationsfähigkeit von Studierenden der Medizin bewerten lässt. Methode: Eine Auswahl von Ärzten, wissenschaftlichen Mitarbeitern und Studierenden des klinischen Studienabschnittes evaluierten im Abstand von mindestens drei Monaten fünf Anamnese-Videos mittels der ins Deutsche übersetzten „CCOG-Kurzversion“. Die Videos bestanden aus je einem Gespräch zwischen einem Studierenden des ersten klinischen Semesters in der Rolle des Arztes bzw. einer approbierten Ärztin und einem Schauspielpatienten – in unterschiedlicher Kommunikationsqualität. Die Auswertung erfolgte deskriptiv nach folgenden Kriterien: Bewertungszeitpunkt, Geschlecht bzw. Gruppe der Rater, Qualität der Videogespräche. Darüberhinaus wurden eine explorative und eine konfirmatorische Faktorenanalyse berechnet und die Retest-Reliabilität (Intra-Rater-Reliabilität) sowie die Intra-Class-Correlation (Inter-Rater-Reliabilität) bestimmt. Ergebnisse: 30 Rater beteiligten sich an der Studie, davon drei als sog. „Goldstan-dard“. Die Gesamtbewertung aller fünf Anamnese-Videos zeigte eine geringfügige Verbesserung in der Notenvergabe beim zweiten Bewertungszeitpunkt. In der Benotung waren professionelle Rater generell etwas strenger als Laien, Ärzte etwas strenger als Studierende und weibliche Rater etwas strenger als männliche. „Gold-standard“ und übrige Rater unterschieden sich bei einzelnen Items bis zu 1,6 Notenpunkten (z.B. beim Item „verbale/nonverbale Unterstützung des Patienten“). In der Originalversion enthält der CCOG 28 Items, die in sechs Skalen (mit jeweils 3 bis 7 Items) zusammengefasst sind. Diese Struktur ließ sich in der hier gerechneten Faktorenanalyse nur bedingt abbilden. Gemäß Eigenvalue > 1 genügten 5 Faktoren zur Abbildung bzw. Aufteilung der Items. Darüber hinaus zeigte sich eine andere Skalenzuordnung als im Original und über die Hälfte der Items (15/28) lud auf demselben Faktor. Auch die Inter-Rater-Übereinstimmung in der Beantwortung einzelner Items war nicht optimal (ICC-Range: 0,05 bis 0,57). Schlussfolgerungen: Die CCOG-Kurzversion zeigte relativ gute Übereinstimmun-gen bei der Retest-Reliabilität. Schwierigkeiten zeigten sich bei der Benotung einiger Items im Vergleich zwischen dem „Goldstandard“ und den übrigen Ratern. Die Skalen-Struktur der Items und die Inter-Rater-Reliabilität sind nur bedingt akzeptabel. Vielleicht hätte eine dreistufige Bewertungsskala oder eine homogene Rater-Gruppe oder auch eine bessere Schulung das Ergebnis der ICC verbessert. Es sollten einige Items gestrichen, sprachlich trennschärfer formuliert oder anders zusammengefasst, neue Items hinzugefügt und Skalen neu strukturiert werden.