This dissertation illustrates how to detect the rater centrality effect in a simulation study that approximates data collected in large scale performance assessment settings. It addresses three research questions that: (1) which of several centrality-detection indices are most sensitive to the difference between effect raters and non-effect raters; (2) how accurate (and inaccurate), in terms of Type I error rate and statistical power, each centrality-detection index is in flagging effect raters; and (3) how the features of the data collection design (i.e., the independent variables including the level of centrality strength, the double-scoring rate, and the number of raters and ratees) influence the accuracy of rater classifications by these centrality-detection indices. The results reveal that the measure-residual correlation, the expected-residual correlation, and the standardized deviation of assigned scores perform better than the point-measure correlation. The mean-square fit statistics, traditionally viewed as potential indicators of rater centrality, perform poorly in terms of differentiating central raters from normal raters. Along with the rater slope index, the mean-square fit statistics did not appear to be sensitive to the rater centrality effect. All of these indices provided reasonable protection against Type I errors when all responses were double scored, and that higher statistical power was achieved when responses were 100% double scored in comparison to only 10% being double scored. With a consideration on balancing both Type I error and statistical power, I recommend the measure-residual correlation and the expected-residual correlation for detecting the centrality effect. I suggest using the point-measure correlation only when responses are 100% double scored. The four parameters evaluated in the experimental simulations had different impact on the accuracy of rater classification. The results show that improving the classification accuracy for non-effect raters may come at a cost of reducing the classification accuracy for effect raters. Some simple guidelines for the expected impact of classification accuracy when a higher-order interaction exists summarized from the analyses offer a glimpse of the "pros" and "cons" in adjusting the magnitude of the parameters when we evaluate the impact of the four experimental parameters on the outcomes of rater classification. / Ph. D.

Legislative pressures are being brought to bear on South African employers to demonstrate that occupational assessment is scientifically valid and culture-fair. The development of valid and reliable performance-based assessment tools will enable employers to meet these requirements. The general aim of this research was to validate a performance-based assessment battery for the placement of sales representatives. A literature survey examined alternative assessment measures and methods of performance measurement, leading to the conclusion that the combination of the work sample as a predictor measure and the managerial rating of performance as a criterion measure offer a practical and cost-effective assessment process to the sales manager. The empirical study involved 54 sales persons working for the Commercial division of an oil marketing company, selling products and services to the commercial and industrial market. By means of the empirical study, a significant correlation was found between performance of sales representatives in terms of the performance-based assessment battery for the entry level of the career ladder and their behaviour in the field as measured by the managerial performance rating instrument. The limitations of the sample, however, prevent the results from being generalised to other organisations. / Industrial & Organisational Psychology / M.A. (Industrial Psychology)

Leadership theories have evolved since their inception and leadership development should also evolve in response to the changing needs of their organizations. There is a gap in the literature on the functionality of the various 360-feedback platforms that are mentioned in literature which makes it difficult to understand what reporting functionality is available in existing systems. The goal of this study was to examine participants' views of a newly developed Adaptive Learning 360-feedback (AL360) treatment platform for leadership development with a focus on self-directed learning for improved accountability. Participants were asked to share their learning development experience in the areas of person, process, and context with suggestions for improvement and how did this development compare to any other leadership development that they had experienced in the past. The AL360 platform was built by applying the research findings of scholars in the areas of leadership, 360-feedback leadership development and educational technologies. The treatment was applied in a business setting for all levels of leadership development. The case study identified creation of leadership of self-awareness and accountability for development without significant intervention of an executive coach. The research findings also list improvement opportunities, limitations, and future considerations.

No description available.

Cette thèse porte sur l’évaluation du leadership par une approche multi-évaluateurs (dite à 360 degrés). Tout d'abord les différents modèles du leadership, allant d'une conception unitaire, à une conception interractionniste puis à une définition en termes d'effet, sont détaillés. Puis en étudiant la question de la mesure, on met en évidence qu'avec la popularité croissante de leurs modèles, certains outils d'évaluation sont devenus des questionnaires psychométriques à part entière. Pourtant, du fait des limites de l’auto-évaluation, une nouvelle approche du leadership à vu le jour: l'évaluation à 360°. Elle évalue les qualités d'un manager en interrogeant les personnes travaillant avec lui (subordonnés, collègues, supérieurs...) et en comparant leurs évaluations avec la propre évaluation du manager. Un des points central de notre recherche a été d’étudier les apports et les limites de cette méthode. Tout d'abord on a vérifié dans quelle mesure les caractéristiques souvent utilisées pour expliquer la variabilité entre les catégories d'observateurs permettaient réellement de comprendre les écarts observés. Ces analyses ont montré que malgré des résultats significatifs, ces variables n’expliquent que très partiellement la variance existante. De par ces conclusions l’accent a été mis, non plus sur les différences inter-groupes, mais sur l'accord et le désaccord au sein des groupes d'observateurs. Enfin, la dernière partie revient aux bases méthodologiques et théoriques de la mesure en cherchant à proposer un modèle psychométriques qui conviendrait aux résultats de questionnaires à 360°, permettant de donner un cadre conceptuel au recours à des évaluateurs multiples. / This thesis is centered on leadership assessment through multi-rater evaluation, commonly known as 360 degrees assessment. First, leadership models were presented, and then, we discussed the measurement aspects of leadership, wherein we observed that some tools became full fledged psychometric assessments owning to the growing popularity of their underlying theory. Nevertheless, the concerns over the inherent limitations of self-report measures continued to be a major challenge in leadership assessment. This led to a new assessment approach called 360 degrees in which the characteristics of leaders are assessed by people working with them (subordinates, colleagues, superiors...) and compared with the leaders' self-appraisals. The focal point of this thesis was to study the advantages as well as the limitations of this approach. The study started with examining how well the variables which are supposed to explain the inter-rater variability were actually helpful in understanding the observed variance among observers. Those analysis yielded significant results despite the fact that those variables could explain only a very limited amount of variance. Following these observations, the study switched its focus from inter-group differences to intra-group / inter-rater agreement and disagreement. Finally, the last part of this thesis gets back to methodological and theoretical basics of measurement theory and proposes a psychometric model that would suit the 360 degrees assessments followed by a conceptual framework for the studies using multi-rater techniques.

Legislative pressures are being brought to bear on South African employers to demonstrate that occupational assessment is scientifically valid and culture-fair. The development of valid and reliable performance-based assessment tools will enable employers to meet these requirements. The general aim of this research was to validate a performance-based assessment battery for the placement of sales representatives. A literature survey examined alternative assessment measures and methods of performance measurement, leading to the conclusion that the combination of the work sample as a predictor measure and the managerial rating of performance as a criterion measure offer a practical and cost-effective assessment process to the sales manager. The empirical study involved 54 sales persons working for the Commercial division of an oil marketing company, selling products and services to the commercial and industrial market. By means of the empirical study, a significant correlation was found between performance of sales representatives in terms of the performance-based assessment battery for the entry level of the career ladder and their behaviour in the field as measured by the managerial performance rating instrument. The limitations of the sample, however, prevent the results from being generalised to other organisations. / Industrial and Organisational Psychology / M.A. (Industrial Psychology)

Purpose: The purpose of this study was to determine the usefulness of SBAR-cued web-based communication skills training and address study participants' perceptions of the training. Specific Aims: Evaluate the usefulness and accuracy of participants to answer prompts of SBAR-cued communication responses. Describe individuals' perspectives of the acceptability of using SBAR patient-HCP communication simulation to better prepare for ED visits during a SCC. Framework: This study was guided by The Theory of Self-Care Management for Sickle Cell Disease (SCMSCD). Design: A secondary analysis was conducted using a qualitative descriptive approach. Inter-rater reliability (IRR) of qualitative data was used to evaluate the usefulness and accuracy of participants to answer prompts of SBAR-cued communication responses. Content analysis was also utilized to describe individuals' perspectives of the acceptability of using SBAR patient-HCP communication simulation to better prepare for ED visits during a SCC. Results: IRR between raters ranged from 64%-94% with predominant themes of (1) Patient-Provider Communication and Interaction, (2) Patients want to be Heard and Believed, (3) Accuracy of the ED Experience and Incorporating the Uniqueness of each Patient and (4) Overall Usefulness of the Video Trainer emerging. Conclusions: This secondary analysis supported how SBAR can be effectively used to assist patients in a SCC to communicate with their HCP. Participants' responses indicated the training module facilitated communication between patients and HCPs.

<div> <div> <div> <p>Personality traits are important factors of psychotherapy for many reasons, as they relate to a variety of clinical outcomes, can complicate treatment, and can also be targets of treatment interventions. Because of its clinical prevalence and impact, it is imperative that therapists are able to effectively assess and treat personality pathology. Previous research has indicated that both client and therapist ratings of personality can provide meaningful information, and this varies across different sessions, but no study to date has examined both client and therapist ratings across the entire therapeutic intervention. There is also limited information on the agreement of client and therapist ratings of personality, as the majority of studies only examine the outset, the end, or a random time point of treatment. Examining only one point in time – or just the beginning and end – misses valuable information regarding possible changes in personality occurring throughout treatment. Using a naturalistic dataset of 128 client-therapist dyads (3,440 observations), the present study examined the longitudinal trajectory of client and therapist ratings of personality change throughout intervention while also accounting for state-level distress. Additionally, the agreement between clients and therapists were examined throughout treatment for any potential patterns of change using rank-order, mean-level, and absolute agreement. Significant patterns of trait change and change in absolute agreement across treatment were assessed using multilevel modeling. Last but not least, the agreement among clients and therapists were examined as potential predictors of therapeutic outcomes, such as engagement and improvement. The results provided evidence for significant decreases in neuroticism that were reported by the client but not therapist that suggest clients might be report decreases in state-level distress rather than true trait change. There were meaningful fluctuations in agreement across treatment, particularly for openness to experience and neuroticism, but the overall agreement – or lack thereof – did not significantly predict client engagement or improvement. Results highlight several clinical implications that are discussed. </p> </div> </div> </div>

No description available.

<p>Introduction</p> <p>The most common complication following invasive cardiac procedures is the development of vascular access site (VAS) bruising. The extent and impact of VAS bruising is poorly understood and minimally reported in the literature. Research into this common post-procedure complication is hindered by the lack of a reliable bruise measurement tool, and the concept that VAS bruising is a minor complication. This mixed methods study examined the inter-rater reliability of two methods to measure VAS bruise size. The embedded qualitative descriptive study explored patient perceptions of VAS bruising.</p> <p>Methods</p> <p>Participants having femoral or radial artery puncture for invasive cardiac procedures were included in this study. Participants reporting VAS bruising completed self measurement of bruise size using two methods, linear measurement and planimetry. The principal investigator and research assistant completed bruise measurements at the same time, and were blinded to participant and each others’ measurements. Following bruise measurement, the principal investigator conducted semi-structured interviews on a convenience sample of participants; including both sexes, a range of ages, and bruise sizes.</p> <p>Results</p> <p>Measurements were completed on 40 participants with VAS bruises. Analysis of inter-rater reliability was done using the intraclass correlation coefficient (ICC), two-way random effects model. The inter-rater reliability for both linear measurement and planimetry between all three measurers was high (.929; .914 respectively). Analysis of participant narratives uncovered three major themes concerns, impact and mediating factors, with several sub-themes.</p> <p>The findings of this study support the reliability of patient VAS bruise measurement using linear measurement and planimetry. The goals and available resources for VAS research may determine the choice of measurement approach. Qualitative descriptive results indicate that patients have concerns related to VAS bruising and that this bruising may impact activities of daily living. Future research examining VAS complications should include evaluation of VAS bruising as significant patient outcome.</p> / Doctor of Philosophy (PhD)