The use of multi-rater feedback as a way to analyze perceptions within the context of job performance and leadership in the Federal Bureau of Investigation (FBI) was examined. Research in this domain is notable as this type of evaluation is now being done with regularity in the private sector and is starting to be utilized more extensively in the public sector, but is still being used to a limited extent in law enforcement. The path of this research examined differences between self-assessments and assessments of others (peers and subordinates) in dimensions of leadership as measured by the same multi-rater instrument at two points in time. This research effort made use of a multi-rater survey instrument called the "Leadership Commitments and Credibility Inventory System (LCCIS)," designed by Keilty, Goldsmith, and Company, which is used in multiple industries and was expanded to capture characteristics considered important to FBI leaders. Results showed high ratings on a five point Likert scale as indicated by mean averages of self and others. Additionally, Z scores, t tests and ANCOVA indicated that FBI supervisors did not overestimate their leadership, as indicated by (1) an overall leadership measure at time two compared to time one, (2) a greater perceptual agreement between others and self existing on second multi-rater assessments than on the initial assessments, and (3) any statistical differences of means in all measured categories at time two versus time one. Various subcategories of the assessment showed a mixture of non-statistically significant results and that subordinates and peers perceived leaders differently. Further, analysis of two unique dimensions of the LCCIS, "Manage Diversity" and "Build Public Trust" showed exceptionally high results. The implications of the present research are that leadership in the FBI, as measured by different dimensions, is strong. Yet, there is no evidence that leaders or others in this organization change their perceptions over time. These findings may point to the need for multi-rater instruments to be used in concert with personal development plans in order to improve the perception of leadership. / Ph. D.

Managerial self-awareness is thought to impact leadership. A multi-rater feedback instrument was used to gather performance data on 70 managers in a large multi-national airline in regards to five leadership dimensions: making sound decisions, driving for results, effective communication, self-management, and innovation. Difference scores between self and direct reports were calculated and used as the operational definition of managerial self-awareness. T-tests were run to examine the difference between high performers and average performers. No significant differences were found. Additionally, correlational measures between the five leadership competencies and the managerial self-awareness measure indicated statistically weak relationships.

Multi-rater tools also referred to as 360-degree feedback tools, are frequently used in addition to traditional supervisory appraisals due to sources (i.e., supervisor, peer, direct report) unique perspectives and opportunities to view different aspects of job performance. Research has found that the differences among sources are most prevalent between self and other ratings, and the direction of agreement is related to overall job performance. Research has typically focused on one form of agreement, the direction of an individual's self-ratings compared to others' ratings. The current study expanded on past research on rater agreement using a data set (n = 215) consisting of multi-rater data for professionals participating in a leadership development process. The study examined the ability to predict job performance with three different measures of self-other agreement (i.e., difference between overall mean scores (difference), mean absolute difference across items (difference), and mean correlation across items (similarity)). The study also examined how the relationships may differ across performance dimensions. The final purpose was to explore how the importance of the performance dimensions, as rated by the participant, may moderate the relationship between self-other agreement and job performance. Partial support for study's hypotheses was found. The direction and difference measures of agreement on the overall multi-rater tool and performance dimensions accounted for a significant amount of the variance in job performance. The relationship between the similarity measure of agreement and job performance, and the moderating effect of importance were not supported in the current study.

Leadership theories have evolved since their inception and leadership development should also evolve in response to the changing needs of their organizations. There is a gap in the literature on the functionality of the various 360-feedback platforms that are mentioned in literature which makes it difficult to understand what reporting functionality is available in existing systems. The goal of this study was to examine participants' views of a newly developed Adaptive Learning 360-feedback (AL360) treatment platform for leadership development with a focus on self-directed learning for improved accountability. Participants were asked to share their learning development experience in the areas of person, process, and context with suggestions for improvement and how did this development compare to any other leadership development that they had experienced in the past. The AL360 platform was built by applying the research findings of scholars in the areas of leadership, 360-feedback leadership development and educational technologies. The treatment was applied in a business setting for all levels of leadership development. The case study identified creation of leadership of self-awareness and accountability for development without significant intervention of an executive coach. The research findings also list improvement opportunities, limitations, and future considerations.