As a process designed to collect information about jobs, job analysis is one of the most fundamental aspects of personnel psychology. It forms the foundation upon which almost every other human resource management component is built, including selection, compensation, performance appraisal, and training program development. Despite the considerable evidence of human fallibility in other judgment processes, many have followed the implicit assumption that job analysis information is accurate without actually examining this proposition. This study considers two potential sources of job analysis rating inaccuracy — the source of the ratings and the type of instrument utilized to collect ratings. By utilizing less job-familiar job analysis raters and shorter, more holistic job analysis instruments, industrial-organizational psychologists have attempted to attenuate the time and costs associated with the job analysis process; however, findings regarding the reliability and accuracy of such practices are questionable. Hypotheses tested in the current study indicated that decomposed measures of job behavior converged to a greater degree with an external job analysis than did holistic measures. Interrater agreements for all types of raters and across all types of instruments are concluded to be inadequate. Potential explanations from the cognitive and social psychological domains for these findings are conjectured and directions for future research are noted. / Ph. D.

The study of interrater agreement and itnerrater reliability attract extensive attention, due to the fact that the judgments from multiple raters are subjective and may vary individually. To evaluate interrater agreement and interrater reliability, five different methods or indices are proposed: percentage of agreement, kappa coefficient, the correlation coefficient, intraclass correlation coefficient (ICC), and generalizability (G) theory. In this study, we introduce and discuss the advantages and disadvantages of these methods to evaluate interrater agreement and reliability. Then we review and explore the rank across these five indices by use of frequency in practice in the past five years. Finally, we illustrate how to use these five methods under different circumstances and provide SPSS and SAS code to analyze interrater agreement and reliability. We apply the methods above to analyze the data from Parent-Child Interaction System of global ratings (PARCHISY), and conclude as follows: (1) ICC is the most often used method to evaluate interrater reliability in recent five years, while generalizability theory is the least often used method. The G coefficients provide similar interrater reliability with weighted kappa and ICC on most items, based on the criteria. (2) When the reliability is high itself, different methods provide consistent indication on interrater reliability based on different criteria. If the reliability is not consistent among different methods, both ICC and G coefficient will provide better interrater reliability based on the criteria, and they also provide consistent results.

No description available.

Despite the importance of job specifications to much of industrial and organizational psychology, little is known of their reliability or validity. Because job specifications are developed based on input from subject matter experts, interrater agreement is a necessary condition for their validity. The purpose of the present research is to examine the validity of job specifications by assessing the level of agreement in ratings and the effects of occupational tenure, occupational complexity, and the abstractness of rated worker requirements. Based on the existing literature, it was hypothesized that (1) agreement will be worse than acceptable levels, (2) agreement will be higher among those with longer tenure, (3) agreement will be lower in more complex occupations, (4) the effect of occupational tenure will be more pronounced in complex than simple occupations, (5) agreement will be higher on more abstract items, and (6) agreement will be lowest for concrete KSAOs in complex occupations. These hypotheses were tested using ratings from 38,041 incumbents in 61 diverse occupations in the Federal government. Consistent with Hypothesis 1, agreement failed to reach acceptable levels in nearly every case, whether measured with the awg or various forms of the rwg agreement indices. However, tenure, occupational complexity, and item abstractness had little effect on ratings, whether agreement was measured with rwg or awg. The most likely explanation for these null findings is that the disagreement reflected a coarse classification system that overshadowed the effects of tenure, complexity, and abstractness. The existence of meaningful subgroups within a single title threatens the content validity of job specifications: the extent to which they include all relevant and predictive KSAOs. Future research must focus on the existence of such subgroups, their consequences, and ways of identifying them. / Ph. D.

An analysis of interrater agreement across multiple respondents on two anecdotal assessments, the Motivation Assessment Scale (MAS) and the Functional Analysis Screening Tool (FAST), was completed for an individual who displayed aggressive behavior. The results of the assessments indicated high agreement across assessments and respondents that the problem behavior was maintained by social positive reinforcement in the form of contingent delivery of tangible items. By contrast, a subsequent experimental analysis indicated that the behavior was maintained by escape from demands. A treatment was implemented based on the functional analysis outcomes to determine if the functional analysis had correctly identified the maintaining variable of the aberrant behavior. Results of the treatment analysis showed significant reductions in the occurrence of aberrant behavior suggesting that the MAS and FAST may not have accurately identified the maintaining variable of the aberrant behavior.

This thesis presents a draft of a diagnostic tool, intented to use to analyze mediated learning experience ("MLE") in the interactions of parents and school aged children. The first chapter deals with historical roots of MLE and theoretical concepts, which inspired Reuver Feuerstein, author of MLE. Next chapters of the thesis are giving insight to MLE parameters and connecting the parameters to the contemporary knowledge from psychological practice. The thesis is also focused on MISC approach, created by Pnina Klein, which hasn't been described yet in the czech psychological literature, even though it is based on MLE and it is commonly used in foreign research and interventional programmes. The thesis describes the structure of MLE and possibilities of it's uses. At the end of the theoretical part of the thesis, methods, based on MLE theory, used in the Czech republic, are described. Contemporary methods of analyzing interactions of parents and children are also mentioned. Research project of the thesis was focused on designing a draft of a specific form of a diagnostic tool Observed Interactions of Mediated Learning Experience (Pozorované Interakce Zkušenosti Zprostředkovaného Učení - "PIZZU"). It's contents and graphic form was created in cooperation with pedagogues and psychologists. Creating PIZZU...

No description available.