The purpose of this dissertation is to examine how the method used to a score situational judgment test (SJT) affects the validity of the SJT both in the presence of other predictors and as a single predictor of task performance. To this end, I compared the summed score approach of scoring SJTs with item response theory and multivariate items response theory. Using two samples and three sets of analyses, I found that the method used to score SJTs influences the validity of the test and that IRT and MIRT show promise for increasing SJT validity. However, no individual scoring method produced the highest amount of validity across all sets of analyses. In line with previous research, SJTs added incremental validity in the presence of GMA and personality and, again, the method used to score the SJT affected the incremental validity. A relative weights analysis was performed for each scoring method across all the sets of analyses showing that, depending on the scoring method, SJT score may account for more criterion variance than either GMA or personality. However, it is likely that the samples were influenced by range restriction present in the incumbent samples.

No description available.

Thesis (MComm (Industrial Psychology))--University of Stellenbosch, 2009. / ENGLISH ABSTRACT: The last two decades have witnessed an upsurge in the research and use of psychometric tests to aid in the prediction of managerial performance. Currently the most prevailing predictor constructs of managerial performance are cognitive ability, personality, and experience. However, researchers and practitioners are still looking for ways in which to maximise the prediction of managerial performance. In recent years, Situational Judgement Tests (SJTs) have become an increasingly popular selection tool. SJTs are multidimensional psychometric instruments designed to assess an individual’s judgement concerning work-related situations. Evidence to date indicates that SJTs are valid predictors of performance, especially for managerial positions in which interpersonal interactions are important. The main objective of this study was to examine whether SJTs significantly add to the prediction of managerial performance over other measures used for managerial selection, such as measures of cognitive ability and personality. Measures of specific cognitive abilities, personality and a SJT were administered to branch managers in a South African retail bank (N = 124) to investigate the ability of the measures to predict managerial performance. Managerial performance was measured using three measures; Performance Ranking, a Behavioural Observation Scale (BOS) and an Overall Performance Rating. Hierarchical multiple regression was used to investigate the relationship between the predictor composites and the managerial performance measures. Findings reveal different prediction patterns for the three criteria. A multiple correlation coefficient of .442 (p > .05) was obtained when predicting Performance Ranking measures, .308 (p < .05) was obtained for predicting the Behavioural Observation Scale (BOS) measure, and .318 (p > .05) was obtained when predicting the Overall Performance Rating measure. Therefore, only when predicting the BOS measure, the SJT provided incremental validity over cognitive ability and personality measures. Consequently, the average of the scores of the three criterion measures, i.e., the Managerial Performance Composite, was used to evaluate the a priori hypotheses. A multiple correlation of .366 (p > .05) was obtained for predicting the Managerial Performance Composite criterion. Results therefore indicate that the SJT did not exhibit meaningful or statistically significant incremental prediction over cognitive ability and personality to predict the composite managerial performance measure. / AFRIKAANSE OPSOMMING: Die laaste twee dekades het ‘n toename in die gebruik van psigometriese toetse in die voorspelling van bestuurdersprestasie waargeneem. Tans is kognitiewe vermoë, persoonlikheid en ervaring die mees algemene voorspellingskonstrukte vir bestuurdersprestasie. Navorsers en praktisyns is egter op soek na maniere om die voorspelling van bestuurdersprestasie te verbeter. ‘n Onlangse verwikkeling is dat “Situational Judgement Tests” (SJTs) toeneem in gewildheid as seleksie-metode. SJTs is multi-dimensionele psigometriese toetse wat ontwerp is om ‘n individu se oordeelsvermoë ten opsigte van werksverwante situasies te assesseer. Navorsing tot op hede wys dat SJTs geldige voorspellers van prestasie is, veral vir bestuursposisies waarin interpersoonlike interaksies belangrik is. Die hoofdoel van hierdie studie was om te ondersoek of SJTs betekenisvolle waarde toevoeg tot die voorspelling van bestuurdersprestasie bo die gebruik van ander meetinstrumente wat vir bestuurskeuring gebruik word, soos metings van kognitiewe vermoë en persoonlikheid. Vir hierdie doel, is takbestuurders in ‘n Suid Afrikaanse bank (N = 124) se kognitiewe vermoëns, persoonlikheid en situasionele beoordelingsvermoë getoets om die vermoë van die meetinstrumente om bestuurdersprestasie te voorspel, te ondersoek. Bestuurdersprestasie was deur middel van drie meetinstrumente bepaal; prestasie-rangordening (“Performance Ranking”), ‘n gedragsobservasieskaal (“Behavioural Observation Scale”) en ‘n algehele prestasiebeoordelingsmeting (“Overall Performance Rating”). Hiërargiese meervoudige regressie-ontleding was gebruik om die verhouding tussen die voorspellers en die bestuurdersprestasiemetings te ondersoek. Verskillende voorspellingspatrone is vir die drie meetinstrumente gevind. ‘n Meervoudige korrelasie koeffisiënt van .308 (p < .05) is vir die voorspelling van die BOS meting verkry, terwyl .442 (p > .05) en .308 (p < .05) onderskeidelik vir die voorspelling van die prestasie-rangordening en algehele prestasiebeoordelingsmeting verkry is. Gevolglik het slegs die BOS meting inkrementele geldigheid getoon. Die gemiddeld van hierdie drie metings se tellings is gebruik om ‘n bestuurdersprestasie-kombinasietelling “Managerial Performance Composite” te skep wat gebruik is om die finale besluit rakende die a priori hipoteses te maak. ‘n Meervoudige korrelasie van .366 (p >.05) was gevind ten einde die bestuurdersprestasie-kombinasietelling te voorspel aan die hand van die voorspellers.

Some basic but desirable employability skills are antecedents of job performance. The Workplace Skills Survey (WSS) is a 48-item situational judgment test (SJT) used to assess non-technical workplace skills for both entry-level and experienced workers. Unfortunately, the psychometric evidence for use of its scores is far from adequate. The purpose of current study was two-fold: (a) to examine the proposed structure of WSS scores using confirmatory factor analysis (CFA), and (b) to explore the WSS item functioning and performance using item response theory (IRT). A sample of 1,018 Jamaican unattached youth completed the WSS instrument as part of a longitudinal study on the efficacy of a youth development program in Jamaica. Three CFA models were tested for the construct validity of WSS scores. Parameter estimations of item difficulty, item discrimination, and examinee’s proficiency estimations were obtained with item response theory (IRT) and plotted in item characteristics curves (ICCs) and item information curves (IICs). Results showed that the WSS performed quite well as a whole and provided precise measurement especially for respondents at latent trait levels of -0.5 and +1.5. However, some modifications of some items were recommended. CFA analyses showed supportive evidence of the one-factor construct model, while the six-factor model and higher-order model were not achieved. Several directions for future research are suggested.

Industrial and Organisational Psychology / M. Com. (Industrial and Organisational Psychology )

Industrial and Organisational Psychology / M. Com. (Industrial and Organisational Psychology )