Back to search

Central Florida

The Effects Of Scoring Technique On Situational Judgment Test Validity

Abstract

dc:description.abstract

Situational Judgment Tests (SJTs) are frequently used by organizations as a face-valid selection measure with low adverse impact and a relatively strong relationship with relevant criteria. Despite their common use, there remain several research questions regarding the theoretical foundations and characteristics of SJTs. Additionally, developments in SJT scoring provide fertile ground for research to validate new scoring techniques to better predict criteria of interest. Motowidlo and his colleagues (2006) recently developed a scoring technique for SJTs based on the principle of Implicit Trait Policies (ITPs) which are implicit beliefs concerning the effectiveness of different behavioral choices that demonstrate varying levels of targeted traits. Individuals high in these targeted traits will rate item responses that demonstrate high levels of that particular trait as more effective. Taking into consideration this new method, and also considering the multitude of scoring methods already available to test developers, it logically follows that these different scoring methods will have different correlations with constructs of interest, and that by using this new method it may be possible to achieve a much higher correlation with personality. The effects of scoring technique on relationships between SJT scores and constructs of interest such as personality will in turn have effects on the criterion validity of the SJT. This research explored how scoring methods affected the relationship SJT scores have with general mental ability, personality traits, typical performance, and maximum performance. Results indicated significant differential validity as a function of the respondents‟ race. For minority participants, SJT scores predicted "maximum performance ratings" in a simulation exercise but not "typical performance ratings" provided by familiar peers. However, the reverse was true for Caucasian participants. The two scoring methods demonstrated differential validity. However, the nature of these differences varied as a function of the iv performance dimension in question (i.e., agreeableness, extraversion). Implications for future research will be discussed as well as the practical implications of these findings.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Miller, Daniel S
Contributors dc:contributor
  • Jentsch, Kimberly

Subjects

dc:subject × 3

Rights

Language dc:language
English

Identifiers

dc:identifier.*
Identifier
CFE0004484

Chain of custody

source
Harvested from
Central Florida
Base URL
stars.library.ucf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Miller, Daniel S. The Effects Of Scoring Technique On Situational Judgment Test Validity. https://stars.library.ucf.edu/etd/1778