Doctor of Philosophy (PhD) / New scoring and test construction methods for emotional intelligence (EI) are suggested as alternatives for current practice, where most tests are scored by group judgment and are in ratings-based format. Both the ratings-based format and the proportion-based scores resulting from group judgments may act as method effects, obscuring relationships between EI tests, and between EI and intelligence. In addition, scoring based on standards rather than group judgments add clarity to the meaning of test scores. For these reasons, two new measures of emotional intelligence (EI) are constructed: (1) the Situational Test of Emotional Understanding (STEU); and (2) the Situational Test of Emotion Management (STEM). Following test construction, validity evidence is collected from four multi-variate studies. The STEU’s items and a standards-based scoring system are developed according to empirically derived appraisal theory concerning the structure of emotion [Roseman, 2001]. The STEM is developed as a Situational Judgment Test (SJT) with situations representing sadness, fear and anger in work life and personal life settings. Two qualitative studies form the basis for the STEM’s item development: (1) content analysis of responses to semi-structured interviews with 31 psychology undergraduates and 19 community volunteers; and (2) content analysis of free responses to targeted vignettes created from these semi-structured interviews (N = 99). The STEM may be scored according to two expert panels of emotions researchers, psychologists, therapists and life coaches (N = 12 and N = 6). In the first multi-variate study (N = 207 psychology undergraduates), both STEU and STEM scores relate strongly to vocabulary test scores and moderately to Agreeableness but no other dimension from the five-factor model of personality. STEU scores predict psychology grade and an emotionally-oriented thinking style after controlling vocabulary and personality test scores (ΔR2 = .08 and .06 respectively). STEM scores did not predict academic achievement but did predict emotionally-oriented thinking and life satisfaction (ΔR2 = .07 and .05 for emotionally-oriented thinking and .04 for life satisfaction). In the second multi-variate study, STEU scores predict lower levels of state anxiety, and STEM scores predict lower levels of state anxiety, depression, and stress among 149 community volunteers from Sydney, Australia. In the third multi-variate study (N = 181 psychology undergraduates), Strategic EI, fluid intelligence (Gf) and crystallized intelligence (Gc) were each measured with three indicators, allowing these constructs to be assessed at the latent variable level. Nested structural equation models show that Strategic EI and Gc form separate latent factors (Δχ2(1) = 12.44, p < .001). However, these factors relate very strongly (r = .73), indicating that Strategic EI may be a primary mental ability underlying Gc. In this study, STEM scores relate to emotionally-oriented thinking but not loneliness, life satisfaction or state stress, and STEU scores do not relate to any of these. STEM scores are significantly and meaningfully higher for females (d = .80), irrespective of gender differences in verbal ability or personality, or whether expert scores are derived from male or female experts. The fourth multi-variate study (N = 118 psychology undergraduates) distinguishes an EI latent factor (indicated by scores on the STEU, STEM and two emotion recognition ability measures) from a general cognitive ability factor (indicated by three intelligence measures; Δχ2(1) = 10.49, p < .001), although again cognitive ability and EI factors were strongly related (r = .66). Again, STEM scores were significantly higher for females (d = .44) and both STEU and STEM relate to Agreeableness but not to any other dimension from the five-factor model of personality. Taken together, results suggest that: (1) STEU and STEM scores are reasonably reliable and valid tests of EI; (2) EI tests assess slightly different constructs to existing measures of Gc, but more likely form a new primary mental ability within Gc than an entirely separate construct; and (3) the female superiority for EI tests may prove useful for addressing adverse impact in applied settings (e.g., selection for employment, promotion or educational opportunities), particularly given that many current assessment tools result in a male advantage.

The aim of this study was to develop a leadership measure using situational judgment test (SJT) methodology and to evaluate both construct and criterion-related validity of the developed SJT with respect to a well established measure of leadership, the Leadership Opinion Questionnaire (LOQ). In this study, it was hypothesized that task-oriented leadership-based SJT (SJT-T) is positively related to the supervisors&rsquo / and subordinates&rsquo / ratings of task-oriented leadership as assessed by the LOQ and relationship-oriented SJT (SJT-L) would be positively related to the supervisors&rsquo / and subordinates&rsquo / ratings of the relationship-oriented leadership as assessed by the LOQ. The data were collected from supervisors (N = 87) and their subordinates (N = 160) in a government organization in Ankara. The results indicated that the expected division of SJT-T and SJT-R was not possible. Therefore, rather than developing two SJTs measuring task- and relationship-oriented leadership, a decision was made to develop a general SJT-L measuring context-specific leadership and the hypotheses were tested on an exploratory basis without making a distinction between SJT-T and SJT-R. It was found that the relationship between the SJT-L and supervisors&rsquo / self ratings for task-oriented LOQ was significant but in the unexpected direction. However, the relationship between the SJT-L and subordinates&rsquo / ratings for their supervisors&rsquo / relationship-oriented leadership, using the LOQ, was positive and significant. Lastly, SJT-L was found to be a significant and unique predictor of subordinates&rsquo / ratings of leadership performance. Limitations of the study are acknowledged and results are discussed along with some suggestions for future research.

One of the primary goals of personality assessment is to provide meaningful information regarding an individual’s characteristic way of thinking, feeling, and behaving. Given the interaction between the individual and the context, however, there is much debate as to how well personality tests do what they intend. In this paper, the limitations of text-based personality assessments are examined, and the use of virtual simulations as an alternative to conventional tests is explored. A research study is proposed comparing a virtual test with a written test on a variety of criteria. Modern technology and the growing popularity of gaming suggest that researchers may find virtual simulations as a more immersive, flexible, and accurate forms of assessment. / text

Tidigare forskning har varit oenig gällande vad ett SJT (situational judgment test) mäter. Studiens syfte är att undersöka vad SJT mäter genom att korrelera SJT-poängen med poängen på personlighets- respektive begåvningstest. Ytterligare en frågeställning är huruvida SJT skulle kunna vara ett användbart komplement till övriga arbetspsykologiska test. Totalt genomförde 130 personer fyra olika test; ett SJT, ett personlighetstest samt två begåvningstest (ett logiskt och ett matematiskt test). Resultatet av en regressionsanalys, med poängen på conscientiousness, agreeableness och g-faktorn som prediktorer av SJT-poäng, gav R = .31 och Adj R2 = .07. I linje med ett flertal andra studier blev conscientiousness och g-faktorn signifikanta prediktorer till SJT. Ytterligare studier krävs dock för att bekräfta vilket konstrukt SJT faktiskt mäter.

Doctor of Philosophy (PhD) / New scoring and test construction methods for emotional intelligence (EI) are suggested as alternatives for current practice, where most tests are scored by group judgment and are in ratings-based format. Both the ratings-based format and the proportion-based scores resulting from group judgments may act as method effects, obscuring relationships between EI tests, and between EI and intelligence. In addition, scoring based on standards rather than group judgments add clarity to the meaning of test scores. For these reasons, two new measures of emotional intelligence (EI) are constructed: (1) the Situational Test of Emotional Understanding (STEU); and (2) the Situational Test of Emotion Management (STEM). Following test construction, validity evidence is collected from four multi-variate studies. The STEU’s items and a standards-based scoring system are developed according to empirically derived appraisal theory concerning the structure of emotion [Roseman, 2001]. The STEM is developed as a Situational Judgment Test (SJT) with situations representing sadness, fear and anger in work life and personal life settings. Two qualitative studies form the basis for the STEM’s item development: (1) content analysis of responses to semi-structured interviews with 31 psychology undergraduates and 19 community volunteers; and (2) content analysis of free responses to targeted vignettes created from these semi-structured interviews (N = 99). The STEM may be scored according to two expert panels of emotions researchers, psychologists, therapists and life coaches (N = 12 and N = 6). In the first multi-variate study (N = 207 psychology undergraduates), both STEU and STEM scores relate strongly to vocabulary test scores and moderately to Agreeableness but no other dimension from the five-factor model of personality. STEU scores predict psychology grade and an emotionally-oriented thinking style after controlling vocabulary and personality test scores (ΔR2 = .08 and .06 respectively). STEM scores did not predict academic achievement but did predict emotionally-oriented thinking and life satisfaction (ΔR2 = .07 and .05 for emotionally-oriented thinking and .04 for life satisfaction). In the second multi-variate study, STEU scores predict lower levels of state anxiety, and STEM scores predict lower levels of state anxiety, depression, and stress among 149 community volunteers from Sydney, Australia. In the third multi-variate study (N = 181 psychology undergraduates), Strategic EI, fluid intelligence (Gf) and crystallized intelligence (Gc) were each measured with three indicators, allowing these constructs to be assessed at the latent variable level. Nested structural equation models show that Strategic EI and Gc form separate latent factors (Δχ2(1) = 12.44, p < .001). However, these factors relate very strongly (r = .73), indicating that Strategic EI may be a primary mental ability underlying Gc. In this study, STEM scores relate to emotionally-oriented thinking but not loneliness, life satisfaction or state stress, and STEU scores do not relate to any of these. STEM scores are significantly and meaningfully higher for females (d = .80), irrespective of gender differences in verbal ability or personality, or whether expert scores are derived from male or female experts. The fourth multi-variate study (N = 118 psychology undergraduates) distinguishes an EI latent factor (indicated by scores on the STEU, STEM and two emotion recognition ability measures) from a general cognitive ability factor (indicated by three intelligence measures; Δχ2(1) = 10.49, p < .001), although again cognitive ability and EI factors were strongly related (r = .66). Again, STEM scores were significantly higher for females (d = .44) and both STEU and STEM relate to Agreeableness but not to any other dimension from the five-factor model of personality. Taken together, results suggest that: (1) STEU and STEM scores are reasonably reliable and valid tests of EI; (2) EI tests assess slightly different constructs to existing measures of Gc, but more likely form a new primary mental ability within Gc than an entirely separate construct; and (3) the female superiority for EI tests may prove useful for addressing adverse impact in applied settings (e.g., selection for employment, promotion or educational opportunities), particularly given that many current assessment tools result in a male advantage.

A Predictor Response Process model (see Ployhart, 2006) and research findings were leveraged to formulate research questions about, and generate construct validity evidence for, a new situational judgment test (SJT) designed to measure declarative and strategic knowledge. The first question asked if SJT response instructions (i.e., 'Should Do', 'Would Do') moderated the validity of an SJT in a maximum performance context. The second question asked what the upper-bound criterion-related validity coefficient is for SJTs in talent selection contexts in which typical performance is the criterion of interest. The third question asked whether the SJT used in the present study was fair for gender and ethnic-based subgroups according to Cleary's (1968) definition of test fairness. Participants were randomly assigned to complete an SJT with either 'Should Do' or 'Would Do' response instructions and their maximum decision making performance outcomes were captured during a moderate fidelity poker simulation. The findings of this study suggested knowledge, as measured by the SJT, interacted with response instructions when predicting aggregate and average performance outcomes such that the 'Should Do' SJT had stronger criterion-related validity coefficients than the 'Would Do' version. The findings also suggested the uncorrected upper-bound criterion-related validity coefficient for SJTs in selection contexts is at least moderate to strong ([beta] = .478). Moreover, the SJT was fair according to Cleary's definition of test fairness. The implications of these findings are discussed.

With the onset of COVID-19 and the anti-Asian bias that has followed, there is growing importance in creating empirically valid measures of racial prejudice. In general, there has been limited research on workplace experiences for Asian Americans, despite this group being one of the fastest-growing racial minorities in the United States. Existing measures of prejudice can be susceptible to social desirability response biases, as they tend to focus on more cognitive and affective components of attitudes, rather than behavioral expectations. The goal of this study was to develop and provide initial validation for a behavior-based measure of racial prejudice called the East Asian American Situational Judgment Test (EAA-SJT). SJTs are designed to test behavioral judgments through context-based, realistic scenarios and close-ended plausible response options (Weekley & Ployhart, 2013; Whetzel et al., 2020). There has been some evidence that SJTs measuring latent constructs may produce stronger predictive accuracy of criteria compared to traditionally used, context-independent Likert scale measures (Peus et al., 2013; Teng et al., 2020). These traditional measures, often in the form of cognitive/affective survey questions around racial attitudes, can often lead to higher instances of social desirability response bias (Huddy & Feldman, 2009; Weber et al., 2014), in which participants may not answer questions truthfully due to social concerns. The themes for the EAA-SJT scenarios and response options were based on microaggression research on Asian Americans from Sue et al., (2007a), and the proposed factor structure was predicated on research from Hauenstein et al., (2014), who were the first to develop SJTs measuring latent prejudicial attitudes toward African Americans and women. To provide initial evidence for the validity of the EAA-SJT, 400 participants from a Qualtrics survey panel completed a 20-minute online survey consisting of demographics, the initial 35-item EAA-SJT, the Asian American Stereotypes scale to test for convergent validity, the Marlowe-Crowne Social Desirability scale to test for discriminant validity, and two locally developed criterion measures to test for criterion-related validity. Results from the exploratory factor analyses (EFAs) provided evidence for a three-factor solution of Challenging, Ambivalence, and Reinforcing microaggressions. There was promising initial validity evidence for the EAA-SJT and evidence towards the incremental validity of the EAA-SJT over existing cognitive/affective measures. Next steps include building off the EFA results from this study and conducting a confirmatory factor analysis to finalize the EAA-SJT. Overall, microaggressions and other forms of racial biases in the workplace can have implications on employee well-being, as well as mental and physical health outcomes. The availability of different types of measurement tools such as the EAA-SJT may allow researchers to better understand prejudicial attitudes towards Asian Americans. / M.S. / Following the onset of the COVID-19 pandemic in 2020, there was a noted rise in anti-Asian hate crimes and instances of discrimination against Asian Americans (Jeung et al., 2021, Levin, 2021, Ruiz et al., 2020). This study aims to develop and provide initial validation for a measure of racial prejudice against East Asian Americans. This measure, the East Asian American Situational Judgement Test (EAA-SJT), provided scenarios of microaggressions against East Asian Americans along with four response options of behavioral expectations in reaction to those scenarios. The EAA-SJT can be taken by a member of any racial group and reframes the focus on the attitudes of those who potentially engage in microaggressions rather than only those who are affected by it. Preliminary evidence was found in support of the validity of the EAA-SJT.

No description available.

A new approach was taken to identify a specific construct or dimension being measured by a video-based situational judgment test (VBSJT). Appropriate exertion of control was specifically explored in relation to a VBSJT test designed for entry-level selection of law enforcement officers. Ratings from ten law enforcement experts were utilized to identify this construct. The VBSJT items scored toward overexertion of control were significantly related to performance (r = .23) in a sample of 334 incumbent police officers, capturing a large portion of the effective variance of the test which had an overall validity of r = .34. Situational variables within the items were then compared to ratings of exertion of control within a sample of 5426 applicants. General provocation toward overexertion of control and ethnicity significantly affected appropriate exertion of control. Gender and likeability also had significant impact on appropriate exertion, but the practical significance was limited. Specific character manipulations (i.e., rudeness, aggressiveness, pleasantness, cooperativeness, sympathy, and suspiciousness) also had a significant impact on appropriate exertion of control. Specific information manipulations (i.e., warrants, complaints, contemptible crimes and laws being broken) also had an impact on appropriate exertion of control. Some unexpected findings suggest that the character manipulations may actually override the effect of other provocation. The overexertion of control scale was also applied to test hypotheses about the likely behavior of police officers. It was found that the location of the organization had an affect on overexertion of control. Contrary to the hypothesis, suburban locations had more overexertion of control than urban locations. Length of tenure for police officers did not have an effect on overexertion of control. This difference did not affect validity across organizations. Implications and further research are discussed. / Ph. D.

Doctor of Philosophy / Department of Psychological Sciences / Satoris S. Culbertson / Developing effective strategic thinkers in an organization requires a dedication to early identification, selection, development, and practice, along with the conceptual understanding and measurement tools to make it happen. In support of this, the current research focused on three objectives: 1) establishing theoretical support across multiple disciplines for the concept of a strategic thinking mindset, 2) developing the Strategic Thinking Mindset Test (STMT) using situational judgment test methodology for the U.S. Army, and 3) evaluating the results of a pilot test of the STMT for reliability and construct validity. The STMT focuses on three characteristics of a strategic mindset: Flexibility, Humility, and Inclusiveness. These characteristics were derived from themes found common to strategic thinking literature across the disciplines of psychology, management, and military science. In all stages of this research, officers and non-commissioned officers of the U.S. Army served as participants. The first three stages involve the development and keying of content for the STMT. A sample of 125 participants provided scenarios in Stage 1. In Stage 2, 75 participants gave feedback on the scenarios and provided realistic response options. In Stage 3, 224 participants rated the response options according to expression of the characteristic and effectiveness. In Stage 4, the pilot version of the STMT was administered to 229 participants, along with several other measures used to establish construct validity evidence. The results of the pilot test revealed that, although there is some evidence supporting the construct validity of the STMT as a three-factor test of flexibility, humility, and inclusiveness, the overall profile of evidence suggests that the construct(s) being measured are unclear. Low inter-item correlations contribute to a low internal consistency in the measure, which further limits the STMT’s use as a predictor. The pilot test revealed interesting results related to cognitive ability, specifically a negative relationship between the mindset and cognitive ability under best/worst response instructions, rather than a positive relationship or no relationship, as was the case under most/least likely response instructions. Future research recommendations are discussed in the areas of SJT development, scoring, format, and the further refinement and measurement of the strategic thinking mindset.