Doctor of Philosophy / Department of Psychology / Ronald G. Downey / Entrepreneurship represents an important path to job creation, product development and organizational competitive advantage. Therefore, the identification and retention of entrepreneurial talent is of primary importance. The Entrepreneurial Orientation Profile Inventory (EOPI) was developed to evaluate the Proactiveness, Innovativeness and Risk-Taking dimensions of Entrepreneurial Orientation using a situational judgment test (SJT) testing methodology. The current research outlines the initial development of the testing items and provides a preliminary review of the process used to develop a scoring key and evaluate the psychometric properties of the measure among two independent samples. Study 1 focused on developing a key to score and evaluate data in subsequent samples. In Study 1, 49 adult workers provided ratings regarding the most and least effective response to 12 business-related scenarios designed to measure the Proactiveness, Innovativeness and Risk-Taking dimensions of Entrepreneurial Orientation. Interrater consistency analyses were conducted to determine the correct rank order of the response options within the most and least effective response conditions. In the most effective condition, raters reached consensus on the correct ranking of the response options for 7 of the 12 items. In the least effective condition, raters reached consensus on the correct ranking of the response options for 9 of the 12 items. The highest ranked response option was identified as the “correct” response and used as a scoring key in Study 2. This finding suggests individuals are generally better at identifying a single best ineffective solution to a business-related problem, but less effective at identifying a single best effective solution to a business-related problem. Thus, when using an SJT format to evaluate business-related problems, asking respondents to identify the least effective responses is likely to provide better identification of a “correct” response. Items for which the adult sample reached agreement were retained for further examination in Study 2. Study 2 was conducted to evaluate the impact of three response option instruction and scoring methodologies (i.e., “most effective”, least effective” or a combined “most and least effective”) on the reliability and validity of the EOPI measure. Using a sample of 188 undergraduate students, the construct and criterion validity of the EOPI measure as a unidimensional composite and at the item level was evaluated. Across the three conditions, the results of the construct and criterion validity analyses generally failed to support the EOPI instrument as an effective method to evaluate Entrepreneurial Orientation at the composite level. The modest correlation coefficients among the criteria variables suggest a potentially broader measurement issue with currently available measures of Entrepreneurship in general. Within the “least effective” response instruction condition, minor significant results were found at the item level. A review of these items provides insight into how modifications of EOPI items may facilitate future item development. Further, the current research also suggests that biographical data may provide insight into the measurement of Entrepreneurial Orientation. A biodata-based unidimensional composite of Entrepreneurial Behavior was found to be both marginally reliable and significantly related to an alternative measure of Entrepreneurial Orientation. The development of additional biodata items that correlate with the current items is likely to improve the psychometric properties of the Entrepreneurial Orientation composite and provide insight into the role of previous experience as a valid and reliable indicator of Entrepreneurial Orientation and Entrepreneurship behaviors.

No description available.

Situational judgment tests are analogous to earlier forms of "high fidelity" simulations such that an ostensible paradox emerges in the consistent finding of criterion-referenced validity but almost complete lack of construct validity evidence. The present study evaluates the extent to which SJT's can demonstrate convergent and discriminant validity by analyzing a SJT from a multitrait-multimethod perspective. A series of hierarchically nested confirmatory factor models were tested. Results indicate that the SJT demonstrates convergent and discriminant validity but also contains non-trivial amounts of construct-irrelevant method variance. Wide variability in the content and validation methods of SJT's are discussed as the reason previous attempts to find construct validity have failed. / Master of Science

As a low fidelity work simulation, Situational Judgment Tests (SJTs) are an affordable and practical way of empirically linking training and on-the-job performance, thereby providing a viable means of evaluating training effectiveness. An issue, when utilizing SJTs, is deciding the appropriate manner in which the SJT should be scored. Traditional SJT scoring methodologies, while successfully utilized for selection and prediction, pose specific challenges when applied to a SJT designed to evaluate the effectiveness of a training program. This study discusses the shortcomings of traditional SJT scoring methodologies when used in the evaluation context. To overcome these challenges, an innovative scoring methodology, the Hybrid methodology, is presented. This study provides the detailed description of the Hybrid scoring key creation, and compares the Hybrid scoring key with two traditional scoring keys (Subject Matter Expert (SME) and Respondent-based scoring keys). Responses from a military training program are utilized to illustrate the distinctive effects of using the three different scoring approaches. The superiority of the hybrid scoring key, due to increased confidence in the key's accuracy, and findings regarding training evaluation are discussed. Future research directions and practical applications of the research are also discussed. / Master of Science

Mental toughness (MT) has been shown to predict outcomes across a variety of high-stress contexts such as athletics, the military, and the workplace. Despite this, researchers have struggled to reach consensus regarding how best to conceptualize and measure MT. Specifically, MT assessments have focused on measuring general MT rather than domain-specific MT. The current study proposes a measurement model of MT grounded in social-cognitive theory and introduced an assessment of MT within a situational judgment test framework to assess MT in the workplace. Participants completed a battery consisting of the new measure as well as measures intended to establish construct validity. Factor analyses suggested a three-factor solution fit the data best. Furthermore, cross-structure analyses indicated that the new assessment avoided common-method bias in responding, as evident by weak correlations with measures of other constructs. / Master of Science / Mental toughness (MT) has been shown to be a resource that buffers against the negative effects of distress and predicts outcomes across a variety of settings, including the workplace. However, widely used self-report MT questionnaires have numerous issues, such as a lack of context. The current study addressed a number of these issues by creating a measure of MT wherein respondents were given a workplace situation and asked the likelihood that they would respond in a variety of manners, thereby assessing MT as relevant to the workplace context. Three factors of MT were most prominent – task persistence, emotional control, and utilization of feedback. The measure introduced in the current study had small associations with existing self-report measures of MT, personality, and distress, suggesting that the new method of measuring MT avoided some issues inherent to self-report responding. This research laid promising groundwork for the future assessment of MT in the workplace.

The focus of the current dissertation was on the construction and validation of a situational judgment test (SJT) assessing lying/dishonesty in the workplace. The scale was designed to have two dimensions corresponding to two needs based on Socioanalytic Theory: (1) the need to get along and (2) the need to get ahead. Three studies were undertaken in order to create items, pilot test items, and assess both the construct and criterion-related validity of the scale. The result of Study 1 was a pilot-tested scale with six SJT items corresponding to each of the two SJT dimension (12 items total). The dimensions of the new SJT are referred to in the paper as LAl (lying to get along) and LAh (lying to get ahead). Study 2 examined convergent and discriminant validity as well as the nomological network of other individual difference variables expected to be related to LAl and/or LAh. The results of Study 2 largely supported the construct validity of the overall scale but many of the personality constructs based on the Five Factor Model (FFM) were not related to LAl or LAh. In Study 3, LAl and LAh were unrelated to several aspects of supervisor rated performance as well turnover intentions and objective career outcomes (e.g., promotions). The primary contribution of the dissertation was the creation of a construct valid SJT measuring lying in organizations whose dimensions could be assessed reliably using coefficient alpha as opposed to test-retest reliability. Future research directions and limitations of the study are discussed in the final section of the dissertation.

The aim of this study was to develop a leadership measure using situational judgment test (SJT) methodology and to evaluate both construct and criterion-related validity of the developed SJT with respect to a well established measure of leadership, the Leadership Opinion Questionnaire (LOQ). In this study, it was hypothesized that task-oriented leadership-based SJT (SJT-T) is positively related to the supervisors&rsquo / and subordinates&rsquo / ratings of task-oriented leadership as assessed by the LOQ and relationship-oriented SJT (SJT-L) would be positively related to the supervisors&rsquo / and subordinates&rsquo / ratings of the relationship-oriented leadership as assessed by the LOQ. The data were collected from supervisors (N = 87) and their subordinates (N = 160) in a government organization in Ankara. The results indicated that the expected division of SJT-T and SJT-R was not possible. Therefore, rather than developing two SJTs measuring task- and relationship-oriented leadership, a decision was made to develop a general SJT-L measuring context-specific leadership and the hypotheses were tested on an exploratory basis without making a distinction between SJT-T and SJT-R. It was found that the relationship between the SJT-L and supervisors&rsquo / self ratings for task-oriented LOQ was significant but in the unexpected direction. However, the relationship between the SJT-L and subordinates&rsquo / ratings for their supervisors&rsquo / relationship-oriented leadership, using the LOQ, was positive and significant. Lastly, SJT-L was found to be a significant and unique predictor of subordinates&rsquo / ratings of leadership performance. Limitations of the study are acknowledged and results are discussed along with some suggestions for future research.

One of the primary goals of personality assessment is to provide meaningful information regarding an individual’s characteristic way of thinking, feeling, and behaving. Given the interaction between the individual and the context, however, there is much debate as to how well personality tests do what they intend. In this paper, the limitations of text-based personality assessments are examined, and the use of virtual simulations as an alternative to conventional tests is explored. A research study is proposed comparing a virtual test with a written test on a variety of criteria. Modern technology and the growing popularity of gaming suggest that researchers may find virtual simulations as a more immersive, flexible, and accurate forms of assessment. / text

Tidigare forskning har varit oenig gällande vad ett SJT (situational judgment test) mäter. Studiens syfte är att undersöka vad SJT mäter genom att korrelera SJT-poängen med poängen på personlighets- respektive begåvningstest. Ytterligare en frågeställning är huruvida SJT skulle kunna vara ett användbart komplement till övriga arbetspsykologiska test. Totalt genomförde 130 personer fyra olika test; ett SJT, ett personlighetstest samt två begåvningstest (ett logiskt och ett matematiskt test). Resultatet av en regressionsanalys, med poängen på conscientiousness, agreeableness och g-faktorn som prediktorer av SJT-poäng, gav R = .31 och Adj R2 = .07. I linje med ett flertal andra studier blev conscientiousness och g-faktorn signifikanta prediktorer till SJT. Ytterligare studier krävs dock för att bekräfta vilket konstrukt SJT faktiskt mäter.

No description available.