Indiana University-Purdue University Indianapolis (IUPUI) / Purpose: Script concordance tests (SCTs) are assessments purported to measure clinical data interpretation. The aims of this research were to (1) test the psychometric properties of SCT items, (2) directly examine the construct validity of SCTs, and (3) explore the concurrent validity of six SCT scoring methods while also considering validity at the item difficulty and item type levels. Methods: SCT scores from a problem solving SCT (SCT-PS; n=522) and emergency medicine SCT (SCT-EM; n=1040) were used to investigate the aims of this research. An item analysis was conducted to optimize the SCT datasets, to categorize items into levels of difficulty and type, and to test for gender biases. A confirmatory factor analysis tested whether SCT scores conformed to a theorized unidimensional factor structure. Exploratory factor analyses examined the effects of six SCT scoring methods on construct validity. The concurrent validity of each scoring method was also tested via a one-way multivariate analysis of variance (MANOVA) and Pearson’s product moment correlations. Repeated measures analysis of variance (ANOVA) and one-way ANOVA tested the discriminatory power of the SCTs according to item difficulty and type. Results: Item analysis identified no gender biases. A combination of moderate model-fit indices and poor factor loadings from the confirmatory factor analysis suggested that the SCTs under investigation did not conform to a unidimensional factor structure. Exploratory factor analyses of six different scoring methods repeatedly revealed weak factor loadings, and extracted factors consistently explained only a small portion of the total variance. Results of the concurrent validity study showed that all six scoring methods discriminated between medical training levels in spite of lower reliability coefficients on 3-point scoring methods. In addition, examinees as MS4s significantly (p<0.001) outperformed their MS2 SCT scores in all difficulty categories. Cross-sectional analysis of SCT-EM data reported significant differences (p<0.001) between experienced EM physicians, EM residents, and MS4s at each level of difficulty. When considering item type, diagnostic and therapeutic items differentiated between all three training levels, while investigational items could not readily distinguish between MS4s and EM residents. Conclusions: The results of this research contest the assertion that SCTs measure a single common construct. These findings raise questions about the latent constructs measured by SCTs and challenge the overall utility of SCT scores. The outcomes of the concurrent validity study provide evidence that multiple scoring methods reasonably differentiate between medical training levels. Concurrent validity was also observed when considering item difficulty and item type.

Indiana University-Purdue University Indianapolis (IUPUI) / Background: Research has repeatedly shown that individuals with severe mental illness desire interpersonal and romantic relationships and that social support (including spousal relationships) is beneficial. In addition, social deficits in mental disorders can often get in the way of developing fulfilling relationships. However, there is little currently available to help clinicians and researchers assess romantic relationship functioning in those with mental illness. The aim of this pilot study was to examine reliability and validity of a new measure of functioning in romantic relationships, the Romantic Relationship Functioning Scale (RRFS). Method: The RRFS was constructed based on theory proposed by Redmond, Larkin, and Harrop (2010). In an analog study, we tested the measure in a sample of college students (N=387), examining reliability, stability over time, factor structure, and relationships with measures of psychopathology and related measures of social functioning to assess convergent validity. Results: The RRFS exhibited a hierarchical four-factor structure, supporting the use of a total score. Although subscales were supported in the factor analysis, other psychometric evidence was weaker, and the use of a total score is advocated. Internal consistency and test-retest reliability were acceptable for the total scale (>.8). The RRFS had moderate to large correlations in the expected direction with all psychopathology measures. In predictive models, overall mental health, social functioning, and fewer interpersonal difficulties predicted higher romantic relationship functioning. Conclusions: The RRFS total score shows preliminary evidence of reliability and validity. The RRFS has potential to be of use in treatment centers for undergraduates and for individuals with diagnosed mental disorders. Future research should further investigate the RRFS subscales and the measure’s performance in clinical samples.

Indiana University-Purdue University Indianapolis (IUPUI) / Background. Traditionally, departmental appointment type (basic science or clinical) and/or degree earned (PhD, MD, or MD-PhD) have served as proxies for how we conceptualize clinical and basic science faculty. However, the landscape in which faculty work has considerably changed and now challenges the meaning of these cohorts. Within this context I introduce a behavior-based role variable that is defined by how faculty spend their time in four academic activities: teaching, research, patient care, and administrative duties. Methods. Two approaches to role were compared to department type and degree earned in terms of their effects on how faculty report their perceptions and experiences of faculty vitality and its related constructs. One approach included the percent of time faculty spent engaged in each of the four academic activities. The second approach included role groups described by a time allocation rubric. This study included faculty from four U.S. medical schools (N = 1,497) and data from the 2011 Indiana University School of Medicine Faculty Vitality Survey. Observed variable path analysis evaluated models that included traditional demographic variables, the role variable, and faculty vitality constructs (e.g., productivity, professional engagement, and career satisfaction). Results. Role group effects on faculty vitality constructs were much stronger than those of percent time variables, suggesting that patterns of how faculty distribute their time are more important than exactly how much time they allocate to single activities. Role group effects were generally similar to, and sometimes stronger than, those of department type and degree earned. Further, the number of activities that faculty participate in is as important a predictor of how faculty experience vitality constructs as their role groups. Conclusions. How faculty spend their time is a valuable and significant addition to vitality models and offers several advantages over traditional cohort variables. Insights into faculty behavior can also show how institutional missions are (or are not) being served. These data can inform hiring practices, development of academic tracks, and faculty development interventions. As institutions continue to unbundle faculty roles and faculty become increasingly differentiated, the role variable can offer a simple way to study faculty, especially across multiple institutions.