The Performance Assessment and Evaluation System (PAES) is used by all major universities in the state of Utah to measure the effective teaching skills of preservice candidates as they progress through their teaching preparation program. The resulting ratings are used to make high-stakes decisions relating to course completion as well as recommendation for licensure. This study analyzes the factor structure and tests for measurement invariance of PAES ratings assigned to 663 elementary education candidates at Brigham Young University across two measurement occasions. The candidates were rated by 30 clinical faculty associates. This study also examines the degree to which differential rater effects impact the PAES ratings of these candidates. A bifactor model, with a general factor measuring effective teaching skills measured through observation, and a specific factor measuring effective teaching skills evaluated through conversation best fit the model. Evidence of measurement invariance was found between evaluations completed for Practicum 1 and Practicum 2 candidates. This study also found that differential rater effects impact the PAES ratings of individual candidates, indicating that a candidate's rating may depend on which rater completed the evaluation. Similar research studies should be conducted to analyze the quality of PAES ratings of teacher candidates in the various secondary education programs at BYU. In addition, since the PAES is used at other teacher preparation colleges and universities in Utah, similar research studies should be conducted to examine the quality of PAES ratings of teacher candidates at these other institutions.

No description available.

The Outcome Questionnaire-45 (OQ-45) is a 45-item instrument designed to be used by psychotherapists to track their clients' distress over time. The OQ-45 is composed of three factors: symptomatic distress, interpersonal relations, and social role performance. Numerous researchers have attempted to replicate this intended three-factor structure in their own data, only to find poor fit. Attempts to find a factor structure that does show adequate fit have been met with mixed, but generally poor, results. Additionally, very little work has been done to establish that the OQ-45 exhibits sufficient longitudinal measurement invariance to allow comparison of OQ-45 scores over time. Notwithstanding these known issues regarding the fit of the OQ-45, it has been adopted widely in many countries and translated into several dozen languages. This study sought to identify a factor structure of the OQ-45 that did exhibit longitudinal measurement invariance. Using a sample of 7,751 clients who made 56,353 visits to Brigham Young University's Counseling and Psychological Services between 1996 and 2017, three factor structures were analyzed using Mplus 8.2 through confirmatory factor analysis: (a) single-factor, (b) intended three-factor, and (c) bifactor models. The bifactor model fit the data best, as determined by standard fit statistics (CFI, TLI, RMSEA, SRMR). However, this bifactor model still had inadequate fit. At this point, exploratory structural equation modeling (ESEM) using target rotation was applied to the bifactor model. This ESEM bifactor model had a dominant general factor and did have good fit to the data. Having selected the ESEM bifactor model, it was then tested to see if it showed longitudinal measurement invariance over five time points (the initial OQ-45 score at the intake appointment, followed by four subsequent appointments). The OQ-45 items were treated as categorical and analyzed using the WLSMV estimator. Four time sequences were examined for configural, metric, and scalar longitudinal invariance: Time 1 to Time 2, Time 1 to Time 3 (inclusive of Time 2), Time 1 to Time 4 (inclusive of Times 2 and 3), and Time 1 to Time 5 (inclusive of Times 2, 3, and 4). The OQ-45, when modeled as an ESEM bifactor model, does exhibit scalar longitudinal measurement invariance. Using a new method developed by Clark (2020), ΔSRMR between adjacent models (configural to metric, metric to scalar) were all below his recommended guideline of .01. This is the first study to find a good fitting model of the OQ- 45 that can be used to assess changes in clients' psychological functioning over time. Total OQ- 45 scores can continue to be used by therapists to monitor their patients with confidence in its longitudinal psychometric properties.

No description available.

Asians consistently report higher social anxiety symptoms but have lower prevalence rates, compared to Westerners. As cultural differences and measurement issues could both be potential sources for the discrepancy, it is important to examine whether score differences between cultural groups are due to measurement issues or genuine underlying differences in social anxiety. This study used 402 participants to examine the construct of social anxiety and measurement invariance of six social anxiety scales using exploratory and confirmatory factor analysis. Results supported scalar invariance of a three-factor bifactor model (comprised of Fear/Avoidance of Social Interaction, Fear of Negative Evaluation, and Taijin Kyofusho/fear of interpersonal relationships). Furthermore, multivariate analysis of covariance and moderation analysis revealed Asian Americans endorsed higher Fear/Avoidance of Social Interaction symptoms, but do not differ in Taijin Kyofusho and Fear of Negative Evaluation symptoms, compared to European Americans. This study showed when measurement bias is minimized, Asians still endorse higher symptoms of Fear/Avoidance of Social Interactions. Moreover, Taijin Kyofusho appears to be an aspect of social anxiety identified in more than one cultural group rather than a culture-related specific syndrome. Thus, it is important for clinicians and researchers to consider Taijin Kyofusho in the evaluation of social anxiety. / School Psychology

Previous research has been limited regarding the measurement of computational thinking, particularly as a learning progression in K-12. This study proposes to apply a multidimensional item response theory (IRT) model to a newly developed measure of computational thinking utilizing both selected response and open-ended polytomous items to establish the factorial structure of the construct, apply the recently introduced composite and structured constructs models, and to investigate the measurement invariance of the assessment between males and females using the means and covariance structures (MACS) approach.

The concept of measurement invariance is essential in ensuring psychological and educational tests are interpreted consistently across diverse groups. This dissertation investigated the practical challenges associated with measurement invariance, specifically on how measurement invariance delta fit indexes are affected by non-normal data. Non-normal data distributions are common in real-world scenarios, yet many statistical methods and measurement invariance delta fit indexes are based on the assumption of normally distributed data. This raises concerns about the accuracy and reliability of conclusions drawn from such analyses. The primary objective of this research is to examine how commonly used delta fit indexes of measurement invariance respond under conditions of non-normality. The present research was built upon Cao and Liang (2022a)’s study to test the sensitivities of a series of delta fit indexes, and further scrutinizes the role of non-normal data distributions. A series of simulation studies was conducted, where data sets with varying degrees of skewness and kurtosis were generated. These data sets were then examined by multi-group confirmatory factor analysis (MGCFA) using the Satorra-Bentler scaled chi-square difference test, a method specifically designed to adjust for non-normality. The performance of delta fit indexes such as the Delta Comparative Fit Index (∆CFI), Delta Standardized Root Mean Square residual (∆SRMR) and Delta Root Mean Square Error of Approximation (∆RMSEA) were assessed. These findings have significant implications for professionals and scholars in psychology and education. They provide constructive information related to key aspects of research and practice in these fields related to measurement, contributing to the broader discussion on measurement invariance by highlighting challenges and offering solutions for assessing model fit in non-normal data scenarios.

Innovative people seize the opportunity to make lives better and more comfortable, which contribute to economy growth and financial gain. Stakeholders study innovativeness of business students, in depth, to understand gender differences, and the factors affecting students' innovativeness. Literature explains how males and females differ in their proneness to guilt and shame. However, a model that explains the dynamic of guilt, shame, and gender on innovativeness will help make policies to improve students' innovativeness. This study describes factor analysis approach to examine the TOSCA-3 subscales guilt, shame, and the DNA instrument of innovativeness. It also describes the measurement invariance across gender for each construct, and for the full measurement model to identify the differences between genders. Moreover, this study examines the total effect of gender on innovativeness, which includes the direct effect, and indirect effect via guilt and shame. The results indicated that guilt is positively associated with innovativeness, and shame and gender are negatively associated with innovativeness. This dissertation can be freely accessed and downloaded from (http://etd.byu.edu/).

The Study Anxiety Inventory (SAI), consisting of the factors of worry and emotionality, was developed to measure college students' self-reported levels of anxiety while studying for an exam. Data from 2002 undergraduate students from four colleges (Arts and Sciences, Engineering, Business, and Education) at a southeastern state university were used to evaluate the validity of the scores from the 16-item Study Anxiety Inventory. Results of confirmatory factor analyses for the two factor model, conducted separately for each college, indicated marginally acceptable fit for the data (median fit measures across the four colleges: CFI =.915, SRMR=.049, RMSEA=.098), a pattern that was repeated for both males and females. Multigroup CFA was used to evaluate the factorial invariance of the SAI across gender within each college. Factor loadings (i.e., pattern coefficients) for the SAI items were not found to be significantly different between males and females (p > .05). Error variances for four items were found to be significantly different between males and females, indicating that there may be some difference in scale reliability by gender. Factor covariances were invariant for all four colleges (p > .05) and factor variances were invariant for all but the worry component for the College of Arts and Sciences where females had significantly greater variability on the worry factor. As was hypothesized, the SAI scores were positively correlated with scores on measures of test anxiety (median r=.74), trait anxiety (median r=.46), active procrastination (median r=.23), and passive procrastination (median r=.29), but negatively correlated with trait curiosity (median r=-.19). Contrary to what was hypothesized, no relationship was demonstrated between study anxiety and study skills and habits (median r=-.03). The nomological network was extended in this study by examining relationships between scores obtained from students on the SAI and measures of active and passive procrastination. This is the first study that systematically examines the factorial invariance of the SAI by gender, which is important because previous research using the SAI has shown men's scores to be consistently lower than women's scores. The results obtained in the current study provide support for gender invariance in a nonclinical population in the situation specific level of anxiety while studying. There is sufficient evidence of validity and reliability (median Cronbach alphas for males and females for the total score were .978 and .980, for worry were .968 and .973, and for emotionality were .947 and .951, respectively) that a researcher should feel confident that the SAI is a psychometrically sound research tool that holds up fairly well across a number of different types of students and that making mean comparisons on the SAI by gender is acceptable.

abstract: In the past, it has been assumed that measurement and predictive invariance are consistent so that if one form of invariance holds the other form should also hold. However, some studies have proven that both forms of invariance only hold under certain conditions such as factorial invariance and invariance in the common factor variances. The present research examined Type I errors and the statistical power of a method that detects violations to the factorial invariant model in the presence of group differences in regression intercepts, under different sample sizes and different number of predictors (one or two). Data were simulated under two models: in model A only differences in the factor means were allowed, while model B violated invariance. A factorial invariant model was fitted to the data. Type I errors were defined as the proportion of samples in which the hypothesis of invariance was incorrectly rejected, and statistical power was defined as the proportion of samples in which the hypothesis of factorial invariance was correctly rejected. In the case of one predictor, the results show that the chi-square statistic has low power to detect violations to the model. Unexpected and systematic results were obtained regarding the negative unique variance in the predictor. It is proposed that negative unique variance in the predictor can be used as indication of measurement bias instead of the chi-square fit statistic with sample sizes of 500 or more. The results of the two predictor case show larger power. In both cases Type I errors were as expected. The implications of the results and some suggestions for increasing the power of the method are provided. / Dissertation/Thesis / M.A. Psychology 2010