Advancements in education and the perceived necessary educational outcomes require advanced psychometric approaches. This is particularly true when we consider noncognitive assessments – assessments that aim to measure soft skills, otherwise untested in classrooms today. Noncognitive skills have gained traction in recent years due to the increased interest in 21st century skills and the realization that these skills impact success as much as technical skills and content knowledge. Despite the growing interest, the appropriate psychometric methods have yet to be applied to such assessments. Because it is difficult to distill noncognitive assessments down to one particular skill, they are inherently multidimensional; however, literature regarding multidimensional applications of psychometric models is lacking. This study aims to address these concerns, using Situational Judgment Tests (SJTs) to assess the feasibility of improving noncognitive assessment adoption across education. This study assesses multiple psychometric models, both uni- and multidimensional, to best understand the internal structure of SJTs as well as explore the interpretability of model results. The results indicate that SJTs are, in fact, multidimensional and are best measured with polytomous Cognitive Diagnostic Models (CDMs). Implications of these results include future opportunities for noncognitive assessments to be utilized in classroom settings, increased feedback and reporting to students, teachers, and parents regarding strengths and weaknesses of students, and pedagogical improvements to noncognitive skill development.

Educational and psychological tests are critical for measuring latent traits, yet their fairness can be compromised by Differential Item Functioning (DIF), where individuals of similar abilities across demographic groups have unequal probabilities of correct responses. To address these challenges, this study introduces the Fair Rasch Model (FRM) and Generalized Fair Rasch Model (GFRM), which integrate fairness regularization into the Rasch model framework to mitigate DIF effects during parameter estimation without requiring prior DIF detection. These models use adjustable hyperparameters to balance fairness and estimation accuracy. Simulation studies demonstrate that FRM and GFRM outperform existing methods in ability estimation, especially under conditions with high DIF magnitude or prevalence. In real data analysis using TIMSS 2015 mathematics assessments, the models minimized gender disparities in ability estimates more effectively than existing approaches. This study advances equitable testing practices, offering a novel approach to addressing DIF in psychometric assessments.