Cognitive Diagnostic Models (CDMs) are a useful way to identify potential areas of intervention for students who may not have mastered various skills and abilities at the same time as their peers. Traditionally, CDMs have been used on narrowly defined classroom tests, such as those for determining whether students are able to use different algebraic principles correctly. In the current study, the Deterministic Input, Noisy "And" Gate model (DINA; Haertel, 1989; Junker&Sijtsma, 2001) and the Compensatory Reparameterized Unified Model (CRUM; Hartz, 2002), as parameterized by the log-linear cognitive diagnosis model (LCDM; Henson, Templin,&Willse, 2009), were used to analyze the utility of pre-defined cognitive components in estimating students' abilities in a broadly defined, standardized mathematics achievement test. The attribute mastery profile distributions were compared; the majority of students was classified into the extremes of no mastery or complete mastery for both the CRUM and DINA models, though greater variability among attribute mastery classifications was obtained by the CRUM.

With the novel data collection tools and diverse item types, computer-based assessments allow to easily obtain more information about an examinee’s response process such as response time (RT) data. This information has been utilized to increase the measurement precision about the latent ability in the response accuracy models. Van der Linden’s (2007) hierarchical speed-accuracy model has been widely used as a joint modelling framework to harness the information from RT and the response accuracy, simultaneously. The strict assumption of conditional independence between response and RT given latent ability and speed is commonly imposed in the joint modelling framework. Recently multiple studies (e.g., Bolsinova & Maris, 2016; Bolsinova, De Boeck, & Tijmstra, 2017a; Meng, Tao, & Chang, 2015) have found violations of the conditional independence assumption and proposed models to accommodate this violation by modelling conditional dependence of responses and RTs within a framework of Item Response Theory (IRT). Despite the widespread usage of Cognitive Diagnostic Models as formative assessment tools, the conditional joint modelling of responses and RTs has not yet been explored in this framework. Therefore, this research proposes a conditional joint response and RT model in CDM with an extended reparametrized higher-order deterministic input, noisy ‘and’ gate (DINA) model for the response accuracy. The conditional dependence is modelled by incorporating item-specific effects of residual RT (Bolsinova et al., 2017a) on the slope and intercept of the accuracy model. The effects of ignoring the conditional dependence on parameter recovery is explored with a simulation study, and empirical data analysis is conducted to demonstrate the application of the proposed model. Overall, modelling the conditional dependence, when applicable, has increased the correct attribute classification rates and resulted in more accurate item response parameter estimates.

As the testing moves from paper-and-pencil to computer-based assessment, both response accuracy (RA) and response time (RT) together provide a potential for improving the performance evaluation and ability estimation of the test takers. Most joint models utilizing RAs and RTs simultaneously assumed an IRT model for the RA measurement at the lower level, among which the hierarchical speed-accuracy (SA) model proposed by van der Linden (2007) is the most prevalent in literature. Zhan et al. (2017) extended the SA model in cognitive diagnostic modeling (CDM) by proposing the hierarchical joint response and times DINA (JRT-DINA) model, but little is known about its generalizability with the presence of missing data. Large-scale assessments are used in educational effectiveness studies to quantify educational achievement, in which the amount of item nonresponses is not negligible (Pohl et al., 2012; Pohl et al., 2019; Rose et al., 2017; Rose et al., 2010) due to lack of proficiency, lack of motivation and/or lack of time. Treating unplanned missingness as ignorable leads to biased sample-based estimates of item and person parameters (R. J. A. Little & Rubin, 2020; Rubin, 1976), therefore, in the past few decades, intensive efforts have been focused on nonignorable missingness (Glas & Pimentel, 2008; Holman & Glas, 2005; Pohl et al., 2019; Rose et al., 2017; Rose et al., 2010; Ulitzsch et al., 2020a, 2020b). However, a great majority of these methods were limited in item nonresponse types and/or model complexity until J. Lu and Wang (2020) incorporated the mixture cure-rate model (Lee & Ying, 2015) and the tree-based IRT framework (Debeer et al., 2017), which inherited a built-in behavior process for item nonresponses thus introduced no additional latent propensity parameters to the joint model. Nevertheless, these approaches were discussed within the IRT framework, and the traditional measurement models could not provide cognitive diagnostic information about attribute mastery. This dissertation first postulates the CDMTree model, an extension of the tree-based RT process model in CDM, and then explores its efficacy through a real data analysis using PISA 2012 computer-based assessment of mathematics data. The follow-up simulation study compares the proposed model to the JRT-DINA model under multiple conditions to deal with various types of nonignorable missingness, i.e. both omitted items (OIs) and not-reached items (NRIs) due to time limits. A fully Bayesian approach is used for the estimation of the model with the Markov chain Monte Carlo (MCMC) method.

Purpose of this study was to propose a simple and effective method for cognitive diagnosis assessment (CDA) without heavy computational demand using Bayesian inference for binomial proportion (BIBP). In real data studies, BIBP was applied to a test data using two different item designs: four and ten attributes. Also, the BIBP method was compared with DINA and LCDM in the diagnosis result using the same four-attribute data set. There were slight differences in the attribute mastery probability estimate among the three model (DINA, LCDM, BIBP), which could result in different attribute mastery pattern. In Simulation studies, it was found that the general accuracy of the BIBP method in the true parameter estimation was relatively high. The DINA estimation showed slightly higher overall correct classification rate but the bigger overall biases and estimation errors than the BIBP estimation. The three simulation variables (Attribute Correlation, Attribute Difficulty, and Sample Size) showed impacts on the parameter estimations of both models. However, they affected differently the two models: Harder attributes showed the higher accuracy of attribute mastery classification in the BIBP estimation while easier attributes was associated with the higher accuracy of the DINA estimation. In conclusion, BIBP appears an effective method for CDA with the advantage of easy and fast computation and a relatively high accuracy of parameter estimation.