No description available.

Background: The use of outcome measures to evaluate upper extremity function after stroke is highly recommended in clinical practice and research. The Chedoke Arm and Hand Activity Inventory (CAHAI) is a recommended measure as it has strong psychometric properties and clinical utility. However, the measure has not been validated in Asia and there are also gaps in the knowledge about the psychometric properties of the CAHAI. Aim & Objectives: This thesis is dedicated to the continued evaluation of the CAHAI with two main objectives: (1) to develop a Singapore version of the CAHAI, and (2) to re-evaluate the original CAHAI using modern test theories. Method: We conducted a study to cross-culturally adapt the CAHAI and evaluated the psychometric properties in a stroke sample in Singapore. Two studies were conducted to re-evaluate the original CAHAI using modern test theories. In the first study, item response theory and Rasch measurement theory were used to evaluate the psychometric properties of the measure. Following which, both measurement theories were used to revise the CAHAI in the second study. Results: Two test items were modified for the Singapore version of the CAHAI, and the measure had good inter-rater reliability (intra-class correlation coefficient = 0.95 – 0.97) and construct validity. The evaluation of the original CAHAI using modern test theories identified three main problems: (1) the scoring scale was not working as intended, (2) local dependency, and (3) the measure was not unidimensional. Revisions to the CAHAI included collapsing the 7-category scale to four categories, deleting two test items, and developing two new shortened versions. Conclusion: The Singapore version of the CAHAI is a valid and culturally relevant outcome measure that can be used to evaluate post-stroke upper extremity function. The original CAHAI was refined into a new 11- and 5-item versions with a 4-category scale which clinicians may find easier to use. / Thesis / Doctor of Philosophy (PhD)

The publisher of the Degrees of Reading Power test of reading comprehension (DRP) calibrate their test using an item response model called the Rasch or one-parameter logistic model. The relationship between the use of the Rasch model in calibration of the DRP and the use of the DRP as a component of the Virginia Literacy Passport Testing Program (LPT) is addressed. Analyses concentrate on sixth grade students who were administered the DRP in 1991. The question that arises is whether the Rasch model is the appropriate model to use to calibrate the DRP in this high-stakes setting. The majority of research that has been reported by the publisher of the DRP to assess the adequacy of the Rasch model have not included direct checks on model assumptions, model features or model predictions. Instead, they have relied almost exclusively on statistical tests in assessment of model fit. This study will assess the adequacy of fitting DRP test data to the Rasch model through direct examination of the assumptions, features and predictions of the IRT model. This is accomplished by comparing the Rasch model to the less restrictive two- and three-parameter logistic models. Robust IRT-based goodness-of-fit techniques are conducted. When the DRP is used in a high stakes setting, guessing is likely for those in jeopardy of failing. Under these circumstances, we must attend to the possibility that guessing may be a factor and thereby calibrate the DRP with the three-parameter model, as this model takes guessing into account. / Ph. D.

Most symptom measures either implicitly or explicitly distinguish between symptom presence and symptom severity. For example, item 2 on the PHQ-9, a commonly used measure of depressive symptoms, asks respondents to rate how much they have been 'feeling down, depressed, or hopeless.' The response options are 0 (Not at all), 1 (Several Days), 2 (More than half the days), and 3 (Nearly every day). Answering 0 indicates that the symptom is not present, and any response greater than 0 suggests the symptom is present. Higher values indicate higher severity of the symptom. Although the response options distinguish between symptom presence and severity, most users of the PHQ-9 score it by assuming that a 0 (i.e., no symptom), lack of symptoms, is the low end of the severity spectrum. However, clinically, there is often a distinction between experiencing symptoms and how severe any one of those symptoms is. Baldwin and Olsen (2023) developed a sequential item-response theory model that can be used to evaluate whether symptom presence and symptom severity should be separated or considered part of the same construct. We applied the sequential model to 3 datasets, a sample of 6242 participants, containing a variety of measures (e.g., Beck Depression Inventory- Second Edition, State Trait Anxiety Inventory, Penn State Worry Questionnaire). The results indicate that the Two-Part model has best overall fit out of the three models (Two-part, Extreme Response, Unique relationship), suggesting that symptom presence and severity should typically be considered distinct constructs. We discuss the implications for scoring and clinical use of symptom measures in light of our results.

As the People’s Republic of China continues its attempt to match the United States as a global hegemon, it has put forth significant efforts to build influence and gain the support of smaller countries around the globe utilizing both hard and soft power. However, the question remains whether these efforts have resulted in greater favorability toward China. This study seeks to answer this question. I use a Bayesian Item Response Theory (IRT) model to generate latent measures of Chinese hard and soft power around the globe using data on Chinese economic aid, alliances, diplomatic exchanges, and security assistance. This is coupled with voting data from the United Nations to determine the nature of the relationship between Chinese influence-building efforts and a nation’s affinity toward China. This research has both practical and theoretical implications. These findings can serve as a guide for policy makers by determining what has worked for China building in their sphere of influence. These findings will also speak more broadly to larger theoretical questions about how superpowers build and exert influence on and what is the impact of competition between great powers can also be drawn, specifically with the IRT latent model to quantify these impacts.

The ability to monitor client change in psychotherapy over time is vital to quality assurance in psychotherapy as well as the continuing improvement of psychotherapy research. Currently there is not a free and comprehensive outcome measure for psychotherapy that meets current research and treatment goals. This study took further steps to validate a suite of measures to aid in treatment and research, theoretically based in the research domain criteria (RDoC) and the phase model of change frameworks. Items previously tested in a community sample were further tested in a clinical population in psychotherapy training clinics and a community clinical sample Data was analyzed using bi-factor confirmatory factor analysis and multidimensional item response theory. Additional exploratory analyses were conducted to explore differential item functioning in these samples.

The recent shift to a culture of competence has permeated several areas of professional psychology, including competency identification, competency-based education training, and competency assessment. A competency framework has also been applied to various programs and specialty areas within psychology, such as clinical, counseling, clinical health, school, cultural diversity, neuro-, gero-, child, and pediatric psychology. Despite the spread of competency focus throughout psychology, few standardized measures of competency assessment have been developed. To the authors' knowledge, only four published studies on measures of competency assessment in psychology currently exist. While these measures demonstrate significant steps in progressing the assessment of confidence, three of these measures were designed for use with individual programs, two of these international (i.e., UK and Taiwan). The current study applied the seminal Competency Benchmarks, via a recently adapted benchmarks form (i.e., Practicum Evaluation form; PEF), to practicum students at the University of North Texas. In addition to traditional supervisor ratings, the present study also involved self-, peer supervisor, and peer supervisee ratings to provide 360-degree evaluations. Item-response theory (IRT) was used to evaluate the psychometric properties of the PEF and inform potential revisions of this form. Supervisor ratings of competency were found to fit the Rasch model specified, lending support to use of the benchmarks framework as assessed by this form. Self- and peer-ratings were significantly correlated with supervisor ratings, indicating that there may be some utility to 360-degree evaluations. Finally, as predicted, foundational competencies were rated as significantly higher than functional competencies, and competencies improved significantly with training. Results of the current study provide clarity about the utility of the PEF and inform our understanding of practicum-level competencies.

Traditional tests for large-scale assessment of mathematics learning have been criticized for several reasons, such as their mismatch between the vision of mathematical competence and the content covered by the test, and their failure to provide relevant information for guiding further learning and instruction. To achieve that large-scale assessments can function as tools for monitoring and improving learning and teaching, one has to move away from the rationale, the constraints, and the practices of traditional tests. As an illustration this paper presents an alternative approach to largescale assessment of elementary school mathematics developed in Flanders, Belgium Using models of item response theory, 14 measurement scales were constructed, each representing a cluster of curriculum standards and covering as a whole the mathematics curriculum relating to numbers, measurement and geometry. A representative sample of 5,763 sixth-graders (12-year-olds) belonging to 184 schools participated in the study. Based on expert judgments a cut-off score was set that determines the minimum level that students must achieve on each scale to master the standards. Overall, the more innovative curriculum standards were mastered less well than the more traditional ones. Few gender differences in performance were observed. The advantages of this approach and its further development are discussed.

Traditional tests for large-scale assessment of mathematics learning have been criticized for several reasons, such as their mismatch between the vision of mathematical competence and the content covered by the test, and their failure to provide relevant information for guiding further learning and instruction. To achieve that large-scale assessments can function as tools for monitoring and improving learning and teaching, one has to move away from the rationale, the constraints, and the practices of traditional tests. As an illustration this paper presents an alternative approach to largescale assessment of elementary school mathematics developed in Flanders, Belgium Using models of item response theory, 14 measurement scales were constructed, each representing a cluster of curriculum standards and covering as a whole the mathematics curriculum relating to numbers, measurement and geometry. A representative sample of 5,763 sixth-graders (12-year-olds) belonging to 184 schools participated in the study. Based on expert judgments a cut-off score was set that determines the minimum level that students must achieve on each scale to master the standards. Overall, the more innovative curriculum standards were mastered less well than the more traditional ones. Few gender differences in performance were observed. The advantages of this approach and its further development are discussed.

Im Rahmen der Item-Response-Theorie evaluiert die kumulative Dissertationsschrift verschiedene Methoden und Modelle zur Identifikation von Kontexteffekten in Large-Scale Assessments. Solche Effekte können etwa in quantitativen empirischen Schulleistungsstudien auftreten und zu verzerrten Item- und Personenparametern führen. Um in Einzelfällen abschätzen zu können, ob Kontexteffekte auftreten und dadurch die Gefahr verzerrter Parameter gegeben ist (und falls ja, in welcher Weise), müssen IRT-Modelle entwickelt werden, die zusätzlich zu Item- und Personeneffekten Kontexteffekte parametrisieren. Solch eine Parametrisierung ist im Rahmen Generalisierter Allgemeiner Linearer Modelle möglich. In der Dissertation werden Positionseffekte als ein Beispiel für Kontexteffekte untersucht, und es werden die statistischen Eigenschaften dieses Messmodells im Rahmen einer Simulationsstudie evaluiert. Hier zeigt sich vor allem die Bedeutung des Testdesigns: Um unverfälschte Parameter zu gewinnen, ist nicht nur ein adäquates Messmodell, sondern ebenso ein adäquates, also ausbalanciertes Testdesign notwendig. Der dritte Beitrag der Dissertation befasst sich mit dem Problem fehlender Werte auf Hintergrundvariablen in Large-Scale Assessments. Als Kontexteffekt wird in diesem Beispiel derjenige Effekt verstanden, der die Wahrscheinlichkeit eines fehlenden Wertes auf einer bestimmten Variablen systematisch beeinflusst. Dabei wurde das Prinzip der multiplen Imputation auf das Problem fehlender Werte auf Hintergrundvariablen übertragen. Anders als bisher praktizierte Ansätze (Dummy-Codierung fehlender Werte) konnten so in einer Simulationsstudie für fast alle Simulationsbedingungen unverfälschte Parameter auf der Personenseite gefunden werden. / The present doctoral thesis evaluates various methods and models of the item response theory to parametrize context effects in large-scale assessments. Such effects may occur in quantitative educational assessments and may cause biased item and person parameter estimates. To decide whether context effects occur in individual cases and lead to biased parameters, specific IRT models have to be developed which parametrize context effects additionally to item and person effects. The present doctoral thesis consists of three single contributions. In the first contribution, a model for the estimation of context effects in an IRT framework is introduced. Item position effects are examined as an example of context effects in the framework of generalized linear mixed models. Using simulation studies, the statistical properties of the model are investigated, which emphasizes the relevance of an appropriate test design. A balanced incomplete test design is necessary not only to obtain valid item parameters in the Rasch model, but to guarantee for unbiased estimation of position effects in more complex IRT models. The third contribution deals with the problem of missing background data in large-scale assessments. The effect which predicts the probability of a missing value on a certain variable, is considered as a context effect. Statistical methods of multiple imputation were brought up to the problem of missing background data in large-scale assessments. In contrast to other approaches used so far in practice (dummy coding of missing values) unbiased population and subpopulation estimates were received in a simulation study for most conditions.