No description available.

No description available.

No description available.

No description available.

The ability to monitor client change in psychotherapy over time is vital to quality assurance in psychotherapy as well as the continuing improvement of psychotherapy research. Currently there is not a free and comprehensive outcome measure for psychotherapy that meets current research and treatment goals. This study took further steps to validate a suite of measures to aid in treatment and research, theoretically based in the research domain criteria (RDoC) and the phase model of change frameworks. Items previously tested in a community sample were further tested in a clinical population in psychotherapy training clinics and a community clinical sample Data was analyzed using bi-factor confirmatory factor analysis and multidimensional item response theory. Additional exploratory analyses were conducted to explore differential item functioning in these samples.

Background: The use of outcome measures to evaluate upper extremity function after stroke is highly recommended in clinical practice and research. The Chedoke Arm and Hand Activity Inventory (CAHAI) is a recommended measure as it has strong psychometric properties and clinical utility. However, the measure has not been validated in Asia and there are also gaps in the knowledge about the psychometric properties of the CAHAI. Aim & Objectives: This thesis is dedicated to the continued evaluation of the CAHAI with two main objectives: (1) to develop a Singapore version of the CAHAI, and (2) to re-evaluate the original CAHAI using modern test theories. Method: We conducted a study to cross-culturally adapt the CAHAI and evaluated the psychometric properties in a stroke sample in Singapore. Two studies were conducted to re-evaluate the original CAHAI using modern test theories. In the first study, item response theory and Rasch measurement theory were used to evaluate the psychometric properties of the measure. Following which, both measurement theories were used to revise the CAHAI in the second study. Results: Two test items were modified for the Singapore version of the CAHAI, and the measure had good inter-rater reliability (intra-class correlation coefficient = 0.95 – 0.97) and construct validity. The evaluation of the original CAHAI using modern test theories identified three main problems: (1) the scoring scale was not working as intended, (2) local dependency, and (3) the measure was not unidimensional. Revisions to the CAHAI included collapsing the 7-category scale to four categories, deleting two test items, and developing two new shortened versions. Conclusion: The Singapore version of the CAHAI is a valid and culturally relevant outcome measure that can be used to evaluate post-stroke upper extremity function. The original CAHAI was refined into a new 11- and 5-item versions with a 4-category scale which clinicians may find easier to use. / Thesis / Doctor of Philosophy (PhD)

The publisher of the Degrees of Reading Power test of reading comprehension (DRP) calibrate their test using an item response model called the Rasch or one-parameter logistic model. The relationship between the use of the Rasch model in calibration of the DRP and the use of the DRP as a component of the Virginia Literacy Passport Testing Program (LPT) is addressed. Analyses concentrate on sixth grade students who were administered the DRP in 1991. The question that arises is whether the Rasch model is the appropriate model to use to calibrate the DRP in this high-stakes setting. The majority of research that has been reported by the publisher of the DRP to assess the adequacy of the Rasch model have not included direct checks on model assumptions, model features or model predictions. Instead, they have relied almost exclusively on statistical tests in assessment of model fit. This study will assess the adequacy of fitting DRP test data to the Rasch model through direct examination of the assumptions, features and predictions of the IRT model. This is accomplished by comparing the Rasch model to the less restrictive two- and three-parameter logistic models. Robust IRT-based goodness-of-fit techniques are conducted. When the DRP is used in a high stakes setting, guessing is likely for those in jeopardy of failing. Under these circumstances, we must attend to the possibility that guessing may be a factor and thereby calibrate the DRP with the three-parameter model, as this model takes guessing into account. / Ph. D.

Most symptom measures either implicitly or explicitly distinguish between symptom presence and symptom severity. For example, item 2 on the PHQ-9, a commonly used measure of depressive symptoms, asks respondents to rate how much they have been 'feeling down, depressed, or hopeless.' The response options are 0 (Not at all), 1 (Several Days), 2 (More than half the days), and 3 (Nearly every day). Answering 0 indicates that the symptom is not present, and any response greater than 0 suggests the symptom is present. Higher values indicate higher severity of the symptom. Although the response options distinguish between symptom presence and severity, most users of the PHQ-9 score it by assuming that a 0 (i.e., no symptom), lack of symptoms, is the low end of the severity spectrum. However, clinically, there is often a distinction between experiencing symptoms and how severe any one of those symptoms is. Baldwin and Olsen (2023) developed a sequential item-response theory model that can be used to evaluate whether symptom presence and symptom severity should be separated or considered part of the same construct. We applied the sequential model to 3 datasets, a sample of 6242 participants, containing a variety of measures (e.g., Beck Depression Inventory- Second Edition, State Trait Anxiety Inventory, Penn State Worry Questionnaire). The results indicate that the Two-Part model has best overall fit out of the three models (Two-part, Extreme Response, Unique relationship), suggesting that symptom presence and severity should typically be considered distinct constructs. We discuss the implications for scoring and clinical use of symptom measures in light of our results.

Este trabalho descreve as etapas de transformação de um exame de proficiência em inglês acadêmico, aplicado via lápis-e-papel, com itens de múltipla escolha administrados segundo o método de Medida de Probabilidade Admissível (Shuford Jr et al., 1966), utilizado no programa de pós-graduação do Instituto de Ciências Matemáticas e de Computação da Universidade de São Paulo (ICMC-USP), em um teste adaptativo informatizado (TAI-PI) baseado em um modelo da Teoria de Resposta ao Item (TRI). Apesar do programa aceitar diversos exames que atestam a proficiência em inglês para indivíduos não-nativos de abrangência e reconhecimento internacionais, como o TOEFL (Test of English as a Foreign Language), IELTS (International English Language Testing System) e CPE (Certicate of Proficiency in English), por exemplo, a sua obrigatoriedade é incoerente em universidades públicas do Brasil devido ao custo que varia de 200 a 300 dólares por exame. O software TAI-PI (Teste Adaptativo Informatizado para Proficiência em Inglês), que foi desenvolvido em Java e SQLite, será utilizado para a avaliação da proficiência em inglês dos alunos do programa a partir do segundo semestre de 2013, de forma gratuita. A metodologia estatística implementada foi definida considerando a história e objetivos do exame e adotou o modelo de resposta gradual unidimensional de Samejima (Samejima, 1969), o critério de Kullback-Leibler para seleção de itens, o método de estimação da esperança a posteriori para os traços latentes (Baker, 2001) e a abordagem Shadow test (Van der Linden e Pashley, 2010) para imposição de restrições (de conteúdo e tamanho da prova) na composição do teste de cada indivíduo. Uma descrição da estrutura do exame, dos métodos empregados, dos resultados das aplicações do TAI-PI a alunos de pós-graduação do ICMC e estudos de classificação dos alunos em aprovados e reprovados, são apresentados neste trabalho, evidenciando a boa qualidade da nova proposta adotada e aprimoramento do exame com a utilização dos métodos de TRI e TAI. / This work describes the steps for converting a linear paper-and-pencil English proficiency test for academic purposes, composed with multiple choice items that are administered following the admissible probability measurement procedure (Shuford Jr et al., 1966), adopted by the graduate program of Institute of Mathematical Sciences and Computing of University of São Paulo (ICMCUSP), Brazil, to a computerized adaptive test (TAI-PI) based on an item response theory model (IRT). Despite the Institute recognizes reliable international English-language exams for academic purposes and non-native speakers, as TOEFL (Test of English as a Foreign Language), IELTS (International English Language Testing System) and CPE (Cambridge English: Proficiency), for instance, it is inconsistent that public universities in Brazil require them as certification because of the cost of approximately US$ 200.00 to US$ 300.00 per exam. The software TAI-PI (computerized adaptive test for English proficiency) was implemented in Java language, used SQLite as database engine, and it shall be offered free of charge for English proficiency assessment of the graduate students from October 2013. The statistical methodology employed for TAI-PI construction was defined considering the history and the aims of the evaluation and adopted the Samejima\'s graded response model (Samejima, 1969), the Kullback-Leibler information criterion for item selection, the expected a posteriori Bayesian estimation for latent trait (Baker, 2001) and shadow test approach (Van der Linden e Pashley, 2010) for test constraints (content and size of the test, for example). A description of the test design, the employed statistical methods, study results of a real application of TAI-PI to graduate students are presented in this work and the validation studies of the new methodology for pass/fail classification, highlighting the good quality of the new evaluation system and examination of improvement with the use of the methods of IRT and CAT.

We propose a new method for measuring teacher grading standards that is based on the Item Response Theory framework, and investigate the relationship between teacher grading standards and student learning in São Paulo State public schools in light of this new approach. We simulate a policy in which student achievement in a standardized examination (Saresp) is used as the passing grade criterion, setting a unique grading standard for each grade and subject that would substitute the current teacher-defined grading. We estimate the optimal standards that maximize student achievement under this policy, and compare them with the standards estimated for each individual teacher. Our estimates indicate that teachers currently apply standards that are, on average, more lenient than the optimal policy standards / Neste trabalho propomos um novo método para a medição do padrão de avaliação dos professores, fundamentado na Teoria de Resposta ao Item. Investigamos, com base no novo método, a relação entre o padrão de avaliação do professor e o aprendizado do aluno. Nós simulamos os potenciais resultados de uma política de aprovação de alunos baseada exclusivamente em um teste padronizado (Saresp), que implicaria em um padrão de avaliação único para cada série e disciplina, em substituição à política atual em que os professores são responsáveis pela definição sobre a aprovação de seus alunos. Estimamos os padrões de avaliação ótimos sob esta política, do ponto de vista da maximização do aprendizado dos alunos, e comparamos estes valores com os padrões de avaliação estimados para cada professor. Nossas estimativas indicam que os professores utilizam atualmente padrões de avaliação que são, em média, mais lenientes do que o padrão de avaliação ótimo estimado para a nova política