Thesis (Ph.D.)--James Madison University, 2010. / Includes bibliographical references.

Thesis advisor: Walter P. Vispoel. Includes bibliographic references (p. 194-196).

Dissertation (Ph.D.)--The University of North Carolina at Greensboro, 2009. / Directed by Richard M. Luecht; submitted to the Dept. of Educational Research Methodology. Title from PDF t.p. (viewed May 17, 2010). Includes bibliographical references (p. 58-62).

Thesis (Ph. D.)--Florida State University, 2006. / Advisor: Akihito Kamata, Florida State University, College of Education, Dept. of Educational Psychology and Learning Systems. Title and description from dissertation home page (viewed June 7, 2006). Document formatted into pages; contains x, 135 pages. Includes bibliographical references.

Thesis (Ph. D.)--University of Missouri-Columbia, 2005. / The entire dissertation/thesis text is included in the research.pdf file; the official abstract appears in the short.pdf file (which also appears in the research.pdf); a non-technical general description, or public abstract, appears in the public.pdf file. Title from title screen of research.pdf file viewed on (July 19, 2006) Vita. Includes bibliographical references.

Thesis (Ph.D.)--Ohio University, August, 2009. / Title from PDF t.p. Includes bibliographical references.

abstract: The existing minima for sample size and test length recommendations for DIMTEST (750 examinees and 25 items) are tied to features of the procedure that are no longer in use. The current version of DIMTEST uses a bootstrapping procedure to remove bias from the test statistic and is packaged with a conditional covariance-based procedure called ATFIND for partitioning test items. Key factors such as sample size, test length, test structure, the correlation between dimensions, and strength of dependence were manipulated in a Monte Carlo study to assess the effectiveness of the current version of DIMTEST with fewer examinees and items. In addition, the DETECT program was also used to partition test items; a second feature of this study also compared the structure of test partitions obtained with ATFIND and DETECT in a number of ways. With some exceptions, the performance of DIMTEST was quite conservative in unidimensional conditions. The performance of DIMTEST in multidimensional conditions depended on each of the manipulated factors, and did suggest that the minima of sample size and test length can be made lower for some conditions. In terms of partitioning test items in unidimensional conditions, DETECT tended to produce longer assessment subtests than ATFIND in turn yielding different test partitions. In multidimensional conditions, test partitions became more similar and were more accurate with increased sample size, for factorially simple data, greater strength of dependence, and a decreased correlation between dimensions. Recommendations for sample size and test length minima are provided along with suggestions for future research. / Dissertation/Thesis / M.A. Educational Psychology 2012

Multiple Choice Questions (MCQs) have long been the backbone of standardized testing in academia and industry. Correspondingly, there is a constant need for the authors of MCQs to write and refine new questions for new versions of standardized tests as well as to support measuring performance in the emerging massive open online courses, (MOOCs). Research that explores what makes a question difficult, or what questions distinguish higher-performing students from lower-performing students can aid in the creation of the next generation of teaching and evaluation tools. In the automated MCQ answering component of this thesis, algorithms query for definitions of scientific terms, process the returned web results, and compare the returned definitions to the original definition in the MCQ. This automated method for answering questions is then augmented with a model, based on human performance data from crowdsourced question sets, for analysis of question difficulty as well as the discrimination power of the non-answer alternatives. The crowdsourced question sets come from PeerWise, an open source online college-level question authoring and answering environment. The goal of this research is to create an automated method to both answer and assesses the difficulty of multiple choice inverse definition questions in the domain of introductory biology. The results of this work suggest that human-authored question banks provide useful data for building gold standard human performance models. The methodology for building these performance models has value in other domains that test the difficulty of questions and the quality of the exam takers.

Introdução: A Teoria de Resposta ao Item constitui-se em um novo grupo de modelos psicométricos de construção, validação e avaliação de instrumentos de aferição em desfechos em saúde. Dentre tais modelos, o Modelo de Rasch se destaca como uma possibilidade de testar a Teoria de Mensuração Conjunta e, portanto, desenvolver escalas de medida com estrutura intervalar. De fato, as robustas estratégias estatísticas utilizadas recentemente demandam que as variáveis apresentem propriedades intervalares para que resultados fidedignos sejam obtidos. Na área da saúde mental, o Modelo de Rasch agrega a possibilidade de que os construtos sejam avaliados quanto à dimensionalidade. Os instrumentos podem ser explorados quanto à invariância dos itens, calibragem, adequação das escalas de resposta e adequação dos itens ao modelo teórico subjacente.Objetivos: Explorar a aplicação do Modelo de Rasch no desenvolvimento de instrumentos de qualidade de vida e atitudes frente ao envelhecimento. Utilizar o Modelo de Rasch na validação cultural de um instrumento de depressão geriátrica, assim como testar a adequação de uma escala de resposta Likert de acordo com o nível educacional, através da aplicação do Modelo de Rasch. Método: Dados internacionais de qualidade de vida, atitudes frente ao envelhecimento e depressão foram utilizados nestes estudos. Os dados são provenientes do Projeto WHOQOL-OLD e AAQ, coordenado pela Organização Mundial de Saúde em 20 países. Uma combinação de Teoria Clássica emPsicometria e Modelo de Rasch foi utilizada para a análise dos dados. O Modelo de Rasch foi implementado através do software RUMM2020. Resultados: A utilização do Modelo de Rasch possibilitou um refinamento dos instrumentos testados, gerando instrumentos com novos formatos mais adequados. No instrumento WHOQOL-OLD, em sua versão em Português, dois domínios dos seis apresentaram interação item-total insuficiente. O remodelamento da escala de resposta e a exclusão de itens de performance inadequada acarretaram melhora da escala. O instrumento Atitudes Frente ao Envelhecimento também necessitou de exclusão de itens e agrupamento de categorias de resposta, em sua versão em Português, para que seu desempenho psicométrico fosse adequado. Em relação ao nível de escolaridade, a análise dos dados gerados pelo instrumento WHOQOL-BREF em uma amostra internacional de idosos por Modelo de Rasch evidenciou que uma escala Likert de cinco pontos não é adequada para sujeitos analfabetos, em oposição aos achados com sujeitos universitários. A escala de depressão geriátrica GDS também foi analisada em uma amostra brasileira. O formato original de 15 itens se mostrou inadequado, e a exclusão de itens é fundamental para que o desempenho deste instrumento seja satisfatório. Por fim, o impacto de sintomas depressivos clínicos e subclínicos na qualidade de vida de idosos foi avaliado. Mesmo níveis subclínicos de depressão mostraram-se significativamente associados a decréscimo de qualidade de vida e atitudes frente ao envelhecimento em uma amostra de 5566 idosos. Conclusão: A aplicação do Modelo de Rasch no desenvolvimento e validação de instrumentos na área da saúde mental mostrou-se marcadamente relevante. Tal modelo proporcionou que as escalas possam ser examinadas em detalhes. A avaliação dos itens de modo individual permite que a sua adequação seja determinada e que o instrumento seja refinado de acordo com as modificações sugeridas pelas análises de Rasch. Por fim, o Modelo de Rasch permite o teste de invariância dos itens, que assume particular importância na utilização transcultural de instrumentos na área da saúde mental. / Background: Item Response Theory represents a novel group of psychometric models for constructing, validating and evaluating health outcome instruments. Among these models, the Rasch Model is able to test the Theory of Conjoint Measurement and, so, develop intervalar scales. In fact, the recently applied robust statistical strategies demand that the variables have an intervalar structure, so that reliable results are obtained. In the mental health field, the Rasch Model assures that the constructs may be tested for dimensionality. The instruments may be explored regarding item invariance, targetting, response scales adequacy and item fit to the underlying theoretical model. Objectives: To explore the role of the Rasch Model for the development of quality of life and attitudes to aging instruments. In addition, to apply the Rasch Model in the cultural validation of a geriatric depression scale, as well as test the suitability of a 5-point Likert response scale according to the educational level. Method: International data on quality of life, attitudes to aging and depression were analyzed in these studies. Data were drawn from the WHOQOL-OLD and AAQ Project, coordinated by the World Health Organization in 20 countries. A combination of Classical Test Theory and Rasch Model was used to data analysis. The Rasch Model was implemented by RUMM2020 software. Results: The application of the Rasch Model determined a refinement of the original instruments, deriving more adequate versions. The Portuguese version of the WHOQOL-OLD instrument presented insufficient item-total interaction in two domains out of six. Rescoring the response scale and deleting items led to 14 improvement of the instrument. The Attitudes to Aging Questionnaire (Portuguese version) has also demanded rescoring and item deletion in order to present adequate psychometric performance. Regarding the educational level, the Rasch analysis of data obtained using the WHOQOL-BREF in an international sample of older adults indicated that a 5-point Likert scale is not suitable for illiterate subjects. In opositte, this Likert scale is suitable for graduate older adults. The Geriatric Depression Scale was explored in a Brazilian sample. Its original 15-item format has proven to be inadequate, and item exclusion is crucial to obtain a satisfactory performance. Finally, the impact of clinical and subclinical depressive symptoms in quality of life was assessed. Even subthreshold symptomatology is significantly associated to impairments in quality of life and attitudes to aging in a sample of 5566 older adults. Conclusion: The application of the Rasch Model to the development and validation of mental health instruments has proven to be highly relevant. This model brings the possibilty of examining scales in depth. The individual assessment of each item makes it possible to determine item fit and, thus, to refine the instruments according to the alterations claimed by the Rasch analysis. Finally, the Rasch Model implements the item invariance test, which is particularly important for utilizing instruments in transcultural investigations

The Chinese government has announced the 2013 Guidelines for developing a national system for early detection of disability among children under 6 years of age. However, given limited resources, challenges exist with developmental measures required in the 2013 Guidelines. In order to meet the needs for a more accurate and cost-efficient measure for developmental assessment, the Ages & Stages Questionnaires:INVENTORY was translated into Simplified Chinese, and validated on a regional sample of 812 Chinese children ages from 1-25 months. Psychometric properties were examined; data from previous studies on the ASQ:INVENTORY in the U.S. were compared to identify differences between the two countries. Results indicated that the Chinese ASQ:INVENTORY was an instrument with sufficient internal consistency, reliability and validity. It was well accepted by parents and professionals in China. Findings suggested that the Chinese ASQ:INVENTORY provides a promising alternative measure for screening and diagnosing developmental delays in young children in China. Implications for future research and implementation are discussed.