Information Systems (IS) research frequently uses survey data to measure the interplay between technological systems and human beings. Researchers have developed sophisticated procedures to build and validate multi-item scales that measure real world phenomena (latent constructs). Most studies use the so-called classical test theory (CTT), which suffers from several shortcomings. We first compare CTT to Item Response Theory (IRT) and subsequently apply a Rasch model approach to measure hedonic aspects of websites. The results not only show which attributes are best suited for scaling hedonic information systems, but also introduce IRT as a viable substitute that overcomes severall shortcomings of CTT. (author´s abstract) / Series: Research Report Series / Department of Statistics and Mathematics

The purpose of this paper is to generalize regression models for repeated categorical data based on maximizing a conditional likelihood. Some existing methods, such as those proposed by Duncan (1985), Fischer (1989), and Agresti (1993, and 1997) are special cases of this latent variable approach, used to account for dependencies in clustered observations. The generalization concerns the incorporation of rather general data structures such as subject-specific time-dependent covariates, a variable number of observations per subject and time periods of arbitrary length in order to evaluate treatment effects on a categorical response variable via a linear parameterization. The response may be polytomous, ordinal or dichotomous. The main tool is the log-linear representation of appropriately parameterized Rasch-type models, which can be fitted using standard software, e.g., R. The proposed method is applied to data from a psychiatric study on the evaluation of psychobiological variables in the therapy of depression. The effects of plasma levels of the antidepressant drug Clomipramine and neuroendocrinological variables on the presence or absence of anxiety symptoms in 45 female patients are analyzed. The individual measurements of the time dependent variables were recorded on 2 to 11 occasions. The findings show that certain combinations of the variables investigated are favorable for the treatment outcome. (author´s abstract) / Series: Research Report Series / Department of Statistics and Mathematics

Setting performance standards is a judgmental process involving human opinions and values as well as technical and empirical considerations and although all cut score decisions are by nature arbitrary, they should not be capricious. Establishing a minimum passing standard is the technical expression of a policy decision and the information gained through standard setting studies inform these policy decisions. To this end, it is necessary to conduct robust examinations of methods and techniques commonly applied to standard setting studies in order to better understand issues that may influence policy decisions. The modified-Angoff method remains one of the most popular methods for setting performance standards in testing and assessment. With this method, is common practice to provide content experts with feedback regarding the item difficulties; however, it is unclear how this feedback affects the ratings and recommendations of content experts. Recent research seems to indicate mixed results, noting that the feedback given to raters may or may not alter their judgments depending on the type of data provided, when the data was provided, and how raters collaborated within groups and between groups. This research seeks to examine issues related to the effects of item-level feedback on the judgment of raters. The results suggest that the most important factor related to item-level feedback is whether or not a Subject Matter Expert (SME) was able to correctly answer a question. If so, then the SMEs tended to rely on their own inherent sense of item difficulty rather than the data provided, in spite of empirical evidence to the contrary. The results of this research may hold implications for how standard setting studies are conducted with regard to the difficulty and ordering of items, the ability level of content experts invited to participate in these studies, and the types of feedback provided.

Introdução: A Teoria de Resposta ao Item constitui-se em um novo grupo de modelos psicométricos de construção, validação e avaliação de instrumentos de aferição em desfechos em saúde. Dentre tais modelos, o Modelo de Rasch se destaca como uma possibilidade de testar a Teoria de Mensuração Conjunta e, portanto, desenvolver escalas de medida com estrutura intervalar. De fato, as robustas estratégias estatísticas utilizadas recentemente demandam que as variáveis apresentem propriedades intervalares para que resultados fidedignos sejam obtidos. Na área da saúde mental, o Modelo de Rasch agrega a possibilidade de que os construtos sejam avaliados quanto à dimensionalidade. Os instrumentos podem ser explorados quanto à invariância dos itens, calibragem, adequação das escalas de resposta e adequação dos itens ao modelo teórico subjacente.Objetivos: Explorar a aplicação do Modelo de Rasch no desenvolvimento de instrumentos de qualidade de vida e atitudes frente ao envelhecimento. Utilizar o Modelo de Rasch na validação cultural de um instrumento de depressão geriátrica, assim como testar a adequação de uma escala de resposta Likert de acordo com o nível educacional, através da aplicação do Modelo de Rasch. Método: Dados internacionais de qualidade de vida, atitudes frente ao envelhecimento e depressão foram utilizados nestes estudos. Os dados são provenientes do Projeto WHOQOL-OLD e AAQ, coordenado pela Organização Mundial de Saúde em 20 países. Uma combinação de Teoria Clássica emPsicometria e Modelo de Rasch foi utilizada para a análise dos dados. O Modelo de Rasch foi implementado através do software RUMM2020. Resultados: A utilização do Modelo de Rasch possibilitou um refinamento dos instrumentos testados, gerando instrumentos com novos formatos mais adequados. No instrumento WHOQOL-OLD, em sua versão em Português, dois domínios dos seis apresentaram interação item-total insuficiente. O remodelamento da escala de resposta e a exclusão de itens de performance inadequada acarretaram melhora da escala. O instrumento Atitudes Frente ao Envelhecimento também necessitou de exclusão de itens e agrupamento de categorias de resposta, em sua versão em Português, para que seu desempenho psicométrico fosse adequado. Em relação ao nível de escolaridade, a análise dos dados gerados pelo instrumento WHOQOL-BREF em uma amostra internacional de idosos por Modelo de Rasch evidenciou que uma escala Likert de cinco pontos não é adequada para sujeitos analfabetos, em oposição aos achados com sujeitos universitários. A escala de depressão geriátrica GDS também foi analisada em uma amostra brasileira. O formato original de 15 itens se mostrou inadequado, e a exclusão de itens é fundamental para que o desempenho deste instrumento seja satisfatório. Por fim, o impacto de sintomas depressivos clínicos e subclínicos na qualidade de vida de idosos foi avaliado. Mesmo níveis subclínicos de depressão mostraram-se significativamente associados a decréscimo de qualidade de vida e atitudes frente ao envelhecimento em uma amostra de 5566 idosos. Conclusão: A aplicação do Modelo de Rasch no desenvolvimento e validação de instrumentos na área da saúde mental mostrou-se marcadamente relevante. Tal modelo proporcionou que as escalas possam ser examinadas em detalhes. A avaliação dos itens de modo individual permite que a sua adequação seja determinada e que o instrumento seja refinado de acordo com as modificações sugeridas pelas análises de Rasch. Por fim, o Modelo de Rasch permite o teste de invariância dos itens, que assume particular importância na utilização transcultural de instrumentos na área da saúde mental. / Background: Item Response Theory represents a novel group of psychometric models for constructing, validating and evaluating health outcome instruments. Among these models, the Rasch Model is able to test the Theory of Conjoint Measurement and, so, develop intervalar scales. In fact, the recently applied robust statistical strategies demand that the variables have an intervalar structure, so that reliable results are obtained. In the mental health field, the Rasch Model assures that the constructs may be tested for dimensionality. The instruments may be explored regarding item invariance, targetting, response scales adequacy and item fit to the underlying theoretical model. Objectives: To explore the role of the Rasch Model for the development of quality of life and attitudes to aging instruments. In addition, to apply the Rasch Model in the cultural validation of a geriatric depression scale, as well as test the suitability of a 5-point Likert response scale according to the educational level. Method: International data on quality of life, attitudes to aging and depression were analyzed in these studies. Data were drawn from the WHOQOL-OLD and AAQ Project, coordinated by the World Health Organization in 20 countries. A combination of Classical Test Theory and Rasch Model was used to data analysis. The Rasch Model was implemented by RUMM2020 software. Results: The application of the Rasch Model determined a refinement of the original instruments, deriving more adequate versions. The Portuguese version of the WHOQOL-OLD instrument presented insufficient item-total interaction in two domains out of six. Rescoring the response scale and deleting items led to 14 improvement of the instrument. The Attitudes to Aging Questionnaire (Portuguese version) has also demanded rescoring and item deletion in order to present adequate psychometric performance. Regarding the educational level, the Rasch analysis of data obtained using the WHOQOL-BREF in an international sample of older adults indicated that a 5-point Likert scale is not suitable for illiterate subjects. In opositte, this Likert scale is suitable for graduate older adults. The Geriatric Depression Scale was explored in a Brazilian sample. Its original 15-item format has proven to be inadequate, and item exclusion is crucial to obtain a satisfactory performance. Finally, the impact of clinical and subclinical depressive symptoms in quality of life was assessed. Even subthreshold symptomatology is significantly associated to impairments in quality of life and attitudes to aging in a sample of 5566 older adults. Conclusion: The application of the Rasch Model to the development and validation of mental health instruments has proven to be highly relevant. This model brings the possibilty of examining scales in depth. The individual assessment of each item makes it possible to determine item fit and, thus, to refine the instruments according to the alterations claimed by the Rasch analysis. Finally, the Rasch Model implements the item invariance test, which is particularly important for utilizing instruments in transcultural investigations

Introdução: A Teoria de Resposta ao Item constitui-se em um novo grupo de modelos psicométricos de construção, validação e avaliação de instrumentos de aferição em desfechos em saúde. Dentre tais modelos, o Modelo de Rasch se destaca como uma possibilidade de testar a Teoria de Mensuração Conjunta e, portanto, desenvolver escalas de medida com estrutura intervalar. De fato, as robustas estratégias estatísticas utilizadas recentemente demandam que as variáveis apresentem propriedades intervalares para que resultados fidedignos sejam obtidos. Na área da saúde mental, o Modelo de Rasch agrega a possibilidade de que os construtos sejam avaliados quanto à dimensionalidade. Os instrumentos podem ser explorados quanto à invariância dos itens, calibragem, adequação das escalas de resposta e adequação dos itens ao modelo teórico subjacente.Objetivos: Explorar a aplicação do Modelo de Rasch no desenvolvimento de instrumentos de qualidade de vida e atitudes frente ao envelhecimento. Utilizar o Modelo de Rasch na validação cultural de um instrumento de depressão geriátrica, assim como testar a adequação de uma escala de resposta Likert de acordo com o nível educacional, através da aplicação do Modelo de Rasch. Método: Dados internacionais de qualidade de vida, atitudes frente ao envelhecimento e depressão foram utilizados nestes estudos. Os dados são provenientes do Projeto WHOQOL-OLD e AAQ, coordenado pela Organização Mundial de Saúde em 20 países. Uma combinação de Teoria Clássica emPsicometria e Modelo de Rasch foi utilizada para a análise dos dados. O Modelo de Rasch foi implementado através do software RUMM2020. Resultados: A utilização do Modelo de Rasch possibilitou um refinamento dos instrumentos testados, gerando instrumentos com novos formatos mais adequados. No instrumento WHOQOL-OLD, em sua versão em Português, dois domínios dos seis apresentaram interação item-total insuficiente. O remodelamento da escala de resposta e a exclusão de itens de performance inadequada acarretaram melhora da escala. O instrumento Atitudes Frente ao Envelhecimento também necessitou de exclusão de itens e agrupamento de categorias de resposta, em sua versão em Português, para que seu desempenho psicométrico fosse adequado. Em relação ao nível de escolaridade, a análise dos dados gerados pelo instrumento WHOQOL-BREF em uma amostra internacional de idosos por Modelo de Rasch evidenciou que uma escala Likert de cinco pontos não é adequada para sujeitos analfabetos, em oposição aos achados com sujeitos universitários. A escala de depressão geriátrica GDS também foi analisada em uma amostra brasileira. O formato original de 15 itens se mostrou inadequado, e a exclusão de itens é fundamental para que o desempenho deste instrumento seja satisfatório. Por fim, o impacto de sintomas depressivos clínicos e subclínicos na qualidade de vida de idosos foi avaliado. Mesmo níveis subclínicos de depressão mostraram-se significativamente associados a decréscimo de qualidade de vida e atitudes frente ao envelhecimento em uma amostra de 5566 idosos. Conclusão: A aplicação do Modelo de Rasch no desenvolvimento e validação de instrumentos na área da saúde mental mostrou-se marcadamente relevante. Tal modelo proporcionou que as escalas possam ser examinadas em detalhes. A avaliação dos itens de modo individual permite que a sua adequação seja determinada e que o instrumento seja refinado de acordo com as modificações sugeridas pelas análises de Rasch. Por fim, o Modelo de Rasch permite o teste de invariância dos itens, que assume particular importância na utilização transcultural de instrumentos na área da saúde mental. / Background: Item Response Theory represents a novel group of psychometric models for constructing, validating and evaluating health outcome instruments. Among these models, the Rasch Model is able to test the Theory of Conjoint Measurement and, so, develop intervalar scales. In fact, the recently applied robust statistical strategies demand that the variables have an intervalar structure, so that reliable results are obtained. In the mental health field, the Rasch Model assures that the constructs may be tested for dimensionality. The instruments may be explored regarding item invariance, targetting, response scales adequacy and item fit to the underlying theoretical model. Objectives: To explore the role of the Rasch Model for the development of quality of life and attitudes to aging instruments. In addition, to apply the Rasch Model in the cultural validation of a geriatric depression scale, as well as test the suitability of a 5-point Likert response scale according to the educational level. Method: International data on quality of life, attitudes to aging and depression were analyzed in these studies. Data were drawn from the WHOQOL-OLD and AAQ Project, coordinated by the World Health Organization in 20 countries. A combination of Classical Test Theory and Rasch Model was used to data analysis. The Rasch Model was implemented by RUMM2020 software. Results: The application of the Rasch Model determined a refinement of the original instruments, deriving more adequate versions. The Portuguese version of the WHOQOL-OLD instrument presented insufficient item-total interaction in two domains out of six. Rescoring the response scale and deleting items led to 14 improvement of the instrument. The Attitudes to Aging Questionnaire (Portuguese version) has also demanded rescoring and item deletion in order to present adequate psychometric performance. Regarding the educational level, the Rasch analysis of data obtained using the WHOQOL-BREF in an international sample of older adults indicated that a 5-point Likert scale is not suitable for illiterate subjects. In opositte, this Likert scale is suitable for graduate older adults. The Geriatric Depression Scale was explored in a Brazilian sample. Its original 15-item format has proven to be inadequate, and item exclusion is crucial to obtain a satisfactory performance. Finally, the impact of clinical and subclinical depressive symptoms in quality of life was assessed. Even subthreshold symptomatology is significantly associated to impairments in quality of life and attitudes to aging in a sample of 5566 older adults. Conclusion: The application of the Rasch Model to the development and validation of mental health instruments has proven to be highly relevant. This model brings the possibilty of examining scales in depth. The individual assessment of each item makes it possible to determine item fit and, thus, to refine the instruments according to the alterations claimed by the Rasch analysis. Finally, the Rasch Model implements the item invariance test, which is particularly important for utilizing instruments in transcultural investigations

Introdução: A Teoria de Resposta ao Item constitui-se em um novo grupo de modelos psicométricos de construção, validação e avaliação de instrumentos de aferição em desfechos em saúde. Dentre tais modelos, o Modelo de Rasch se destaca como uma possibilidade de testar a Teoria de Mensuração Conjunta e, portanto, desenvolver escalas de medida com estrutura intervalar. De fato, as robustas estratégias estatísticas utilizadas recentemente demandam que as variáveis apresentem propriedades intervalares para que resultados fidedignos sejam obtidos. Na área da saúde mental, o Modelo de Rasch agrega a possibilidade de que os construtos sejam avaliados quanto à dimensionalidade. Os instrumentos podem ser explorados quanto à invariância dos itens, calibragem, adequação das escalas de resposta e adequação dos itens ao modelo teórico subjacente.Objetivos: Explorar a aplicação do Modelo de Rasch no desenvolvimento de instrumentos de qualidade de vida e atitudes frente ao envelhecimento. Utilizar o Modelo de Rasch na validação cultural de um instrumento de depressão geriátrica, assim como testar a adequação de uma escala de resposta Likert de acordo com o nível educacional, através da aplicação do Modelo de Rasch. Método: Dados internacionais de qualidade de vida, atitudes frente ao envelhecimento e depressão foram utilizados nestes estudos. Os dados são provenientes do Projeto WHOQOL-OLD e AAQ, coordenado pela Organização Mundial de Saúde em 20 países. Uma combinação de Teoria Clássica emPsicometria e Modelo de Rasch foi utilizada para a análise dos dados. O Modelo de Rasch foi implementado através do software RUMM2020. Resultados: A utilização do Modelo de Rasch possibilitou um refinamento dos instrumentos testados, gerando instrumentos com novos formatos mais adequados. No instrumento WHOQOL-OLD, em sua versão em Português, dois domínios dos seis apresentaram interação item-total insuficiente. O remodelamento da escala de resposta e a exclusão de itens de performance inadequada acarretaram melhora da escala. O instrumento Atitudes Frente ao Envelhecimento também necessitou de exclusão de itens e agrupamento de categorias de resposta, em sua versão em Português, para que seu desempenho psicométrico fosse adequado. Em relação ao nível de escolaridade, a análise dos dados gerados pelo instrumento WHOQOL-BREF em uma amostra internacional de idosos por Modelo de Rasch evidenciou que uma escala Likert de cinco pontos não é adequada para sujeitos analfabetos, em oposição aos achados com sujeitos universitários. A escala de depressão geriátrica GDS também foi analisada em uma amostra brasileira. O formato original de 15 itens se mostrou inadequado, e a exclusão de itens é fundamental para que o desempenho deste instrumento seja satisfatório. Por fim, o impacto de sintomas depressivos clínicos e subclínicos na qualidade de vida de idosos foi avaliado. Mesmo níveis subclínicos de depressão mostraram-se significativamente associados a decréscimo de qualidade de vida e atitudes frente ao envelhecimento em uma amostra de 5566 idosos. Conclusão: A aplicação do Modelo de Rasch no desenvolvimento e validação de instrumentos na área da saúde mental mostrou-se marcadamente relevante. Tal modelo proporcionou que as escalas possam ser examinadas em detalhes. A avaliação dos itens de modo individual permite que a sua adequação seja determinada e que o instrumento seja refinado de acordo com as modificações sugeridas pelas análises de Rasch. Por fim, o Modelo de Rasch permite o teste de invariância dos itens, que assume particular importância na utilização transcultural de instrumentos na área da saúde mental. / Background: Item Response Theory represents a novel group of psychometric models for constructing, validating and evaluating health outcome instruments. Among these models, the Rasch Model is able to test the Theory of Conjoint Measurement and, so, develop intervalar scales. In fact, the recently applied robust statistical strategies demand that the variables have an intervalar structure, so that reliable results are obtained. In the mental health field, the Rasch Model assures that the constructs may be tested for dimensionality. The instruments may be explored regarding item invariance, targetting, response scales adequacy and item fit to the underlying theoretical model. Objectives: To explore the role of the Rasch Model for the development of quality of life and attitudes to aging instruments. In addition, to apply the Rasch Model in the cultural validation of a geriatric depression scale, as well as test the suitability of a 5-point Likert response scale according to the educational level. Method: International data on quality of life, attitudes to aging and depression were analyzed in these studies. Data were drawn from the WHOQOL-OLD and AAQ Project, coordinated by the World Health Organization in 20 countries. A combination of Classical Test Theory and Rasch Model was used to data analysis. The Rasch Model was implemented by RUMM2020 software. Results: The application of the Rasch Model determined a refinement of the original instruments, deriving more adequate versions. The Portuguese version of the WHOQOL-OLD instrument presented insufficient item-total interaction in two domains out of six. Rescoring the response scale and deleting items led to 14 improvement of the instrument. The Attitudes to Aging Questionnaire (Portuguese version) has also demanded rescoring and item deletion in order to present adequate psychometric performance. Regarding the educational level, the Rasch analysis of data obtained using the WHOQOL-BREF in an international sample of older adults indicated that a 5-point Likert scale is not suitable for illiterate subjects. In opositte, this Likert scale is suitable for graduate older adults. The Geriatric Depression Scale was explored in a Brazilian sample. Its original 15-item format has proven to be inadequate, and item exclusion is crucial to obtain a satisfactory performance. Finally, the impact of clinical and subclinical depressive symptoms in quality of life was assessed. Even subthreshold symptomatology is significantly associated to impairments in quality of life and attitudes to aging in a sample of 5566 older adults. Conclusion: The application of the Rasch Model to the development and validation of mental health instruments has proven to be highly relevant. This model brings the possibilty of examining scales in depth. The individual assessment of each item makes it possible to determine item fit and, thus, to refine the instruments according to the alterations claimed by the Rasch analysis. Finally, the Rasch Model implements the item invariance test, which is particularly important for utilizing instruments in transcultural investigations

Doctor Educationis / One of the educational ideas used in mathematics education to improve mathematics achievement in schools is examination-driven teaching. Its effects have sparked intense debates in different didactic circles regarding its usefulness as a teaching technique. More specifically, researchers have consistently debated whether examination-driven teaching is a good or a bad approach that can be used beneficially for learners’ achievement. In South Africa, the urgent need to uplift the low performances of high school learners in Mathematics has led to a development of a project which is a partnership with the Western Cape Education Department (WCED) and the University of the Western Cape (UWC). This project used examination-driven teaching in the context of a continuous professional development to improve learners’ mathematics scores. Five secondary schools that were opportunistically sampled in the province of the Western Cape were exposed to examination-driven teaching. For evaluation, the project yearly developed and implemented high-stakes Grade 10 end-ofyear mathematics examinations, and the data subjected to analysis were learners’ mathematics scores for 2012, 2013 and 2014. A quantitative approach employing Rasch procedures and some statistical procedures were used to analyse the data. The study intended to answer the following questions: 1) Do learners’ achievement scores in a high-stakes Grade 10 mathematics examinations improve over time when an examination-driven teaching approach is being used as intervention? 2) Does socio-economic status of schools influence mathematics performances in the case of using examination-driven teaching ? 3) Are there differences over time in the achievement of learners in the two different papers comprising the examination? / Okitowamba, O. (2015) Tracking learners’ performances in high-stakes Grade 10 mathematics examinations. PhD thesis. University of the Western Cape. Bellville. http://hdl.handle.net/11394/5655.

In variety testing as well as in psychological assessment, the situation occurs that in a two-way ANOVA-type model with only one replication per cell, analysis is done under the assumption of no interaction between the two factors. Tests for this situation are known only for fixed factors and normally distributed outcomes. In the following we will present five additivity tests and apply them to fixed and mixed models and to quantitative as well as to Bernoulli distributed data. We consider their performance via simulation studies with respect to the type-I-risk and power. Furthermore, two new approaches will be presented, one being a modification of Tukey's test and the other being a new experimental design to test for interactions.

This thesis is submitted to the University of Manchester for the degree of Doctor of Philosophy (PhD). This study developed an experimental small group teaching method in the Realistic Mathematics Education tradition for teaching fractions using models and contexts to year 7 children in Brunei (N=89) whose effectiveness was evaluated using a treatment-control design: the E1 group was given the experimental lessons, the E2 group who was given “normal” lessons taught by the experimenter, and a whole class (E3) group which acted as the control group. The experimental teaching was video recorded and subject to semiotic analysis, aiming to describe the objectifications that realized ‘learning of fractions’ by the groups.The research addresses two research questions:1. How effective was the experimental teaching in helping learners make sense of fractions, with respect to equivalence of fractions and flexibility of unitizing?2. What were the semiotic learning and teaching processes in the experimental group of the RME-like lessons? This study used a mixed method approach with a quasi-experimental design (QED) for the quantitative side, and a semiotic analysis for the qualitative side. Quantitatively, the experimental teachings proved to be relatively effective with an effect size of 0.6 from the pre- to the delayed post-teaching test, compared to the E2 and the control groups.The basic findings pertaining to the semiotic analyses were:a. The mediation of the production of fractions in terms of length, from the production of fractions in terms of the number of parts which led to equivalence of fractions;b. The use of language and gesture help to objectify the equivalence of fractions and the flexibility of unitizing–in some case it involved gesturing to the self;c. The role of the Hour-Foot clock (HFC) as a model in a realistic context; andd. The complexity of the required chains of objectifications reflects the difficulties of the topic.

The purpose of this study was to investigate the effects of various testlet characteristics in terms of an ability parameter recovery under the modality of computerized adaptive test (CAT). Given the popularity of using CATs and the high frequency of emerging testlets into exams as either mixed format or not, it was important to evaluate the various conditions in a testlet-based CAT fitted testlet response theory models. The manipulated factors of this study were testlet size, testlet effect size, testlet composition, and exam format. The performance of each condition was compared with the true thetas which were 81 equally spaced points from -3.0 to +3.0. For each condition, 1,000 times of replication process were conducted with respect to overall bias, overall standard error, overall RMSE, conditional bias, conditional standard error, conditional RMSE, as well as conditional passing rate. The conditional results were presented in the pre-specified intervals. Several significant conclusions were made. Overall, the mean theta estimates over 1,000 replications were close to the true thetas regardless of manipulated conditions. In terms of aggregated overall RMSE, predictable relationships were found in four study factors: A larger amount of error was associated with a longer testlet, a bigger effect size, a random composition, and a testlet only exam format. However, when the aggregated overall bias was considered, only two effects were observed: a large difference among three testlet length conditions, and almost no difference between two testlet composition conditions. As expected, conditional SEMs for all conditions showed a U-shape across the theta scale. The noticeable discrepancy occurred only within the testlet length condition: more error was associated with the condition of the longest testlet length compared to the short and medium length conditions. Conditional passing rate showed little discrepancy among conditions within each facto, so no particular association was found. In general, a short testlet length is better, a small testlet effect size is better, a homogeneous difficulty composition is better, and a mixed format is better in terms of the smaller amount of error found in this study. Other than these obvious findings, some interaction effects were also observed. When the medium or large (i.e., greater than .50) testlet effect was suspicious, it was better to have a short length testlet. It was also found that using a mixed-format exam increased the accuracy of the random difficulty composition. However, this study was limited by several other factors which were controlled to be the same across the conditions: a fixed length exam, no content balancing, and the uniform testlet effects. Consequently, plans for improvements in terms of generalization were also discussed.