A multilevel mixture IRT (MMixIRT) model for DIF analysis has been proposed as a solution to gain greater insight on the source of nuisance factors which reduce the reliability and validity of educational assessments. The purpose of this study was to investigate the efficacy of a MMix2PL model in detecting DIF across a broad set of conditions in hierarchically structured, dichotomous data. Monte Carlo simulation was performed to generate examinee response data with conditions common in the field of education. These include (a) two instrument lengths, (b) nine hierarchically structured sample sizes, (c) four latent class features, and (d) eight distinct DIF characteristics, thus allowing for an examination with 576 unique data conditions. DIF analysis was performed using an iterative IRT-based ordinal logistic regression technique, with the focal group identified through estimation of latent classes from a multilevel mixture model. For computational efficiency in analyzing 50 replications for each condition, model parameters were recovered using maximum likelihood estimation (MLE) with the expectation maximization algorithm. Performance of the MMix2PL model for DIF analysis was evaluated by (a) the accuracy in recovering the true class structure, (b) the accuracy of membership classification, and (c) the sensitivity in detecting DIF items and Type I error rates. Results from this study demonstrate that the model is predominantly influenced by instrument length and separation between the class mean abilities, referred to as impact. Enumeration accuracy improved by an average of 40% when analyzing the short 10-item instrument, but with 100 clusters enumeration accuracy was high regardless of the number of items. Classification accuracy was substantially influenced by the presence of impact. Under conditions with no impact, classification was unsuccessful as the matching between model-based class assignments and examinees' true classes averaged only 53.2%. At best, with impact of one standard deviation, classification accuracy averaged between 66.5% to 70.3%. Misclassification errors were then propagated forward to influence the performance of the DIF analysis. Detection power was poor, averaging only 0.34 across the analysis iterations that reached convergence. Additionally, the short 10-item instrument proved challenging for MLE, a condition in which a Bayesian estimation method appears necessary. Finally, this paper provides recommendations on data conditions which improve performance of the MMix2PL model for DIF analysis. Additionally, suggestions for several improvements to the MMix2PL analysis process, which have potential to improve the feasibility of the model for DIF analysis, are summarized.

Three decision consistency and accuracy (DC/DA) methods, the Livingston and Lewis (LL) method, LEE method, and the Hambleton and Han (HH) method, were evaluated. The purposes of the study were (1) to evaluate the accuracy and robustness of these methods, especially when their assumptions were not well satisfied, (2) to investigate the " true" DC/DA indices in various conditions, and (3) to assess the impact of choice of reliability estimate on the LL method. Four simulation studies were conducted. Study 1 looked at various test lengths. Study 2 focused on local item dependency (LID). Study 3 checked the consequences of IRT model data misfit and Study 4 checked the impact of using different scoring metrics. Finally, a real data study was conducted where no advantages were given to any models or assumptions. The results showed that the factors of LID and model misfit had a negative impact on " true" DA index, and made all selected methods over-estimate DA index. On the contrary, the DC estimates had minimal impacts from the above factors, although the LL method had poorer estimates in short tests and the LEE and HH methods were less robust to tests with a high level of LID. Comparing the selected methods, the LEE and HH methods had nearly identical results across all conditions, while the HH method had more flexibility in complex scoring metrics. The LL method was found sensitive to the choice of test reliability estimate. The LL method with Cronbach's alpha consistently underestimated DC estimates while LL with stratified alpha functioned noticeably better with smaller bias and more robustness in various conditions. Lastly it is hoped to make the software be available soon to permit the wider use of the HH method. The other methods in the study are already well supported by easy to use software

No description available.

No description available.

No description available.

No description available.

No description available.

Die vorliegende Arbeit verknüpft die Theorie der Interaktion psychischer Systeme von Kuhl (2000, 2001) mit dem Modell beruflicher Arbeitsleistung von Tett und Burnett (2003). Unter Anwendung reizorientierter arbeitspsychologischer Stressmodelle werden Hypothesen über einen durch das subjektive Bedrohungs- und Belastungspotenzial der Arbeitssituation moderierten Zusammenhang zwischen dem berufsbezogenen und nach Maßgabe der Item Response Theorie skalierbaren Persönlichkeitsmerkmal Handlungs- vs. Lageorientierung und beruflicher Arbeitsleistung aufgestellt. In drei Befragungen an N = 415, N = 331 sowie N = 49 Berufstätigen wurden querschnittliche Daten zur Hypothesenprüfung erhoben. Berufsbezogene Handlungs- vs. Lageorientierung zeigt sich als valides Subkonstrukt der allgemeinen Handlungs- vs. Lageorientierung, welches gemäß Graded Response Modell von Samejima (1969, 1997) mit 14 Items skalierbar ist. Prospektive berufsbezogene Handlungs- vs. Lageorientierung erklärt in multiplen hierarchischen Regressionsanalysen, im Gegensatz zu allgemeiner Handlungs- vs. Lageorientierung, inkrementell zu Gewissenhaftigkeit, Extraversion und Neurotizismus Anteile kontextueller und aufgabenbezogener Arbeitsleistung. Hypothesenkonträr werden diese Zusammenhänge nur marginal vom subjektiven Belastungspotential der Arbeitssituation moderiert. Die Prädiktorfunktion prospektiver berufsbezogener Handlungs- vs. Lageorientierung für berufliche Arbeitsleistung bleibt auch unter pfadanalytischer Kontrolle eines vorhandenen Common Method Bias erhalten. Die dispositionelle Fähigkeit, durch berufliche Hindernisse gehemmten positiven Affekt vorbewusst gegenregulieren zu können, scheint demnach ein bedeutender Prädiktor beruflicher Arbeitsleistung zu sein, insbesondere bei Führungskräften. Für die berufliche Eignungsbeurteilung ist es damit von diagnostischem Mehrwert, Handlungs- vs. Lageorientierung kontextualisiert zu erheben. Der Einsatz probabilistisch- testtheoretisch konstruierter Skalen steigert dabei die Effizienz des Beurteilungsprozesses. / The current paper combines personality systems interaction theory (Kuhl, 2000, 2001) with the model of job performance by Tett and Burnett (2003). Using established stress models from work psychology it is hypothesized that there is a relation between occupational action state orientation, scalable by means of items response theory, and job performance, which is moderated by the subjective stress level of job characteristics. Three surveys among samples of N = 415, N = 331, and N = 49 professionals yielded cross sectional data for investigating the hypotheses. Occupational action state orientation proves a valid construct which is compatible with Samejima’s (1969, 1997) Graded Response Model using a 14-item scale. As a result of multiple hierarchical regression analyses, the hesitation dimension of specifically occupational, in contrast to general action state orientation is a predictor of both contextual and task performance, incremental to conscientiousness, extraversion, and neuroticism. Contrary to expectations this relation is only marginally moderated by stress-relevant job characteristics. Even when controlling for an occurring common method bias by means of path analysis the occupational hesitation dimension’s predictor role perseveres. Therefore, the dispositional ability in subconsciously regulating inhibited positive affect due to occupational obstacles, seems to be a crucial predictor of job performance, especially regarding leaders. Hence, professional aptitude assessment benefits from assessing action state orientation in a contextualized manner. Application of item response theory-based scales further enhances assessment process efficiency.

The Item response Theory and its Application in the Sociology Abstract The focus of this master's thesis is an item response theory, a modern methodological approach, which models a relationship between a latent variable and measured responses to the particular item. A theoretical introduction, which explains fundamentals of the theory, is followed by a comparison with the classical test theory and suggestions of its possible applications in a field of sociology. The thesis is completed with an illustrative example using real data to evaluate an instrument for measuring subjective quality of working life in the framework of item response theory. Keywords: Item response theory, scale, reliability, model, subjective quality of working life

5 Abstract The theoretical part of the paper focuses on computerized adaptive testing (CAT) and item response theory (IRT). Also included is a chapter comparing IRT with the commonly used classical test theory (CTT). There is also a brief mention of computerized and online testing, as these types of administration differ in many aspects from conventional paper & pencil tests. The goal of this paper was to evaluate the individual ways of eEPI test administration and to compare them with eEPQ tests and self-evaluation. In the practical part the items of the extraversion scale of the Eysenck Personality Inventory (eEPI) were calibrated using a group of 124 respondents. The acquired data were subsequently used to carry out a simulation of computerized adaptive testing, which clearly demonstrated the benefits of this type of testing in comparison to the classical test form. These results were compared with the results of real CAT test administration using the original sample and a new group of respondents (Np=69, Nn=68). The results were highly correlated with the results of the simulated test. Moreover, to verify the validity of the computerized adaptive version of eEOD, the respondents' results in this test were compared with the results in the eEPQ test and in a short self-assessment scale. Finally,...