A new ESSA indicator of school quality and student success provides flexibility to broaden a states’ definition of school and student success. Educational research has found school success is in part determined by a school’s climate and should be considered in improvement/reform strategies (Cohen et al., 2009; Thapa et al., 2013). Yet, school climate research is often difficult and time consuming, and employs a variety of conflicting definitions and dimensions, instruments, and empirical approaches to determining school climate. Given these significant limitations with current measures, the purpose of this study was to validate an instrument measuring school climate based on the four most commonly accepted dimensions of school climate, using items adapted from a well-regarded and established theoretical framework to provide an effective measure for educators and researchers. The sample selected for this study was a portion of teachers who indicated teaching 3rd or 8th grade as their primary teaching assignment (n=500) from the larger study sample (n=4974). A Rasch Rating Scale Model was used to evaluate unidimensionality, item fit and difficulty, reliability, and potential differential item functioning on a 23-item school climate survey. Results of the study showed the instrument was not unidimensional and was split into two subdimensions: student-centered and teacher/school support. All items were retained and displayed appropriate fit. Significant differential item functioning (DIF) was found between 3rd and 8th grade teachers on both subdimensions, further suggesting multidimensionality in the scale. Study findings suggest researchers should be mindful of any school climate instrument not validated at the item level for unidimensionality, and that an instrument may perform differently for teachers at different grade levels.

English language learner (ELL) is a term to describe students who are still acquiring English proficiency. In recent decades, ELLs are a very rapidly growing student group in United States. In school classrooms, ELLs are learning English and their academic subjects simultaneously. It is challenging for them to hear lectures, read textbooks, and complete tests in English despite of their inadequate English language proficiency (Ilich, 2013). As a result, the increasing number of ELLs in public schools has paralleled the increase in ELLs’ low mathematics performance (NCES, 2016). Due to the popularization of international large-scale assessments in the recent decade, it is necessary to analyze their psychometric properties (e.g., reliability, validity) so that those results can provide with evidence-based implications for policymakers. Educational researchers need to assess the validity for subgroups within each country. The Programme for International Student Assessment (PISA), as one of the influential large-scale assessments, allows researchers to investigate academic achievement and group membership from a variety of different viewpoints. The current study was to understand the nature and potential sources of the gaps in mathematics achievement between ELLs and non-ELLs. The nature of achievement gap was examined using three DIF methodologies including Mantel-Haenszel procedure, Rasch analysis, and Hierarchical Generalized Linear Model (HGLM) at the item level instead of total test level. Amon the three methods, HGLM was utilized to examine the potential sources of DIF. This method can take into account of the nested structure of data where items are nested within students, and students nested within schools. At the student level, sources of DIF were investigated through students’ variations in mathematics self-efficacy, language proficiency, and student socioeconomic status. At the school level, school type and school educational resource were investigated as potential sources of DIF after controlling for the student variables. The U.S. sample from PISA 2012 was used, and 76 dichotomously coded items from PISA 2012 mathematics assessment were included to detect DIF effects. Results revealed that ten common items are identified with DIF effects using MH procedure, Rasch analysis, and HGLM. These ten items are all in favor of non-ELLs.The decreasing number of items showing DIF effects in HGLM after controlling for student-level variables revealed mathematics self-efficacy, language proficiency, and SES are potential sources of DIF between ELLs and non-ELLs. In addition, the number of DIF items continued to decrease after controlling for both student and school-level variables. This finding proved that school type and school educational resources were also potential sources of DIF between ELLs and non-ELLs. Findings from this study can help educational researchers, administrators, and policymakers understand the nature of the gap at item level instead of the total test level so that United States can be competitive in middle school mathematics education. This study can also help guide item writers and test developers in the construction of more linguistically accessible assessments for students who are still learning English. The significance of this study lies in the empirical investigation of the gap between ELLs and non-ELLs in mathematics achievement at an item level and from perspectives of both students and schools.

<p>The main purpose of this thesis was to evaluate and test the Job Stress Survey (JSS, Spielberger, 1991; Spielberger & Vagg, 1999), a self-report instrument which assesses workplace stress. In the thesis a thorough evaluation is made of JSS scales and items, and the relations to health, particularly musculoskeletal complaints. The aim of Study I was to evaluate the factor structure and the psychometric properties of a Swedish version of the JSS. The instrument was distributed to medical service personal and metal industry workers (n=1186). Factor analyses demonstrated a good resemblance between the present version and the American original version. The results also showed that the internal consistencies, as well as the test-retest reliabilities of the scales are high, and the concurrent validity are good. Study II examined work-related stress measured by JSS for the subgroups of gender, industry workers and medical service personnel, and special attention was given to the problem of differential item functioning (DIF) on these subgroups. The main findings were that both gender and occupation has a substantial impact on specific sources of work-related stress assessed by JSS scales and individual items. The result of the DIF analyses showed no item bias in the gender subgroup, but for the occupational subgroups there where items showing DIF in two of the scales. These items do not jeopardize the conclusions made on scale level since the number of items showing DIF are too few to make an impact on the overall result on the different scales. In Study III the relation between self-reported stress and health, particularly musculoskeletal problems were examined longitudinally in two metal industry factories. Results showed high levels of stress and musculoskeletal complaints in these factories and significant and strong relationships between the JSS scales and musculoskeletal, as well as psychosocial ratings. Lack of Organizational Support was found to be more related to musculoskeletal pain than Job Pressure. Longitudinal differences were found between the factories and between different types of musculoskeletal complaints. The general conclusions from the studies are that the present version of JSS shows a good resemblance with the American original, and that JSS is a useful instrument for studying relationships between stress and health.</p>

The main purpose of this thesis was to evaluate and test the Job Stress Survey (JSS, Spielberger, 1991; Spielberger &amp; Vagg, 1999), a self-report instrument which assesses workplace stress. In the thesis a thorough evaluation is made of JSS scales and items, and the relations to health, particularly musculoskeletal complaints. The aim of Study I was to evaluate the factor structure and the psychometric properties of a Swedish version of the JSS. The instrument was distributed to medical service personal and metal industry workers (n=1186). Factor analyses demonstrated a good resemblance between the present version and the American original version. The results also showed that the internal consistencies, as well as the test-retest reliabilities of the scales are high, and the concurrent validity are good. Study II examined work-related stress measured by JSS for the subgroups of gender, industry workers and medical service personnel, and special attention was given to the problem of differential item functioning (DIF) on these subgroups. The main findings were that both gender and occupation has a substantial impact on specific sources of work-related stress assessed by JSS scales and individual items. The result of the DIF analyses showed no item bias in the gender subgroup, but for the occupational subgroups there where items showing DIF in two of the scales. These items do not jeopardize the conclusions made on scale level since the number of items showing DIF are too few to make an impact on the overall result on the different scales. In Study III the relation between self-reported stress and health, particularly musculoskeletal problems were examined longitudinally in two metal industry factories. Results showed high levels of stress and musculoskeletal complaints in these factories and significant and strong relationships between the JSS scales and musculoskeletal, as well as psychosocial ratings. Lack of Organizational Support was found to be more related to musculoskeletal pain than Job Pressure. Longitudinal differences were found between the factories and between different types of musculoskeletal complaints. The general conclusions from the studies are that the present version of JSS shows a good resemblance with the American original, and that JSS is a useful instrument for studying relationships between stress and health.

This dissertation served two purposes. The first was to develop a method of detecting differential item functioning (DIF) within tests containing both dichotomously and polytomously scored items. The second was related to gender and aimed a) to investigate if those items that were functioning differently for girls and boys showed any characteristic properties and, if so, b) determine if these properties could be used to predict which items would be flagged for D1F. The method development was based on the Mantel-Haenszel (MH) method used for dichotmously scored items. By dichotomizing the polytomously scored items both types of item could be compared on the same statistical level as either solved or non-solved items. It was not possible to compare the internal score structures for the two gender groups, only overall score differences were detected. By modelling the empirical item characteristic curves it was possible to develop a MH method for identifying nonuniform DIF. Both internal and external ability criteria were used. Total test score with no purification was used as the internal criterion. Purification was not done for validity reasons, no items were judged as biased. Teacher set marks were used as external criteria. The marking scale had to be transformed for either boys or girls since a comparison of scores for boys and girls with the same marks showed that boys always got higher mean scores. The results of the two MH analyses based on internal and external criterion were compared with results from P-SIBTEST. All three methods corresponded well although P-SIBTEST flagged considerably more items in favour of the reference group (boys) which exhibited a higher overall ability. All 200 items included in the last 15 annual national tests in physics were analysed for DIF and classified by ten criteria The most significant result was that items in electricity were, to a significantly higher degree, flagged as DIF in favour of girls whilst items in mechanics were flagged in favour of boys. Items in other content areas showed no significant pattern. Multiple-Choice items were flagged in favour of boys. Regardless of the degree of significance by which items from different content areas were flagged on a group level it was not possible to predict which single item would be flagged for DIF. The most probable prediction was always that an item was neutral. Some possible interpretations of DIF as an effect of multidimen-sionality were discussed as were some hypotheses about the reasons why boys did better in mechanics and girls in electricity. / digitalisering@umu

abstract: The study examined how ATFIND, Mantel-Haenszel, SIBTEST, and Crossing SIBTEST function when items in the dataset are modelled to differentially advantage a lower ability focal group over a higher ability reference group. The primary purpose of the study was to examine ATFIND's usefulness as a valid subtest selection tool, but it also explored the influence of DIF items, item difficulty, and presence of multiple examinee populations with different ability distributions on both its selection of the assessment test (AT) and partitioning test (PT) lists and on all three differential item functioning (DIF) analysis procedures. The results of SIBTEST were also combined with those of Crossing SIBTEST, as might be done in practice. ATFIND was found to be a less-than-effective matching subtest selection tool with DIF items that are modelled unidimensionally. If an item was modelled with uniform DIF or if it had a referent difficulty parameter in the Medium range, it was found to be selected slightly more often for the AT List than the PT List. These trends were seen to increase as sample size increased. All three DIF analyses, and the combined SIBTEST and Crossing SIBTEST, generally were found to perform less well as DIF contaminated the matching subtest, as well as when DIF was modelled less severely or when the focal group ability was skewed. While the combined SIBTEST and Crossing SIBTEST was found to have the highest power among the DIF analyses, it also was found to have Type I error rates that were sometimes extremely high. / Dissertation/Thesis / Doctoral Dissertation Educational Psychology 2014

Differential item functioning (DIF) is present when examinees who have the same level of a trait have a different probability of correctly answering a test item intended to measure that trait (Shepard & Averill, 1981). The following study is a DIF analysis of the Herrmann Brain Dominance Instrument (HBDI), a preference profiling instrument developed by Herrmann International to help individuals identify their dominant preferences and then classify their level of dominance into four preference quadrants. Examinees who completed the American English version of the instrument were classified as the reference group and examinees of the International English version were classified as the focal group. Out of 105 items, 11 were manifesting a large amount of DIF and were flagged for further review. The POLYSIBTEST procedure was used to carry out the DIF analysis. POLYSIBTEST is an extension of the SIBTEST procedure, which is a conceptually simple method for analyzing DIF that uses a latent trait measure rather than an observed total score. The latent trait measure helps detect both uniform and nonuniform DIF and the POLYSIBTEST procedure is used for both dichotomous and polytomous items. Each of the four preference quadrants were analyzed separately to reduce incorrect findings as a result of ipsative scoring. The process used to complete the DIF analysis was documented so that additional language groups may be analyzed by Herrmann International.

A multilevel mixture IRT (MMixIRT) model for DIF analysis has been proposed as a solution to gain greater insight on the source of nuisance factors which reduce the reliability and validity of educational assessments. The purpose of this study was to investigate the efficacy of a MMix2PL model in detecting DIF across a broad set of conditions in hierarchically structured, dichotomous data. Monte Carlo simulation was performed to generate examinee response data with conditions common in the field of education. These include (a) two instrument lengths, (b) nine hierarchically structured sample sizes, (c) four latent class features, and (d) eight distinct DIF characteristics, thus allowing for an examination with 576 unique data conditions. DIF analysis was performed using an iterative IRT-based ordinal logistic regression technique, with the focal group identified through estimation of latent classes from a multilevel mixture model. For computational efficiency in analyzing 50 replications for each condition, model parameters were recovered using maximum likelihood estimation (MLE) with the expectation maximization algorithm. Performance of the MMix2PL model for DIF analysis was evaluated by (a) the accuracy in recovering the true class structure, (b) the accuracy of membership classification, and (c) the sensitivity in detecting DIF items and Type I error rates. Results from this study demonstrate that the model is predominantly influenced by instrument length and separation between the class mean abilities, referred to as impact. Enumeration accuracy improved by an average of 40% when analyzing the short 10-item instrument, but with 100 clusters enumeration accuracy was high regardless of the number of items. Classification accuracy was substantially influenced by the presence of impact. Under conditions with no impact, classification was unsuccessful as the matching between model-based class assignments and examinees' true classes averaged only 53.2%. At best, with impact of one standard deviation, classification accuracy averaged between 66.5% to 70.3%. Misclassification errors were then propagated forward to influence the performance of the DIF analysis. Detection power was poor, averaging only 0.34 across the analysis iterations that reached convergence. Additionally, the short 10-item instrument proved challenging for MLE, a condition in which a Bayesian estimation method appears necessary. Finally, this paper provides recommendations on data conditions which improve performance of the MMix2PL model for DIF analysis. Additionally, suggestions for several improvements to the MMix2PL analysis process, which have potential to improve the feasibility of the model for DIF analysis, are summarized.

No description available.

No description available.