The purpose of this descriptive, ex post facto study was to examine possible relationships between demographic, pre-programmatic, and programmatic factors with success in a final Advanced Medical-Surgical nursing course and the National Council Licensure Examination for Registered Nurses, as well as their predictive abilities. Data were obtained from the academic records of 209 full-time and part-time nursing program graduates who completed an upper-division baccalaureate nursing degree at a mid-Atlantic private urban university. Descriptive and inferential analyses were utilized to discover possible relationships between the two dependent variables and the various independent variables in groupings suggested by Carroll's (1963) model of School Learning. Two models were derived to predict success in the nursing program as evidenced by successful completion of the final Advanced Medical-Surgical nursing course and passing the NCLEX-RN on the first attempt. The first model identified age at entrance to the nursing program and repeating a science course as the two factors that explained approximately 49% of the variance in the Advanced Medical-Surgical course performance. The second model was able to predict 97.2% correctly those graduates who would be successful on the NCLEX-RN, and only identify 43.8% of those candidates likely to fail. The overall classification ability by the model was 89%. Implications for nursing educators are that more attention needs to be given to admission policies/procedures, and that students entering nursing programs require routine standardized evaluation, identification and remediation of nursing content gaps in order to be successful throughout their nursing studies and ultimately on the NCLEX-RN. / Educational Psychology

A rating design is a pre-specified plan for collecting ratings. The best design for a rater-mediated assessment both psychometrically and from the perspective of fairness is a fully-crossed design in which all objects are rated by all raters. An incomplete rating design is one in which all objects are not rated by all raters, instead each object is rated by an assigned subset of raters usually to reduce the time and/or cost of the assessment. Human raters have varying propensities to rate severely or leniently. One method of compensating for rater severity is the many-facets Rasch model (MFRM). However, unless the incomplete rating design used to gather the ratings is appropriately linked, the results of the MFRM analysis may not be on the same scale and therefore may not be fairly compared. Given non-trivial numbers of raters and/or objects to rate, there are numerous possible incomplete designs with various levels of linkage. The literature provides little guidance on the extent to which differently linked rating designs might affect the results of a MFRM analysis. Eighty different subsets of data were extracted from a pre-existing fully-crossed rating data set originally gathered from 24 essays rated by eight raters. These subsets represented 20 different incomplete rating designs and four specific assignments of raters to essays. The subsets of rating data were analyzed in Facets software to investigate the effects of incomplete rating designs on the MFRM results. The design attributes related to linkage that were varied in the incomplete designs include (a) rater coverage: the number of raters-per-essay, (b) repetition-size: the number of essays rated in one repetition of the sub-design pattern, (c) design structure: the linking network structure of the incomplete design, and (d) rater order: the specific assignments of raters to essays. A number of plots and graphs were used to visualize the incomplete designs and the rating results. Several measures including the observed and fair averages for raters and essays from the 80 MFRM analyses were compared against the fair averages for the fully-crossed design. Results varied widely depending on different combinations of design attributes and rater orders. Rater coverage had the overall largest effect, with rater order producing larger ranges of values for sparser designs. Many of the observed averages for raters and essays more closely approximated the results from the fully-crossed design than did the adjusted fair-averages, particularly for the more sparsely linked designs. The stability of relative standing measures was unexpectedly low.

The Rasch model was employed to analyze the psychometric properties of a diagnostic reading assessment and then create five short forms (n = 10, 16, 22, 28, 34 items) with an optimal test information function. The goal was to develop a universal screening measure that second grade teachers can use to identify advanced and at-risk readers to enhance Response-to-Intervention frameworks. These groups were targeted because both will need differentiated instruction in order to improve reading skills. The normative dataset of a national reading test developed with classical test theory methods was used to estimate person and item parameters. The measurement precision and classification accuracy of each short form was evaluated with the second grade students in the normative sample. Compared with full bank scores, all short forms produced highly correlated scores. The degree to which each short form identified exceptional readers was also analyzed. In consideration of classification accuracy and time-efficiency, the findings were most robust for the 10-item form. / text

Background: The Alliance for Patient Medication Safety (APMS) helps community pharmacies comply with continuous quality improvement requirements of many states and third party payment contracts through use of their Pharmacy Quality Commitment (PQC) program. Aims: To assess changes in the incidence of prescription Quality Related Events (QREs) and adoption of patient safety culture attitudes after guided PQC implementation. Methods: Twenty-one pharmacies were randomized to standard PQC practices (control) or guided PQC implementation (treatment). Pharmacy staff completed retrospective pre and post safety culture questionnaires. Negative binomial mixed and linear regression analyses were employed to examine changes in QREs rates and attitudes, respectively. Rasch analysis was used to assess questionnaire validity and reliability. Results: During the 2-month study period 3,343 QREs were reported to the online PQC system. At baseline, no difference in the average QRE reporting rate was identified between groups. Treatment group QRE reporting rates differed in one category post treatment (more incorrect safety caps QREs reported in treatment group, p<0.034). Seventy-one employees completed the questionnaire (70% response). Attitude improvement in the treatment group was 36% greater than in control (p<0.0001). The questionnaire demonstrated acceptable reliability and validity evidence. Conclusions: Guided PQC implementation increased reporting of certain QREs and increased the adoption of patient safety culture attitudes among staff pharmacy.

This document is a supplementary text to "An Empirical Power Analysis of Quasi-Exact Tests for the Rasch Model: Measurement Invariance in Small Samples" by Koller, Maier, & Hatzinger (to be published in Methodology, ISSN-L 1614-1881), which covers all technical details regarding the simulation and its results. First, the simulation scenarios and the introduction of differential item functioning (DIF) are described. Next, the different populations' distributions that were investigated are discussed, and finally, actual type-I-error rates and empirical power are displayed for all simulated scenarios. (authors' abstract) / Series: Research Report Series / Department of Statistics and Mathematics

The ceiling and basal rules of the Wechsler Adult Intelligence Scale – Fourth Edition (WAIS-IV; Wechsler, 2008) only function as intended if subtest items proceed in order of difficulty. While many aspects of the WAIS-IV have been researched, there is no literature about subtest item difficulty and precise item difficulty values are not available. The WAIS-IV was developed within the framework of Classical Test Theory (CTT) and item difficulty was most often determined using p-values. One limitation of this method is that item difficulty values are sample dependent. Both standard error of measurement, an important indicator of reliability, and p-values change when the sample changes. A different framework within which psychological tests can be created, analyzed and refined is called Item Response Theory (IRT). IRT places items and person ability onto the same scale using linear transformations and links item difficulty level to person ability. As a result, IRT is said to be produce sample-independent statistics. Rasch modeling, a form of IRT, is one parameter logistic model that is appropriate for items with only two response options and assumes that the only factors affecting test performance are characteristics of items, such as their difficulty level or their relationship to the construct being measured by the test, and characteristics of participants, such as their ability levels. The partial credit model is similar to the standard dichotomous Rasch model, except that it is appropriate for items with more than two response options. Proponents of standard dichotomous Rasch model argue that it has distinct advantages above both CTT-based methods as well as other IRT models (Bond & Fox, 2007; Embretson & Reise, 2000; Furr & Bacharach, 2013; Hambleton & Jones, 1993) because of the principle of monotonicity, also referred to as specific objectivity, the principle of additivity or double cancellation, which “establishes that two parameters are additively related to a third variable” (Embretson & Reise, 2000, p. 148). In other words, because of the principle of monotonicity, in Rasch modeling, probability of correctly answering an item is the additive function of individuals’ ability, or trait level, and the item’s degree of difficulty. As ability increases, so does an individual’s probability of answering that item. Because only item difficulty and person ability affect an individual’s chance of correctly answering an item, inter-individual comparisons can be made even if individuals did not receive identical items or items of the same difficulty level. This is why Rasch modeling is referred to as a test-free measurement. The purpose of this study was to apply a standard dichotomous Rasch model or partial credit model to the individual items of seven core perceptual, verbal and working memory subtests of the WAIS-IV: Block Design, Matrix Reasoning, Visual Puzzles, Similarities, Vocabulary, Information, Arithmetic Digits Forward, Digits Backward and Digit Sequencing. Results revealed that WAIS-IV subtests fall into one of three categories: optimally ordered, near optimally ordered and sub-optimally ordered. Optimally ordered subtests, Digits Forward and Digits Backward, had no disordered items. Near optimally ordered subtests were those with one to three disordered items and included Digit Sequencing, Arithmetic, Similarities and Block Design. Sub-optimally ordered subtests consisted of Matrix Reasoning, Visual Puzzles, Information and Vocabulary, with the number of disordered items ranging from six to 16. Two major implications of the result of this study were considered: the impact on individuals’ scores and the impact on overall test administration time. While the number of disordered items ranged from 0 to 16, the overall impact on raw scores was deemed minimal. Because of where the disordered items occur in the subtest, most individuals are administered all the items that they would be expected to answer correctly. A one-point reduction in any one subtest is unlikely to significantly affect overall index scores, which are the scores most commonly interpreted in the WAIS-IV. However, if an individual received a one-point reduction across all subtests, this may have a more noticeable impact on index scores. In cases where individuals discontinue before having a chance to answer items that were easier, clinicians may consider testing the limits. While this would have no impact on raw scores, it may provide clinicians with a better understanding of individuals’ true abilities. Based on the findings of this study, clinicians may consider administering only certain items in order to test the limits, based on the items’ difficulty value. This study found that the start point for most subtests is too easy for most individuals. For some subtests, most individuals may be administered more than 10 items that are too easy for them. Other than increasing overall administration time, it is not clear what impact, of any, this has. However, it does suggest the need to reevaluate current start items so that they are the true basal for most people. Future studies should break standard test administration by ignoring basal and ceiling rules to collect data on more items. In order to help clarify why some items are more or less difficult than would be expected given their ordinal rank, future studies should include a qualitative aspect, where, after each subtest, individuals are asked describe what they found easy and difficult about each item. Finally, future research should examine the effects of item ordering on participant performance. While this study revealed that only minimal reductions in index scores likely result from the prematurely stopping test administration, it is not known if disordering has other impacts on performance, perhaps by increasing or decreasing an individual’s confidence.

No description available.

A validity study can be used to investigate the effectiveness of an exam and reveal both its strengths and weaknesses. This study concerns an investigation of the writing portfolio Level Achievement Test (LAT) at the English Language Center (ELC) of Brigham Young University (BYU). The writing portfolios of 251 students at five proficiency levels were rated by 11 raters. Writing portfolios consisted of two coursework essays, a self-reflection assignment, and a 30-minute timed essay. Quantitative methods included an analysis with Many-Facet Rasch Model (MFRM) software, called FACETS, which looked for anomalies in levels, classes, examinees, raters, writing criteria, and the rating scale categories. Qualitative methods involved a rater survey, rater Think Aloud Protocols (TAPs), and rater interviews. Results indicated that the exam has a high degree of validity based on the MFRM analysis. The survey and TAPs revealed that although raters follow a similar pattern for rating portfolios, they differed both in the time they took to rate portfolios and in the degree to which they favored the rating criteria. This may explain some of the discrepancies in the MFRM rater analysis. Conclusions from the MFRM analysis, surveys, TAPs, and interviews were all used to make recommendations to improve the rating process of the LAT, as well as to strengthen the relationship between LAT rating and classroom teaching and grading.

No description available.

De nombreux chercheurs et cliniciens sont sceptiques quant à la validité des questionnaires autoadministrés, lorsqu’utilisés auprès d’une population carcérale (Gendreau, Irvine et Knight, 1973), surtout si celle-ci est composée de délinquants sexuels (Marshall et Hall, 1995). La sensibilité des sujets investigués jumelés à la transparence des questions expose l’évaluateur à la possibilité que le participant dissimule et modifie ses réponses (Tierney et McCabe, 2001). L’objectif de ce projet est de comprendre les processus impliqués dans la performance des participants à une évaluation autoadministrée. Les données de 282 délinquants sexuels ayant complétés le Multidimensional Inventory of Development, Sex, and Agression (MIDSA) ont été analysées afin de mieux comprendre l’interaction entre les échelles de désirabilité sociale, les temps de latence et les coefficients d’ajustement du modèle de Rasch. La convergence des analyses de temps de latence et des échelles de désirabilité sociale semble indiquer que certains participants dissimuleraient consciemment leurs réponses. Notamment, les participants détectés par les échelles de désirabilité sociales sembleraient répondre de manière de plus lente aux échelles d’évaluations, et certains d’entre eux offriraient des patrons de réponses incohérents à la prescription du modèle de Rasch. Les hypothèses permettant d’expliquer les potentiels mécanismes liés à la dissimulation seront discutées. / Many researchers and clinicians are skeptical towards the validity of self-reported assessment used within a forensic population (Gendreau, Irvine, & Knight, 1973), especially when used within a sexual offender population (Marshall & Hall, 1995). The sensitivity of the topics addressed as well as the transparency of the questions expose the evaluator to possible dissimulations or distortions in the subject responses (Tierney & McCabe, 2001). Responses of 282 sexual offenders who completed the Multidimensional Inventory of Development, Sex, and Aggression (MIDSA) were analyzed in order to understand the interaction between response time, social desirability scales and Rasch person-fit scores in order to account for response distortion. The convergence of latency times and social desirability scales seems to indicate that certain participants consciously manipulate their responses. Notably, participants detected by social desirability scales seem to respond more slowly to particular items. Other participants may answer in ways that are incoherent with the predicted responses according to the Rasch model. Hypotheses as to potential mechanisms underlying this process will be discussed.