Research has indicated accredited, U.S. higher education-governed intensive English programs (IEPs) often struggle financially due to a scarcity of resources (namely students) because of political and global economic factors and increased competition (ICEF Monitor, 2017; IIE, 2017; Ladika, 2018; Soppelsa, 2015). However, few IEPs advertise online language acquisition (OLA) courses despite the increase in online study methods at the higher education institutes governing the programs and its use by competitors. The purpose of this study was to determine the status and extent of OLA diffusion in U.S. IEPs, how IEP directors and faculty perceived OLA, and whether they perceived themselves to be the leaders in its diffusion. Drawing on Rogers’ (1962) diffusion of innovation framework to inform the instrument methodology, this study employed a quantitative, cross-sectional survey. The study used the Rasch measurement model (1960) as the framework informing the instrument’s design and analysis. All 249 executive directors and 2,492 faculty in the 249 accredited, higher education-governed IEPs were invited to participate in the study, and 328 directors and faculty from 121 IEPs opted to do so. Major findings revealed 40.5% had experimented with online courses within the last five years, and 24.8% offered it currently. The Winsteps dimensionality analysis showed each of the six innovation characteristics performed as a separate strand supporting the dimension of OLA adoption potential. The Wright map and item measures revealed respondents perceived OLA visibility (1.52 logits) as the most difficult-to-endorse characteristic followed by complexity (0.48 logits). The least challenging characteristic was articulated benefits (-0.39 logits), and the easiest item was technology confidence (-1.21 logits) followed by technology clusters (-0.65 logits). Regarding leadership in promoting OLA adoption, 53.2% of the sample claimed they were involved in its leadership at some level, and 31.1% reported leadership involvement at institutes currently lacking online English courses. This study suggests respondents found OLA to be beneficial for their IEP with articulable results. Cost and technology confidence were not viewed as prohibitive, but respondents lacked confidence that OLA would lead to increased enrollment. Because of the high level of OLA leadership in their IEP, the adoption of online language courses appears to be moving in an upward trajectory.

Background: This thesis is was designed to evaluate aspects of reliability and validity of the School Version of the Assessment of Motor and Process Skills (School AMPS) (Fisher, Bryze, Hume, & Griswold, 2007), an observation-based evaluation of quality of occupational performance when children perform schoolwork tasks in school settings. The long term goal was to contribute to knowledge about children at risk or with mild disabilities who experience difficulties with occupational performance in school settings, and describe how the School AMPS can be used when a true top−down process of planning and implementing school-based occupational therapy services is implemented in a Swedish context. Methods: In Study I, two different split-half methods and were used to estimate reliability of the School AMPS measures. These were cross-validated using Rasch equivalent of Cronbach’s alpha. The standard error of measurement (m) was also calculated. In Studies II and III, many-facet Rasch analyses and/or relevant inferential statistics (e.g., ANOVA, tests) were used to examine for evidence of validity based on (1) internal structure related to differential item functioning (DIF), (2) relations to other variables (sensitivity) in terms of comparing groups (typically-developing children vs. children with mild disabilities), and (3) consequences of testing (benefits of testing) in terms of test fairness. In Study IV, ANOVA and tests were used to examine relations to other variables in terms of sensitivity of the School AMPS measures for detecting change based on repeated School AMPS evaluations pre- and post-interventions. Results: The three methods for estimating reliability of the School AMPS measures yielded high reliability coefficient estimates (≥0.73) and low ms. Minimal DIF was identified, and despite minimal DIF, the School AMPS measures were found to be free of differential test functioning. The School AMPS measures were sensitive enough to detect differences between groups as well as changes following consultative occupational therapy services provided in natural school settings. Conclusions: The results support the reliability and validity of the School AMPS scales and measures when used to evaluate quality of occupational performance in school settings. The results are also of clinical importance as they provide evidence that occupational therapists can have confidence in the School AMPS measures when they are used in the process of making decisions about individual students, planning interventions, and later perform follow-up evaluations to measure the outcomes. We also have objective evidence that children with mild disabilities demonstrate diminished quality of "doing" when performing schoolwork tasks. The potential long term benefits of such evidence may be to support or justify the need for children with mild disabilities to receive occupational therapy services within school settings in Sweden; and through collaboration with teachers, plan and implement better targeted and more effective interventions.

Background: The main objective of this study was to investigate older people’s physical activity, their participation in various life situations, and their perceptions of their own health. This included an exploration of potential influences of urban versus rural residency on these outcomes, an evaluation of the measurement properties of a balance confidence scale, and an examination of the proposed usefulness of the International Classification of Functioning, Disability and Health (ICF) as a conceptual framework to facilitate analysis and understanding of selected outcomes. Methods: The study design was cross-sectional, population-based, with random selection from the national register of one urban and two rural municipalities in Northern Iceland. There were 186 participants, all community-dwelling, aged 65 to 88 years (mean = 73.8), and 48% of the group were women. The participation rate was 79%. Data was collected in 2004, in face-to-face interviews and through various standardized assessments. The main outcomes were total physical activity; leisure-time, household, and work-related physical activity; participation frequency and perceived participation restrictions; and self-rated health. Other assessments represented aspects of the ICF body functions, activities, environmental factors and personal factors. Moreover, Rasch analysis methods were applied to examine and modify the Activities-specific Balance Confidence (ABC) scale and the ICF used as a conceptual framework throughout the study. Results: The total physical activity score was the same for urban and rural people and the largest proportion of the total physical activity behavior was derived from the household domain. Rural females received the highest scores of all in household physical activity and rural males were more physically active than the others in the work-related domain. However, leisure-time physical activity was more common in urban than rural communities. A physically active lifestyle, urban living, a higher level of cognition, younger age, and fewer depressive symptoms were all associated with more frequent participation. Rural living and depressive symptoms were associated with perceived participation restrictions. Moreover, perceived participation restrictions were associated with not being employed and limitations in advanced lower extremity capacity. Both fewer depressive symptoms and advanced lower extremity capacity also increased the likelihood of better self-rated health, as did capacity in upper extremities, older age, and household physical activity. Rasch rating scale analysis indicated a need to modify the ABC to improve its psychometric properties. The modified ABC was then used to measure balance confidence which, however, was found not to play a major role in explaining participation or self-rated health. Finally, the ICF was useful as a conceptual framework for mapping various components of functioning and health and to facilitate analyses of their relationships. Conclusions: The results highlighted the commonalities and differences in factors associated with participation frequency, perceived participation restrictions, and self-rated health in old age. Some of these factors, such as advanced lower extremity capacity, depressive symptoms, and physical activity pattern should be of particular interest for geriatric physical therapy due to their potential for interventions. While the associations between depressive symptoms, participation, and self-rated health are well known, research is needed on the effects of advanced lower extremity capacity on participation and self-rated health in old age. The environment (urban versus rural) also presented itself as an important contextual variable to be aware of when working with older people’s participation and physically active life-style. Greater emphasis should be placed on using Rasch measurement methods for improving the availability of quality scientific measures to evaluate various aspects of functioning and health among older adults. Finally, a coordinated implementation of a conceptual framework such as ICF may further advance interdisciplinary and international studies on aging, functioning, and health.

The ability to perform everyday life occupations is a critical component in both evaluation and intervention for persons with mental retardation (MR). While the ability to perform personal and instrumental activities of daily living (ADL) has always been important for occupational therapy (OT) practice, there is an absence in OT literature and research with a focus on ADL and persons with MR. The overall aim of this thesis was to evaluate the validity of the Assessment of Motor and Process Skills (AMPS) for evaluation and intervention of ADL ability for persons with MR. In order to evaluate the evidence of validity of the AMPS ability measures based on relation to level of MR, two groups of participants with MR were evaluated with the AMPS (,#=22; #= 39). The results indicated expected moderate relationships between ADL motor and ADL process ability measures and level of MR, despite different methods used for evaluating level of MR. The results also indicated that the results of the AMPS evaluation could be used to directly describe and measure the consequences in performance of ADL tasks for persons with different levels of MR. The evidence of validity of the AMPS was further examined in a study including participants with different types of developmental disabilities (e.g., MR, cerebral palsy, spina bifida) (#=1724). An application of many-faceted Rasch analysis was used to examine goodness-of-fit of the responses for the tasks, skill items, and participants included in the study. All tasks and all items except one demonstrated acceptable goodness-of-fit to the model on the ADL motor and ADL process scales. An expected proportion of participants demonstrated acceptable goodness-of-fit on the ADL motor scale. On the ADL process scale, a slightly lower proportion of participants than expected demonstrated acceptable goodness-of-fit. The results indicated further that persons with more severe levels of MR and persons with more limited ADL process abilities demonstrated different response patterns across tasks and possibly items. The evidence of validity of the internal structure of the AMPS scales was also evaluated between persons with mild and moderate MR (#=178; #=170). Group specific ADL motor and ADL process skill item hierarchies were generated using many-faceted Rasch analyses and compared. The hierarchies of ADL motor and ADL process skill items remained stable across groups, indicating evidence of validity of the AMPS scales when used to evaluate persons with MR. The results also indicated that although participants with moderate MR demonstrated overall lower mean ADL motor and ADL process ability, they did perform some specific ADL motor and ADL process skills at a similar level as persons with mild MR. Finally, the utility of the AMPS ability measures for detecting change were examined in an intervention study including three female participants with moderate MR. The study was based on a single case design and evaluated the effectiveness of a structured occupational therapy intervention program. Improvements were found for the participants in relation to the implementation of the program, but the pattern of changes were different between the participants and across the dependent variables. ADL process ability was the only variable that improved across all participants. The results supported the ADL process abilities as sensitive measures for detecting changes in ADL ability of persons with MR. In conclusion, the results of these studies contribute to the evidence of validity of the AMPS ability measures and scales, specifically in relation to the evaluation of persons with MR. The finding that an OT program resulted in improved ADL process ability also suggest that the results of the AMPS can be used to plan as well as evaluate outcomes of OT practice. Further research is also suggested in order to improve validity evidence and utility of the AMPS when used with persons with MR. / <p>Diss. (sammanfattning) Umeå : Umeå universitet, 2003</p>

Occupational engagement among older people is important to investigate as older people are the fastest growing segment in our society, and because occupational engagement is viewed within occupational therapy as one of the basic premises for health. Three perspectives of engagement are highlighted in this thesis: evaluation of occupational engagement, the repertoire of occupational engagement, and finally, the relation between occupational engagement and life satisfaction. The overall aim of this thesis was to study aspects of occupational engagement among older people, with a special focus on evaluation of the experiences of an occupation-based group programme, evaluation of leisure, the leisure repertoire, and the relation between occupational engagement and life satisfaction. The thesis is comprised of four studies which all contribute in different ways to an increasing understanding of occupational engagement among older people. In the first study (Study I), three older persons participated in a group activity programme and were interviewed about their experiences of occupational engagement. The qualitative interviews were done with each participant after each group session, in total 15 interviews. The other three studies (Studies II-IV) were based on a subgroup of a population studied in a cross-sectional population-based study, the Umeå 85+ study. Very old people with an MMSE score of 20 or more were included in Studies II, III, and IV (n=156). During home visits, they were interviewed about their occupational engagement (ADL and leisure) and their life satisfaction. The qualitative interviews analysed using a Grounded theory approach, revealed two different dimensions of experiences while engaged in a group programme. The participants described experiences of activation, with a creative force and a place for learning, but also experiences of transformation with reflection, adaptation, and finally, a personal synthesis. Evaluation of occupational engagement through measurement using the modified NPS Interest Checklist (MNPS) was made possible using Rasch analysis. The results revealed preliminary evidence for internal scale validity and person response validity. Scale and person reliability were Rasch equivalents of Cronbach alpha of .98 for items and .66 to .75 for persons, respectively. In their leisure repertoire, very old people were more likely to endorse Social and Cultural activities and least likely to endorse Ballgames and Equipment sport. Traditional gender differences and some differences between older persons in rural versus urban areas and between persons with different cognitive levels were also found. Finally, significant correlations were found between life satisfaction and both engagement in ADL (r =.31) and engagement in leisure (r =.34) among very old people. A forced entry regression revealed that both variables together explained slightly more (12.4%) than leisure alone (11.2%). As a conclusion and in relation to evaluation of occupational engagement during therapy, the experiences of engagement are described by the respondents from both a perspective of action and a perspective of inner reflection, and together they might support the developmental process among older people. Through using Rasch analysis, it was possible to convert ordinal data into linear measures and also to organize leisure occupations into a hierarchical repertoire of engagement. This repertoire gives further understanding for specific tasks and about the general relation between leisure dimensions. Finally, the contribution of occupational engagement to life satisfaction is likely essential, but explains only about 12% of total life satisfaction among very old people.

This study investigated how 2 different rating conditions, the controlled rating condition (CRC) and the uncontrolled rating condition (URC), effected rater behavior and the reliability of a performance assessment (PA) known as the Missionary Teaching Assessment (MTA). The CRC gives raters the capability to manipulate (pause, rewind, fast-forward) video recordings of an examinee's performance as they rate while the URC does not give them this capability (i.e., the rater must watch the recording straight through without making any manipulations). Few studies have compared the effect of these two rating conditions on ratings. Ryan et al. (1995) analyzed the impact of the CRC and URC on the accuracy of ratings, but few, if any, have analyzed its impact on reliability. The Missionary Teaching Assessment is a performance assessment used to assess the teaching abilities of missionaries for the Church of Jesus Christ of Latter-day Saints at the Missionary Training Center. In this study, 32 missionaries taught a 10-minute lesson that was recorded and later rated by trained raters based on a rubric containing 5 criteria. Each teaching sample was rated by 4 of 6 raters. Two of the 4 ratings were rated using the CRC and 2 using the URC. Camtasia Studio (2010), a screen capture software, was used to record when raters used any type of manipulation. The recordings were used to analyze if raters manipulated the recordings and if so, when and how frequently. Raters also performed think-alouds following a random sample of the ratings that were performed using the CRC. These data revealed that when raters had access to the CRC they took advantage of it the majority of the time, but they differed in how frequently they manipulated the recordings. The CRC did not add an exorbitant amount of time to the rating process. The reliability of the ratings was analyzed using both generalizability theory (G theory) and many-facets Rasch measurement (MFRM). Results indicated that, in general, the reliability of the ratings obtained from the 2 rating conditions were not statistically significantly different from each other. The implications of these findings are addressed.

The purpose of this study is to explore a potentially more practical approach to direct writing assessment using computer algorithms. Traditional rubric rating (RR) is a common yet highly resource-intensive evaluation practice when performed reliably. This study compared the traditional rubric model of ESL writing assessment and many-facet Rasch modeling (MFRM) to comparative judgment (CJ), the new approach, which shows promising results in terms of reliability and validity. We employed two groups of raters”novice and experienced”and used essays that had been previously double-rated, analyzed with MFRM, and selected with fit statistics. We compared the results of the novice and experienced groups against the initial ratings using raw scores, MFRM, and a modern form of CJ”randomly distributed comparative judgment (RDCJ). Results showed that the CJ approach, though not appropriate for all contexts, can be valid and as reliable as RR while requiring less time to generate procedures, train and norm raters, and rate the essays. Additionally, the CJ approach is more easily transferable to novel assessment tasks while still providing context-specific scores. Results from this study will not only inform future studies but can help guide ESL programs to determine which rating model best suits their specific needs.

No description available.

No description available.

[pt] Nesta tese, trabalhou-se com a modelagem Rasch visando a apresentar alternativas mais práticas e de melhor qualidade em termos de medida, para dois cenários distintos. O primeiro está relacionado ao fato de que medir conhecimento é algo muito complexo e de difícil entendimento para profissionais que não são da área da psicometria. Por meio de experimentos envolvendo modelos da família Rasch, apresentamos a aplicabilidade e as potencialidades dessa modelagem para atender a novas demandas de avaliação em larga escala no Brasil. O segundo cenário relaciona-se à busca de medir, de modo o mais imparcial possível, itens de produção escrita, em que a nota recebida pelos alunos é influenciada pela subjetividade dos corretores, ou seja, corretores lenientes beneficiam alunos e corretores severos penalizam alunos. Diante desses dois cenários, esta tese tem os seguintes objetivos: (i) trazer para o âmbito das avaliações realizadas no Brasil uma modelagem matemática mais simples que aquela atualmente adotada, visando uma melhor comunicação com os professores, e; (ii) a possibilidade de operar não apenas com itens de múltipla escolha, corrigidos de forma automática, mas também com itens de produção escrita, em que a subjetividade dos corretores (severidade) é controlada pelo modelo psicométrico, gerando medidas de melhor qualidade. Para isso, utilizou-se a modelagem Rasch com multifacetas, abordando, por meio de casos práticos, as vantagens dessa modelagem em relação a outras metodologias atualmente adotadas no país. Assim, para a alcançarmos o primeiro objetivo, confrontamos a modelagem Rasch com multifacetas com a modelagem de três parâmetros logísticos em um estudo de efeito contexto em testes compostos por diferentes modelos de cadernos e com mais de uma disciplina avaliada por caderno e, para o segundo, comparamos as medidas de proficiência através da Rasch com multifacetas com as notas médias das duplas correções dadas pelos corretores aos alunos em testes do tipo redação. A partir dos resultados encontrados, concluímos que a Rasch com multifacetas pode ser utilizada de forma alternativa ou concomitante com as avaliações que utilizam a modelagem de três parâmetros logísticos, produzindo resultados mais rápidos e de entendimento mais fácil por parte dos professores e que, no caso de redações, as proficiências obtidas pela Rasch com multifacetas apresentaram medidas com melhores indicadores de fidedignidade e validade, quando comparadas com as medidas de notas via Teoria Clássica do Teste, sendo, portanto, uma alternativa mais viável para esse tipo de avaliação. Conclui-se essa tese apresentando situações de empregabilidade das metodologias estudadas. / [en] In this thesis, we worked with Rasch modeling, aiming to present more practical alternatives and better quality in terms of measurement, for two different scenarios. The first one is related to the fact that measuring knowledge is something very complex and difficult to understand for professionals who are not in the psychometrics area. Through experiments involving the Rasch family models, we present the applicability and the potentiality of this model to adequately comply with the new demands of the large-scale evaluation in Brazil. The second scenario is related to the search of measuring, in the most impartial way possible, written production items which grade received by the subjectivity of the raters (severity), that is, lenient raters benefit students and severe raters penalize them. In view of these two scenarios, this thesis has the following objectives: (i) to bring to the scope of the evaluations carried out in Brazil a simpler mathematical modeling than the currently adopted, aiming at a better communication with the teachers; and (ii) the possibility of operating not only with multiple choice items, corrected automatically, but also with written production items, in which the subjectivity of the raters (severity) is controlled by the psychometric model, generating better quality measures. For this, Many-Facet Rasch Measurement was used, approaching, through practical cases, the advantages of this modeling in relation to other methodologies currently adopted in the country. Thus, in order to reach the first objective, we confronted Many-Facet Rasch Measurement with the modeling of three logistic parameters in a study of context effect in tests composed by different models of test books and with more than one discipline evaluated by test book and, for the second one, we compared the measures of proficiency through the Many-Facet Rasch Measurement with the average scores of the double corrections given by the raters to the students in tests of the essay type. From the results found, we conclude that the Many-Facet Rasch Measurement can be used in an alternative or concomitant way with the evaluations that use the three logistic parameters model, producing faster results and easier to understand by the teachers and that, in the case of essays, the measures of proficiency obtained by Many-Facet Rasch Measurement presented measures with better reliability and validity indicators, when compared to the grading measures through the Classical Theory of Testing, being, therefore, a more viable alternative for this type of evaluation. This thesis concludes with situations of usability of the methodologies studied.