No description available.

This study explores, from a test validity perspective, the extent to which the two variables of topic and background knowledge of topic have an effect on spoken performance in language proficiency interviews. It is argued that in assessment contexts where topics are randomly assigned to test takers, it is necessary to demonstrate that topics of tasks and the level of background knowledge that test takers brings to these topics do not exert an undue influence on test results. Otherwise, a validity threat may be introduced to the test. Data were collected from 82 Farsi speakers of English who performed on ten different topics, across three task types. Participants’ background knowledge of topics was elicited using self- report questionnaires while C-tests were used as a measure of general English language proficiency. Four raters assigned scores to spoken performances using rating scales. Semi- structured interviews were carried out with raters, upon completion of the rating process. A mixed- methods strategy of inquiry was adopted where findings from the quantitative analyses of score data (using Multi-Faceted Rasch Measurement, multiple regression and descriptive statistics) were synthesised with the results of the qualitative analyses of rater interviews and test takers’ content of speech in addressing the foci of the study. The study’s main findings showed that the topics used in the study exhibited difficulty measures which were statistically distinct i.e. topics, within a given task type, could not be considered parallel. However, the size of the differences in topic difficulties was too small to have a large practical effect on scores. Participants’ different levels of background knowledge were shown to have a consistent, systematic and statistically significant effect on performance with low levels of background knowledge posing the highest level of challenge for test takers and vice versa. Nevertheless, these statistically significant differences in background knowledge levels failed to translate into practically significant differences, as the size of the differences were too small to have a large impact on performance scores. Results indicated that, compared to general language proficiency which accounted for approximately 60% of the variance in spoken performance scores, background knowledge only explained about 1-3% of the variance. Qualitative analyses of data suggested lack of background knowledge to be associated with topic abandonment, disengagement from topic-related questions, and fewer opportunities for test takers to elaborate on topics. It was also associated with negative affective influence on test takers, particularly lower proficiency individuals. Taken together, the findings have theoretical, methodological and practical implications for second language speaking performance assessment.

South African organisations face the challenge of creating organisations that will engage employees in ways that allow for the optimisation of their strengths. This can be achieved by following a strength-based approach (SBA). An SBA aims to achieve optimisation of human functioning, where talents and strengths are the focus and weaknesses are understood and managed. Although previous research suggests that an SBA has positive influences on individual and organisational outcomes, no instrument exists within the South African context that measures employees’ perception of the extent to which they believe their organisation makes use of their strengths and talents. Recently, a new scale was developed to address this need. However, no studies have been done to see how well the items of this scale function. The objectives of this research were to 1) conceptualise an SBA according to literature, 2) determine whether the items in the SBA scale are unidimensional, 3) to determine the internal validity and reliability of the new SBA scale, and 4) make recommendations for future research. A cross-sectional quantitative research design was used whereby online and hardcopy versions of the questionnaire were distributed to participants. A sample of 699 participants was collected from the Gauteng and North-West provinces. Rasch analysis was used to determine the reliability and validity of the items. Acceptable item reliability was found. Both the item and person separation indices were acceptable. Mean infit and outfit indices for both person and item were acceptable. The seven-point frequency-based Likert scale worked satisfactorily overall, although categories 0, 1, and 2 were under-utilised. Finally, the infit and outfit statistics for all eight items functioned satisfactorily, except for one item. Recommendations are made for practice as well as for future research. / Thesis (MA (Industrial Psychology))--North-West University, Potchefstroom Campus, 2012

There is a distinct lack of research into the concept of acute mental health crisis. Without investigating the concept of crisis itself, it is not possible to appreciate the attributes of crisis so that it can be measured. This has hampered the development of good psychometric tools for crisis. The aim of this research was to develop the first standardised, valid and reliable measure for the assessment of people presenting to Crisis Resolution and Home Treatment (CRHT) teams. This research utilised qualitative and quantitative research techniques to develop a crisis measure starting with a comprehensive investigation into the concept of acute mental health crisis to identify an item pool and clinically credible item rating scale. A prototype crisis measure was developed and piloted in two NHS CRHTs and data collected. This data was analysed to identify the key areas of crisis assessment (the subscales), a flexible rating scale and scoring system creating a measure named the Crisis Risk and Adaptive Functioning Tool (CRAFT). The CRAFT provides patient crisis profiles highlighting areas of strength, resilience, weakness and vulnerability. A thorough investigation of crisis was completed with CRHT staff and patients through interviews and focus groups. An initial 143 item pool and clinically credible item rating scale were identified and developed into a prototype pilot crisis measure. This measure utilised a flexible rating approach encapsulating both risk and protective factors believed to reflect clinical practice. The measure was piloted and the data analysed to assess the structure of the crisis measure’s item pool using the statistical techniques of Principal Component Analysis (PCA) and Rasch analysis. These analyses resulted in a 66 item measure with 8 unidimensional subscales including; 1) Crisis Recovery Indicators, 2) Adaptive Decision Making, 3) Risk of Harm to Self, 4) Mediating Factors, 5) Daily Structure, 6) Risk of Harm to Others, 7) Feelings and Affect, and 8) Basic Needs. The total variance explained by these 8 subscales was 67.6% with excellent internal reliability as indicated by a Cronbach’s alpha coefficient of 0.98 (p<0.001) and temporal reliability indicated by Spearman’s correlation of 0.971 (p<0.001, one tailed). This suggests that this measure has a strong internal structure and provides stable outcomes over time at both the subscale and global overall measurement levels. Receiver Operator Characteristic curve analysis supported the identification of cut-offs to indicate low, moderate and high levels of crisis and were shown to have good levels of sensitivity and specificity for the crude discrimination between individuals who require CRHT treatment and individuals who do not require CRHT treatment (sensitivity 0.89 and specificity 0.73) and for accurately discriminating between the basic treatment levels of low, moderate and high (sensitivity 0.80; specificity 0.69). One of the great advantages of utilising the Rasch model is that it supports the identification of key characteristics from an item pool. Application of the subscales and the overall measure to the Rasch model identified items that were most representative of underlying constructs and risk, highlighting items of essential essence for assessing crisis in the context of community treatment. These items may act as useful clinical and risk indicators for community assessment. After considering the evidence from the PCA and Rasch analysis for the underpinning construct, the measure was named the Crisis Risk and Adaptive Functioning Tool (CRAFT) to encapsulate both the risk and adaptive functioning (coping and management) aspects measured by the tool. There are a number of clinical implications resulting from the development of CRAFT for the assessment of crisis. This research clearly identifies 8 key areas for crisis assessment and the specific items that describe them. This promises to be a powerful clinical tool as it clarifies the main areas of concern and importance for crisis assessment and provides mental health professionals with a means of assessing and monitoring patients experiencing crisis. In addition to the clinical benefits offered by the CRAFT, it provides an approach to assessing and monitoring crisis to support further research in the area of acute mental health crisis. This research offers significant steps towards the development of a quality measure for crisis assessment. However, it is acknowledged that the process of measurement development is never complete. It simply evolves over time with the aim of coming closer to the valued direction.

In second language (L2) listening assessment, various factors have the potential to impact the validity of listening test items (Brindley & Slatyer, 2002; Buck & Tatsuoka, 1998; Freedle & Kostin, 1999; Nissan, DeVincenzi, & Tang, 1996; Read, 2002; Shohamy & Inbar, 1991). One relatively unexplored area to date is who controls the aural input. In traditional standardized listening tests, an administrator controlled recording is played once or twice. In real world or classroom listening, however, listeners can sometimes request repetition or clarification. Allowing listeners to control the aural input thus has the potential to add test authenticity but requires careful design of the input and expected response as well as an appropriate computer interface. However, if candidates feel less anxious, allowing control of listening input may enhance examinees' experience and still reflect their listening proficiency. Comparing traditional and self paced (i.e., examinees having the opportunity to start, stop, and move the audio position) delivery of multiple choice comprehension items, my research inquiry is whether self paced listening can be a sufficiently reliable and valid measure of examinees' listening ability. Data were gathered from 100 prospective and current university ESL students. They were administered computer based multiple choice listening tests: 10 identical once played items, followed by 33 items in three different conditions: 1) administrator paced input with no audio player visible, 2) self paced with a short time limit, and 3) self paced with a longer time limit. Many facet Rasch (1960/1980) modeling was used to compare the difficulty and discrimination of the items across conditions. Results indicated items on average were similar difficulty overall but discriminated best in self paced conditions. Furthermore, the vast majority of examinees reported they preferred self paced listening. The quantitative results were complemented by follow up stimulated recall interviews with eight participants who took 22 additional test items using screen capture software to explore whether and when they paused and/or repeated the input. Frequency of and reasons for self pacing did not follow any particular pattern by proficiency level. Examinees tended to play more than once but not two full times through, even without limited time. Implications for listening instruction and classroom assessment, as well as standardized testing, are discussed.

A prevalência de crianças com atrasos no desenvolvimento é estimada em torno de 16 a 18%, sendo que menos de um terço dessas crianças são identificadas por serviços básicos de saúde. Estima-se que no Brasil a taxa de identificação de crianças com problemas do desenvolvimento é ainda menor uma vez que existe uma lacuna de instrumentos validados e normatizados para crianças de idades entre zero a seis anos de idade. A identificação de atrasos e comprometimentos do desenvolvimento cognitivo de crianças é essencial já que quanto mais cedo houver encaminhamento para intervenção, maiores são os benefícios. A escala cognitiva do Inventário Dimensional de Avaliação do Desenvolvimento Infantil (IDADI) é composta por 106 itens que avaliam o desempenho em funções como formação de conceitos, simbolização, abstração, percepção, atenção, velocidade de processamento da informação, processamento visoespacial, solução de problemas e memória. O presente estudo objetivou investigar as evidências de validade relacionadas à estrutura interna da escala cognitiva do IDADI e descrever as transformações das estimativas de desenvolvimento cognitivo ao longo das faixas etárias da amostra. 1336 crianças foram avaliadas a partir das respostas das mães aos itens do inventário. Análises de Rasch indicaram consistência interna plenamente satisfatória e ótima discriminação dos participantes. Os itens forneceram uma medida compreensível do traço latente avaliado indicando validade de construto e precisão da escala. As faixas etárias tiveram diferenças significativas com grande tamanho de efeito e análises de agrupamento apontaram para diferentes padrões de associação entre desenvolvimento e idade ao longo do tempo. Deste modo, a escala cognitiva do IDADI se mostrou um instrumento válido e útil capaz de avaliar os diferentes estágios do desenvolvimento cognitivo infantil. / The prevalence of children with developmental delays is estimated to range from 16% to 18%, but less than one-third of these children are identified by their health care providers. In Brazil, underdetection of children with developmental problems is even greater due to the limited number of standardized screening and assessment tools with appropriate normative data. Early detection of delays and disabilities is essential, because the earlier the intervention, the greater the benefit. In order to address this issue, the cognitive scale of the Inventário Dimensional de Avaliação do Desenvolvimento Infantil (IDADI; Dimensional Inventory for Child Development Assessment) was developed. It consists of 106 items that assess, via mother report, cognitive development on symbolization, abstraction, perception, attention, information processing speed, visuospatial processing, problem solving and memory. A sample of 1336 Brazilian children took part on the study. Rasch analysis indicated satisfactory discrimination. Items provided a comprehensive measure of the latent trait indicating construct validity and scale reliability. Age groups had significant differences with large effect size. Cluster analyzes indicated different patterns of association between cognitive development and age over time. Thus, the results point out that the cognitive scale of IDADI can be considered a valid and useful tool, capable to assess the different stages of child cognitive development.

Diversos estudos têm demonstrado associação entre o temperamento e os níveis de ansiedade pré-operatória em crianças. Entretanto, a natureza desta associação não é consenso. Como a maioria desses estudos utilizou o questionário EASI (emocionalidade, atividade, sociabilidade e impulsividade) como instrumento para mensurar o temperamento, o presente estudo examina em profundidade as propriedades psicométricas do mesmo utilizando a análise de Rasch, para determinar se o instrumento mede de forma adequada o temperamento de crianças. Além disso, outras análises são aplicadas para explorar novas estruturas fatoriais do instrumento original. A habilidade da nova versão do EASI para discriminar diferentes níveis de ansiedade no período préoperatório imediato em crianças também foi investigada. Trata-se de um estudo transversal, com amostra de conveniência, onde foram incluídas crianças entre dois e seis anos agendadas para procedimentos cirúrgicos eletivos ambulatoriais. Os cuidadores das crianças completaram um formulário sócio-demográfico, o questionário EASI e uma Escala Análogo-Visual (EAV) de 100mm. Também, a Escala de Ansiedade Pré-Operatória de Yale (mYPAS) foi aplicada imediatamente antes da administração da medicação préanestésica, pelo anestesista pediátrico. Uma vez que o escore médio de compreensão das questões, medido pela EAV, foi superior a 80mm, considerou-se que o nível de compreensão da versão traduzida do EASI foi adequado. O questionário foi respondido por 110 cuidadores. Os resultados da análise de Rasch evidenciaram que as quatro subescalas da estrutura original apresentam desempenho inadequado (especialmente baixo índice de classificação correta dos aspectos do temperamento). A análise de componentes principais gerou uma solução com dois fatores. O Fator 1 é composto de itens de atividade e impulsividade, e o Fator 2 é predominantemente composto por itens das sub-escalas de sociabilidade e atividade. As sub-escalas originais do EASI não se correlacionaram com os escores da mYPAS, à exceção da sociabilidade (r=-0,449; P <0,001). O Fator 1 apresentou correlação positiva (r=0,239; P =0,0034) com a mYPAS, enquanto o Fator 2 apresentou correlação negativa (r=-0,404, P <0,0001). A análise de Rasch indicou que o instrumento original teve poder de discriminação insuficiente. Dentre os dois fatores propostos, os doze itens que compõem o Fator 1 apresentam perfil estatístico adequado, com alto poder de discriminação e relevância clínica. O Fator 2 não tem poder de discriminação adequado no seu formato atual, e ainda necessita de ajustes. / Some studies have shown correlation between temperament and preoperative anxiety levels in children. However, the nature of the association is not consensual. As most of these studies have used the EASI (emotionality, activity, sociability and impulsivity) questionnaire as an instrument to assess temperament, the present study examines the psychometric properties of this questionnaire in depth using the Rasch analysis to determine whether it is suitable for measuring children’s temperament. In addition, further analyses are carried out to explore potentially new factorial structures of the original instrument. The ability to discriminate between different levels of a child’s anxiety in the immediate preoperative period was also examined. It is a cross-sectional study, with a convenience sample of children aged between 2 and 6 years, to be submitted to outpatient elective surgeries. Children’s caregivers completed a socio-demographic form, the EASI questionnaire and a 0-100mm Analogue-Visual Scale (VAS). In addition, the Yale Preoperative Anxiety Scale (mYPAS) was performed immediately prior to the administration of pre-anesthetic medication by the pediatric anesthetist. As the mean comprehension level of the EASI-questions on VAS score was above 80mm, the translation of EASI was considered adequate. The questionnaire was answered by 110 caregivers. Results from Rasch analysis showed that the four subscales have inadequate perfomance (especially low person separation indices). Principal component analysis yielded a two-factor solution. Factor 1 is composed of activity and impulsivity items, and factor 2 is predominantly composed of items from the sociability and activity subscales. The original EASI subscales did not correlate with m-YPAS scores, with exception of sociability (r=-0.449; P <0.001). Factor 1 had a positive correlation (r=0.239; P=0.0034) with m-YPAS, while factor 2 showed a negative correlation (r=-0.404, P <0.0001). Rasch analysis indicated that the original EASI instrument has insufficient separation power. The twelve items that compose Factor 1 show adequate fit statistics, high separation power, and clinical meaning. Factor 2 is not sufficiently powerful in its current state, and still requires refinements.

The overall aim of this thesis was to examine the construct validity and the reliability of the instrument DOA; Dialogue about working ability. The theoretical foundation for DOA is the Model of Human Occupation. The instrument is divided into two sections; client self-assessment and professional assessment focusing on the individual’s working ability. The assessments are followed by a dialogue to distinguish goals for the return to-work process based on the client’s own preferences. In line with the theoretical base, the concept ‘work’ refers to both paid work and other productive activities related to work. In Study I, 21 professionals and 126 clients participated from three clinics in Sweden. All clients were involved as outpatients in psychiatric work rehabilitation in county councils or community services. The results from the Rasch analysis test indicated that items are well separated from each other and generally work together in five unidimensional continuums with the objective to measure abilities related to work. Twenty-nine of 34 items showed goodness-of-fit statistics, which means acceptable infit MnSq values from &gt;0.6 to &lt;1.4 in association with Z values from –2 to 2, and that the five misfit items have to be revised. Thirty-four outpatient clients and 14 professionals from four clinics within psychiatric work rehabilitation in Sweden participated in Study II. Two reliability aspects were studied: test-retest and inter rater reliability. The statistical analyses used were Spearman Rank Correlation test and Percentage of agreement (PA). Generally, the Spearman Rank Correlation test showed acceptable significant correlations between test retest since a greater part of items correlated “moderate to good” (r = 0.51 – 0.75). In the inter raters´ test PA was good, ranging from 93.1 % to 96.5 %, which is well above the acceptable criteria of 80 %. There were no significance differences between the raters. The study has shown that one item has to revised. The findings verify that DOA has potential as a means of making valid and reliable assessments of working ability, as described by the Model of Human Occupation, out of both clients´ and occupational therapists´ perspective.

<p>The overall aim of this thesis was to examine the construct validity and the reliability of the instrument DOA; Dialogue about working ability. The theoretical foundation for DOA is the Model of Human Occupation. The instrument is divided into two sections; client self-assessment and professional assessment focusing on the individual’s working ability. The assessments are followed by a dialogue to distinguish goals for the return to-work process based on the client’s own preferences. In line with the theoretical base, the concept ‘work’ refers to both paid work and other productive activities related to work.</p><p>In Study I, 21 professionals and 126 clients participated from three clinics in Sweden. All clients were involved as outpatients in psychiatric work rehabilitation in county councils or community services. The results from the Rasch analysis test indicated that items are well separated from each other and generally work together in five unidimensional continuums with the objective to measure abilities related to work. Twenty-nine of 34 items showed goodness-of-fit statistics, which means acceptable infit MnSq values from >0.6 to <1.4 in association with Z values from –2 to 2, and that the five misfit items have to be revised.</p><p>Thirty-four outpatient clients and 14 professionals from four clinics within psychiatric work rehabilitation in Sweden participated in Study II. Two reliability aspects were studied: test-retest and inter rater reliability. The statistical analyses used were Spearman Rank Correlation test and Percentage of agreement (PA). Generally, the Spearman Rank Correlation test showed acceptable significant correlations between test retest since a greater part of items correlated “moderate to good” (r = 0.51 – 0.75). In the inter raters´ test PA was good, ranging from 93.1 % to 96.5 %, which is well above the acceptable criteria of 80 %. There were no significance differences between the raters. The study has shown that one item has to revised.</p><p>The findings verify that DOA has potential as a means of making valid and reliable assessments of working ability, as described by the Model of Human Occupation, out of both clients´ and occupational therapists´ perspective.</p>

South African organisations face the challenge of creating organisations that will engage employees in ways that allow for the optimisation of their strengths. This can be achieved by following a strength-based approach (SBA). An SBA aims to achieve optimisation of human functioning, where talents and strengths are the focus and weaknesses are understood and managed. Although previous research suggests that an SBA has positive influences on individual and organisational outcomes, no instrument exists within the South African context that measures employees’ perception of the extent to which they believe their organisation makes use of their strengths and talents. Recently, a new scale was developed to address this need. However, no studies have been done to see how well the items of this scale function. The objectives of this research were to 1) conceptualise an SBA according to literature, 2) determine whether the items in the SBA scale are unidimensional, 3) to determine the internal validity and reliability of the new SBA scale, and 4) make recommendations for future research. A cross-sectional quantitative research design was used whereby online and hardcopy versions of the questionnaire were distributed to participants. A sample of 699 participants was collected from the Gauteng and North-West provinces. Rasch analysis was used to determine the reliability and validity of the items. Acceptable item reliability was found. Both the item and person separation indices were acceptable. Mean infit and outfit indices for both person and item were acceptable. The seven-point frequency-based Likert scale worked satisfactorily overall, although categories 0, 1, and 2 were under-utilised. Finally, the infit and outfit statistics for all eight items functioned satisfactorily, except for one item. Recommendations are made for practice as well as for future research. / Thesis (MA (Industrial Psychology))--North-West University, Potchefstroom Campus, 2012