Introduction: The Antimicrobial Self-assessment Toolkit for Acute NHS Trusts (ASAT) was developed by a pharmacist reference group of an Advisory Non-Departmental Public Body on Antimicrobial Resistance and Healthcare Associated Infections (ARHAI). It was developed in conjunction with the Department of Health. The primary purpose of the ASAT is to identify and to measure the methods of implementation of antimicrobial stewardship programmes in acute NHS trusts. The face validity was previously tested by ARHAI. The overall aims of this programme of work were to investigate the validity of the ASAT and to make iterative changes to improve its validity. Ethical approval was not required for this PhD project because it was categorised as service evaluation by the LREC. Also, ethical approval from the University of Manchester Research Ethics Committee was deemed unnecessary at the time of the PhD project due to the nature of the data collected. Methods: A mixed methodology approach utilising a sequential exploratory strategy was used to investigate the validity of the ASAT. This PhD project was composed of four sequential studies which resulted in iterative changes to the ASAT, that is, from ASAT v15a to ASAT v18. In Study 1, cognitive interviews were conducted with eight antimicrobial pharmacists in order to investigate the content validity of ASAT v15a. In Study 2, both cognitive interviews and semi-structured interviews were conducted with 10 clinical microbiologists in order to investigate the content validity of ASAT v16. In Study 3, Rasch modelling and analyses using the Partial Credit Model (PCM) were conducted on the responses to ASAT v17 from 33 NHS trusts across England. In Study 4, simple OLS regression analyses were conducted using the NHS trust ‘ability’ estimates or calibrations and Clostridium difficile (CDI) rates of participating NHS trusts in order to investigate model fit and the predictive validity of the ASAT. Results: The cognitive interviews conducted in study 1 indicated that AMPs encountered cognitive difficulties along the cognitive processing pathway in response to ASAT v15a. These difficulties included comprehension in 27 (32.5%) questions and response generation/formatting in 13 (15.7%) questions. Also respondents indicated that the role of clinical microbiologists in ASPs was underrepresented in ASAT v15a. The interviews conducted in Study 2 were confirmatory in nature as they reflected the findings of Study 1. For example terms such as ‘formulary’ and ‘policy’ were misinterpreted by respondents. Rasch modelling and analysis showed that there were items within ASAT v17 which were underfitting and overfitting the Partial Credit Model. Item fit was investigated after removal of these items which resulted in improved fit for domains 2 and 5. ASAT v18 was developed after these analyses and was included items that were productive for measurement. On examination of the OLS regression analyses conducted in Study 4, it was seen that there was poor model fit and very limited predictive validity of the model. Conclusion: The iterative methodology utilised to investigate the validity and subsequently improve the ASAT was effective in establishing content and construct validity. However, the predictive validity of the ASAT was limited. This may be due to the outcome variable chosen for the OLS regression modelling. A more sensitive outcome measure such as compliance to treatment or prophylaxis guidelines may have been more effective at establishing predictive validity. The findings of this programme of work highlighted that there is further work required to validate the ASAT such as the determination of the appropriate weights and scores for ASAT domains and also the determination of the appropriate outcomes measures to determine the efficacy of ASPs. It is recommended that further validity testing should be conducted before a further iteration of the ASAT is used as a set of quality standards or as a hospital benchmarking tool

BACKGROUND:No consistent relationship exists between pre-treatment expectations and therapeutic benefit from various complementary and alternative medicine (CAM) therapies in clinical trials. However, many different expectancy measures have been used in those studies, with no validated questionnaires clearly focused on CAM and pain. We undertook cognitive interviews as part of a process to develop and validate such a questionnaire.METHODS:We reviewed questions about expectations of benefits of acupuncture, chiropractic, massage, or yoga for pain. Components of the questions - verbs, nouns, response options, terms and phrases describing back pain - were identified. Using seven different cognitive interview scripts, we conducted 39 interviews to evaluate how individuals with chronic low back pain understood these individual components in the context of expectancy questions for a therapy they had not yet received. Chosen items were those with the greatest agreement and least confusion among participants, and were closest to the meanings intended by the investigators.RESULTS:The questionnaire drafted for psychometric evaluation had 18 items covering various domains of expectancy. "Back pain" was the most consistently interpreted descriptor for this condition. The most understandable response options were 0-10 scales, a structure used throughout the questionnaire, with 0 always indicating no change, and 10 anchored with an absolute descriptor such as "complete relief". The use of words to describe midpoints was found to be confusing. The word "expect" held different and shifting meanings for participants. Thus paired items comparing "hope" and "realistically expect" were chosen to evaluate 5 different aspects of treatment expectations (back pain / back dysfunction and global effects / impact of back pain on specific areas of life / sleep, mood, and energy / coping). "Impact of back pain" on various areas of life was found to be a consistently meaningful concept, and more global than "interference".CONCLUSIONS:Cognitive interviews identified wordings with considerable agreement among both participants and investigators. Some items widely used in clinical studies had different meanings to participants than investigators, or were confusing to participants. The final 18-item questionnaire is undergoing psychometric evaluation with goals of streamlining as well as identifying best items for use when questionnaire length is constrained.

This dissertation investigated several aspects of middle graders’ mathematical understanding based on representational models. Twenty (11 male, 9 female) sixth grade students were interviewed about their solution strategies and answer justifications when solving difficult mathematics problems. The interview participants represented a stratified demographic sampling of the student body of a culturally diverse middle school in a suburban school district in the southwestern United States. Data from the interviews were analyzed qualitatively. This involved “chunking” cognitive interview transcripts into sections. Major themes were identified and manuscripts were developed around those themes. One theme examined the interviewers’ ethic of care behaviors. Carol Gilligan noted differences in male and female ethic of care behaviors, but it was Nel Noddings who discussed the importance of such behaviors in the educational community. So what impact could the gender of the interviewer have on cognitive interviews? After considering ethic of care behaviors explicated by Hayes, Ryan and Zseller’s (1994) study with middle grades students, the interview transcripts were examined for specific positive and negative ethic of care behaviors. The theme of students’ justifications of mathematical solutions was also selected. The major undertaking involved developing a justification scheme applicable across mathematical strands and grade levels. The justification scheme that emerged was based on the work of Guershon Harel and Larry Sowder. The first-level schemes of Language, Mechanistic, Authoritarian, and Visual were used to classify and define the justifications. Several second-level schemes were also defined. The justification scheme framework was applied to students’ cognitive interview responses on four difficult mathematics problems. The third theme investigated the symbiosis of justification schemes with mathematical representations. This study examined possible links between representational formats and justification scheme categories. The premise of this study was that representations “trigger” students’ choices of justification schemes. Student responses were analyzed as to which aspect of the mathematical representation received the students’ initial attention. The students’ understanding of the representation was pivotal to their solution, as well as the students’ reasoning, or justification, of the answer. Students focused on key aspects of the problem and developed solutions based on that information.

Thesis advisor: Zhushan Li / Higher education institutions are expected to improve the employability-related outcomes of their graduates. Previous assessments of graduate employability have largely focused on assessing the quality of a graduate’s first career-related position after graduating, referred to as their first destination, under the assumption that graduates who secure higher quality first destinations are more highly employable. Previous assessments of first destinations have focused more on simple descriptive information (e.g., monetary compensation, number of hours worked per week, etc.) as opposed to evaluating more complex, multidimensional constructs related to employment quality. Assessing these types of constructs would not only provide institutions with more information on graduate employability outcomes, but could also improve the quality and utility of this information. This study investigated whether the theory of Person-Job (P-J) Fit could be generalized to assess a new self-report employment quality construct for recent Bachelor’s degree graduates called First Destination Fit (FDF). Using a mixed-methods research design, the study investigated the dimensionality of this construct, the extent to which the construct is invariant for graduates who secure employment first destinations versus all other types of first destinations (e.g., continued education, military service, etc.), and the extent to which the construct is associated with similar constructs as P-J Fit. Survey scale items were rigorously developed, evaluated, and refined using both quantitative and qualitative methodologies, including both exploratory and confirmatory factor analysis, as well as cognitive interviews. The results suggested a four-dimensional framework for understanding FDF. These dimensions were Needs-Tasks fit, Needs-Field fit, Previous-Experience-Tasks fit, and Previous-Experience-Field fit. In large part, these dimensions were found to be related to similar constructs as P-J fit has been found to be related to, particularly for graduates with employment-type destinations. However, this four-dimensional framework was found to be better fitting for graduates with employment-type destinations compared to graduates with non-employment destinations. The study contributes a newly developed and rigorously evaluated scale for HEIs to use to collect new, important information about their graduates’ employability. It also began the process of validating this new scale using advanced psychometric testing. Implications for the scale and future directions for research are discussed. / Thesis (PhD) — Boston College, 2021. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.

No description available.

This study aimed to measure teachers’ self-efficacy for teaching students with autism spectrum disorder (ASD). Teacher self-efficacy refers to the belief teachers hold about their ability to affect student learning and has been shown to change teachers’ motivation, stress levels, and amount of given effort (Klassen, Tze, Betts, & Gordon, 2011). Numerous studies have dealt with the measurement of this construct and developed measures that assess teachers in different domains and populations; however, only one study (Ruble, Toland, Birdwhistell, McGrew, & Usher, 2013) has attempted to measure within the population of students with ASD. The purpose of the current study was to develop and pilot a new measure, the Teacher Self-Efficacy for Autism Scale (TSEAS), expand the construct to the target population, and provide sources of validity evidence. Results indicated that the TSEAS represented a unidimensional scale and latent correlation analyses suggested that the TSEAS has a positive correlation with a well-known teacher self-efficacy measure and has a low, positive correlation with a job satisfaction instrument. With continued refinement, the TSEAS can support others who wish to measure this construct and future application of the methods employed in building this scale can be applied to other scales with minor revisions.

<b>Introduction:</b> Health-related websites have developed to be much more than information sites: they are used to exchange experiences and find support as well as information and advice. It is important that health professionals and website developers understand how content may impact users. This thesis documents the development and application of a tool to measure the impact of using health-related websites which contain experiential and/or factual information. <b>Methods:</b> A multi-method study with five stages. Stage 1: Questionnaire items based upon themes relating to the impact of using health-related websites were constructed following qualitative secondary analysis of 93 interviews relating to patient and carer experiences of health and a recent literature review. Items were assessed by an expert panel. Stage 2: Cognitive interviews were carried out to confirm acceptability of items. Stage 3: Item reduction steps were used to reduce the number of items. Stage 4: The validity and reliability of the remaining items were tested using traditional and modern psychometric methods. Stage 5: The new questionnaire was piloted in a randomised controlled trial. <b>Results:</b> Eighty-two items were constructed according to the key themes identified in Stage 1. Following expert and patient refinement, two independent item pools entered psychometric testing. The first item pool related to general views of using the internet in relation to health and the second item pool related to the impact of using a specific health-related website. Sub-scales and summary scores were found to have high construct validity, internal consistency and test-retest reliability. The questionnaire showed high completion rates and low counts of missing data in a trial setting. <b>Conclusion:</b> Analysis confirmed good psychometric properties in the eHIQ-Part 1 (11 items) and the eHIQ-Part 2 (26 items). Preliminary findings of trial data demonstrate the acceptability and feasibility of including the eHIQ in randomised controlled trials. This tool will enable the measurement of the impact of health-related websites containing various styles of information and support across a range conditions and facilitate their accurate evaluation in clinical trials.

Background: New genomic tests are being developed to predict an individual’s risk of cancer recurrence by analyzing the expression of multiple genes. However, it is unclear how to report the test results so that they would be most useful to clinicians. A mail-out questionnaire has the potential to help a) describe physicians’ attitudes towards the clinical use of new genomic tests, b) determine what information physicians prefer to have included in the test reports, and c) explore how physicians think the test results would impact their treatment recommendations. Objectives: To design such a questionnaire that could be used in the eventual large-scale survey, and to ensure that the questionnaire a) is comprehensible, b) has face validity, c) appears interesting to, and d) does not place undue response burden on, the target population. Methods: The first draft, based on a specific genomic test for breast cancer recurrence (Oncotype DX) and on two case scenarios, was created. Cognitive interviews with practicing oncologists were conducted to identify problems in the questionnaire. The evaluation involved face-to-face interviews with Kingston oncologists who treat breast cancer, followed by telephone interviews with medical oncologists who treat breast cancer in other places in Ontario. Three-to-four oncologists were included in each round of interviewing after which the questionnaire was revised based on that round’s recommendations. Additional rounds of interviews were conducted until no new problems/issues were raised in one entire round. Results: A medium-length questionnaire was drafted. Four rounds of interviews were conducted with no new problems/issues being raised in the fourth round. Most of the problems identified in the questionnaire related to comprehensibility, followed by logical issues which detected fundamental problems in the questionnaire design. There was no evidence of fatigue or disinterest in participants and they deemed the response burden reasonable. Conclusion: The results suggest that the proposed questionnaire is comprehensible and has face validity. Additionally, it appears to be an interesting questionnaire to, and would not place undue burden on, the target population. Thus, the questionnaire is now ready for the field administration. / Thesis (Master, Community Health & Epidemiology) -- Queen's University, 2009-05-05 17:23:10.551

The high prevalence of substance use disorders is well-documented among criminal offenders. Drug courts are specialty judicial programs designed to: 1) improve public safety outcomes; 2) reduce criminal recidivism and substance abuse among offenders with substance use disorders; and 3) better utilize scarce criminal justice and treatment resources. Drug courts operate through partnerships between the criminal justice, behavioral health and public health systems. Offenders participate in an intensive regimen of substance abuse treatment and case management while under close judicial supervision. Drug courts' effectiveness in reducing criminal recidivism and drug use has been documented through numerous primary studies as well as meta-analytic reviews. The task remains now to determine the causal mechanisms of drug courts. The current study conducted preliminary activities to develop a measure to assess drug court structures and practices based upon the Ten Key Components of drug courts (NADCP, 1997). The creation and use of such a measure is necessary to the understanding of how drug courts work, why and how best to invest scarce judicial and treatment resources to optimize drug court participant and program outcomes. An iterative process was conducted such that results from previous activities informed subsequent steps in the measurement development process. Participants consisted of a convenience sample of drug court personnel at three local drug courts as well as academic experts in drug courts and measurement. Preliminary measurement development activities included: 1) a comprehensive review of the literature; 2) semi-structured interviews with key stakeholders to inform item development; 3) construction of a draft survey protocol; 4) expert reviews of the draft survey protocol and initial item pool to assess item construct and content validity, response format and clarity; 5) pile sort activity, wherein participants sorted items into piles, one for each measure sub-construct and one `other' pile; 6) exploratory factor analyses based on a joint-proportion matrix derived from pile sort activity data on which items best represent measure sub-constructs; 7) cognitive interviews completed by key stakeholders to review items retained from exploratory factor analyses; and 8) final revisions to the item pool based upon results from cognitive interviews. The item pool developed through the current research will be used as the basis for a future large-scale pilot test to determine the true factor structure underlying the preliminary measure developed. Results of this future research are expected to identify similarities and differences in the underlying factor structure compared to the Ten Key Components.

This diploma thesis deals with the rating scales, which are widely used in the attitude measurement, namely with the impact of their length (number of scale points) on measurement, both in terms of cognitive processes in the minds of respondents and impact on the data. Hypotheses, formulated on the basis of the literature and tested using data obtained from a split ballot experiment and cognitive interviews, are related to the comparability of results, distribution of responses and task demands. The Net Promoter Score from the environment of market research, expressing respondent's willingness to recommend a service or company used, is used as an application example. It originally uses 0 to 10 eleven-point scale which is here compared to five-point scale from 1 to 5 using various recoding and rescaling techniques. Analysis shows that rating on eleven-point and five-point scales means two different cognitive tasks for respondents (eleven-point scale is a bit more demanding but more accurate in respondents perception) and that the data differ from each other in terms of distribution of responses - on short range scale, midpoint and extreme positive category are selected more often, while on eleven-point scale, milder positive points are used more often.