Trustworthy clinical practice guidelines assist health care professionals in selecting the management options that optimize patient health outcomes. The development of trustworthy guidelines requires the consideration of many aspects and the involvement of multiple contributors, often working in groups. The guideline panel plays the key role in the development process as it is responsible for prioritizing topics that should be covered as part of the guideline effort, formulating questions, reviewing the evidence, developing and agreeing on the recommendations, and endorsing the final guideline document. Ensuring transparency throughout the process by appropriately organizing and documenting panel activities is an essential standard that is used to assess the credibility of a developed guideline and its resulting recommendations. The adoption of conceptual frameworks that systematically guides panel members in their decision-making process (e.g. the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks) can aid in the formulation of methodologically sound recommendations. In this dissertation, I used the example of a guideline on diagnosis and treatment of autism spectrum disorders to describe how rigorous research methods can support guideline panels in the development process from early stages to the formulation of recommendations. In another prominent guideline development effort with the American Society of Hematology, I have identified two steps in the process where panel members may benefit from further support and addressed these gaps by conceptualizing and developing novel approaches. The first approach comprises modelling baseline risk estimates for patient-important outcomes when only surrogate data is available. The second approach proposes a method to estimate decision thresholds for judgments on health benefits and harms using the GRADE EtD framework. While these approaches are tailored to address specific guideline panel needs, guideline methodologists could use the underlying concepts to find solutions to aid guideline panels in other steps of the development process. / Thesis / Doctor of Philosophy (PhD) / Clinical practice guidelines assist health care professionals in selecting management options that can best improve the health outcomes of their patients. The development of trustworthy guidelines is a complex process that requires the contribution of several entities. The guideline panel, which typically comprises different experts (clinicians, patient representatives, experts in research methodologies) plays the key role in this process as it is responsible for selecting the most important questions to address in the guideline, reviewing the evidence supporting an option, agreeing on the recommendations, and endorsing the final guideline document. To ensure that the process of developing guidelines is transparent and that the recommendations are credible, it is important that panel activities are well documented and follow rigorous methods. Structured frameworks, such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) approach, have been developed to systematically guide the panel members and to minimize the error that could be introduced while making decisions. In this thesis, I describe the development of an approach and its application for comprehensive guideline development by the Italian National Health Institute, to describe rigorous guideline development and propose two novel approaches to further assist panel members in enhancing their guideline development. The first of these two enhancements to guideline development describes how to derive a modelled estimate of the risk of having certain health conditions when this data is not directly available in the medical literature. The second of the two enhancements is a method to support guideline panels in judging how substantial the desirable and undesirable effects of health interventions are. Both approaches were tailored to fit specific needs but can be adapted to inform the improvement of other steps in the guideline development process.

Doctor of Philosophy / Similar to their colleagues throughout the world, nurses and midwives in New South Wales (NSW), Australia, welcome evidencebased practice (EBP) as a means to improve patient or client outcomes. This thesis explores the way nurses and midwives understand evidence for EBP and aims to determine whether members of these professions currently have the knowledge and skills necessary to implement evidence‐based care. Three separate studies were conducted to explore NSW nurses’ readiness for EBP. Attitudes, knowledge and skill were investigated using an EBP questionnaire returned by 383 nurses. The views of 23 nursing opinion leaders were elicited during qualitative in‐depth interviews, and their ideas on maximising the potential for future nurses to confidently engage in EBP were explored. Current approaches to teaching EBP in undergraduate nursing programs were investigated by examining documents issued by NSW nursing education providers. The results demonstrate many differences between the ways NSW nurses currently understand evidence for EBP, and a range of approaches to teaching EBP in undergraduate nursing programs. Under current conditions, nurses graduating from universities in NSW commence practice with varying levels of preparation for EBP and enter into a professional arena that is itself struggling to cope with the concepts and language of this approach to improving healthcare. v Evidence for the effectiveness of EBP is slowly accumulating and despite some small positive signs, the collective results of this thesis suggest that current educational approaches are not capable of producing the kind of results that are both necessary and desirable for the promotion of evidence‐based nursing practice in NSW. Articulating a commitment to EBP, using a common language and a consistent approach are among the recommendations made for the future promotion of EBP in nursing education.

In the early 1990s claims were made that in all areas of health care, “30-40% of patients do not receive treatments of proven effectiveness”, and, “20-25% of patients have treatments that are unnecessary or potentially harmful”. Many such practices were diffused prior to the acceptance of modern evidence-based standards of clinical- and cost-effectiveness. I define disinvestment in the context of health care as the processes of withdrawing (partially or completely) resources from any existing health care practices, procedures, technologies or pharmaceuticals that are deemed to deliver little or no health gain relative to their cost, and thus are not efficient health resource allocations. Arguably disinvestment has been central to Evidence-Based Medicine(EBM) for well over a decade yet despite general advances in EBM, this topic remains relatively unexplored. This thesis examines the ongoing challenges that exist within the Australian context relating to effective disinvestment. Upper airway surgical procedures for the treatment of adult Obstructive Sleep Apnoea Syndrome (OSA) are used as a case study to contextualise these challenges. This thesis has six sections: 1. A review of the literature outlines developments in EBM broadly and provides a detailed background to OSA, including the numerous treatment options for the condition. This review examines evidence that highlights the importance of ‘highly effective treatment’ over ‘subtherapeutic treatment’ as a necessity to confer improved health outcomes in OSA. It is argued that claims of surgical success inherent in most published results of surgery effectiveness fail to assimilate contemporary evidence for clinically significant indicators of success. 2. Section two comprises the first reported meta-analysis in this area. It presents the pooled success rates of surgery according to various definitions. Specifically, when the traditional ‘surgical’ definition of success is applied the pooled success rate for Phase I (i.e. soft palate) surgical procedures is 55% (that is 45% fail). However, using a more stringent definition (endorsed by the peak international sleep medicine body), success is reduced to 13% (that is 87% fail). Similarly for Phase II (i.e. hard palate) procedures success rates decrease from 86% to 43% respectively when moving from a surgical to a medical definition of success. That various medical specialties differentially define treatment success, I argue, creates uncertainty for observers and non-clinical participants in this debate (eg policy stakeholders and patients). This represents a barrier to disinvestment decisions. 3. Results are presented from a clinical audit of surgical cases conducted as a component of this thesis. Both clinical effectiveness and procedural variability of surgery are reported. A unique methodology was utilised to capture data from multiple centres. It is the first time such a methodology has been reported to measure procedural variability alongside clinical effectiveness (inclusive of a comparative treatment arm). The observed cohort (n=94) received 41 varying combinations of surgery in an attempt to treat OSA. Results on effectiveness demonstrate an overall physiological success rate of 13% (according to the most stringent definition; phases I and II combined). This demonstration of procedural variability combined with limited effectiveness highlights clinical uncertainty in the application of surgical procedures. 4. Section four outlines how a qualitative phase of enquiry, directed at exploring the perspectives and experiences of surgery recipients, was approved by three independent research ethics review boards but was not supported by a small group of surgeons, resulting in the project being canceled. Potential consequences of this for impeding health services research (HSR) are discussed. 5. Two sets of results are reported from a qualitative phase of enquiry (semi-structured interviews) involving senior Australian health policy stakeholders. The first results are of policy stakeholders’ perspectives on the surgical meta-analysis and clinical audit studies in 2 and 3 above. The second results are from an extended series of questions relating to challenges and direction for effecting disinvestment mechanisms in Australia. Stakeholder responses highlight that Australia currently has limited formal systems in place to support disinvestment. Themes include how defining and proving inferiority of health care practices is not only conceptually difficult but also is limited by data availability and interpretation. Also, as with any policy endeavour there is the ever-present need to balance multiple interests. Stakeholders pointed to a need, and a role, for health services and policy research to build methodological capacity and decision support tools to underpin disinvestment. 6. A final discussion piece is presented that builds on all previous sections and summarises the specific challenges that exist for disinvestment, including those methodological in nature. The thesis concludes with potential solutions to address these challenges within the Australian and international context. Systematic policy approaches to disinvestment represent one measure to further improve equity, efficiency, quality of care, as well as sustainability of resource allocation. / http://proxy.library.adelaide.edu.au/login?url= http://library.adelaide.edu.au/cgi-bin/Pwebrecon.cgi?BBID=1297655 / Thesis (Ph.D.) -- University of Adelaide, School of Population Health and Clinical Practice, 2007

In the early 1990s claims were made that in all areas of health care, “30-40% of patients do not receive treatments of proven effectiveness”, and, “20-25% of patients have treatments that are unnecessary or potentially harmful”. Many such practices were diffused prior to the acceptance of modern evidence-based standards of clinical- and cost-effectiveness. I define disinvestment in the context of health care as the processes of withdrawing (partially or completely) resources from any existing health care practices, procedures, technologies or pharmaceuticals that are deemed to deliver little or no health gain relative to their cost, and thus are not efficient health resource allocations. Arguably disinvestment has been central to Evidence-Based Medicine(EBM) for well over a decade yet despite general advances in EBM, this topic remains relatively unexplored. This thesis examines the ongoing challenges that exist within the Australian context relating to effective disinvestment. Upper airway surgical procedures for the treatment of adult Obstructive Sleep Apnoea Syndrome (OSA) are used as a case study to contextualise these challenges. This thesis has six sections: 1. A review of the literature outlines developments in EBM broadly and provides a detailed background to OSA, including the numerous treatment options for the condition. This review examines evidence that highlights the importance of ‘highly effective treatment’ over ‘subtherapeutic treatment’ as a necessity to confer improved health outcomes in OSA. It is argued that claims of surgical success inherent in most published results of surgery effectiveness fail to assimilate contemporary evidence for clinically significant indicators of success. 2. Section two comprises the first reported meta-analysis in this area. It presents the pooled success rates of surgery according to various definitions. Specifically, when the traditional ‘surgical’ definition of success is applied the pooled success rate for Phase I (i.e. soft palate) surgical procedures is 55% (that is 45% fail). However, using a more stringent definition (endorsed by the peak international sleep medicine body), success is reduced to 13% (that is 87% fail). Similarly for Phase II (i.e. hard palate) procedures success rates decrease from 86% to 43% respectively when moving from a surgical to a medical definition of success. That various medical specialties differentially define treatment success, I argue, creates uncertainty for observers and non-clinical participants in this debate (eg policy stakeholders and patients). This represents a barrier to disinvestment decisions. 3. Results are presented from a clinical audit of surgical cases conducted as a component of this thesis. Both clinical effectiveness and procedural variability of surgery are reported. A unique methodology was utilised to capture data from multiple centres. It is the first time such a methodology has been reported to measure procedural variability alongside clinical effectiveness (inclusive of a comparative treatment arm). The observed cohort (n=94) received 41 varying combinations of surgery in an attempt to treat OSA. Results on effectiveness demonstrate an overall physiological success rate of 13% (according to the most stringent definition; phases I and II combined). This demonstration of procedural variability combined with limited effectiveness highlights clinical uncertainty in the application of surgical procedures. 4. Section four outlines how a qualitative phase of enquiry, directed at exploring the perspectives and experiences of surgery recipients, was approved by three independent research ethics review boards but was not supported by a small group of surgeons, resulting in the project being canceled. Potential consequences of this for impeding health services research (HSR) are discussed. 5. Two sets of results are reported from a qualitative phase of enquiry (semi-structured interviews) involving senior Australian health policy stakeholders. The first results are of policy stakeholders’ perspectives on the surgical meta-analysis and clinical audit studies in 2 and 3 above. The second results are from an extended series of questions relating to challenges and direction for effecting disinvestment mechanisms in Australia. Stakeholder responses highlight that Australia currently has limited formal systems in place to support disinvestment. Themes include how defining and proving inferiority of health care practices is not only conceptually difficult but also is limited by data availability and interpretation. Also, as with any policy endeavour there is the ever-present need to balance multiple interests. Stakeholders pointed to a need, and a role, for health services and policy research to build methodological capacity and decision support tools to underpin disinvestment. 6. A final discussion piece is presented that builds on all previous sections and summarises the specific challenges that exist for disinvestment, including those methodological in nature. The thesis concludes with potential solutions to address these challenges within the Australian and international context. Systematic policy approaches to disinvestment represent one measure to further improve equity, efficiency, quality of care, as well as sustainability of resource allocation. / http://proxy.library.adelaide.edu.au/login?url= http://library.adelaide.edu.au/cgi-bin/Pwebrecon.cgi?BBID=1297655 / Thesis (Ph.D.) -- University of Adelaide, School of Population Health and Clinical Practice, 2007

In the early 1990s claims were made that in all areas of health care, “30-40% of patients do not receive treatments of proven effectiveness”, and, “20-25% of patients have treatments that are unnecessary or potentially harmful”. Many such practices were diffused prior to the acceptance of modern evidence-based standards of clinical- and cost-effectiveness. I define disinvestment in the context of health care as the processes of withdrawing (partially or completely) resources from any existing health care practices, procedures, technologies or pharmaceuticals that are deemed to deliver little or no health gain relative to their cost, and thus are not efficient health resource allocations. Arguably disinvestment has been central to Evidence-Based Medicine(EBM) for well over a decade yet despite general advances in EBM, this topic remains relatively unexplored. This thesis examines the ongoing challenges that exist within the Australian context relating to effective disinvestment. Upper airway surgical procedures for the treatment of adult Obstructive Sleep Apnoea Syndrome (OSA) are used as a case study to contextualise these challenges. This thesis has six sections: 1. A review of the literature outlines developments in EBM broadly and provides a detailed background to OSA, including the numerous treatment options for the condition. This review examines evidence that highlights the importance of ‘highly effective treatment’ over ‘subtherapeutic treatment’ as a necessity to confer improved health outcomes in OSA. It is argued that claims of surgical success inherent in most published results of surgery effectiveness fail to assimilate contemporary evidence for clinically significant indicators of success. 2. Section two comprises the first reported meta-analysis in this area. It presents the pooled success rates of surgery according to various definitions. Specifically, when the traditional ‘surgical’ definition of success is applied the pooled success rate for Phase I (i.e. soft palate) surgical procedures is 55% (that is 45% fail). However, using a more stringent definition (endorsed by the peak international sleep medicine body), success is reduced to 13% (that is 87% fail). Similarly for Phase II (i.e. hard palate) procedures success rates decrease from 86% to 43% respectively when moving from a surgical to a medical definition of success. That various medical specialties differentially define treatment success, I argue, creates uncertainty for observers and non-clinical participants in this debate (eg policy stakeholders and patients). This represents a barrier to disinvestment decisions. 3. Results are presented from a clinical audit of surgical cases conducted as a component of this thesis. Both clinical effectiveness and procedural variability of surgery are reported. A unique methodology was utilised to capture data from multiple centres. It is the first time such a methodology has been reported to measure procedural variability alongside clinical effectiveness (inclusive of a comparative treatment arm). The observed cohort (n=94) received 41 varying combinations of surgery in an attempt to treat OSA. Results on effectiveness demonstrate an overall physiological success rate of 13% (according to the most stringent definition; phases I and II combined). This demonstration of procedural variability combined with limited effectiveness highlights clinical uncertainty in the application of surgical procedures. 4. Section four outlines how a qualitative phase of enquiry, directed at exploring the perspectives and experiences of surgery recipients, was approved by three independent research ethics review boards but was not supported by a small group of surgeons, resulting in the project being canceled. Potential consequences of this for impeding health services research (HSR) are discussed. 5. Two sets of results are reported from a qualitative phase of enquiry (semi-structured interviews) involving senior Australian health policy stakeholders. The first results are of policy stakeholders’ perspectives on the surgical meta-analysis and clinical audit studies in 2 and 3 above. The second results are from an extended series of questions relating to challenges and direction for effecting disinvestment mechanisms in Australia. Stakeholder responses highlight that Australia currently has limited formal systems in place to support disinvestment. Themes include how defining and proving inferiority of health care practices is not only conceptually difficult but also is limited by data availability and interpretation. Also, as with any policy endeavour there is the ever-present need to balance multiple interests. Stakeholders pointed to a need, and a role, for health services and policy research to build methodological capacity and decision support tools to underpin disinvestment. 6. A final discussion piece is presented that builds on all previous sections and summarises the specific challenges that exist for disinvestment, including those methodological in nature. The thesis concludes with potential solutions to address these challenges within the Australian and international context. Systematic policy approaches to disinvestment represent one measure to further improve equity, efficiency, quality of care, as well as sustainability of resource allocation. / http://proxy.library.adelaide.edu.au/login?url= http://library.adelaide.edu.au/cgi-bin/Pwebrecon.cgi?BBID=1297655 / Thesis (Ph.D.) -- University of Adelaide, School of Population Health and Clinical Practice, 2007

Doctor of Philosophy / Similar to their colleagues throughout the world, nurses and midwives in New South Wales (NSW), Australia, welcome evidencebased practice (EBP) as a means to improve patient or client outcomes. This thesis explores the way nurses and midwives understand evidence for EBP and aims to determine whether members of these professions currently have the knowledge and skills necessary to implement evidence‐based care. Three separate studies were conducted to explore NSW nurses’ readiness for EBP. Attitudes, knowledge and skill were investigated using an EBP questionnaire returned by 383 nurses. The views of 23 nursing opinion leaders were elicited during qualitative in‐depth interviews, and their ideas on maximising the potential for future nurses to confidently engage in EBP were explored. Current approaches to teaching EBP in undergraduate nursing programs were investigated by examining documents issued by NSW nursing education providers. The results demonstrate many differences between the ways NSW nurses currently understand evidence for EBP, and a range of approaches to teaching EBP in undergraduate nursing programs. Under current conditions, nurses graduating from universities in NSW commence practice with varying levels of preparation for EBP and enter into a professional arena that is itself struggling to cope with the concepts and language of this approach to improving healthcare. v Evidence for the effectiveness of EBP is slowly accumulating and despite some small positive signs, the collective results of this thesis suggest that current educational approaches are not capable of producing the kind of results that are both necessary and desirable for the promotion of evidence‐based nursing practice in NSW. Articulating a commitment to EBP, using a common language and a consistent approach are among the recommendations made for the future promotion of EBP in nursing education.

Thesis (M.P.H.)--University of Texas Health Science Center at Houston, School of Public Health, 2008. / Source: Masters Abstracts International, Volume: 46-05, page: 2673. Adviser: Linda Lloyd. Includes bibliographical references.

This thesis explores the transformations of evidence in health research synthesis studies – studies that bring together evidence from a number of research reports on the same/ similar topic. It argues that health research synthesis is a broad and intriguing field in a state of pre-formation, in spite of the fact that it may appear well established if equated with its exemplar method – the systematic review inclusive of meta-analysis. Transformations of evidence are processes by which pieces of evidence are modified from what they are in the primary study report into what is needed in the synthesis study while, supposedly, having their integrity fully preserved. Such processes have received no focused attention in the literature. Yet they are key to the validity and reliability of synthesis studies. This work begins to describe them and explore their frequency, scope and drivers. A ‘meta-scientific’ perspective is taken, where ‘meta-scientific’ is understood to include primarily ideas from the philosophy of science and methodological texts in health research, and, to a lesser extent, social studies of science and psychology of science thinking. A range of meta-scientific ideas on evidence and factors that shape it guide the analysis of processes of “data extraction” and “coding” during which much evidence is transformed. The core of the analysis involves the application of an extensive Analysis Framework to 17 highly heterogeneous research papers on cancer. Five non-standard ‘injunctions’ complement the Analysis Framework – for comprehensiveness, extensive multiple coding, extreme transparency, combination of critical appraisal and critique, and for first coding as close as possible to the original and then extending towards larger transformations. Findings suggest even lower credibility of the current overall model of health research synthesis than initially expected. Implications are discussed and a radical vision for the future proposed.

Background Selective reporting of outcomes within a published study based on their nature or direction can result in systematic differences between reported and unreported data. Direct evidence of outcome reporting bias is limited to case reports. Objective To study empirically the nature of outcome reporting bias in randomised controlled trials (RCTs). Methods Three cohorts of RCTs were identified: PubMed-indexed RCTs published in December 2000; trial protocols approved by a Danish ethics committee from 1994-95; and trial protocols funded by a government agency in Canada from 1990-98. Data on reported and unreported outcomes were recorded from all trial publications and a survey of authors. An outcome was considered incompletely reported if insufficient data were presented for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial, and then pooled using a random effects meta-analysis. Protocols and publications were also reviewed for discrepancies in primary outcome reporting. Results 519 trials with 10,557 outcomes, 102 trials with 3613 outcomes, and 48 trials with 1390 outcomes were identified for the PubMed, ethics committee, and funding agency cohorts respectively. 22%-35% of outcomes per parallel group study were, on average, incompletely reported for meta-analysis. Fully reported outcomes had a two- to three-fold higher odds of being statistically significant compared to incompletely reported outcomes. The most common reasons given for omitting outcomes included a lack of clinical importance, lack of statistical significance, and space constraints. Major discrepancies between primary outcomes in protocols and publications were found in one half of trials. Discussion and conclusions The reporting of trial outcomes is frequently inadequate for meta-analysis; is biased to favour statistical significance; and is inconsistent with pre-specified protocol outcomes. Unacknowledged modifications to outcomes specified in trial protocols constitute scientific misconduct. Meta-analyses may therefore produce inflated and unreliable estimates of treatment effect.

OBJECTIVE: Hospital systems and regulating agencies enforce strict guidelines barring personal items from entering the operating room (OR) - touting surgical site infections (SSIs) and patient safety as the rationale. We sought to determine whether or not evidence supporting this recommendation exists by reviewing available literature. BACKGROUND DATA: Rules and guidelines that are not evidence based may lead to increased hospital expenses and limitations on healthcare provider autonomy. METHODS: PubMed, Embase, Scopus, Cochrane Library, Web of Science, and CINAHL were searched in order to find articles that correlated personal items in the OR to documented SSIs. Articles that satisfied the following criteria were included: (1) studies looking at personal items in the OR, such as handbags, purses, badges, pagers, backpacks, jewelry phones, and eyeglasses, but not just OR equipment; and (2) the primary outcome measure was infection at the surgical site. RESULTS: Seventeen articles met inclusion criteria and were evaluated. Of the 17, the majority did not determine if personal items increased risk for SSIs. Only one article examined the correlation between a personal item near the operative site and SSI, concluding that wedding rings worn in the OR had no impact on SSIs. Most studies examined colonization rates on personal items as potential infection risk; however, no personal items were causally linked to SSI in any of these studies. CONCLUSION: There is no objective evidence to suggest that personal items in the OR increase risk for SSIs.