published_or_final_version / Medical Sciences / Master / Master of Medical Sciences

Background Selective reporting of outcomes within a published study based on their nature or direction can result in systematic differences between reported and unreported data. Direct evidence of outcome reporting bias is limited to case reports. Objective To study empirically the nature of outcome reporting bias in randomised controlled trials (RCTs). Methods Three cohorts of RCTs were identified: PubMed-indexed RCTs published in December 2000; trial protocols approved by a Danish ethics committee from 1994-95; and trial protocols funded by a government agency in Canada from 1990-98. Data on reported and unreported outcomes were recorded from all trial publications and a survey of authors. An outcome was considered incompletely reported if insufficient data were presented for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial, and then pooled using a random effects meta-analysis. Protocols and publications were also reviewed for discrepancies in primary outcome reporting. Results 519 trials with 10,557 outcomes, 102 trials with 3613 outcomes, and 48 trials with 1390 outcomes were identified for the PubMed, ethics committee, and funding agency cohorts respectively. 22%-35% of outcomes per parallel group study were, on average, incompletely reported for meta-analysis. Fully reported outcomes had a two- to three-fold higher odds of being statistically significant compared to incompletely reported outcomes. The most common reasons given for omitting outcomes included a lack of clinical importance, lack of statistical significance, and space constraints. Major discrepancies between primary outcomes in protocols and publications were found in one half of trials. Discussion and conclusions The reporting of trial outcomes is frequently inadequate for meta-analysis; is biased to favour statistical significance; and is inconsistent with pre-specified protocol outcomes. Unacknowledged modifications to outcomes specified in trial protocols constitute scientific misconduct. Meta-analyses may therefore produce inflated and unreliable estimates of treatment effect.

Objective. The Cochrane risk of bias tool (CRoB) is one of the most widely used tools for assessing the risk of bias of clinical trials. However, it was criticized for its poor inter-rater reliability, lack of clear and detailed guidelines for its application, and no clear distinguishing between reporting quality from real quality in implementation. This study aims to develop a framework (or improved CRoB, iCRoB) so as to improve the inter-rater reliability of the CRoB in its first 4 domains: sequence generation, allocation concealment, blinding of participants and personnel, and blinding of outcome assessment, through providing: i) a structured pathway for assessing risk of bias assessment; and ii) a comprehensive dictionary of scenarios for each domain. / Methods. The study is consisted of 4 steps: / i) Step 1: Develop a step-by-step structured pathway for assessing the risk of bias. / ii) Step 2: Identify and summarize possible scenarios that are used in literature to describe a domain in clinical trials by using a qualitative content analysis approach. A random sample of 100 Cochrane systematic reviews (SRs) was taken from the Cochrane Database of Systematic Reviews. Each review was carefully scrutinized for this purpose. / iii) Step 3: Merge the scenarios identified from the sample with those already provided in the CRoB. The combined list of scenarios extends the current coverage of the CRoB and forms a more comprehensive dictionary of scenarios for use in the future. The bias assessment pathway and the new dictionary of scenarios in combination are the new components added or contribution to the CRoB to form the iCRoB. / v) Step 4: Conduct a randomized controlled study that allocated at random 8 raters equally into either using the CRoB or our new iCRoB. 150 clinical trials were randomly selected from the fore-mentioned 100 SRs for the inter-rater reliability comparison. Both inter-rater reliability among individual raters (measured with Fleiss’ κ) and that across rater pairs (measured with weighted Cohen’s κ) were computed. Data analyses were conducted by using STATA version 13.0. / Results. A structured pathway for systematically assessing bias was designed, which helps classify a study into one of 5 categories for each risk of bias domain based on the information provided in the report of a trial: Category A: a trial reports in details how a bias reduction method was conducted and it is also deemed by the assessor to be conducted adequately; Category B: a trial reports in details how a bias reduction method was conducted but it is deemed by the assessor to be conducted inadequately; Category C: a trial reports that a bias reduction method was conducted but no detailed description was given which can be used to judge whether it was done adequately; Category D: a trial reports that a bias reduction method was not conducted; Category E: a trial does not mention at all whether or not a bias reduction method was conducted. / A total of 34, 36, 26 and 20 scenarios were generated for sequence generation, allocation concealment, blinding of participants and personnel, and blinding of outcome assessment, respectively. We extended the current CRoB list of scenarios by a number of 20, 23, 26 and 20 respectively for the 4 bias reduction domains. / Our trial results showed that the iCRoB had a higher inter-rater reliability across rater pairs than the original CRoB for every bias reduction domain. The weighted κ was 0.71 and 0.81 for sequence generation respectively for CRoB and iCRoB; 0.53 and 0.61 for allocation concealment respectively for CRoB and iCRoB; 0.56 for blinding of participants and personnel in CRoB, 0.68 for blinding of participants and 0.70 for blinding of personnel ini CRoB; and 0.19 and 0.43 for blinding of outcome assessment respectively for CRoB and iCRoB. / Conclusion. We developed the iCRoB including a standard pathway and extended substantively the dictionary of scenarios for making the judgement on risk of bias in the reports of clinical trials. Our iCRoB showed a higher reliability than the current CRoB in all the domains examined. The iCRoB can be recommended for improving the assessment of bias in clinical trials. / 目的：考柯藍偏倚風險評估工具(CRoB)是最廣泛應用的用於評價臨床試驗偏倚風險的工具之一。然而，CRoB 有以下三個缺陷：評價者間信度低，缺乏明確和詳細的應用說明和沒有明確區分報告質量和方法學質量。本研究擬制定一個新的工具iCRoB 用以提高CRoB 前4 項指標的評價者間信度。這4項指標分別為隨機序列生成，分配隱藏，對研究對象和研究者實施盲法，和對結局評估者實施盲法。本研究通過以下2 點實現這一目的：i) 提供一個結構化路徑用以評估偏倚風險；ii) 為每個研究指標提供一個廣泛包含偏倚風險評估相關描述場景的字典。 / 方法：本研究包含以下4 個步驟： / 第1 步：制定一個用以評估偏倚風險的結構化路徑。 / 第2 步：從考柯藍系統綜述數據庫中隨機抽取100 篇系統綜述，應用定性內容分析法從中確定並總結出臨床試驗中與偏倚風險相關的可能的描述場景。 / 第3 步：將從100 個樣本中總結的描述場景與CRoB 中已有的場景合併，從而擴大CRoB 的描述場景的覆蓋範圍，得到一個更廣泛包含偏倚風險評估相關描述場景字典。偏倚風險評估的結構化路徑和包含場景描述的字典共同形成了本研究中新制定的iCRoB，用以評估臨床試驗的偏倚風險。 / 第4 步：在一個隨機對照研究中，8 名評價者被隨機平均分配至CRoB 組或者iCRoB 組。在上述100個系統綜述所納入的臨床試驗中隨機抽取150 個臨床試驗用以比較CRoB 和iCRoB 的評價者間信度。評價者間信度的比較包括個體評價者間信度（用Fleiss’κ 測量）和配對評價者間信度(用加權Cohen’s κ 測量)的比較。數據採用Stata 13.0 進行統計分析。 / 結果：本研究成功的制定了一個用於系統評價偏倚風險的結構化路徑，在該結構化路徑中，每個偏倚風險相關的指標在一個臨床研究中將分為以下5 類： / A 類：臨床試驗詳細描述了預防偏倚的措施的實施，根據描述可以判定該措施的實施能預防偏倚的產生; B 類：臨床試驗詳細描述了預防偏倚的措施的實施，根據描述可以判定該措施的實施不能預防偏倚的產生; C 類：臨床試驗報告採取了預防偏倚的措施，但未描述這一過程如何實施，從而無法判斷其實施是否正確; D 類：臨床試驗報告沒有採取任何預防偏倚的措施; E 類：臨床試驗沒有報告是否採取了預防偏倚的措施。 / 本研究分別為隨機序列生成，分配隱藏，對研究對象和研究者實施盲法，和對結局評估者實施盲法收集了34，36，26 和20 個描述場景。與CRoB 提供的描述場景比較，iCRoB 分別為隨機序列生成，分配隱藏，對研究對象和研究者實施盲法，和對結局評估者實施盲法增加了20，23，26 和20 個新的描述場景。 / 隨機對照試驗結果顯示，iCRoB 中每個研究指標的配對評價者間信度均高於CRoB，其中，隨機序列生成加權κ 為0.71（CRoB）和0.81（iCRoB），分配隱藏加權κ 為0.53（CRoB）和0.61（iCRoB），對研究對象和研究者實施盲法加權κ 為0.56（CRoB），對研究對象實施盲法加權κ 為0.68（iCRoB），對研究者實施盲法加權κ 為0.70（iCRoB），對結局評估者實施盲法加權κ 為0.19（CRoB）和0.43（iCRoB）。 / 結論：本研究通過制定一個由偏倚風險評估的結構化路徑和包含場景描述的字典組成的iCRoB，用以改善CRoB 中對臨床試驗中隨機序列生成，分配隱藏，對研究對象和研究者實施盲法，和對結局評估者實施盲法偏倚風險評估過程。相比於CRoB，iCRoB 在每個研究指標中均顯示出更好的配對評價者間信度。這些結果證明評價者間信度可以通過提供結構化偏倚風險評估路徑和更全面的描述場景字典而提高。 / Wu, Xinyin. / Thesis Ph.D. Chinese University of Hong Kong 2014. / Includes bibliographical references (leaves 93-105). / Abstracts also in Chinese. / Title from PDF title page (viewed on 09, September, 2016). / Wu, Xinyin. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only.y066 / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only.