Background: Many randomized controlled trials (RCTs) and meta-analyses include analyses of effect modification (also known as subgroup, interaction, or moderation analyses). Methodologists have widely acknowledged the challenges in deciding whether an apparent effect modification is credible or likely the result of chance or bias. Various sets of credibility criteria are available (Chapter 2 provides an example) but are inconsistent, vague in wording, lack guidance for deciding on overall credibility, and have not been systematically tested.
Objective: To systematically develop a formal instrument to assess the credibility of effect modification analyses (ICEMAN) in RCTs and meta-analyses of RCTs.
Methods: Key steps in the development process included 1) a systematic survey of the literature to identify available criteria, rationales, and previous instruments, 2) a formal consensus study among 10 leading experts, and 3) a formal user-testing study to refine the instrument based on interviews with trial investigators, systematic reviewer authors, and journal editors who applied drafts of the instrument to published claims of effect modification.
Results: The systematic survey identified 150 relevant publications, 36 candidate credibility criteria with associated rationales, and 30 existing checklists (Chapter 3). The consensus study consisted of two main video conferences and multiple rounds of written discussion. The user-testing involved 17 users (including systematic review authors, trial investigators, and journal editors) who suggested substantial improvements based on detailed interviews. The final instrument provides separate versions for RCTs (five core questions) and meta-analyses (eight core questions) with explicit response options, and an overall credibility rating ranging from very low to high credibility. A detailed manual provides rationales, supporting references, examples from the literature, and suggestions for use in combination with other quality appraisal tools and reporting (Chapter 4).
Discussion: ICEMAN is a rigorously developed instrument to evaluate claims of effect modification and addresses the main limitations of previous approaches. / Thesis / Doctor of Philosophy (PhD) / Randomized controlled trials and meta-analyses provide the best available evidence to evaluate whether effects of a therapy vary among individual patients. Efforts to decide whether treatment effects differ across patients are important and frequently done but difficult to interpret. The fundamental challenge is to decide whether apparent differences in effect are real or due to chance. To aid this decision, experts have suggested various sets of credibility criteria, all with important limitations. This thesis documents how we systematically addressed the limitations of previous approaches. Key steps were a systematic survey of the available credibility criteria, a consensus study among leading methodologists, and a formal user-testing study. The result is a new instrument for assessing the credibility of effect modification analyses (ICEMAN).
| Identifer | oai:union.ndltd.org:mcmaster.ca/oai:macsphere.mcmaster.ca:11375/24375 |
| Date | January 2019 |
| Creators | Schandelmaier, Stefan |
| Contributors | Guyatt, Gordon, Health Research Methodology |
| Source Sets | McMaster University |
| Language | English |
| Detected Language | English |
| Type | Thesis |
Page generated in 0.0019 seconds