Systematic reviews (SR) and meta-analysis (MA) of randomised controlled trials (RCT) are the trustworthy sources of evidence. However, most systematic reviews focus on pair-wise comparisons. Network-meta-analysis (NMA) offers quantitative methods of integrating data from all the available comparisons of many different treatments for each outcome. In a systematic review of interventions, Summary of Findings (SoF) tables present the main findings of a review in a transparent and simple form. However, it is unknown how to present NMA findings in a tabular format. Moreover, systematic reviews and meta-analysis of interventions can summarize bodies of evidence from randomized and non-randomized studies (NRS). Integrating both sources of evidence in a single study can be challenging particularly in the context of assessing the certainty of the evidence, as well as presenting findings of both RCTs and NRS sources of evidence. In our study, we described how 276 NMA were conducted and how authors reported their main findings. We also conducted 32 interviews with users of NMAs and we designed two final NMA-SoF tables. Furthermore, we conducted two systematic reviews that included RCTs and NRS to address methodological challenges. Based on our results, we developed two NMA-SoF table formats to report the main findings of NMAs. The final format was appealing for users and allowed them to better understand NMA findings. Assessment of quality of individual NRS remains challenging and further research is needed to increase its appropriateness in systematic reviews of NRS. We determined that quality assessment of individual NRS was particularly challenging to implement due to the complexity of NRS evaluation tools. Our evaluation revealed that effect estimates of RCTs and NRS were better presented separately. / Thesis / Doctor of Philosophy (PhD) / Systematic reviews (SR) are a summary of studies that address a particular clinical question. Frequently, SRs are complemented with a statistical aggregation of results of individual studies to produce a single estimate. Summary of findings (SoF) tables are designed to present the most relevant information of systematic reviews and meta-analysis. However, it is unknown how to present network meta-analysis (NMA) findings in SoF tables. Another challenge relates to the integration of randomized controlled trials (RCTs) and non-randomized (NRS) studies. Methodological challenges in systematic reviews need to be addressed through careful research. In our study, we appraised how NMA were conducted, and how they presented their main findings. We designed two versions of SoF tables to present NMA findings. Moreover, we conducted two systematic reviews that included RCTs and NRS to address potential challenges in analyzing and presenting their findings.

Background: Many randomized controlled trials (RCTs) and meta-analyses include analyses of effect modification (also known as subgroup, interaction, or moderation analyses). Methodologists have widely acknowledged the challenges in deciding whether an apparent effect modification is credible or likely the result of chance or bias. Various sets of credibility criteria are available (Chapter 2 provides an example) but are inconsistent, vague in wording, lack guidance for deciding on overall credibility, and have not been systematically tested. Objective: To systematically develop a formal instrument to assess the credibility of effect modification analyses (ICEMAN) in RCTs and meta-analyses of RCTs. Methods: Key steps in the development process included 1) a systematic survey of the literature to identify available criteria, rationales, and previous instruments, 2) a formal consensus study among 10 leading experts, and 3) a formal user-testing study to refine the instrument based on interviews with trial investigators, systematic reviewer authors, and journal editors who applied drafts of the instrument to published claims of effect modification. Results: The systematic survey identified 150 relevant publications, 36 candidate credibility criteria with associated rationales, and 30 existing checklists (Chapter 3). The consensus study consisted of two main video conferences and multiple rounds of written discussion. The user-testing involved 17 users (including systematic review authors, trial investigators, and journal editors) who suggested substantial improvements based on detailed interviews. The final instrument provides separate versions for RCTs (five core questions) and meta-analyses (eight core questions) with explicit response options, and an overall credibility rating ranging from very low to high credibility. A detailed manual provides rationales, supporting references, examples from the literature, and suggestions for use in combination with other quality appraisal tools and reporting (Chapter 4). Discussion: ICEMAN is a rigorously developed instrument to evaluate claims of effect modification and addresses the main limitations of previous approaches. / Thesis / Doctor of Philosophy (PhD) / Randomized controlled trials and meta-analyses provide the best available evidence to evaluate whether effects of a therapy vary among individual patients. Efforts to decide whether treatment effects differ across patients are important and frequently done but difficult to interpret. The fundamental challenge is to decide whether apparent differences in effect are real or due to chance. To aid this decision, experts have suggested various sets of credibility criteria, all with important limitations. This thesis documents how we systematically addressed the limitations of previous approaches. Key steps were a systematic survey of the available credibility criteria, a consensus study among leading methodologists, and a formal user-testing study. The result is a new instrument for assessing the credibility of effect modification analyses (ICEMAN).

The relationship between human health and nutrition is complex and limited widely accepted guidance on proper methods of evidence synthesis is available for nutritional issues. While concepts and methods of evidence synthesis in pharmacological treatments can be mostly applied to nutritional interventions, characteristics unique to the nutrition- and dietetics-related topics can lead to distinct challenges that may not be encountered in evidence synthesis of traditional medical interventions. In addition to traditional methods for pooling the results, state-of-the-art methodologies such as GRADE or network meta-analysis, while being widely used in many medical fields, their use in the field of nutrition and food science is surprisingly rare. This thesis begins with the assessment of methodological quality of available public health guidelines on sugar intake to determine the extent to which nutritional guidelines follow currently available guidance in evidence synthesis and making practice recommendations. Subsequently, we present two examples of proper implementation of evidence synthesis methods in standard pairwise meta-analysis and indirect treatment comparison and handling of relevant challenges including applications of GRADE approach. Further, this thesis presents a network meta-analysis in the field of nutrition and child health in which the challenges of conducting multiple treatment comparison are tackled and a new approach for presenting and making conclusion from network meta-analysis results is proposed. / Thesis / Doctor of Philosophy (PhD)

Network meta-analysis (NMA) synthesizes all available direct (head-to-head) and indirect evidence on the comparative effectiveness of at least three treatments and provides coherent estimates of their relative effects. Ranking probabilities are commonly used to summarize these estimates and provide comparative rankings of treatments. However, the reliability of ranking probabilities as summary measures has not been formally established and treatments are often ranked for each outcome separately. This thesis aims to address methodological gaps and limitations in current literature by providing alternative methods for evaluating the robustness of treatment ranks, establishing comparative rankings, and integrating ranking probabilities across multiple outcomes. These novel tools, addressing three specific objectives, are developed in three papers. The first paper presents a conceptual framework for quantifying the robustness of treatments ranks and for elucidating potential sources of lack of robustness. Cohen’s kappa is proposed for quantifying the agreement between two sets of ranks based on NMAs of the full data and a subset of the data. A leave one-study-out strategy was used to illustrate the framework with empirical data from published NMAs, where ranks based on the surface under the cumulative ranking curve (SUCRA) were considered. Recommendations for using this strategy to evaluate sensitivity or robustness to concerning evidence are given. When two or more cumulative ranking curves cross, treatments with large probabilities of ranking the best, second best, third best, etc. may rank worse than treatments with smaller corresponding probabilities based on SUCRA. This limitation of SUCRA is addressed in the second paper through the proposal of partial SUCRA (pSUCRA) as an alternative measure for ranking treatments. pSUCRA is adopted from the partial area under the receiver operating characteristic curve in diagnostic medicine and is derived to summarize relevant regions of the cumulative ranking curve. Knowledge users are often faced with the challenge of making sense of large volumes of NMA results presented across multiple outcomes. This may be further complicated if the comparative rankings on each outcome contradict each other, leading to subjective final decisions. The third paper addresses this limitation through a comprehensive methodological framework for integrating treatments’ ranking probabilities across multiple outcomes. The framework relies on the area inside spie charts representing treatments’ performances on all outcomes, while also incorporating the outcomes’ relative importance. This approach not only provides an objective measure of the comparative ranking of treatments across multiple outcomes, but also allows graphical presentation of the results, thereby facilitating straightforward interpretation. All contributions in this thesis provide objective means to improve the use of comparative treatment rankings in NMA. Further extensive evaluations of these tools are required to assess their validity in empirical and simulated networks of different size and sparseness. / Thesis / Doctor of Philosophy (PhD) / Decisions on how to best treat a patient should be informed by all relevant evidence comparing the benefits and harms of available options. Network meta-analysis (NMA) is a statistical method for combining evidence on at least three treatments and produces a coherent set of results. Nevertheless, NMA results are typically presented separately for each health outcome (e.g., length of hospital stay, mortality) and the volume of results can be overwhelming to a knowledge user. Moreover, the results can be contradictory across multiple outcomes. Statistics that facilitate the ranking of treatments may aid in easing this interpretative burden while limiting subjectivity. This thesis aims to address methodological gaps and limitations in current ranking approaches by providing alternative methods for evaluating the robustness of treatment ranks, establishing comparative rankings, and integrating ranking probabilities across multiple outcomes. These contributions provide objective means to improve the use of comparative treatment rankings in NMA.

Us as humans are colonised by many microbial communities (the human microbiome) that interact with and regulate the host's physiology, and have been linked with several diseases. The high number of interactions that intercurre between the microbiome and the host requires rigorous statistical approaches to link any condition of interest to microbiome data. Many publicly available microbiome datasets are available that allow to study such interactions. However, strong inconsistencies are found among the reported associations when looking at the same condition in different studies. On the road to consistent statistical microbiome analyses that rely on public data, lack of standardisation and availability are barriers to define reproducible and generalisable associations. The main aim of my PhD was the development of meta-analytical approaches to identify microbial signatures as general hallmarks of health versus disease, integrating diverse cohorts and conditions. During my PhD training, I first explored the associations between the oral microbiome and peri-implantitis, an oral disease of dental implants, in which I defined a microbial signature discriminating diseased from control samples. I further developed and applied discriminative models to multiple colorectal cancer (CRC) cohorts, showing that the microbial signature defined on CRC samples is shared across different populations. To be able to further generalised microbial signature with host's conditions through a meta-analysis approach, I collected and analysed 20,533 public metagenomes from 90 cohorts, that are available through the curatedMetagenomicData (cMD) version 3, an R package providing standardised taxonomic and functional profiles and manually curated metadata. The cMD3 resource was used to derive an easy-to-compute oral-to-gut introgression score that I found systematically associated in a large meta-analysis of twelve diseases and with ageing. Finally, I applied the meta-analysis approach to study diet interventions in mice, exploiting a novel approach able to profile the unexplored fraction of microbiomes, and showing associations driven by previously uncharacterised species. Overall, this thesis contributes to strengthening the links between human and animal microbiomes in normal and altered host conditions.

The overall effectiveness of computer-assisted instruction (CAI) for higher order learning in technical education and training was determined through the meta-analysis approach. Studies that had investigated the effectiveness of CAI as compared to traditional instruction were selected from major databases in the civilian and military sectors. The selection criteria were: (a) instruction was in the area of technical education and training, (b) a comparison was made between a group of students that received computer-assisted instruction with another group that was taught in the traditional manner, (c) student learning in both groups was measured in some form, and (d) quantitative results on criterion measures were provided. The common comparison metric chosen to indicate the effect size was the standardized mean difference. Additionally, a determination was made of the difference in CAI effectiveness between studies categorized into: (a) CAI type -- intelligent CAI and ordinary CAI; (b) nature of CAI treatment -- replacement and supplemental; (c) subject assignment -- random groups, intact groups, and assignments other than the preceding two groups; (d) educational level -- secondary / postsecondary, university, and adult military training; and (e) setting -- civilian and military. The overall effect size of CAI was found to be 0.35, implying that the average student in the traditional class would have improved from the 50th percentile to the 64th percentile if the student had been provided with CAI. Intelligent CAI was found to be significantly more effective than ordinary CAI. / Ph. D.

Conduct disorder in children and adolescents has developed into a very costly problem with severe negative consequences to individuals, families, and communities. A void exists in the literature in that no summaries have been found which compare the effectiveness of the leading treatment modalities for conduct disorder. The purpose of this study is to conduct a meta-analysis comparing three psychotherapeutic interventions for the treatment of conduct disorders in children and adolescents: 1) cognitive interventions, 2) parent management training, and 3) psychopharmacological intervention (i.e. the use of lithium carbonate and the use of stimulants [e.g., methylphenidate, dextroamphetamine, and pemoline]). The inclusion criteria for this meta-analysis accepted 26 studies for data collection and analysis. / Ph. D.

Tree species migration as a result of climate change may alter the composition of trees in local communities. Shifts in tree diversity, stand age, species predominance and the overall number of trees are potential changes. Community tree programs may also change the characteristics of local trees through planting or preservation efforts, but these programs may also mitigate the effects of climate induced tree migration. Numerous hedonic property value studies have estimated the implicit price of tree amenities associated with residential properties. Quantitative analysis of the results from multiple studies valuing trees can identify if the relationship between implicit price and tree amenities extended across these studies. The results of the meta-regression found systematic variation was present across positive implicit prices for local tree cover. The scarcity, age and type of local trees were also significantly related to the implicit price of amenity tree cover. The amenity tree cover findings suggest that county tree canopy cover of about 42% optimizes implicit price. Recent extreme weather events and ownership of trees contributed to negative implicit prices. These results may assist in planning and goal setting for community tree programs to mitigate the effects of climate induced tree migration. / Master of Science

Social presence has drawn great attention in the last three decades. A large number of studies attempted to prove that social presence exerted an effect on distance learning through including more interaction. This integrative review provided a comprehensive summary of current studies on social presence, identified problems in measuring social presence, and evaluated the effects of social presence on learning. Data were collected from 189 social presence studies in the area of distance education from 1976 to 2012. Data were analyzed qualitatively followed by a quantitative meta-analysis. This study revealed that social presence was still illusive and difficult to define. Due to its ambiguity, many doubts exist related to the measurement of social presence. The results of this study suggest future researchers should be cautious when advocating the importance of social presence in distance learning. / Ph. D.

Over the past decade there has been a growing investment in professional development (PD) programs for K-12 mathematics teachers. Researchers and policy makers often inquire about the effect of professional development programs and whether they are having a positive impact on students' learning, and which type of programs aid in students ' academic improvements. In view of the continued concern about the low achievement of U.S. students in mathematics there is a strong need to better understand the effect of PD on student learning and more specifically, which components of PD are more beneficial and are likely to enhance students' math learning. This study offers evidence in an effort fill the gap in the literature by examining the relationship between professional development and student achievement. A meta-analysis methodology was used to synthesize quantitatively and aggregate the results of the prior studies that have used scientific methods and reported numerical results. The time period for the selection of studies is from 2003 to 2014. Findings in this study confirm earlier literature and offers guidance about three effective components (duration, content, and intervention modality) in professional development programs that have been found to have a positive relationship with students' achievement. The results also highlighted the importance of content focus, sufficient duration and multiple modalities of professional development programs aiding teachers in their ability to become more effective in the classroom leading to increased students' achievement in math. / Ph. D.