Research-based decision making has been advanced as a way for professionals to make a determination about the effectiveness of a potential treatment. However, informed consumers of research need to be able to determine what constitutes evidence-based practices and what criteria can be used to determine if evidence-based practices have been met. This study was a synthesis of research that involved a critical review of the empirical research studies reported in Volume 47 of the Journal of Speech, Language, and Hearing Research (JSLHR) published in 2004. This methodological research synthesis evaluated (a) the research designs used in the JSLHR studies, (b) information and rationale used to inform population validity assessment decisions, and (c) the extent to which the sampling designs, population validity rating, data analysis procedures, and the specification of generalizations and conclusions provide sufficient evidence to determine an overall rating of conclusion validity. Results indicated that less than one-fifth of the 105 research synthesis population of studies used experimental research designs. Additionally, the vast majority of the research synthesis population of studies (83.8%) were observational research designs. Only five studies out of the research synthesis population of studies (4.8%) were determined to have high population validity. In contrast, 84.8 percent of the research synthesis population of studies were found to have low population validity. That is, the studies did not contain adequate information or description of the essential sampling concerns. The vast majority or 75.3 percent of the research synthesis population of studies were rated as having low conclusion validity. Approximately one-fifth of the 105 research synthesis study population (22 studies or 20.9%) were found to have moderate conclusion validity while less than five percent of the total studies (4 of 105 studies or 3.8%) were found to have high conclusion validity. A meaningful relationship between population validity ratings and conclusion validity ratings was established. Since 81 of 105 studies have identical ratings for both population and conclusion validity, the accuracy of the prediction model developed for this study is 77.1 percent.

This thesis contributes to the research field of causal inference, where the effect of a treatment on an outcome is of interest is concerned. Many such effects cannot be estimated through randomised experiments. For example, the effect of higher education on future income needs to be estimated using observational data. In the estimation, assumptions are made to make individuals that get higher education comparable with those not getting higher education, to make the effect estimable. Another assumption often made in causal inference (both in randomised an nonrandomised studies) is that the treatment received by one individual has no effect on the outcome of others. If this assumption is not met, the meaning of the causal effect of the treatment may be unclear. In the first paper the effect of college choice on income is investigated using Swedish register data, by comparing graduates from old and new Swedish universities. A semiparametric method of estimation is used, thereby relaxing functional assumptions for the data. One assumption often made in causal inference in observational studies is that individuals in different treatment groups are comparable, given that a set of pretreatment variables have been adjusted for in the analysis. This so called unconfoundedness assumption is in principle not possible to test and, therefore, in the second paper we propose a Bayesian sensitivity analysis of the unconfoundedness assumption. This analysis is then performed on the results from the first paper. In the third paper of the thesis, we study profile likelihood as a tool for semiparametric estimation of a causal effect of a treatment. A semiparametric version of the Bayesian sensitivity analysis of the unconfoundedness assumption proposed in Paper II is also performed using profile likelihood. The last paper of the thesis is concerned with the estimation of direct and indirect causal effects of a treatment where interference between units is present, i.e., where the treatment of one individual affects the outcome of other individuals. We give unbiased estimators of these direct and indirect effects for situations where treatment probabilities vary between individuals. We also illustrate in a simulation study how direct and indirect causal effects can be estimated when treatment probabilities need to be estimated using background information on individuals.

Osteoporotic fractures are a major and increasing clinical and public health concern internationally. Identification of individuals at high risk for fragility fractures may enable us to target preventive interventions more effectively. In this thesis, I aimed to evaluate novel risk factors for osteoporosis and develop a fracture risk assessment model among the middle-aged and older people. I used data from the European Prospective Investigation into Cancer (EPIC)-Norfolk study, which is a large population-based prospective study started in 1993. About 25,000 men and women were assessed at baseline and about 15,000 of them returned for a second examination 4 years later. All participants are followed up to the present for clinical events including fractures. My work is in two parts. For the first part, I examined the risk of fracture associated with some novel or less well studied risk factors. These risk factors included change in height over time, respiratory function, physical activity and body fat mass. We found that men and women with annual height loss >0.5 cm are at increased risk of hip and any fracture (relative risk=1.9 (95% CI 1.3-2.7) per cm/year height loss). One litre lower forced expiratory volume in 1 second (FEV1) was associated with a 2-fold risk of hip fracture in men and women. We also observed a non-linear association, independent of body mass index, between increasing body fat mass and lower fracture risk in women but not in men. I performed a systematic review and meta-analysis of studies evaluating the association between physical activity and hip fractures. Using a new validated questionnaire in EPIC-Norfolk, we observed varying relationships between physical activity in different domains of life and fracture risk in men and women. For the second part of the thesis, I developed a biostatistical model to calculate 10-year risk of developing a fracture among EPIC-Norfolk study participants. This model incorporates clinical and radiological assessments known to be associated with fractures and can be extended to other risk factors assessed in other prospective cohorts. This helps clinicians to achieve a better estimate of the prospective risk of fracture in their patients. I applied this model to compare the predictive value of two different clinical assessment methods for osteoporosis, namely dual-energy X-ray absorptiometry (DXA) and quantitative ultrasound (QUS). We found that that the predictive power of QUS is comparable to, and independent of, predictive power of DXA. In summary, my studies have added to our knowledge about some novel and easy-to-use risk factors of osteoporosis and proposed a practical method to merge and utilise data from different risk factors for estimation of fracture risk in individuals.

Tetralogy of Fallot (TOF) is the most common cyanotic congenital cardiac defect with a global annual incidence of 40,000 cases. Advances in surgery and perioperative care led to improvements in perioperative mortality and, thus, a growing number of survivors. TOF survivors often suffer from complications related to a failing right ventricle. Follow-up studies evaluating TOF repair strategies suggest an association between the type of surgical repair strategy and late right ventricular health. However, surgical practices remain unchanged and led by institution-level biases. The body of evidence addressing outcomes based on TOF surgical repair strategy is weak and controversies persists on the management of these patients.  This thesis comprises 6 chapters that form the foundation of a multi-centre research program on outcomes after TOF surgical repair. The program uses various methodologies to generate evidence with a vision to change surgical practices.  Chapter 1 is an introduction providing background on TOF and contemporary areas of controversy.  Chapter 2 presents the results of a retrospective analysis evaluating the use of early echocardiogram parameters in predicting late cardiac magnetic resonance imaging evaluation of the right ventricle.  Chapter 3 presents the results of a retrospective cohort exploring the association between TOF repair strategy and development of right bundle branch block.  Chapter 4 presents the results of a multinational survey aiming to explore contemporary biases in TOF surgical repair strategy selection.  Chapter 5 presents the background, rationale, design and baseline cohort characteristics of the Tetralogy of Fallot for Life (TOF LIFE) study. The study is a multi-centre inception cohort study with a follow-up period of 2 years.  Finally, Chapter 6 discusses the conclusion, limitations, and future implications of this research program. / Thesis / Doctor of Science (PhD)

<p>This thesis makes contributions to the statistical research field of causal inference in observational studies. The results obtained are directly applicable in many scientific fields where effects of treatments are investigated and yet controlled experiments are difficult or impossible to implement.</p><p>In the first paper we define a partially specified directed acyclic graph (DAG) describing the independence structure of the variables under study. Using the DAG we show that given that unconfoundedness holds we can use the observed data to select minimal sets of covariates to control for. General covariate selection algorithms are proposed to target the defined minimal subsets.</p><p>The results of the first paper are generalized in Paper II to include the presence of unobserved covariates. Morevoer, the identification assumptions from the first paper are relaxed.</p><p>To implement the covariate selection without parametric assumptions we propose in the third paper the use of a model-free variable selection method from the framework of sufficient dimension reduction. By simulation the performance of the proposed selection methods are investigated. Additionally, we study finite sample properties of treatment effect estimators based on the selected covariate sets.</p><p>In paper IV we investigate misspecifications of parametric models of a scalar summary of the covariates, the propensity score. Motivated by common model specification strategies we describe misspecifications of parametric models for which unbiased estimators of the treatment effect are available. Consequences of the misspecification for the efficiency of treatment effect estimators are also studied.</p>

Propensity score analysis has been used to minimize the selection bias in observational studies to identify causal relationships. A propensity score is an estimate of an individual's probability of being placed in a treatment group given a set of covariates. Propensity score analysis aims to use the estimate to create balanced groups, akin to a randomized experiment. This study used Monte Carlo methods to examine the appropriateness of using propensity score methods to achieve balance between groups on observed covariates and reproduce treatment effect estimates in multilevel studies. Specifically, this study examined the extent to which four different propensity score estimation models and three different propensity score conditioning methods produced balanced samples and reproduced the treatment effects with clustered data. One single-level logistic model and three multilevel models were investigated. Conditioning methods included: (a) covariance adjustment, (b) matching, and (c) stratification. Design factors investigated included: (a) level-1sample size, (b) level-2 sample size, (c) level-1 covariate relationship to treatment, (d) level-2 covariate relationship to treatment, (e) level-1 covariate relationship to outcome, (f) level-2 covariate relationship to outcome, and (g) population effect size. The results of this study suggest the degree to which propensity score analyses are able to create balanced groups and reproduce treatment effect estimates with clustered data is largely dependent upon the propensity score estimation model and conditioning method selected. Overall, the single-level logistic and random intercepts models fared slightly better than the more complex multilevel models while covariance adjustment and matching methods tended to be more stable in terms of balancing groups than stratification. Additionally, the results indicate propensity score analysis should not be conducted with small samples. Finally, this study did not identify an estimation model or conditioning method that was consistently able to create adequately balanced groups and reproduce treatment effect estimates.

This thesis makes contributions to the statistical research field of causal inference in observational studies. The results obtained are directly applicable in many scientific fields where effects of treatments are investigated and yet controlled experiments are difficult or impossible to implement. In the first paper we define a partially specified directed acyclic graph (DAG) describing the independence structure of the variables under study. Using the DAG we show that given that unconfoundedness holds we can use the observed data to select minimal sets of covariates to control for. General covariate selection algorithms are proposed to target the defined minimal subsets. The results of the first paper are generalized in Paper II to include the presence of unobserved covariates. Morevoer, the identification assumptions from the first paper are relaxed. To implement the covariate selection without parametric assumptions we propose in the third paper the use of a model-free variable selection method from the framework of sufficient dimension reduction. By simulation the performance of the proposed selection methods are investigated. Additionally, we study finite sample properties of treatment effect estimators based on the selected covariate sets. In paper IV we investigate misspecifications of parametric models of a scalar summary of the covariates, the propensity score. Motivated by common model specification strategies we describe misspecifications of parametric models for which unbiased estimators of the treatment effect are available. Consequences of the misspecification for the efficiency of treatment effect estimators are also studied.

This thesis develops techniques for adjusting for selection bias using Gaussian process models. Selection bias is a key issue both in sample surveys and in observational studies for causal inference. Despite recently emerged techniques for dealing with selection bias in high-dimensional or complex situations, use of Gaussian process models and Bayesian hierarchical models in general has not been explored. Three approaches are developed for using Gaussian process models to estimate the population mean of a response variable with binary selection mechanism. The first approach models only the response with the selection probability being ignored. The second approach incorporates the selection probability when modeling the response using dependent Gaussian process priors. The third approach uses the selection probability as an additional covariate when modeling the response. The third approach requires knowledge of the selection probability, while the second approach can be used even when the selection probability is not available. In addition to these Gaussian process approaches, a new version of the Horvitz-Thompson estimator is also developed, which follows the conditionality principle and relates to importance sampling for Monte Carlo simulations. Simulation studies and the analysis of an example due to Kang and Schafer show that the Gaussian process approaches that consider the selection probability are able to not only correct selection bias effectively, but also control the sampling errors well, and therefore can often provide more efficient estimates than the methods tested that are not based on Gaussian process models, in both simple and complex situations. Even the Gaussian process approach that ignores the selection probability often, though not always, performs well when some selection bias is present. These results demonstrate the strength of Gaussian process models in dealing with selection bias, especially in high-dimensional or complex situations. These results also demonstrate that Gaussian process models can be implemented rather effectively so that the benefits of using Gaussian process models can be realized in practice, contrary to the common belief that highly flexible models are too complex to use practically for dealing with selection bias.

Les études de pharmacoépidémiologie consistent à étudier l’effet de médicaments en vie réelle, et sont menées de plus en plus souvent sur bases de données médico-administratives. Ce sont principalement des études observationnelles, et sont donc soumises à des biais liés à des facteurs de confusion. Ces facteurs ne sont pas toujours recueillis dans les bases de données médico-administratives qui sont implémentées à d’autres fins que la recherche. Des plans expérimentaux self-controlled designs (où le patient est son propre témoin, et dont les principaux sont le case-crossover et le self-controlled case-series) permettent d’étudier l’effet transitoire d'expositions brèves sur des évènements à début brutal. Ils sont soumis à certaines conditions d’application. Ils ont la particularité de réaliser des comparaisons sur différentes périodes plutôt que sur différents groupes de patients, permettant ainsi de prendre en compte des facteurs de confusion, y compris non mesurés, et qui ne varient pas entre les périodes observées. Ces méthodes ont montré leur utilité pour pallier l’absence de randomisation, et leur utilisation est recommandée quand leurs conditions d’application sont remplies. Nous avons étudié la fréquence d’utilisation des self-controlled designs en pharmacoépidémiologie sur bases de données, les opportunités manquées d’utilisation et leur usage approprié au regard de leurs conditions d’application, ainsi que la qualité de l’information rapportée dans les articles. Nous avons montré que leur utilisation est rare, que 15% des articles correspondent à des situations d’opportunité où ces méthodes auraient pu être implémentées, que 34% des case-crossover et 13% des self-controlled case-series étaient appliqué de façon inapproprié, et que pour 16% des articles la méthode aurait pu être adaptée pour être valide. Un usage plus approprié permettrait de contribuer à l’investigation en pharmacoépidémiologie tout en bénéficiant des avantages de ces méthodes en particulier sur bases de données de santé. / Pharmacoepidemiology consists in the study of efficacy or safety of drugs in real life, with the use more and more frequently of medico-administrative databases. Study designs are generally observational, thus they are prone to confounding bias. Confounders are not systematically collected in databases, which are implemented for other purposes than research. Self-controlled designs (mainly represented by case-crossover and self-controlled case-series, and in which the patient acts as his own control), have been developed for the study of intermittent exposure with short-term effect on abrupt onset event. They require that validity assumptions being fulfilled. They consist in the comparison over different periods, rather than different groups of patients, thus allowing for confounding factors, also if not measured, which are invariant over observed periods. Such designs have been proved useful in observational studies in the absence of randomization, and their implementation is recommended in case of validity assumptions are fulfilled. We studied their frequency of use in pharmacoepidemiology in healthcare databases, missed opportunities for use, inappropriate use with respect to validity assumptions, as well as quality of reporting. We showed that self-controlled designs are rarely used, that opportunity for use was founds in 15% of articles where such methods could have been implemented, that 34% of case-crossover and 13% of self-controlled case series were inappropriately used, and that the method could have been adapted to be valid in 16% of articles. A more appropriate use of self-controlled designs could contribute to improve investigation in pharmacoepidemiology, while beneficiating from their advantages, especially in healthcare databases.

Thesis advisor: Henry I. Braun / Sponsored by OECD, PIAAC represents the first attempt to assess adult problem solving in technology-rich environments (PS-TRE) on an international scale that is comparable cross-culturally and cross-nationally. The objectives of this study are to study (1) the distributions of PS-TRE proficiency scores across 14 selected countries and (2) within each country, the associations between PS-TRE proficiency scores and the different formats of adult education and training (AET) participation. Using data on full-time professionals (at least 25 years old) from these countries, propensity score weighting was applied to estimate the associations between the different formats of AET participation and their PS-TRE proficiency scores. To place these estimates in context, parallel analyses were conducted – one with the sample of full-time associates in the 14 selected countries and the other with full-time professionals’ Literacy and Numeracy proficiency scores as measured by PIAAC. The results showed that after controlling for socio-demographic background, occupational categories, use of key information-processing skills (both at home and at work), as well as use of generic workplace skills, no consistent pattern was found across the 14 selected countries. At the individual country level, scattered significant relationships were identified. For example, in Denmark, both formats of AET participation (vs. None) are significantly and positively associated with full-time professionals’ PS-TRE proficiency scores and their probability of scoring in the top quartile of the PS-TRE distribution (p < .01). While in the United States, Formal AET (vs. None) is significantly and positively associated with full-time associates’ PS-TRE proficiency scores and their probability of scoring in the top quartile of the PS-TRE distribution (p < .01). The variations in relationships between the different formats of AET participation and working adults’ skills proficiency across domains and samples indicate the necessity of conducting qualitative research on AET programs in individual countries. Furthermore, to provide recommendations tailored to the specific needs of each country, a fine-grained classification of AET programs based on the OECD guideline was suggested. / Thesis (PhD) — Boston College, 2020. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.