This study is aimed to unpack the ‘black box’ that connects the grade-3 test-based retention policy with students’ academic outcomes. I theorized that the policy effects on teaching and learning may be modified by instructional capacity, but are unlikely to occur through enhancing teachers’ capability to teach. Analyzing the Early Childhood Longitudinal Study Kindergarten cohort (ECLS-K) dataset, I first explored the relationship between the test-based retention policy and instructional capacity as indicated by teacher expectations of students’ learning capability and then investigated whether and how the expectations moderated the policy effects on instructional time reallocation, student academic performance, and student self-perceived academic competence and interests. To remove the selection bias associated with the non-experimental data, I applied a novel propensity score-based causal inference method, the marginal mean weighting through stratification (MMW-S) method and extended it to a causal analysis that approximates a randomization of schools to the test-based retention policy followed by a randomization of classes to teachers with different levels of expectations. Consistent with my theory, I found that the test-based retention policy had no effects on teacher expectations. Although the policy uniformly increased the time allocated to math instruction, it produced no significant changes in students’ overall performance and overall self-perception in math. In addition, I found that students responded differently to the test-based retention policy depending on the expectations they received from the grade-3 teachers. The results suggested some benefits of positive expectations over negative and indifferent expectations in moderating the policy effects, including more access to advanced content, higher learning gains of average-ability students, and more resilient student learning over a long term. However, the results also showed that having positive expectations alone is not sufficient for academic improvement under the high-stakes policy. If implemented by a positive-expectation teacher, the policy could be detrimental to students’ learning in the nontested subject or to their learning of basic reading/math skills. It would as well place the bottom-ability students at a disadvantage. The findings have significant implications for the ongoing high-stakes testing debate, for school improvement under the current accountability reform, and for research of teacher effectiveness.

This study is aimed to unpack the ‘black box’ that connects the grade-3 test-based retention policy with students’ academic outcomes. I theorized that the policy effects on teaching and learning may be modified by instructional capacity, but are unlikely to occur through enhancing teachers’ capability to teach. Analyzing the Early Childhood Longitudinal Study Kindergarten cohort (ECLS-K) dataset, I first explored the relationship between the test-based retention policy and instructional capacity as indicated by teacher expectations of students’ learning capability and then investigated whether and how the expectations moderated the policy effects on instructional time reallocation, student academic performance, and student self-perceived academic competence and interests. To remove the selection bias associated with the non-experimental data, I applied a novel propensity score-based causal inference method, the marginal mean weighting through stratification (MMW-S) method and extended it to a causal analysis that approximates a randomization of schools to the test-based retention policy followed by a randomization of classes to teachers with different levels of expectations. Consistent with my theory, I found that the test-based retention policy had no effects on teacher expectations. Although the policy uniformly increased the time allocated to math instruction, it produced no significant changes in students’ overall performance and overall self-perception in math. In addition, I found that students responded differently to the test-based retention policy depending on the expectations they received from the grade-3 teachers. The results suggested some benefits of positive expectations over negative and indifferent expectations in moderating the policy effects, including more access to advanced content, higher learning gains of average-ability students, and more resilient student learning over a long term. However, the results also showed that having positive expectations alone is not sufficient for academic improvement under the high-stakes policy. If implemented by a positive-expectation teacher, the policy could be detrimental to students’ learning in the nontested subject or to their learning of basic reading/math skills. It would as well place the bottom-ability students at a disadvantage. The findings have significant implications for the ongoing high-stakes testing debate, for school improvement under the current accountability reform, and for research of teacher effectiveness.

On December 26th, 2004, 280,000 people lost their lives. A massive earthquake struck Indonesia, triggering a tsunami that affected several countries, including Thailand. The disaster had important implications for health status of Thai citizens, as well as health system planning, and thus underscores the need to study its long-term effect. This dissertation examined the patterns, determinants, and spatial analysis of health service utilization following the tsunami in Thailand. The primary aim was to determine whether tsunami-affected status (personal injury or property loss) and distance to a health facility (public health center or hospital) influenced health service utilization. The study population included Thai citizens (aged 14+), living in the tsunami-affected Thai provinces: Phuket, Phang Nga, Krabi, and Ranong. Study participants were randomly selected from the ‘affected’ and ‘unaffected’ populations. One and two years after the tsunami, participants were interviewed in-person on demographic and socio-economic factors, disaster impact, health status, and health service utilization. Five types of health services were examined: outpatient services, inpatient services, home visits, medications, and informal (unpaid) care. Distance to a health facility was calculated using Geographic Information System’s Network Analyst. The Grossman model of the demand for health care and a distance decay concept provided the foundation for this study. A propensity score method and a two-part model were used to examine the study objectives. There were 1,889 participants. One year after the tsunami, individuals affected by property loss were more likely to use medications than unaffected participants. Two years after the tsunami, individuals with personal injury were more likely to use outpatient services, medications, and informal care than unaffected participants. Distance to a health facility was associated with the use of medications and informal care. The results confirmed the long-term effect of a tsunami. This dissertation may assist the decision- and policy-makers in the identification of those most likely to use health services and in the request of health resources to the affected areas. The patterns, determinants, and spatial analysis of health service utilization found in this study may not be specific to a tsunami and may provide insights on post-disaster contexts of other natural disasters.

On December 26th, 2004, 280,000 people lost their lives. A massive earthquake struck Indonesia, triggering a tsunami that affected several countries, including Thailand. The disaster had important implications for health status of Thai citizens, as well as health system planning, and thus underscores the need to study its long-term effect. This dissertation examined the patterns, determinants, and spatial analysis of health service utilization following the tsunami in Thailand. The primary aim was to determine whether tsunami-affected status (personal injury or property loss) and distance to a health facility (public health center or hospital) influenced health service utilization. The study population included Thai citizens (aged 14+), living in the tsunami-affected Thai provinces: Phuket, Phang Nga, Krabi, and Ranong. Study participants were randomly selected from the ‘affected’ and ‘unaffected’ populations. One and two years after the tsunami, participants were interviewed in-person on demographic and socio-economic factors, disaster impact, health status, and health service utilization. Five types of health services were examined: outpatient services, inpatient services, home visits, medications, and informal (unpaid) care. Distance to a health facility was calculated using Geographic Information System’s Network Analyst. The Grossman model of the demand for health care and a distance decay concept provided the foundation for this study. A propensity score method and a two-part model were used to examine the study objectives. There were 1,889 participants. One year after the tsunami, individuals affected by property loss were more likely to use medications than unaffected participants. Two years after the tsunami, individuals with personal injury were more likely to use outpatient services, medications, and informal care than unaffected participants. Distance to a health facility was associated with the use of medications and informal care. The results confirmed the long-term effect of a tsunami. This dissertation may assist the decision- and policy-makers in the identification of those most likely to use health services and in the request of health resources to the affected areas. The patterns, determinants, and spatial analysis of health service utilization found in this study may not be specific to a tsunami and may provide insights on post-disaster contexts of other natural disasters.

<p>This dissertation address two significant challenges in the causal inference workflow for Big Observational Data. The first is designing Big Observational Data with high-dimensional and heterogeneous covariates. The second is performing uncertainty quantification for estimates of causal estimands that are obtained from the application of black box machine learning algorithms on the designed Big Observational Data. The methodologies developed by addressing these challenges are applied for the design and analysis of Big Observational Data from a large public university in the United States. </p> <h4>Distributed Design</h4> <p>A fundamental issue in causal inference for Big Observational Data is confounding due to covariate imbalances between treatment groups. This can be addressed by designing the study prior to analysis. The design ensures that subjects in the different treatment groups that have comparable covariates are subclassified or matched together. Analyzing such a designed study helps to reduce biases arising from the confounding of covariates with treatment. Existing design methods, developed for traditional observational studies consisting of a single designer, can yield unsatisfactory designs with sub-optimum covariate balance for Big Observational Data due to their inability to accommodate the massive dimensionality, heterogeneity, and volume of the Big Data. We propose a new framework for the distributed design of Big Observational Data amongst collaborative designers. Our framework first assigns subsets of the high-dimensional and heterogeneous covariates to multiple designers. The designers then summarize their covariates into lower-dimensional quantities, share their summaries with the others, and design the study in parallel based on their assigned covariates and the summaries they receive. The final design is selected by comparing balance measures for all covariates across the candidates and identifying the best amongst the candidates. We perform simulation studies and analyze datasets from the 2016 Atlantic Causal Inference Conference Data Challenge to demonstrate the flexibility and power of our framework for constructing designs with good covariate balance from Big Observational Data.</p> <h4>Designed Bootstrap</h4> <p>The combination of modern machine learning algorithms with the nonparametric bootstrap can enable effective predictions and inferences on Big Observational Data. An increasingly prominent and critical objective in such analyses is to draw causal inferences from the Big Observational Data. A fundamental step in addressing this objective is to design the observational study prior to the application of machine learning algorithms. However, the application of the traditional nonparametric bootstrap on Big Observational Data requires excessive computational efforts. This is because every bootstrap sample would need to be re-designed under the traditional approach, which can be prohibitive in practice. We propose a design-based bootstrap for deriving causal inferences with reduced bias from the application of machine learning algorithms on Big Observational Data. Our bootstrap procedure operates by resampling from the original designed observational study. It eliminates the need for additional, costly design steps on each bootstrap sample that are performed under the standard nonparametric bootstrap. We demonstrate the computational efficiency of this procedure compared to the traditional nonparametric bootstrap, and its equivalency in terms of confidence interval coverage rates for the average treatment effects, by means of simulation studies and a real-life case study.</p> <h4>Case Study</h4> <p>We apply the distributed design and designed bootstrap methodologies in a case study involving institutional data from a large public university in the United States. The institutional data contains comprehensive information about the undergraduate students in the university, ranging from their academic records to on-campus activities. We study the causal effects of undergraduate students’ attempted course load on their academic performance based on a selection of covariates from these data. Ultimately, our real-life case study demonstrates how our methodologies enable researchers to effectively use straightforward design procedures to obtain valid causal inferences with reduced computational efforts from the application of machine learning algorithms on Big Observational Data.</p> <p><br></p>