The work presented in this dissertation is driven by three fundamental questions in climate science: (1) What is the natural variability of our climate system? (2) What components of this variability are predictable? (3) How does climate change affect variability and predictability? Determining the variability and predictability of the chaotic and nonlinear climate system is an inherently challenging problem. Climate scientists face the additional complications from limited and error-filled observational data of the true climate system and imperfect dynamical climate models used to simulate the climate system. This dissertation contains four chapters, each of which explores at least one of the three fundamental questions by providing novel approaches to address the complications. Chapter 1 examines the uncertainty in the observational record. As surface temperature data is among the highest quality historical records of the Earth’s climate, it is a critical source of information about the natural variability and forced response of the climate system. However, there is still uncertainty in global and regional mean temperature series due to limited and inaccurate measurements. This chapter provides an assessment of the global and regional uncertainty in temperature from 1880-present in the NASA Goddard Institute for Space Studies (GISS) Surface Temperature Analysis (GISTEMP). Chapter 2 extends the work of Chapter 1 to the regional spatial scale and monthly time scale. An observational uncertainty ensemble of historical global surface temperature is provided for easy use in future studies. Two applications of this uncertainty ensemble are discussed. First, an analysis of recent global and Arctic warming shows that the Arctic is warming four times faster than the rest of the global, updating the oft-provided statistic that Arctic warming is double that of the global rate. Second, the regional uncertainty product is used to provide uncertainty on country-level temperature change estimates from 1950-present. Chapter 3 investigates the impacts of the El Niño-Southern Oscillation (ENSO) on seasonal precipitation globally. In this study, novel methodology is developed to detect ENSO-precipitation teleconnections while accounting for missing data in the CRU TS historical precipitation dataset. In addition, the predictability of seasonal precipitation is assessed through simple empirical forecasts derived from the historical impacts. These simple forecasts provide significant skill over climatological forecasts for much of the globe, suggesting accurate predictions of ENSO immediately provide skillful forecasts of precipitation for many regions. Chapter 4 explores the role of initialization shock in long-lead ENSO forecasts. Initialized predictions from the CMIP6 decadal prediction project and uninitialized predictions using an analogue prediction method are compared to assess the role of model biases in climatology and variability on long-lead ENSO predictability. Comparable probabilistic skill is found in the first year between the model-analogs and the initialized dynamical forecasts, but the initialized dynamical forecasts generally show higher skill. The presence of skill in the initialized dynamical forecasts in spite of large initialization shocks suggest that initialization of the subsurface ocean may be a key component of multi-year ENSO skill. Chapter 5 brings together ideas from the previous chapters through an attribution of historical temperature variability to various anthropogenic and natural sources of variability. The radiative forcing due to greenhouse gas emissions is necessary to explain the observed variability in temperature nearly everywhere on the land surface. Regional fingerprints of anthropogenic aerosols are detected as well as the impact of major sources of natural variability such as ENSO and Atlantic Multidecadal Variability (AMV).

This dissertation examines the use of statistical models for prediction. Examples are drawn from public policy and chosen because they represent pressing problems facing U.S. governments at the local, state, and federal level. The first five chapters provide examples where the perfunctory use of linear models, the prediction tool of choice in government, failed to produce reasonable predictions. Methodological flaws are identified, and more accurate models are proposed that draw on advances in statistics, data science, and machine learning. Chapter 1 examines skyscraper construction, where the normality assumption is violated and extreme value analysis is more appropriate. Chapters 2 and 3 examine presidential approval and voting (a leading measure of civic participation), where the non-collinearity assumption is violated and an index model is more appropriate. Chapter 4 examines changes in temperature sensitivity due to global warming, where the linearity assumption is violated and a first-hitting-time model is more appropriate. Chapter 5 examines the crime rate, where the independence assumption is violated and a block model is more appropriate. The last chapter provides an example where simple linear regression was overlooked as providing a sensible solution. Chapter 6 examines traffic fatalities, where the linear assumption provides a better predictor than the more popular non-linear probability model, logistic regression. A theoretical connection is established between the linear probability model, the influence score, and the predictivity.