Background: The COVID-19 pandemic posed an unparalleled challenge to worldwide public health systems, characterized by its high transmissibility and the initial absence of accessible testing, treatments, and vaccines. The deficiency in public awareness and the scarcity of readily available public health information regarding this century's disaster further intensified the critical need for innovative solutions to bridge these gaps. In response, Shaman Labs1,2, leveraging its deep expertise in forecasting for influenza3, Ebola, and various SARS viruses, initiated the development of country-wide COVID-19 projections within weeks following the WHO's declaration of the pandemic4–6. Almost immediately thereafter, it became necessary to create a sophisticated online platform—a system capable of displaying county-specific COVID-19 forecasts, including daily estimated infections, cases, and deaths. This platform was designed to allow users to select any county, state, or national geography and compare it with another, under various scenarios of social distancing measures. Additionally, the architecture of this system was required to facilitate the regular integration of updated data, ensuring the tool's ongoing relevance and utility. Columbia University's data visualization system aimed to communicate epidemiological forecasts to various stakeholders. At the onset of the COVID-19 pandemic, amid escalating uncertainty and the pressing need for reliable data, Dr. Rundle played a pivotal role in briefing key stakeholders on the unfolding crisis. His efforts were directed towards providing Congressman Ron Johnson, Chairman of the U.S. Senate Committee on Homeland Security & Governmental Affairs, and Congresswoman Anna Eshoo, as well as their staff, with up-to-date projections and analyses derived from the Classic Data Visualization tools. Dr. Rundle’s consultative role extended to a diverse array of institutions including the U.S. Army Corps of Engineers, the U.S. Air Force, and the Federal Reserve Board, as well as advising private entities such as Pfizer, MetLife, and Unilever. His expertise facilitated informed planning and response efforts across various levels of government and sectors, underscoring the critical role of sophisticated data visualization from the earliest stages of the pandemic. This Integrated Learning Experience (ILE) examines the development and implementation of the Time Machine platform, focusing on its application in visualizing and analyzing COVID-19 epidemiological forecasts. The study explores methods for improving forecast data presentation, analysis, and accuracy assessment. Methods: The body of this work unfolds through a series of critical chapters that collectively address the multifaceted functionality and impact of the Time Machine platform. Initially, the work focuses on the construction of the Time Machine platform, a web-based R interactive user interface coupled with cloud-based database system, specifically tailored for the intuitive visualization of epidemiological forecasts, detailing the technical and design considerations essential for enabling users to interpret complex data more effectively. Following this, the implementation of a rigorous data-discovery framework is presented, examining case reporting inconsistencies across different regions, using low-level GitHub and Windows scripting technologies, thereby highlighting the significance of accurate data collection and the impact of discrepancies on public health decisions. The narrative then transitions to the implementation of advanced statistical models, such as strictly proper scoring and weighted interval scoring, to assess the accuracy of the forecasts provided by the Time Machine platform, using a dedicated R library and testing with the help of MS Excel sandbox, underscoring the importance of reliable predictions in the management of public health crises. Lastly, a detailed analysis is conducted, encompassing countrywide data (3142 counties) over an extended period (147 weeks), utilizing Generalized Estimating Equations (GEE) to identify key predictors that influence forecast accuracy, offering valuable insights into the factors that either enhance or detract from the reliability of epidemiological predictions. Results: The deployment of the Classic Data Visualization and the subsequent evolution of the Time Machine platform have significantly advanced epidemiological forecast visualization capabilities. The Time Machine platform was designed with an automated data refresh system, allowing for regular updates of epidemiological forecast data and reported actuals. The project developed tools for monitoring and evaluating the quality of public health reporting, aiming to improve the accuracy and timeliness of data used in public health decisions. Additionally, the research implemented methods for standardizing forecast accuracy assessments, including the normalization of scores to enable comparisons across different geographical scales. These approaches were designed to support both local and national-level pandemic response efforts. The accuracy analyses throughout different phases of the pandemic revealed a 42% improvement in forecast accuracy from Phase 1 to Phase 7. Larger populations (27% increase per unit increase on a base-10 logarithmic scale) and higher county-level activity (45% increase from the lowest to the highest quartile) resulted in better estimations. Additionally, the analysis highlighted the significant impact of reporting quality on forecast accuracy. On the other hand, the study identified the challenges in predicting case surges, showing a 27% decline in accuracy during periods of rising infections compared to declining periods. The regression results highlight the potential benefits of improving data collection and providing timely feedback to forecasting teams. Conclusion: This study demonstrates the potential of advanced data visualization and accuracy measurement techniques in improving epidemiological forecasting. The findings suggest that factors such as urbanicity, case reporting quality, and pandemic phase significantly influence forecast accuracy. Further research is needed to refine these models and enhance their applicability across various public health scenarios.

Thesis (Ed.D.)--Teachers College, Columbia University. / Typescript. Sponsor: Karl W. Bigelow. Dissertation Committee: R. Freeman Butts, Ralph R. Field, . Type C project. Includes bibliographical references (leaves 419-437).