Advances in water resources modeling are improving the information that can be supplied to support decisions that affect the safety and sustainability of society, but these advances result in models being more computationally demanding. To facilitate the use of cost- effective computing resources to meet the increased demand through high-throughput computing (HTC) and cloud computing in modeling workflows and web applications, I developed a comprehensive Python toolkit that provides the following features: (1) programmatic access to diverse, dynamically scalable computing resources; (2) a batch scheduling system to queue and dispatch the jobs to the computing resources; (3) data management for job inputs and outputs; and (4) the ability for jobs to be dynamically created, submitted, and monitored from the scripting environment. To compose this comprehensive computing toolkit, I created two Python libraries (TethysCluster and CondorPy) that leverage two existing software tools (StarCluster and HTCondor). I further facilitated access to HTC in web applications by using these libraries to create powerful and flexible computing tools for Tethys Platform, a development and hosting platform for web-based water resources applications. I tested this toolkit while collaborating with other researchers to perform several modeling applications that required scalable computing. These applications included a parameter sweep with 57,600 realizations of a distributed, hydrologic model; a set of web applications for retrieving and formatting data; a web application for evaluating the hydrologic impact of land-use change; and an operational, national-scale, high- resolution, ensemble streamflow forecasting tool. In each of these applications the toolkit was successful in automating the process of running the large-scale modeling computations in an HTC environment.

The interactive nature of web applications or “web apps” makes it an excellent medium for conveying complex scientific concepts to lay audiences and creating decision support tools that harness cutting edge modeling techniques. However, the technical expertise required to develop them represents a barrier for would-be developers. The barrier can be characterized by the following hurdles that developers must overcome: (1) identify, select, and install software that meet the spatial and computational capabilities commonly required for water resources modeling; (2) orchestrate the use of multiple FOSS and FOSS4G projects and navigate their differing application programming interfaces (APIs); (3) learn the multi-language programming skills required for modern web development; and (4) develop a web-safe and fully featured web site to host the app. This research has resulted in two primary products that effectively lower the barrier to water resources web app development: (1) a literature review of free and open source software (i.e. software review) and (2) Tethys Platform. The software review included earth science web apps that were published in the peer-reviewed literature in the last decade and it was performed to determine which FOSS4G and FOSS web software has been used to develop such web apps. The review highlights 11 FOSS4G software projects and 9 FOSS projects for web development that were used to develop 45 earth sciences web apps, which constitutes a significantly reduced list of possible software projects that could be used to meet the needs of water resources web app development—greatly lowering the barrier for entry to water resources web development. While the software review addresses the hurdle of identifying FOSS software to provide a web framework and spatial data capabilities for water resources web apps, there are still other hurdles that needed to be overcome to make development more viable. Tethys Platform was developed to address these other hurdles and streamline the development of water resources web apps. It includes (1) a suite of free and open source software that address the unique data and computational needs common to water resources web app development, (2) a Python software development kit for incorporating the functionality of each software element into web apps and streamlining their development, and (3) a customizable web portal that is used to deploy the completed web apps. Tethys Platform has been used to develop a broad array of web apps for water resources modeling and decision support.

Warning systems with the ability to predict floods days in advance can benefit tens of millions of people. Because of these potential impacts there have been efforts to improve prediction systems such as the United States’ Advanced Hydrologic Prediction Service and European-developed Global Flood Awareness System. However, these projects are currently limited to relatively coarse resolutions. This thesis presents a method for downscaling and routing global runoff forecasts generated by the European Centre for Medium-Range Weather Forecasts using the Routing Application for Parallel computatIon of Discharge program that make possible orders of magnitude increases in the density of the resolution of stream forecasts. The processing method involves using the Amazon Web Services to distribute execution in a cloud-computing environment to make it possible to solve for large watersheds with high-density stream networks. Using the Amazon Web Services, the number of streams that can be used in the downscaling process in a twelve-hour period is approximated to be close to five million. In addition, an application for visualizing large high-density stream networks has been created using the Tethys Platform of water resources modeling developed as part of the CI-WATER NSF grant. The web application is tested with the HUC-2 Region 12 watershed network with over 67,000 reaches and is able to display analyzed results to the user for each reach.

Since 2002, NASA's GRACE Satellite mission has allowed scientists of various disciplines to analyze and map the changes in Earth's total water storage on a global scale. Although the raw data is available to the public, the process of viewing, manipulating, and analyzing the GRACE data can be difficult for those without strong technological backgrounds in programming or geospatial software. This is particularly true for water managers in developing countries, where GRACE data could be a valuable asset for sustainable water resource management. To address this problem, I have a developed a utility for subsetting GRACE data to particular regions of interest and I have packaged that utility in a web app that allows water managers to quickly and easily visualize GRACE data these regions. Using the GLDAS-Noah Land Surface Model, the total water storage for the regions derived from the raw GRACE data is decomposed into surface water, soil moisture, and groundwater components. The GRACE Groundwater Subsetting Tool is easily deployed, open-source, and provides access to all of the major signal processing solutions available for the total water storage data. The application has been successfully applied to both developed and developing countries in various parts of the world, including the Central Valley region in California, Bangladesh, the La Plata River Basin in South America, and the SERVIR Hindu Kush Himalaya region. The groundwater data in this application has proven capable of monitoring groundwater use based on drought trends as well as agricultural demand in a number of locations and can assist in uniting decision makers and water users in the mission of sustainably managing the world's groundwater resources.

This dissertation presents the design, development and validation of new data integration methods for mapping the extent of snow cover based on open access ground station measurements, remote sensing images, volunteer observer snow reports, and cross country ski track recordings from location-enabled mobile devices. The first step of the data integration procedure includes data discovery, data retrieval, and data quality control of snow observations at ground stations. The WaterML R package developed in this work enables hydrologists to retrieve and analyze data from multiple organizations that are listed in the Consortium of Universities for the Advancement of Hydrologic Sciences Inc (CUAHSI) Water Data Center catalog directly within the R statistical software environment. Using the WaterML R package is demonstrated by running an energy balance snowpack model in R with data inputs from CUAHSI, and by automating uploads of real time sensor observations to CUAHSI HydroServer. The second step of the procedure requires efficient access to multi-temporal remote sensing snow images. The Snow Inspector web application developed in this research enables the users to retrieve a time series of fractional snow cover from the Moderate Resolution Imaging Spectroradiometer (MODIS) for any point on Earth. The time series retrieval method is based on automated data extraction from tile images provided by a Web Map Tile Service (WMTS). The average required time for retrieving 100 days of data using this technique is 5.4 seconds, which is significantly faster than other methods that require the download of large satellite image files. The presented data extraction technique and space-time visualization user interface can be used as a model for working with other multi-temporal hydrologic or climate data WMTS services. The third, final step of the data integration procedure is generating continuous daily snow cover maps. A custom inverse distance weighting method has been developed to combine volunteer snow reports, cross-country ski track reports and station measurements to fill cloud gaps in the MODIS snow cover product. The method is demonstrated by producing a continuous daily time step snow presence probability map dataset for the Czech Republic region. The ability of the presented methodology to reconstruct MODIS snow cover under cloud is validated by simulating cloud cover datasets and comparing estimated snow cover to actual MODIS snow cover. The percent correctly classified indicator showed accuracy between 80 and 90% using this method. Using crowdsourcing data (volunteer snow reports and ski tracks) improves the map accuracy by 0.7 – 1.2 %. The output snow probability map data sets are published online using web applications and web services.