This master's thesis deals with possible applications of Bayesian networks. The theoretical part is mainly of mathematical nature. At first, we focus on general probability theory and later we move on to the theory of Bayesian networks and discuss approaches to inference and to model learning while providing explanations of pros and cons of these techniques. The practical part focuses on applications that demand learning a Bayesian network, both in terms of network parameters as well as structure. These applications include general benchmarks, usage of Bayesian networks for knowledge discovery regarding the causes of criminality and exploration of the possibility of using a Bayesian network as a spam filter.

Advances in data-driven applications, particularly artificial intelligence and deep learning, are driving the explosive growth of computation and communication in today’s data centers and high-performance computing (HPC) systems. Increasingly, system performance is not constrained by the compute speed at individual nodes, but by the data movement between them. This calls for innovative architectures, smart connectivity, and extreme bandwidth densities in interconnect designs. Silicon photonics technology leverages mature complementary metal-oxide-semiconductor (CMOS) manufacturing infrastructure and is promising for low cost, high-bandwidth, and reconfigurable interconnects. Flexible and high-performance photonic switched architectures are capable of improving the system performance. The work in this dissertation explores various photonic interconnected systems and the associated optical switching functionalities, hardware platforms, and novel architectures. It demonstrates the capabilities of silicon photonics to enable efficient deep learning training. We first present field programmable gate array (FPGA) based open-loop and closed-loop control for optical spectral-and-spatial switching of silicon photonic cascaded micro-ring resonator (MRR) switches. Our control achieves wavelength locking at the user-defined resonance of the MRR for optical unicast, multicast, and multiwavelength-select functionalities. Digital-to-analog converters (DACs) and analog-to-digital converters (ADCs) are necessary for the control of the switch. We experimentally demonstrate the optical switching functionalities using an FPGA-based switch controller through both traditional multi-bit DAC/ADC and novel single-wired DAC/ADC circuits. For system-level integration, interfaces to the switch controller in a network control plane are developed. The successful control and the switching functionalitiesachieved are essential for system-level architectural innovations as presented in the following sections. Next, this thesis presents two novel photonic switched architectures using the MRR-based switches. First, a photonic switched memory system architecture was designed to address memory challenges in deep learning. The reconfigurable photonic interconnects provide scalable solutions and enable efficient use of disaggregated memory resources for deep learning training. An experimental testbed was built with a processing system and two remote memory nodes using silicon photonic switch fabrics and system performance improvements were demonstrated. The collective results and existing high-bandwidth optical I/Os show the potential of integrating the photonic switched memory to state-of-the-art processing systems. Second, the scaling trends of deep learning models and distributed training workloads are challenging network capacities in today’s data centers and HPCs. A system architecture that leverages SiP switch-enabled server regrouping is proposed to tackle the challenges and accelerate distributed deep learning training. An experimental testbed with a SiP switch-enabled reconfigurable fat tree topology was built to evaluate the network performance of distributed ring all-reduce and parameter server workloads. We also present system-scale simulations. Server regrouping and bandwidth steering were performed on a large-scale tapered fat tree with 1024 compute nodes to show the benefits of using photonic switched architectures in systems at scale. Finally, this dissertation explores high-bandwidth photonic interconnect designs for disaggregated systems. We first introduce and discuss two disaggregated architectures leveraging extreme high bandwidth interconnects with optically interconnected computing resources. We present the concept of rack-scale graphics processing unit (GPU) disaggregation with optical circuit switches and electrical aggregator switches. The architecture can leverage the flexibility of high bandwidth optical switches to increase hardware utilization and reduce application runtimes. A testbed was built to demonstrate resource disaggregation and defragmentation. In addition, we also present an extreme high-bandwidth optical interconnect accelerated low-latency communication architecture for deep learning training. The disaggregated architecture utilizes comb laser sources and MRR-based cross-bar switching fabrics to enable an all-to-all high bandwidth communication with a constant latency cost for distributed deep learning training. We discuss emerging technologies in the silicon photonics platform, including light source, transceivers, and switch architectures, to accommodate extreme high bandwidth requirements in HPC and data center environments. A prototype hardware innovation - Optical Network Interface Cards (comprised of FPGA, photonic integrated circuits (PIC), electronic integrated circuits (EIC), interposer, and high-speed printed circuit board (PCB)) is presented to show the path toward fast lanes for expedited execution at 10 terabits. Taken together, the work in this dissertation demonstrates the capabilities of high-bandwidth silicon photonic interconnects and innovative architectural designs to accelerate deep learning training in optically connected data center and HPC systems.

No description available.

No description available.

In this dissertation I study the benefits that machine learning can bring to problems of Sustainable Development in the field of hydrology. Specifically, in Chapter 1 I investigate how predictable groundwater depletion is across India and to what extent we can learn from the model’s predictions about underlying drivers. In Chapter 2, I joined a competition to predict the amount of water in snow in the western United States using satellite imagery and convolutional neural networks. Lastly, in Chapter 3 I examine how cloud cover impacts the machine learning model’s predictions and explore how cloudiness impacts the successes and limitation of the popular uncertainty quantification method known as Monte Carlo dropout. Food production in many parts of the world relies on groundwater resources. In many regions, groundwater levels are declining due to a combination of anthropogenic abstraction, localized meteorological and geological characteristics, and climate change. Groundwater in India is characteristic of this global trend, with an agricultural sector that is highly dependent on groundwater and increasingly threatened by abstraction far in excess of recharge. The complexity of inputs makes groundwater depletion highly heterogeneous across space and time. However, modeling this heterogeneity has thus far proven difficult. In Chapter 1 using random forest models and high-resolution feature importance methods, we demonstrate a recent shift in the predictors of groundwater depletion in India and show an improved ability to make predictions at the district-level across seasons. We find that, as groundwater depletion begins to accelerate across India, deep-well irrigation use becomes 250% more important from 1996-2014, becoming the most important predictor of depletion in the majority of districts in northern and central India. At the same time, even many of the districts that show gains in groundwater levels show an increasing importance of deep irrigation. Analysis shows widespread decreases in crop yields per unit of irrigation over our time period, suggesting decreasing marginal returns for the largely increasing quantities of groundwater irrigation used. Because anthropogenic and natural drivers of groundwater recharge are highly localized, understanding the relationship between multiple variables across space and time is inferentially challenging, yet extremely important. Our granular, district-focused models of groundwater depletion rates can inform decision-making across diverse hydrological conditions and water use needs across space, time, and groups of constituents. In Chapter 2 I reflect on competing in the U.S. Bureau of Reclamation’s snow water equivalent prediction competition (Snowcast Showdown). This project was a joint effort with Isabella Smythe and we ended the competition scoring roughly 45th out of over 1000 teams on the public leaderboard. In this chapter I outline our approach and discuss the competition format, model building, and examine alternative approaches taken by other competitors. Similarly I consider the success and limitations of our own satellite-based approach and consider future improvements to iterate upon our model. In Chapter 3 I study the black-box deep learning model built on MODIS imagery to estimate snow water equivalent (SWE) made for the competition discussed in Chapter 2. Specifically, I here investigate a major component of uncertainty in my remotely-sensed images: cloud cover which completely disrupts viewing of the surface in the visible spectrum. To understand the impact of cloud-driven missingness, I document how and where clouds occur in the dataset. I then use Monte Carlo dropout - a popular method of quantifying uncertainty in deep learning models - to learn how well the method captures the aleatoric errors unique to remote sensing with cloud cover. Next, I investigate how the underlying filters of the convolutional neural network appear using the guided backprop technique and draw conclusions regarding what features in the images the model was using to make its predictions. Lastly, I investigate what forms of validation best estimated the true generalization error in Chapter 2 using ordinary least squares (OLS) and the elastic-net technique. These three chapters show that machine learning has an important place in the future of hydrology, however the tools that it brings are still difficult to interpret. Moreover, future work is still needed to bring these predictive advancements to scientific standards of understanding. This said, the increases to accuracy brought by the new techniques can currently make a difference to people’s lives who will face greater water scarcity as climate change accelerates.

In recent years, due to the increasing efficiency and availability of information technologies for collecting massive amounts of data (e.g., smart meters and sensors), a variety of advanced technologies and decision-making strategies in the civil engineering sector have shifted in leaps and bounds to a data-driven manner. While there is still no consensus in industry and academia on the latest advances, challenges, and trends in some innovative data-driven methods related to, e.g., deep learning and neural networks, it is undeniable that these techniques have been proven to be considerably effective in helping our academics and engineers solve many real-life tasks related to the smart city framework. This dissertation systematically presents the investigation and development of the cutting-edge data-driven methods related to two specific areas of civil engineering, namely, Residential Electricity Modeling (REM) and Structural Health Monitoring (SHM). For both components, the presentation of this dissertation starts with a brief review of classical data-driven methods used in particular problems, gradually progresses to an exploration of the related state-of-the-art technologies, and eventually lands on our proposed novel data-driven strategies and algorithms. In addition to the classical and state-of-the-art modeling techniques focused on these two areas, this dissertation also put great emphasis on the proposed effective feature extraction and selection approaches. These approaches are aimed to optimize model performance and to save computational resources, for achieving the ideal characterization of the information embedded in the collected raw data that is most relevant to the problem objectives, especially for the case of modeling deep neural networks. For the problems on REM, the proposed methods are validated with real recorded data from multi-family residential buildings, while for SHM, the algorithms are validated with data from numerically simulated systems as well as real bridge structures.

Software permeates every facet of our lives, improving their convenience and efficiency, and its sphere of influence continues to expand, leading to novel applications and services. However, as software grows in complexity, it increasingly exposes vulnerabilities within the intricate landscape of security threats. Program analysis emerges as a pivotal technique for constructing software that is secure, reliable, and efficient. Despite this, existing methodologies predominantly rely on rules and heuristics, which necessitate substantial manual tuning to accommodate the diverse components of software. In this dissertation, I introduce our advancements in data-driven program analysis, a novel approach in which we employ machine learning techniques to comprehend both the structures and behaviors of programs, thereby enhancing the analysis and security of software applications. Besides focusing on traditional software, I also elaborate on our work in the systematic testing and formal verification of learned software components, including neural networks. I commence by detailing a succession of studies centered on the ambitious goal of learning execution-aware program representations. This is achieved by training large language models to understand program execution semantics. I illustrate that the models equipped with execution-aware pre-training attain state-of-the-art results in a range of program analysis tasks, such as detecting semantically similar code, type inference, memory dependence analysis, debugging symbol recovery, and generating invariants. Subsequently, I outline our approach to learning program structures and dependencies for disassembly and function boundary recovery, which are building blocks for downstream reverse engineering and binary analysis tasks. In the final part of this dissertation, I delve into DeepXplore, the inaugural white-box testing framework designed for deep learning systems, and VeriVis, a pioneering verification framework capable of proving the robustness guarantee of neural networks with only black-box access, extending beyond norm-bounded input transformations.

Carbon emissions associated with deep learning and high-performance computing have reached critical levels and must be addressed to mitigate the potential damage to the environment. Optical solutions have been widely accepted as a necessary part of any comprehensive intervention, primarily in the form of ultra-broadband wavelength-division multiplexing (WDM) optical interconnects to connect spatially distanced compute nodes and, in the further term, as dedicated photonic deep learning accelerators and photonic quantum computers. Silicon photonic interconnects provides the most promising platform for satisfying the required performance, device density, and total wafer throughput by leveraging the same mature complementary metal–oxide–semiconductor (CMOS) infrastructure used to fabricate modern electronic chips. However, implementing these links at scale requires unprecedented levels of integration density in the associated silicon photonic integrated circuit (PICs). The potential explosion in PIC density poses a significant design challenge towards guaranteeing that designers are capable of both an exhaustive design space exploration and rigorous design optimization within reasonable design cycles. Higher level design abstractions—that is, representations of designs that accurately capture system behavior while simultaneously reducing model complexity—are needed for moreefficient design and optimization of PICs. This work contributes two novel design abstractions for the rapid optimization of ultra-high-bandwidth silicon photonic interconnects. The first contribution is a novel process variation-aware compact model of strip waveguides that is suitable for circuit-level simulation of waveguide-based process design kit (PDK) elements. The model is shown to describe both loss and—using a novel expression for the thermo-optic effect in high index contrast materials—the thermo-optic behavior of strip waveguides. Experimental results prove the reported model can self-consistently describe waveguide phase, loss, and thermo-optic behavior across all measured devices over an unprecedented range of optical bandwidth, waveguide widths, and temperatures. The second contribution is a generalized abstraction for designing WDM links in the multi-freespectral range (FSR) regime, a technique for avoiding aliasing while using microresonators with FSRs smaller than the total optical bandwidth of the link. Extensive simulation and experimental results prove that the aforementioned abstractions described collectively provide a powerful toolset for rapid interconnect design and optimization. The advances in this thesis demonstrate the utility of higher-level design abstractions for fully realizing the potential silicon photonics holds for keeping pace with ever-growing bandwidth demands computing systems in the post-Moore’s Law era and beyond.

Bug bounties are an increasingly popular way of performing penetration tests of web applications. User statistics of bug bounty platforms show that a lot of hackers struggle to find bugs. This report explores a way of using Markov decision processes and reinforcement learning to help hackers find vulnerabilities in web applications by building a tool that suggests attack surfaces to examine and vulnerability reports to read to get the relevant knowledge. The attack surfaces, vulnerabilities and reports are all derived from a taxonomy of web vulnerabilities created in a collaborating project. A Markov decision process (MDP) was defined, this MDP includes the environment, different states of knowledge and actions that can take a user from one state of knowledge to another. To be able to suggest the best possible next action to perform, the MDP uses a policy that describes the value of entering each state. Each state is given a value that is called Q-value. This value indicates how close that state is to another state where a vulnerability has been found. This means that a state has a high Q-value if the knowledge gives a user a high probability of finding a vulnerability and vice versa. This policy was created using a reinforcement learning algorithm called Q-learning. The tool was implemented as a web application using Java Spring Boot and ReactJS. The resulting tool is best suited for new hackers in the learning process. The current version is trained on the indexed reports of the vulnerability taxonomy but future versions should be trained on user behaviour collected from the tool. / Bug bounties är ett alltmer populärt sätt att utföra penetrationstester av webbapplikationer. Användarstatistik från bug bounty-plattformar visar att många hackare har svårt att hitta buggar. Denna rapport undersöker ett sätt att använda Markov-beslutsprocesser och förstärkt inlärning för att hjälpa hackare att hitta sårbarheter i webbapplikationer genom att bygga ett verktyg som föreslår attackytor att undersöka och sårbarhetsrapporter att läsa för att tillgodogöra sig rätt kunskaper. Attackytor, sårbarheter och rapporter är alla hämtade från en taxonomi över webbsårbarheter skapad i ett samarbetande projekt. En Markovbeslutsprocess (MDP) definierades. Denna MDP inkluderar miljön, olika kunskapstillstånd och handlingar som kan ta användaren från ett kunskapstillstånd till ett annat. För kunna föreslå nästa handling på bästa möjliga sätt använder MDPn en policy som beskriver värdet av att träda in i alla de olika tillstånden. Alla tillstånd ges ett värde som kallas Q-värde. Detta värde indikerar hur nära ett tillstånd har till ett annat tillstånd där en sårbarhet har hittats. Detta betyder att ett tillstånd har ett högt Q-värde om kunskapen ger användaren en hög sannolikhet att hitta en sårbarhet och vice versa. Policyn skapades med hjälp av en typ av förstärkt inlärningsalgoritm kallad Q-inlärning. Verktyget implementerades som en webbapplikation med hjälp av Java Spring Boot och ReactJS. Det resulterande verktyget är bäst lämpat för nya hackare i inlärningsstadiet. Den nuvarande versionen är tränad på indexerade rapporter från sårbarhetstaxonomin men framtida versioner bör tränas på användarbeteende insamlat från verktyget.

There is a growing number of applications of machine learning and deep learning in quantitative and computational finance. In this thesis, we focus on two of them. In the first application, we employ machine learning and deep learning in derivative pricing. The models considering jumps or stochastic volatility are more complicated than the Black-Merton-Scholes model and the derivatives under these models are harder to be priced. The traditional pricing methods are computationally intensive, so machine learning and deep learning are employed for fast pricing. I n Chapter 2, we propose a method for pricing American options under the variance gamma model. We develop a new fast and accurate approximation method inspired by the quadratic approximation to get rid of the time steps required in finite difference and simulation methods, while reducing the error by making use of a machine learning technique on pre-calculated quantities. We compare the performance of our method with those of the existing methods and show that this method is efficient and accurate for practical use. In Chapters 3 and 4, we propose unsupervised deep learning methods for option pricing under Lévy process and stochastic volatility respectively, with a special focus on barrier options in Chapter 4. The unsupervised deep learning approach employs a neural network as the candidate option surface and trains the neural network to satisfy certain equations. By matching the equation and the boundary conditions, the neural network would yield an accurate solution. Special structures called singular terms are added to the neural networks to deal with the non-smooth and discontinuous payoff at the strike and barrier levels so that the neural networks can replicate the asymptotic behaviors of options at short maturities. Unlike supervised learning, this approach does not require any labels. Once trained, the neural network solution yields fast and accurate option values. The second application focuses on financial time series simulation utilizing deep learning techniques. Simulation extends the limited real data for training and evaluation of trading strategies. It is challenging because of the complex statistical properties of the real financial data. In Chapter 5, we introduce two generative adversarial networks, which utilize the convolutional networks with attention and the transformers, for financial time series simulation. The networks learn the statistical properties in a data-driven manner and the attention mechanism helps to replicate the long-range dependencies. The proposed models are tested on the S&P 500 index and its option data, examined by scores based on the stylized facts and are compared with the pure convolutional network, i.e. QuantGAN. The attention-based networks not only reproduce the stylized facts, including heavy tails, autocorrelation and cross-correlation, but also smooth the autocorrelation of returns.