Opportunistic communication techniques have shown to provide significant performance improvements in centralised random access wireless networks. The key mechanism of opportunistic communication is to send back-to-back data packets whenever the channel quality is deemed "good". Recently there have been attempts to introduce opportunistic communication techniques in distributed wireless networks such as wireless ad hoc networks. In line of this research, we propose a new paradigm of medium access control, called Channel MAC based on the channel randomness and opportunistic communication principles. Scheduling in Channel MAC depends on the instance at which the channel quality improves beyond a threshold, while neighbouring nodes are deemed to be silent. Once a node starts transmitting, it will keep transmitting until the channel becomes "bad". We derive an analytical throughput equation of the proposed MAC in a multiple access environment and validate it by simulations. It is observed that Channel MAC outperforms IEEE 802.11 for all probabilities of good channel condition and all numbers of nodes. For higher number of nodes, Channel MAC achieves higher throughput at lower probabilities of good channel condition increasing the operating range. Furthermore, the total throughput of the network grows with increasing number of nodes considering negligible propagation delay in the network. A scalable channel prediction scheme is required to implement the practical Channel MAC protocol in practice. We propose a mean-value based channel prediction scheme, which provides prediction with enough accuracy to be used in the Channel MAC protocol. NS2 simulation result shows that the Channel MAC protocol outperforms the IEEE 802.11 in throughput due to its channel diversity mechanism in spite of the prediction errors and packet collisions. Next, we extend the Channel MAC protocol to support multi-rate communications. At present, two prominent multi-rate mechanisms, Opportunistic Auto Rate (OAR) and Receiver Based Auto Rate (RBAR) are unable to adapt to short term changes in channel conditions during transmission as well as to use optimum power and throughput during packet transmissions. On the other hand, using channel predictions, each source-destinations pair in Channel MAC can fully utilise the non-fade durations. We combine the scheduling of Channel MAC and the rate adaptive transmission based on the channel state information to design the 'Rate Adaptive Channel MAC' protocol. However, to implement the Rate adaptive Channel MAC, we need to use a channel prediction scheme to identify transmission opportunities as well as auto rate adaptation mechanism to select rates and number of packets to transmit during those times. For channel prediction, we apply the scheme proposed for the practical implementation of Channel MAC. We propose a "safety margin" based technique to provide auto rate adaptation. Simulation results show that a significant performance improvement can be achieved by Rate adaptive Channel MAC as compared to existing rate adaptive protocols such as OAR.

Opportunistic communication techniques have shown to provide significant performance improvements in centralised random access wireless networks. The key mechanism of opportunistic communication is to send back-to-back data packets whenever the channel quality is deemed "good". Recently there have been attempts to introduce opportunistic communication techniques in distributed wireless networks such as wireless ad hoc networks. In line of this research, we propose a new paradigm of medium access control, called Channel MAC based on the channel randomness and opportunistic communication principles. Scheduling in Channel MAC depends on the instance at which the channel quality improves beyond a threshold, while neighbouring nodes are deemed to be silent. Once a node starts transmitting, it will keep transmitting until the channel becomes "bad". We derive an analytical throughput equation of the proposed MAC in a multiple access environment and validate it by simulations. It is observed that Channel MAC outperforms IEEE 802.11 for all probabilities of good channel condition and all numbers of nodes. For higher number of nodes, Channel MAC achieves higher throughput at lower probabilities of good channel condition increasing the operating range. Furthermore, the total throughput of the network grows with increasing number of nodes considering negligible propagation delay in the network. A scalable channel prediction scheme is required to implement the practical Channel MAC protocol in practice. We propose a mean-value based channel prediction scheme, which provides prediction with enough accuracy to be used in the Channel MAC protocol. NS2 simulation result shows that the Channel MAC protocol outperforms the IEEE 802.11 in throughput due to its channel diversity mechanism in spite of the prediction errors and packet collisions. Next, we extend the Channel MAC protocol to support multi-rate communications. At present, two prominent multi-rate mechanisms, Opportunistic Auto Rate (OAR) and Receiver Based Auto Rate (RBAR) are unable to adapt to short term changes in channel conditions during transmission as well as to use optimum power and throughput during packet transmissions. On the other hand, using channel predictions, each source-destinations pair in Channel MAC can fully utilise the non-fade durations. We combine the scheduling of Channel MAC and the rate adaptive transmission based on the channel state information to design the 'Rate Adaptive Channel MAC' protocol. However, to implement the Rate adaptive Channel MAC, we need to use a channel prediction scheme to identify transmission opportunities as well as auto rate adaptation mechanism to select rates and number of packets to transmit during those times. For channel prediction, we apply the scheme proposed for the practical implementation of Channel MAC. We propose a "safety margin" based technique to provide auto rate adaptation. Simulation results show that a significant performance improvement can be achieved by Rate adaptive Channel MAC as compared to existing rate adaptive protocols such as OAR.

<p> The Bias-Variance Trade-off, where restricting the size of a hypothesis class can limit the generalization error of a model, is a canonical problem in Machine Learning, and a particular issue for high-variance models like Neural Networks that do not have enough parameters to enter the interpolating regime. Regularization techniques add bias to a model to lower testing error at the cost of increasing training error. This paper applies quantum circuits as activation functions in order to regularize a Feed-Forward Neural Network. The network using Quantum Activation Functions is compared against a network of the same dimensions except using Rectified Linear Unit (ReLU) activation functions, which can fit any arbitrary function. The Quantum Activation Function network is then shown to have comparable training performance to ReLU networks, both with and without regularization, for the tasks of binary classification, polynomial regression, and regression on a multicollinear dataset, which is a dataset whose design matrix is rank-deficient. The Quantum Activation Function network is shown to achieve regularization comparable to networks with L2-Regularization, the most commonly used method for neural network regularization today, with regularization parameters in the range of λ ∈ [.1, .5], while still allowing the model to maintain enough variance to achieve low training error. While there are limitations to the current physical implementation of quantum computers, there is potential for future architecture, or hardware-based, regularization methods that leverage the aspects of quantum circuits that provide lower generalization error. </p>

<p dir="ltr">New system-level codes are being developed for advanced reactors for safety analysis and licensing purposes. Thermal-hydraulics of advanced reactors is a challenging problem due to complex flow scenarios assisted by free jets and stratified flows that lead to turbulent mixing. For these reasons, the 0D or 1D models used for reactor plena in traditional safety analysis codes like RELAP cannot capture the physics accurately and introduce a large degree of modeling uncertainty. System-level calculation codes based on the advection-diffusion equation neglect turbulent fluctuations. These fluctuations are extremely important as they introduce higher-order moments, which are responsible for vortex stretching and the passage of energy to smaller scales. Alternatively, extremely detailed simulations with velocity coupling from the Navier-Stokes equations are able to capture turbulence effects accurately using DNS. These solutions are accurate because they resolve the flow into the smallest possible length and time scales (Kolmogorov scale) important to the flow, which makes DNS computationally expensive for simple geometries and impossible at the system level.</p><p dir="ltr">The flow field can be described through a reduced-order model using the principles of statistical mechanics. Statistical mechanics-based methods provide a method for extracting statistics from data and modeling that data using easily represented differential equations. The Kramers-Moyal (KM) expansion method can be used as a subgrid-scale (SGS) closure for solving the momentum equation. The stochastic Burgers equation is solved using DNS, and the DNS solutions are used to calculate the KM coefficients, which are then implemented as an SGS closure model. The KM method outperforms traditional methods in capturing the multi-scale behavior of Burgers turbulence. The functional dependencies of the KM coefficients are also uniform for several boundary conditions, meaning the closure model can be extended to multiple flow scenarios. </p><p dir="ltr">For the case of the Navier-Stokes equations, each particle trajectory tends to follow some scaling law. Kolmogorov hypothesized that the flow velocity field follows a -5/3 scaling in the inertial region where Markovian characteristics can be invoked to model the interaction between eddies of adjacent sizes. This law holds true in the inertial region where the flow is Markovian. For scalar turbulence, the scaling laws are affected by thermal diffusion. If a fluid has a Prandtl number close to one, the thermal behavior is dominated by momentum, so the spectra for velocity and temperature are similar. For small Prandtl number fluids, such as liquid metals, the thermal diffusion dominates the lower scales and the slope of the spectrum shifts from the -5/3 slope to a -3 slope, also called the Batchelor region. System-level thermal hydraulics codes need to be able to capture these behaviors for a range of Prandtl number fluids. The KM-based model can also be used as a surrogate for velocity or temperature fluctuations in scalar turbulence. Using DNS solutions for turbulent channel flow, the KM model is used to provide a surrogate for temperature and velocity signals at different wall locations in the channel for Pr = 0.004, Pr = 0.025, and Pr = 0.71. The KM surrogate matches well for all wall locations, but is not able to capture the viscous dissipation in the velocity signal, or the thermal dissipation in the low Prandtl number cases. The dissipation can be captured by implementing a Gaussian filter.</p><p dir="ltr">Statistical mechanics-based methods are not limited to modeling turbulence in a reactor. Renewable power generation, such as wind, can be modeled using the Ornstein-Uhlenbeck (OU) method, which allows the long-term trends and short-term fluctuations of wind power to be decoupled. This allows for large fluctuations in wind power to be scaled down to a level that a reactor can accommodate safely. </p><p dir="ltr">Since statistical mechanics methods are based in physics, the calculated coefficients provide some information about the inputted signal. In a high-temperature gas-cooled reactor, strong heating can cause flow that is expected to be turbulent to show laminar characteristics. This laminarization results in reduced heat removal. The KM coefficients can be used to classify the laminarization from probed velocity signals more effectively than traditional statistical analyses.</p>

<p dir="ltr">Statistical arbitrage is a prevalent trading strategy which takes advantage of mean reverse property of spreads constructed from pairs or portfolios of assets. Utilizing statistical models and algorithms, statistical arbitrage exploits and capitalizes on the pricing inefficiencies between securities or within asset portfolios. </p><p dir="ltr">In chapter 2, We propose a framework for constructing diversified portfolios with multiple pairs trading strategies. In our approach, several pairs of co-moving assets are traded simultaneously, and capital is dynamically allocated among different pairs based on the statistical characteristics of the historical spreads. This allows us to further consider various portfolio designs and rebalancing strategies. Working with empirical data, our experiments suggest the significant benefits of diversification within our proposed framework.</p><p dir="ltr">In chapter 3, we explore an optimal timing strategy for the trading of price spreads exhibiting mean-reverting characteristics. A sequential optimal stopping framework is formulated to analyze the optimal timings for both entering and subsequently liquidating positions, all while considering the impact of transaction costs. Then we leverages a refined signature optimal stopping method to resolve this sequential optimal stopping problem, thereby unveiling the precise entry and exit timings that maximize gains. Our framework operates without any predefined assumptions regarding the dynamics of the underlying mean-reverting spreads, offering adaptability to diverse scenarios. Numerical results are provided to demonstrate its superior performance when comparing with conventional mean reversion trading rules.</p><p dir="ltr">In chapter 4, we introduce an innovative model-free and reinforcement learning based framework for statistical arbitrage. For the construction of mean reversion spreads, we establish an empirical reversion time metric and optimize asset coefficients by minimizing this empirical mean reversion time. In the trading phase, we employ a reinforcement learning framework to identify the optimal mean reversion strategy. Diverging from traditional mean reversion strategies that primarily focus on price deviations from a long-term mean, our methodology creatively constructs the state space to encapsulate the recent trends in price movements. Additionally, the reward function is carefully tailored to reflect the unique characteristics of mean reversion trading.</p>

Analyzing the structure of a dataset is a challenging problem in high-dimensions as the volume of the space increases at an exponential rate and typically, data becomes sparse in this high-dimensional space. This poses a significant challenge to machine learning methods which rely on exploiting structures underlying data to make meaningful inferences. This dissertation proposes the <i>n</i>-TARP method as a building block for high-dimensional data analysis, in both supervised and unsupervised scenarios.<div><br></div><div>The basic element, <i>n</i>-TARP, consists of a random projection framework to transform high-dimensional data to one-dimensional data in a manner that yields point separations in the projected space. The point separation can be tuned to reflect classes in supervised scenarios and clusters in unsupervised scenarios. The <i>n</i>-TARP method finds linear separations in high-dimensional data. This basic unit can be used repeatedly to find a variety of structures. It can be arranged in a hierarchical structure like a tree, which increases the model complexity, flexibility and discriminating power. Feature space extensions combined with <i>n</i>-TARP can also be used to investigate non-linear separations in high-dimensional data.<br></div><div><br></div><div>The application of <i>n</i>-TARP to both supervised and unsupervised problems is investigated in this dissertation. In the supervised scenario, a sequence of <i>n</i>-TARP based classifiers with increasing complexity is considered. The point separations are measured by classification metrics like accuracy, Gini impurity or entropy. The performance of these classifiers on image classification tasks is studied. This study provides an interesting insight into the working of classification methods. The sequence of <i>n</i>-TARP classifiers yields benchmark curves that put in context the accuracy and complexity of other classification methods for a given dataset. The benchmark curves are parameterized by classification error and computational cost to define a benchmarking plane. This framework splits this plane into regions of "positive-gain" and "negative-gain" which provide context for the performance and effectiveness of other classification methods. The asymptotes of benchmark curves are shown to be optimal (i.e. at Bayes Error) in some cases (Theorem 2.5.2).<br></div><div><br></div><div>In the unsupervised scenario, the <i>n</i>-TARP method highlights the existence of many different clustering structures in a dataset. However, not all structures present are statistically meaningful. This issue is amplified when the dataset is small, as random events may yield sample sets that exhibit separations that are not present in the distribution of the data. Thus, statistical validation is an important step in data analysis, especially in high-dimensions. However, in order to statistically validate results, often an exponentially increasing number of data samples are required as the dimensions increase. The proposed <i>n</i>-TARP method circumvents this challenge by evaluating statistical significance in the one-dimensional space of data projections. The <i>n</i>-TARP framework also results in several different statistically valid instances of point separation into clusters, as opposed to a unique "best" separation, which leads to a distribution of clusters induced by the random projection process.<br></div><div><br></div><div>The distributions of clusters resulting from <i>n</i>-TARP are studied. This dissertation focuses on small sample high-dimensional problems. A large number of distinct clusters are found, which are statistically validated. The distribution of clusters is studied as the dimensionality of the problem evolves through the extension of the feature space using monomial terms of increasing degree in the original features, which corresponds to investigating non-linear point separations in the projection space.<br></div><div><br></div><div>A statistical framework is introduced to detect patterns of dependence between the clusters formed with the features (predictors) and a chosen outcome (response) in the data that is not used by the clustering method. This framework is designed to detect the existence of a relationship between the predictors and response. This framework can also serve as an alternative cluster validation tool.<br></div><div><br></div><div>The concepts and methods developed in this dissertation are applied to a real world data analysis problem in Engineering Education. Specifically, engineering students' Habits of Mind are analyzed. The data at hand is qualitative, in the form of text, equations and figures. To use the <i>n</i>-TARP based analysis method, the source data must be transformed into quantitative data (vectors). This is done by modeling it as a random process based on the theoretical framework defined by a rubric. Since the number of students is small, this problem falls into the small sample high-dimensions scenario. The <i>n</i>-TARP clustering method is used to find groups within this data in a statistically valid manner. The resulting clusters are analyzed in the context of education to determine what is represented by the identified clusters. The dependence of student performance indicators like the course grade on the clusters formed with <i>n</i>-TARP are studied in the pattern dependence framework, and the observed effect is statistically validated. The data obtained suggests the presence of a large variety of different patterns of Habits of Mind among students, many of which are associated with significant grade differences. In particular, the course grade is found to be dependent on at least two Habits of Mind: "computation and estimation" and "values and attitudes."<br></div>

<p>Gaussian Process Regression (GPR) is a widely used surrogate model in efficient global optimization (EGO) due to its capability to provide uncertainty estimates in the prediction. The cost of creating a GPR model for large data sets is high. On the other hand, neural network (NN) models scale better compared to GPR as the number of samples increase. Unfortunately, the uncertainty estimates for NN prediction are not readily available. In this work, a scalable algorithm is developed for EGO using NN-based prediction and uncertainty (EGONN). Initially, two different NNs are created using two different data sets. The first NN models the output based on the input values in the first data set while the second NN models the prediction error of the first NN using the second data set. The next infill point is added to the first data set based on criteria like expected improvement or prediction uncertainty. EGONN is demonstrated on the optimization of the Forrester function and a constrained Branin function and is compared with EGO. The convergence criteria is based on the maximum number of infill points in both cases. The algorithm is able to reach the optimum point within the given budget. The EGONN is extended to handle constraints explicitly and is utilized for aerodynamic shape optimization of the RAE 2822 airfoil in transonic viscous flow at a free-stream Mach number of 0.734 and a Reynolds number of 6.5 million. The results obtained from EGONN are compared with the results from gradient-based optimization (GBO) using adjoints. The optimum shape obtained from EGONN is comparable to the shape obtained from GBO and is able to eliminate the shock. The drag coefficient is reduced from 200 drag counts to 114 and is close to 110 drag counts obtained from GBO. The EGONN is also extended to handle uncertainty quantification (uqEGONN) using prediction uncertainty as an infill method. The convergence criteria is based on the relative change of summary statistics such as mean and standard deviation of an uncertain quantity. The uqEGONN is tested on Ishigami function with an initial sample size of 100 samples and the algorithm terminates after 70 infill points. The statistics obtained from uqEGONN (using only 170 function evaluations) are close to the values obtained from directly evaluating the function one million times. uqEGONN is demonstrated on to quantifying the uncertainty in the airfoil performance due to geometric variations. The algorithm terminates within 100 computational fluid dynamics (CFD) analyses and the statistics obtained from the algorithm are close to the one obtained from 1000 direct CFD based evaluations.</p>

<p>Uncertainty analysis is of great use both for calculating outputs that are more akin to real<br> flight, and for optimization to more robust shapes. However, implementation of uncertainty<br> has been a longstanding challenge in the field of aerodynamics due to the computational cost<br> of simulations. Geometric uncertainty in particular is often left unexplored in favor of uncer-<br> tainties in freestream parameters, turbulence models, or computational error. Therefore, this<br> work proposes a method of geometric uncertainty analysis for aerodynamic shapes that miti-<br> gates the barriers to its feasible computation. The process takes a two- or three-dimensional<br> shape and utilizes a combination of multifidelity meshes and Gaussian process regression<br> (GPR) surrogates in a multifidelity Monte Carlo (MFMC) algorithm. Multifidelity meshes<br> allow for finer sampling with a given budget, making the surrogates more accurate. GPR<br> surrogates are made practical to use by parameterizing major factors in geometric uncer-<br> tainty with only four variables in 2-D and five in 3-D. In both cases, two parameters control<br> the heights of steps that occur on the top and bottom of airfoils where leading and trailing<br> edge devices are attached. Two more parameters control the height and length of waves<br> that can occur in an ideally smooth shape during manufacturing. A fifth parameter controls<br> the depth of span-wise skin buckling waves along a 3-D wing. Parameters are defined to<br> be uniformly distributed with a maximum size of 0.4 mm and 0.15 mm for steps and waves<br> to remain within common manufacturing tolerances. The analysis chain is demonstrated<br> with two test cases. The first, the RAE2822 airfoil, uses transonic freestream parameters<br> set by the ADODG Benchmark Case 2. The results show a mean drag of nearly 10 counts<br> above the deterministic case with fixed lift, and a 2 count increase for a fixed angle of attack<br> version of the case. Each case also has small variations in lift and angle of attack of about<br> 0.5 counts and 0.08◦, respectively. Variances for each of the three tracked outputs show that<br> more variability is possible, and even likely. The ONERA M6 transonic wing, popular due<br> to the extensive experimental data available for computational validation, is the second test<br> case. Variation is found to be less substantial here, with a mean drag increase of 0.5 counts,<br> and a mean lift increase of 0.1 counts. Furthermore, the MFMC algorithm enables accurate<br> results with only a few hours of wall time in addition to GPR training. </p>

<div>The rise of artificial intelligence (AI) hinges on the efficient training of modern deep neural networks (DNNs) for non-convex optimization and uncertainty quantification, which boils down to a non-convex Bayesian learning problem. A standard tool to handle the problem is Langevin Monte Carlo, which proposes to approximate the posterior distribution with theoretical guarantees. However, non-convex Bayesian learning in real big data applications can be arbitrarily slow and often fails to capture the uncertainty or informative modes given a limited time. As a result, advanced techniques are still required.</div><div><br></div><div>In this thesis, we start with the replica exchange Langevin Monte Carlo (also known as parallel tempering), which is a Markov jump process that proposes appropriate swaps between exploration and exploitation to achieve accelerations. However, the na\"ive extension of swaps to big data problems leads to a large bias, and the bias-corrected swaps are required. Such a mechanism leads to few effective swaps and insignificant accelerations. To alleviate this issue, we first propose a control variates method to reduce the variance of noisy energy estimators and show a potential to accelerate the exponential convergence. We also present the population-chain replica exchange and propose a generalized deterministic even-odd scheme to track the non-reversibility and obtain an optimal round trip rate. Further approximations are conducted based on stochastic gradient descents, which yield a user-friendly nature for large-scale uncertainty approximation tasks without much tuning costs. </div><div><br></div><div>In the second part of the thesis, we study scalable dynamic importance sampling algorithms based on stochastic approximation. Traditional dynamic importance sampling algorithms have achieved successes in bioinformatics and statistical physics, however, the lack of scalability has greatly limited their extensions to big data applications. To handle this scalability issue, we resolve the vanishing gradient problem and propose two dynamic importance sampling algorithms based on stochastic gradient Langevin dynamics. Theoretically, we establish the stability condition for the underlying ordinary differential equation (ODE) system and guarantee the asymptotic convergence of the latent variable to the desired fixed point. Interestingly, such a result still holds given non-convex energy landscapes. In addition, we also propose a pleasingly parallel version of such algorithms with interacting latent variables. We show that the interacting algorithm can be theoretically more efficient than the single-chain alternative with an equivalent computational budget.</div>