University of Waterloo
Enhancing Machine Learning Algorithms Using Low-Discrepancy Sequences
Abstract
dc:description.abstractThis thesis investigates the application of randomised quasi-Monte Carlo (RQMC) methods in settings where the smoothness and structural conditions underpinning classical RQMC theory are rarely satisfied. The applications studied in this thesis form a progression that stretches the notion of what RQMC can do. To start, Adaptive Quasi-Regression (AQR) is introduced to improve the efficiency of Quasi-Regression (QR) for black-box model interpretability in high-dimensional settings. By exploiting hierarchical structure and adaptive cut-off procedures, AQR selects only the coefficients deemed important, substantially reducing the runtime of fitting QR models using RQMC point sets. In a similar vein, we treat RQMC as a tool for variance reduction in numerical integration, but apply it to copula sampling, where tractable inverse transforms are generally unavailable. We introduce the novel idea of Normalising Flows (NFs) to model target copula distributions, enabling the generation of RQMC samples where it was previously infeasible. NFs are shown to learn faster and approximate target copulas more accurately than previous generative approaches. Adaptive importance sampling methods are further introduced to correct for the bias inherent in generative model approximations, recovering accuracy at minimal cost to variance. We break from the traditional application of RQMC by extending it beyond the continuous variable paradigm to the discrete setting, enabling RQMC sampling for Bernoulli random variables. Novel methods combining RQMC with an adaptive sampling framework are introduced, negatively correlating successive samples to reduce variance in gradient estimation for combinatorial optimisation. The RQMC variance-reduction intuition is shown to carry over to discrete variables, with a discrete variational autoencoder converging in fewer training steps to a better model fit than its MC counterpart. In reinforcement learning, Hybrid QMC and Latin Supercube Sampling strategies achieve higher average rewards in fewer training steps than MC sampling. Finally, departing entirely from the integration perspective, the geometric space-filling properties of low-discrepancy point sets are exploited to define a superior pixel-ordering strategy for autoregressive image models. A novel extension, ConvNADE-Beta-Colour, is introduced to model continuous images efficiently using Beta distributions. Conditioning on pixels selected via Sobol' point sets, rather than random orderings, yields measurably more accurate image reconstructions. Collectively, these contributions demonstrate that the value of low-discrepancy point sets extends well beyond classical integration error bounds, with meaningful gains achievable across interpretability, generative modelling, discrete optimisation, and image reconstruction.
Degree
thesis:*- Grantor dc:publisher
- University of Waterloo
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Emmett-Iwaniw, Ambrose
Rights
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/10012/23938