Back to search

University of Waterloo

Symbolic Regression and Sequence Modelling with Conditional and Dynamic Language Models

Abstract

dc:description.abstract

In an era where the boundaries of machine learning are continuously being pushed, this thesis presents two more advancements in the field of deep learning and artificial intelligence, with a focus on symbolic regression and dynamic training methodologies for neural networks. The first major contribution, SymbolicGPT, introduces a novel approach to symbolic regression using a transformer-based language model. This model significantly outperforms traditional methods by leveraging the strengths of probabilistic language models for improved accuracy and efficiency. The second theme of this thesis revolves around dynamic training methodologies, aimed at enhancing the adaptability and computational efficiency of neural networks under varying constraints. Within this framework, we introduce DyLoRA and SortedNet as key innovations. DyLoRA offers a dynamic, search-free low-rank adaptation technique, enabling models to adjust their complexity on-the-fly without extensive retraining. SortedNet proposes a generalized framework for embedding multiple neural network architectures within a single model, facilitating efficient model selection and adaptation. Extending SortedNet, SortedLLama applies these principles to large language models, demonstrating efficient dynamic inference capabilities.

Degree

thesis:*
Grantor dc:publisher
University of Waterloo
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Valipour, Mojtaba
Advisor dc:contributor.advisor
  • Ghodsi, Ali

Subjects

dc:subject × 8

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/10012/20630

Chain of custody

source
Harvested from
University of Waterloo
Base URL
dspacemainprd01.lib.uwaterloo.ca/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Valipour, Mojtaba. Symbolic Regression and Sequence Modelling with Conditional and Dynamic Language Models. University of Waterloo, 2024. http://hdl.handle.net/10012/20630