<p dir="ltr">Deep neural networks (DNNs) have demonstrated outstanding performance in numerous tasks such as image recognition and natural language processing. However, in dynamic systems modeling, the tasks of estimating and uncovering the potentially nonlinear structure of systems represented by ordinary differential equations (ODEs) pose a significant challenge. In this dissertation, we employ DNNs to enable precise and efficient parameter estimation of dynamic systems. In addition, we introduce a highly flexible neural ODE model to capture both nonlinear and sparse dependent relations among multiple functional processes. Nonetheless, DNNs are susceptible to overfitting and often struggle to accurately assess predictive uncertainty despite their widespread success across various AI domains. The challenge of defining meaningful priors for DNN weights and characterizing predictive uncertainty persists. In this dissertation, we present a novel neural adaptive empirical Bayes framework with a new class of prior distributions to address weight uncertainty.</p><p dir="ltr">In the first part, we propose a precise and efficient approach utilizing DNNs for estimation and inference of ODEs given noisy data. The DNNs are employed directly as a nonparametric proxy for the true solution of the ODEs, eliminating the need for numerical integration and resulting in significant computational time savings. We develop a gradient descent algorithm to estimate both the DNNs solution and the parameters of the ODEs by optimizing a fidelity-penalized likelihood loss function. This ensures that the derivatives of the DNNs estimator conform to the system of ODEs. Our method is particularly effective in scenarios where only a set of variables transformed from the system components by a given function are observed. We establish the convergence rate of the DNNs estimator and demonstrate that the derivatives of the DNNs solution asymptotically satisfy the ODEs determined by the inferred parameters. Simulations and real data analysis of COVID-19 daily cases are conducted to show the superior performance of our method in terms of accuracy of parameter estimates and system recovery, and computational speed.</p><p dir="ltr">In the second part, we present a novel sparse neural ODE model to characterize flexible relations among multiple functional processes. This model represents the latent states of the functions using a set of ODEs and models the dynamic changes of these states utilizing a DNN with a specially designed architecture and sparsity-inducing regularization. Our new model is able to capture both nonlinear and sparse dependent relations among multivariate functions. We develop an efficient optimization algorithm to estimate the unknown weights for the DNN under the sparsity constraint. Furthermore, we establish both algorithmic convergence and selection consistency, providing theoretical guarantees for the proposed method. We illustrate the efficacy of the method through simulation studies and a gene regulatory network example.</p><p dir="ltr">In the third part, we introduce a class of implicit generative priors to facilitate Bayesian modeling and inference. These priors are derived through a nonlinear transformation of a known low-dimensional distribution, allowing us to handle complex data distributions and capture the underlying manifold structure effectively. Our framework combines variational inference with a gradient ascent algorithm, which serves to select the hyperparameters and approximate the posterior distribution. Theoretical justification is established through both the posterior and classification consistency. We demonstrate the practical applications of our framework through extensive simulation examples and real-world datasets. Our experimental results highlight the superiority of our proposed framework over existing methods, such as sparse variational Bayesian and generative models, in terms of prediction accuracy and uncertainty quantification.</p>

In this thesis we construct a new model of the immune response to the growth of breast cancer cells and investigate the impact of certain drug therapies on the cancer. We use delay differential equations to model the interaction of breast cancer cells with the immune system. The new model is constructed by combining two previous models. The first model accounts for different cell cycles and includes terms to evaluate drug treatments, but ignores quiescent tumor cells. The second model includes quiescent cells, but ignores the immune response and drug treatments. The new model is obtained by combining and modifying these two models to account for quiescent cells, immune cells and includes drug intervention terms. This new model is used to evaluate the effects of pulsed applications of the drug Paclitaxel for models with and without quiescent cells. We use sensitivity equation methods to analyze the sensitivity of the model with respect to the initial number of immune cytotoxic T-cells. Numerical experiments are conducted to compare the model predictions to observed behavior. / Master of Science

Infectious diseases caused by bacteria, fungi, viruses and parasites have affected humans historically. Infectious diseases remain a major cause of premature death and a public health concern globally with increased mortality and significant economic burden. Unvaccinated individuals, people with suppressed and compromised immune systems are at higher risk of suffering from infectious diseases. In spite of significant advancements in infectious diseases research, the control or treatment process faces challenges. The mucosal immune system plays a crucial role in safeguarding the body from harmful pathogens, while being constantly exposed to the environment. To develop treatment options for infectious diseases, it is vital to understand the immune responses that occur during infection. The two infectious diseases presented here are: i) Helicobacter pylori infection and ii) human immunodeficiency (HIV) and human papillomavirus (HPV) co-infection. H pylori, is a bacterium that colonizes the stomach and causes gastric cancer in 1-2% but is beneficial for protection against allergies and gastroesophageal diseases. An estimated 85% of H pylori colonized individuals show no detrimental effects. HIV is a virus that causes AIDS, one of the deadliest and most persistent epidemics. HIV-infected patients are at an increased risk of co-infection with HPV, and report an increased incidence of oral cancer. The goal of this thesis is to elucidate the host immune responses in infectious diseases via the use of computational and mathematical models. First, the thesis reviews the need for computational and mathematical models to study the immune responses in the course of infectious diseases. Second, it presents a novel sensitivity analysis method that identifies important parameters in a hybrid (agent-based/equation-based) model of H. pylori infection. Third, it introduces a novel model representing the HIV/HPV coinfection and compares the simulation results with a clinical study. Fourth, it discusses the need of advanced modeling technologies to achieve a personalized systems wide approach and the challenges that can be encountered in the process. Taken together, the work in this dissertation presents modeling approaches that could lead to the identification of host immune factors in infectious diseases in a predictive and more resource-efficient manner. / Doctor of Philosophy / Infectious diseases caused by bacteria, fungi, viruses and parasites have affected humans historically. Infectious diseases remain a major cause of premature death and a public health concern globally with increased mortality and significant economic burden. These infections can occur either via air, travel to at-risk places, direct person-to-person contact with an infected individual or through water or fecal route. Unvaccinated individuals, individuals with suppressed and compromised immune system such as that in HIV carriers are at higher risk of getting infectious diseases. In spite of significant advancements in infectious diseases research, the control and treatment of these diseases faces numerous challenges. The mucosal immune system plays a crucial role in safeguarding the body from harmful pathogens, while being exposed to the environment, mainly food antigens. To develop treatment options for infectious diseases, it is vital to understand the immune responses that occur during infection. In this work, we focus on gut immune system that acts like an ecosystem comprising of trillions of interacting cells and molecules, including membars of the microbiome. The goal of this dissertation is to develop computational models that can simulate host immune responses in two infectious diseases- i) Helicobacter pylori infection and ii) human immunodeficiency virus (HIV)-human papilloma virus (HPV) co-infection. Firstly, it reviews the various mathematical techniques and systems biology based methods. Second, it introduces a "hybrid" model that combines different mathematical and statistical approaches to study H. pylori infection. Third, it highlights the development of a novel HIV/HPV coinfection model and compares the results from a clinical trial study. Fourth, it discusses the challenges that can be encountered in adapting machine learning based computational technologies. Taken together, the work in this dissertation presents modeling approaches that could lead to the identification of host immune factors in infectious diseases in a predictive and more resourceful way.

Interval analysis is an essential tool in the construction of validated numerical solutions of Initial Value Problems (IVP) for Ordinary (ODE) and Partial (PDE) Differential Equations. A validated solution typically consists of guaranteed lower and upper bounds for the exact solution or set of exact solutions in the case of uncertain data, i.e. it is an interval function (enclosure) containing all solutions of the problem. IVP for ODE: The central point of discussion is the wrapping effect. A new concept of wrapping function is introduced and applied in studying this effect. It is proved that the wrapping function is the limit of the enclosures produced by any method of certain type (propagate and wrap type). Then, the wrapping effect can be quantified as the difference between the wrapping function and the optimal interval enclosure of the solution set (or some norm of it). The problems with no wrapping effect are characterized as problems for which the wrapping function equals the optimal interval enclosure. A sufficient condition for no wrapping effect is that there exist a linear transformation, preserving the intervals, which reduces the right-hand side of the system of ODE to a quasi-isotone function. This condition is also necessary for linear problems and "near" necessary in the general case. Hyperbolic PDE: The Initial Value Problem with periodic boundary conditions for the wave equation is considered. It is proved that under certain conditions the problem is an operator equation with an operator of monotone type. Using the established monotone properties, an interval (validated) method for numerical solution of the problem is proposed. The solution is obtained step by step in the time dimension as a Fourier series of the space variable and a polynomial of the time variable. The numerical implementation involves computations in Fourier and Taylor functoids. Propagation of discontinuo~swaves is a serious problem when a Fourier series is used (Gibbs phenomenon, etc.). We propose the combined use of periodic splines and Fourier series for representing discontinuous functions and a method for propagating discontinuous waves. The numerical implementation involves computations in a Fourier hyper functoid. / Mathematical Sciences / D. Phil. (Mathematics)

Mathematical models have been used to capture the behaviour of biological systems, from low-level biochemical reactions to multi-scale whole-organ models. Models are typically based on experimentally-derived data, attempting to reproduce the observed behaviour through mathematical constructs, e.g. using Ordinary Differential Equations (ODEs) for spatially-homogeneous systems. These models are developed and published as mathematical equations, yet are of such complexity that they necessitate computational simulation. This computational model development is often performed in an ad hoc fashion by modellers who lack extensive software engineering experience, resulting in brittle, inefficient model code that is hard to extend and reuse. Several Domain Specific Languages (DSLs) exist to aid capturing such biological models, including CellML and SBML; however these DSLs are designed to facilitate model curation rather than simplify model development. We present research into the application of techniques from software engineering to this domain; starting with the design, development and implementation of a DSL, termed Ode, to aid the creation of ODE-based biological models. This introduces features beneficial to model development, such as model verification and reproducible results. We compare and contrast model development to large-scale software development, focussing on extensibility and reuse. This work results in a module system that enables the independent construction and combination of model components. We further investigate the use of software engineering processes and patterns to develop complex modular cardiac models. Model simulation is increasingly computationally demanding, thus models are often created in complex low-level languages such as C/C++. We introduce a highly-efficient, optimising native-code compiler for Ode that generates custom, model-specific simulation code and allows use of our structured modelling features without degrading performance. Finally, in certain contexts the stochastic nature of biological systems becomes relevant. We introduce stochastic constructs to the Ode DSL that enable models to use Stochastic Differential Equations (SDEs), the Stochastic Simulation Algorithm (SSA), and hybrid methods. These use our native-code implementation and demonstrate highly-efficient stochastic simulation, beneficial as stochastic simulation is highly computationally intensive. We introduce a further DSL to model ion channels declaratively, demonstrating the benefits of DSLs in the biological domain. This thesis demonstrates the application of software engineering methodologies, and in particular DSLs, to facilitate the development of both deterministic and stochastic biological models. We demonstrate their benefits with several features that enable the construction of large-scale, reusable and extensible models. This is accomplished whilst providing efficient simulation, creating new opportunities for biological model development, investigation and experimentation.

When traditional linearised theory is used to study free-surface flows past a surface-piercing object or over an obstruction in a stream, the geometry of the object is usually lost, having been assumed small in one or several of its dimensions. In order to preserve the nonlinear nature of the geometry, asymptotic expansions in the low-Froude or low-Bond limits can be derived, but here, the solution invariably predicts a waveless free-surface at every order. This is because the waves are in fact, exponentially small, and thus beyond-all-orders of regular asymptotics; their formation is a consequence of the divergence of the asymptotic series and the associated Stokes Phenomenon. In this thesis, we will apply exponential asymptotics to the study of two new problems involving nonlinear geometries. In the first, we examine the case of free-surface flow over a step including the effects of both gravity and surface tension. Here, we shall see that the availability of multiple singularities in the geometry, coupled with the interplay of gravitational and cohesive effects, leads to the discovery of a remarkable new set of solutions. In the second problem, we study the waves produced by bluff-bodied ships in low-Froude flows. We will derive the analytical form of the exponentially small waves for a wide range of hull geometries, including single-cornered and multi-cornered ships, and then provide comparisons with numerical computations. A particularly significant result is our confirmation of the thirty-year old conjecture by Vanden-Broeck & Tuck (1977) regarding the impossibility of waveless single-cornered ships.

The performance modelling of large-scale systems using discrete-state approaches is fundamentally hampered by the well-known problem of state-space explosion, which causes exponential growth of the reachable state space as a function of the number of the components which constitute the model. Because they are mapped onto continuous-time Markov chains (CTMCs), models described in the stochastic process algebra PEPA are no exception. This thesis presents a deterministic continuous-state semantics of PEPA which employs ordinary differential equations (ODEs) as the underlying mathematics for the performance evaluation. This is suitable for models consisting of large numbers of replicated components, as the ODE problem size is insensitive to the actual population levels of the system under study. Furthermore, the ODE is given an interpretation as the fluid limit of a properly defined CTMC model when the initial population levels go to infinity. This framework allows the use of existing results which give error bounds to assess the quality of the differential approximation. The computation of performance indices such as throughput, utilisation, and average response time are interpreted deterministically as functions of the ODE solution and are related to corresponding reward structures in the Markovian setting. The differential interpretation of PEPA provides a framework that is conceptually analogous to established approximation methods in queueing networks based on meanvalue analysis, as both approaches aim at reducing the computational cost of the analysis by providing estimates for the expected values of the performance metrics of interest. The relationship between these two techniques is examined in more detail in a comparison between PEPA and the Layered Queueing Network (LQN) model. General patterns of translation of LQN elements into corresponding PEPA components are applied to a substantial case study of a distributed computer system. This model is analysed using stochastic simulation to gauge the soundness of the translation. Furthermore, it is subjected to a series of numerical tests to compare execution runtimes and accuracy of the PEPA differential analysis against the LQN mean-value approximation method. Finally, this thesis discusses the major elements concerning the development of a software toolkit, the PEPA Eclipse Plug-in, which offers a comprehensive modelling environment for PEPA, including modules for static analysis, explicit state-space exploration, numerical solution of the steady-state equilibrium of the Markov chain, stochastic simulation, the differential analysis approach herein presented, and a graphical framework for model editing and visualisation of performance evaluation results.

Capillaries in tumours are often severely buckled (in a plane perpendicular to the axis) and / or chaotic in their direction. We develop a model of these phenomena using nonlinear solid mechanics. Our model focusses on the immediate surrounding of a capillary. The vessel and surrounding tissue are modelled as concentric annulii. The growth is dependent on the concentration of a nutrient (oxygen) diffusing from the vessel into the tumour interstitium. The stress is modelled using a multiplicative decomposition of the deformation gradient F=F_e F_g. The stress is determined by substituting the elastic deformation gradient F_e (which gives the deformation gradient from the hypothetical configuration to the current configuration) into a hyperelastic constitutive model as per classical solid mechanics. We use a Blatz-Ko model, parameterised using uniaxial compression experiments. The entire system is in quasi-static equilibrium, with the divergence of the stress tensor equal to zero. We determine the onset of buckling using a linear stability analysis. We then investigate the postbuckling behaviour by introducing higher order perturbations in the deformation and growth before using the Fredholm Alternative to obtain the magnitude of the buckle. Our results demonstrate that the growth-induced stresses are sufficient for the capillary to buckle in the absence of external loading and / or constraints. Planar buckling usually occurs after 2-5 times the cellular proliferation timescale. Buckles with axial variation almost always go unstable after planar buckles. Buckles of fine wavelength are initially preferred by the system, but over time buckles of large wavelength become energetically more favourable. The tumoural hoop stress T_{ThetaTheta} is the most invariant (Eulerian) variable at the time of buckling: it is typically of the order of the tumoural Young's Modulus when this occurs.

Swarms of animals exhibit aggregations whose behavior is a challenge for mathematicians to understand. We analyze this behavior numerically and analytically by using the pairwise interaction model known as the Morse potential. Our goal is to prove the global stability of the candidate local minimizer in 1D found in A Primer of Swarm Equilibria. Using the calculus of variations and eigenvalues analysis, we conclude that the candidate local minimizer is a global minimum with respect to all solution smaller than its support. In addition, we manage to extend the global stability condition to any solutions whose support has a single component. We are still examining the local minimizers with multiple components to determine whether the candidate solution is the minimum-energy configuration.

Mathematical models apply to a multitude physiological processes and are used to make predictions and analyze outcomes of these processes. Specifically, in the medical field, a mathematical model uses a set of initial conditions that represents a physiological state as input and a set of parameter values are used to describe the interaction between variables being modeled. These models are used to analyze possible outcomes, and assist physicians in choosing the most appropriate treatment options for a particular situation. We aim to use mathematical modeling to analyze the dynamics of processes involved in the inflammatory process. First, we create a model of hematopoiesis, the processes of creating new blood cells. We analyze stem cell collection regimens and statistically sample parameter space in order to create a model accounts for the dynamics of multiple patients. Next, we modify an existing model of the wound healing response by introducing a variable for two inflammatory cell types. We analyze the timing of the inflammatory response and introduce the presence of systemic estrogen in the model, as there is evidence that the presence of estrogen leads to a more efficient wound healing response. Last, we mathematically model the gas exchange process in the lungs and body in order to lay the foundation for a model of the inflammatory response in the lung under conditions of mechanical ventilation. We introduce normal and ventilation breathing waveforms and a third state of hemoglobin in a closed loop partial differential equations model. We account for gas exchange in the lung and body compartments in addition to introducing a third discretized well-mixing compartment between the two. We use ordinary and partial differential equations to model these systems over one or more independent variables, as well as classical analysis techniques and computational methods to analyze systems. Statistical sampling is also used to investigate parameter values in order for the mathematical models developed to account for patient-to-patient variability. This alters the traditional mathematical model, which yields a single set of parameter values that represent one instance of the physiology, into a mathematical model that accounts for many different instances of physiology.}