Spatiotemporal Predictive Learning (SPL) is an essential research topic involving many practical and real-world applications, e.g., motion detection, video generation, precipitation forecasting, and traffic flow prediction. The problems and challenges of this field come from numerous data characteristics in both time and space domains, and they vary depending on the specific task. For instance, spatial analysis refers to the study of spatial features, such as spatial location, latitude, elevation, longitude, the shape of objects, and other patterns. From the time domain perspective, the temporal analysis generally illustrates the time steps and time intervals of data points in the sequence, also known as interval recording or time sampling. Typically, there are two types of time sampling in temporal analysis: regular time sampling (i.e., the time interval is assumed to be fixed) and the irregular time sampling (i.e., the time interval is considered arbitrary) related closely to the continuous-time prediction task when data are in continuous space. Therefore, an efficient spatiotemporal predictive method has to model spatial features properly at the given time sampling types. In this thesis, by taking advantage of Machine Learning (ML) and Deep Learning (DL) methods, which have achieved promising performance in many complicated computational tasks, we propose three DL-based models used for Spatiotemporal Sequence Prediction (SSP) with several types of time sampling. First, we design the Trajectory Gated Recurrent Unit Attention (TrajGRU-Attention) with novel attention mechanisms, namely Motion-based Attention (MA), to improve the performance of the standard Convolutional Recurrent Neural Networks (ConvRNNs) in the SSP tasks. In particular, the TrajGRU-Attention model can alleviate the impact of the vanishing gradient, which leads to the blurry effect in the long-term predictions and handle both regularly sampled and irregularly sampled time series. Consequently, this model can work effectively with different scenarios of spatiotemporal sequential data, especially in the case of time series with missing time steps. Second, by taking the idea of Neural Ordinary Differential Equations (NODEs), we propose Trajectory Gated Recurrent Unit integrating Ordinary Differential Equation techniques (TrajGRU-ODE) as a continuous time-series model. With Ordinary Differential Equation (ODE) techniques and the TrajGRU neural network, this model can perform continuous-time spatiotemporal prediction tasks and generate resulting output with high accuracy. Compared to TrajGRU-Attention, TrajGRU-ODE benefits from the development of efficient and accurate ODE solvers. Ultimately, we attempt to combine those two models to create TrajGRU-Attention-ODE. NODEs are still in their early stage of research, and recent ODE-based models were designed for many relatively simple tasks. In this thesis, we will train the models with several video datasets to verify the ability of the proposed models in practical applications. To evaluate the performance of the proposed models, we select four available spatiotemporal datasets based on the complexity level, including the MovingMNIST, MovingMNIST++, and two real-life datasets: the weather radar HKO-7 and KTH Action. With each dataset, we train, validate, and test with distinct types of time sampling to justify the prediction ability of our models. In summary, the experimental results on the four datasets indicate the proposed models can generate predictions properly with high accuracy and sharpness. Significantly, the proposed models outperform state-of-the-art ODE-based approaches under SSP tasks with different circumstances of interval recording.

Immunotherapy, by enhancing the endogenous anti-tumor immune responses, is showing promising results for the treatment of numerous cancers refractory to conventional therapies. However, its effectiveness for advanced castration-resistant prostate cancer remains unsatisfactory and new therapeutic strategies need to be developed. To this end, mathematical modeling provides a quantitative framework for testing in silico the efficacy of new treatments and combination therapies, as well as understanding unknown biological mechanisms. In this dissertation we present two mathematical models of prostate cancer immunotherapy defined as systems of ordinary differential equations. The first work, introduced in Chapter 2, provides a mathematical model of prostate cancer immunotherapy which has been calibrated using data from pre-clinical experiments in mice. This model describes the evolution of prostate cancer, key components of the immune system, and seven treatments. Numerous combination therapies were evaluated considering both the degree of tumor inhibition and the predicted synergistic effects, integrated into a decision tree. Our simulations predicted cancer vaccine combined with immune checkpoint blockade as the most effective dual-drug combination immunotherapy for subjects treated with androgen-deprivation therapy that developed resistance. Overall, this model serves as a computational framework to support drug development, by generating hypotheses that can be tested experimentally in pre-clinical models. The Chapter 3 is devoted to the description of a human prostate cancer mathematical model. The potential effect of immunotherapies on castration-resistant form has been analyzed. In particular, the model includes the dendritic vaccine sipuleucel-T, the only currently available immunotherapy option for advanced prostate cancer, and the ipilimumab, a drug targeting the cytotoxic T-lymphocyte antigen 4 , exposed on the CTLs membrane, currently under Phase II clinical trial. From a mathematical analysis of a simplified model, it seems likely that, under continuous administration of ipilimumab, the system lies in a bistable situation where both the no-tumor equilibrium and the high-tumor equilibrium are attractive. The schedule of periodic treatments could then determine the outcome, and mathematical models could help in deciding an optimal schedule.

<p>We study bifurcations of periodic travelling waves in granular dimer chains from the anti-continuum limit, when the mass ratio between the light and heavy beads tends to zero. We show that every limiting periodic wave is uniquely continued with respect to the mass ratio parameter and the periodic waves with the wavelength larger than a certain critical value are spectrally stable. Numerical computations are developed to study how this solution family is continued to the limit of equal mass ratio between the beads, where periodic travelling waves of granular monomer chains exist.</p> / Master of Science (MSc)

<p dir="ltr">Deep neural networks (DNNs) have demonstrated outstanding performance in numerous tasks such as image recognition and natural language processing. However, in dynamic systems modeling, the tasks of estimating and uncovering the potentially nonlinear structure of systems represented by ordinary differential equations (ODEs) pose a significant challenge. In this dissertation, we employ DNNs to enable precise and efficient parameter estimation of dynamic systems. In addition, we introduce a highly flexible neural ODE model to capture both nonlinear and sparse dependent relations among multiple functional processes. Nonetheless, DNNs are susceptible to overfitting and often struggle to accurately assess predictive uncertainty despite their widespread success across various AI domains. The challenge of defining meaningful priors for DNN weights and characterizing predictive uncertainty persists. In this dissertation, we present a novel neural adaptive empirical Bayes framework with a new class of prior distributions to address weight uncertainty.</p><p dir="ltr">In the first part, we propose a precise and efficient approach utilizing DNNs for estimation and inference of ODEs given noisy data. The DNNs are employed directly as a nonparametric proxy for the true solution of the ODEs, eliminating the need for numerical integration and resulting in significant computational time savings. We develop a gradient descent algorithm to estimate both the DNNs solution and the parameters of the ODEs by optimizing a fidelity-penalized likelihood loss function. This ensures that the derivatives of the DNNs estimator conform to the system of ODEs. Our method is particularly effective in scenarios where only a set of variables transformed from the system components by a given function are observed. We establish the convergence rate of the DNNs estimator and demonstrate that the derivatives of the DNNs solution asymptotically satisfy the ODEs determined by the inferred parameters. Simulations and real data analysis of COVID-19 daily cases are conducted to show the superior performance of our method in terms of accuracy of parameter estimates and system recovery, and computational speed.</p><p dir="ltr">In the second part, we present a novel sparse neural ODE model to characterize flexible relations among multiple functional processes. This model represents the latent states of the functions using a set of ODEs and models the dynamic changes of these states utilizing a DNN with a specially designed architecture and sparsity-inducing regularization. Our new model is able to capture both nonlinear and sparse dependent relations among multivariate functions. We develop an efficient optimization algorithm to estimate the unknown weights for the DNN under the sparsity constraint. Furthermore, we establish both algorithmic convergence and selection consistency, providing theoretical guarantees for the proposed method. We illustrate the efficacy of the method through simulation studies and a gene regulatory network example.</p><p dir="ltr">In the third part, we introduce a class of implicit generative priors to facilitate Bayesian modeling and inference. These priors are derived through a nonlinear transformation of a known low-dimensional distribution, allowing us to handle complex data distributions and capture the underlying manifold structure effectively. Our framework combines variational inference with a gradient ascent algorithm, which serves to select the hyperparameters and approximate the posterior distribution. Theoretical justification is established through both the posterior and classification consistency. We demonstrate the practical applications of our framework through extensive simulation examples and real-world datasets. Our experimental results highlight the superiority of our proposed framework over existing methods, such as sparse variational Bayesian and generative models, in terms of prediction accuracy and uncertainty quantification.</p>

Infectious diseases caused by bacteria, fungi, viruses and parasites have affected humans historically. Infectious diseases remain a major cause of premature death and a public health concern globally with increased mortality and significant economic burden. Unvaccinated individuals, people with suppressed and compromised immune systems are at higher risk of suffering from infectious diseases. In spite of significant advancements in infectious diseases research, the control or treatment process faces challenges. The mucosal immune system plays a crucial role in safeguarding the body from harmful pathogens, while being constantly exposed to the environment. To develop treatment options for infectious diseases, it is vital to understand the immune responses that occur during infection. The two infectious diseases presented here are: i) Helicobacter pylori infection and ii) human immunodeficiency (HIV) and human papillomavirus (HPV) co-infection. H pylori, is a bacterium that colonizes the stomach and causes gastric cancer in 1-2% but is beneficial for protection against allergies and gastroesophageal diseases. An estimated 85% of H pylori colonized individuals show no detrimental effects. HIV is a virus that causes AIDS, one of the deadliest and most persistent epidemics. HIV-infected patients are at an increased risk of co-infection with HPV, and report an increased incidence of oral cancer. The goal of this thesis is to elucidate the host immune responses in infectious diseases via the use of computational and mathematical models. First, the thesis reviews the need for computational and mathematical models to study the immune responses in the course of infectious diseases. Second, it presents a novel sensitivity analysis method that identifies important parameters in a hybrid (agent-based/equation-based) model of H. pylori infection. Third, it introduces a novel model representing the HIV/HPV coinfection and compares the simulation results with a clinical study. Fourth, it discusses the need of advanced modeling technologies to achieve a personalized systems wide approach and the challenges that can be encountered in the process. Taken together, the work in this dissertation presents modeling approaches that could lead to the identification of host immune factors in infectious diseases in a predictive and more resource-efficient manner. / Doctor of Philosophy / Infectious diseases caused by bacteria, fungi, viruses and parasites have affected humans historically. Infectious diseases remain a major cause of premature death and a public health concern globally with increased mortality and significant economic burden. These infections can occur either via air, travel to at-risk places, direct person-to-person contact with an infected individual or through water or fecal route. Unvaccinated individuals, individuals with suppressed and compromised immune system such as that in HIV carriers are at higher risk of getting infectious diseases. In spite of significant advancements in infectious diseases research, the control and treatment of these diseases faces numerous challenges. The mucosal immune system plays a crucial role in safeguarding the body from harmful pathogens, while being exposed to the environment, mainly food antigens. To develop treatment options for infectious diseases, it is vital to understand the immune responses that occur during infection. In this work, we focus on gut immune system that acts like an ecosystem comprising of trillions of interacting cells and molecules, including membars of the microbiome. The goal of this dissertation is to develop computational models that can simulate host immune responses in two infectious diseases- i) Helicobacter pylori infection and ii) human immunodeficiency virus (HIV)-human papilloma virus (HPV) co-infection. Firstly, it reviews the various mathematical techniques and systems biology based methods. Second, it introduces a "hybrid" model that combines different mathematical and statistical approaches to study H. pylori infection. Third, it highlights the development of a novel HIV/HPV coinfection model and compares the results from a clinical trial study. Fourth, it discusses the challenges that can be encountered in adapting machine learning based computational technologies. Taken together, the work in this dissertation presents modeling approaches that could lead to the identification of host immune factors in infectious diseases in a predictive and more resourceful way.

In this thesis we construct a new model of the immune response to the growth of breast cancer cells and investigate the impact of certain drug therapies on the cancer. We use delay differential equations to model the interaction of breast cancer cells with the immune system. The new model is constructed by combining two previous models. The first model accounts for different cell cycles and includes terms to evaluate drug treatments, but ignores quiescent tumor cells. The second model includes quiescent cells, but ignores the immune response and drug treatments. The new model is obtained by combining and modifying these two models to account for quiescent cells, immune cells and includes drug intervention terms. This new model is used to evaluate the effects of pulsed applications of the drug Paclitaxel for models with and without quiescent cells. We use sensitivity equation methods to analyze the sensitivity of the model with respect to the initial number of immune cytotoxic T-cells. Numerical experiments are conducted to compare the model predictions to observed behavior. / Master of Science

Interval analysis is an essential tool in the construction of validated numerical solutions of Initial Value Problems (IVP) for Ordinary (ODE) and Partial (PDE) Differential Equations. A validated solution typically consists of guaranteed lower and upper bounds for the exact solution or set of exact solutions in the case of uncertain data, i.e. it is an interval function (enclosure) containing all solutions of the problem. IVP for ODE: The central point of discussion is the wrapping effect. A new concept of wrapping function is introduced and applied in studying this effect. It is proved that the wrapping function is the limit of the enclosures produced by any method of certain type (propagate and wrap type). Then, the wrapping effect can be quantified as the difference between the wrapping function and the optimal interval enclosure of the solution set (or some norm of it). The problems with no wrapping effect are characterized as problems for which the wrapping function equals the optimal interval enclosure. A sufficient condition for no wrapping effect is that there exist a linear transformation, preserving the intervals, which reduces the right-hand side of the system of ODE to a quasi-isotone function. This condition is also necessary for linear problems and "near" necessary in the general case. Hyperbolic PDE: The Initial Value Problem with periodic boundary conditions for the wave equation is considered. It is proved that under certain conditions the problem is an operator equation with an operator of monotone type. Using the established monotone properties, an interval (validated) method for numerical solution of the problem is proposed. The solution is obtained step by step in the time dimension as a Fourier series of the space variable and a polynomial of the time variable. The numerical implementation involves computations in Fourier and Taylor functoids. Propagation of discontinuo~swaves is a serious problem when a Fourier series is used (Gibbs phenomenon, etc.). We propose the combined use of periodic splines and Fourier series for representing discontinuous functions and a method for propagating discontinuous waves. The numerical implementation involves computations in a Fourier hyper functoid. / Mathematical Sciences / D. Phil. (Mathematics)

Mathematical models have been used to capture the behaviour of biological systems, from low-level biochemical reactions to multi-scale whole-organ models. Models are typically based on experimentally-derived data, attempting to reproduce the observed behaviour through mathematical constructs, e.g. using Ordinary Differential Equations (ODEs) for spatially-homogeneous systems. These models are developed and published as mathematical equations, yet are of such complexity that they necessitate computational simulation. This computational model development is often performed in an ad hoc fashion by modellers who lack extensive software engineering experience, resulting in brittle, inefficient model code that is hard to extend and reuse. Several Domain Specific Languages (DSLs) exist to aid capturing such biological models, including CellML and SBML; however these DSLs are designed to facilitate model curation rather than simplify model development. We present research into the application of techniques from software engineering to this domain; starting with the design, development and implementation of a DSL, termed Ode, to aid the creation of ODE-based biological models. This introduces features beneficial to model development, such as model verification and reproducible results. We compare and contrast model development to large-scale software development, focussing on extensibility and reuse. This work results in a module system that enables the independent construction and combination of model components. We further investigate the use of software engineering processes and patterns to develop complex modular cardiac models. Model simulation is increasingly computationally demanding, thus models are often created in complex low-level languages such as C/C++. We introduce a highly-efficient, optimising native-code compiler for Ode that generates custom, model-specific simulation code and allows use of our structured modelling features without degrading performance. Finally, in certain contexts the stochastic nature of biological systems becomes relevant. We introduce stochastic constructs to the Ode DSL that enable models to use Stochastic Differential Equations (SDEs), the Stochastic Simulation Algorithm (SSA), and hybrid methods. These use our native-code implementation and demonstrate highly-efficient stochastic simulation, beneficial as stochastic simulation is highly computationally intensive. We introduce a further DSL to model ion channels declaratively, demonstrating the benefits of DSLs in the biological domain. This thesis demonstrates the application of software engineering methodologies, and in particular DSLs, to facilitate the development of both deterministic and stochastic biological models. We demonstrate their benefits with several features that enable the construction of large-scale, reusable and extensible models. This is accomplished whilst providing efficient simulation, creating new opportunities for biological model development, investigation and experimentation.

When traditional linearised theory is used to study free-surface flows past a surface-piercing object or over an obstruction in a stream, the geometry of the object is usually lost, having been assumed small in one or several of its dimensions. In order to preserve the nonlinear nature of the geometry, asymptotic expansions in the low-Froude or low-Bond limits can be derived, but here, the solution invariably predicts a waveless free-surface at every order. This is because the waves are in fact, exponentially small, and thus beyond-all-orders of regular asymptotics; their formation is a consequence of the divergence of the asymptotic series and the associated Stokes Phenomenon. In this thesis, we will apply exponential asymptotics to the study of two new problems involving nonlinear geometries. In the first, we examine the case of free-surface flow over a step including the effects of both gravity and surface tension. Here, we shall see that the availability of multiple singularities in the geometry, coupled with the interplay of gravitational and cohesive effects, leads to the discovery of a remarkable new set of solutions. In the second problem, we study the waves produced by bluff-bodied ships in low-Froude flows. We will derive the analytical form of the exponentially small waves for a wide range of hull geometries, including single-cornered and multi-cornered ships, and then provide comparisons with numerical computations. A particularly significant result is our confirmation of the thirty-year old conjecture by Vanden-Broeck & Tuck (1977) regarding the impossibility of waveless single-cornered ships.

The performance modelling of large-scale systems using discrete-state approaches is fundamentally hampered by the well-known problem of state-space explosion, which causes exponential growth of the reachable state space as a function of the number of the components which constitute the model. Because they are mapped onto continuous-time Markov chains (CTMCs), models described in the stochastic process algebra PEPA are no exception. This thesis presents a deterministic continuous-state semantics of PEPA which employs ordinary differential equations (ODEs) as the underlying mathematics for the performance evaluation. This is suitable for models consisting of large numbers of replicated components, as the ODE problem size is insensitive to the actual population levels of the system under study. Furthermore, the ODE is given an interpretation as the fluid limit of a properly defined CTMC model when the initial population levels go to infinity. This framework allows the use of existing results which give error bounds to assess the quality of the differential approximation. The computation of performance indices such as throughput, utilisation, and average response time are interpreted deterministically as functions of the ODE solution and are related to corresponding reward structures in the Markovian setting. The differential interpretation of PEPA provides a framework that is conceptually analogous to established approximation methods in queueing networks based on meanvalue analysis, as both approaches aim at reducing the computational cost of the analysis by providing estimates for the expected values of the performance metrics of interest. The relationship between these two techniques is examined in more detail in a comparison between PEPA and the Layered Queueing Network (LQN) model. General patterns of translation of LQN elements into corresponding PEPA components are applied to a substantial case study of a distributed computer system. This model is analysed using stochastic simulation to gauge the soundness of the translation. Furthermore, it is subjected to a series of numerical tests to compare execution runtimes and accuracy of the PEPA differential analysis against the LQN mean-value approximation method. Finally, this thesis discusses the major elements concerning the development of a software toolkit, the PEPA Eclipse Plug-in, which offers a comprehensive modelling environment for PEPA, including modules for static analysis, explicit state-space exploration, numerical solution of the steady-state equilibrium of the Markov chain, stochastic simulation, the differential analysis approach herein presented, and a graphical framework for model editing and visualisation of performance evaluation results.