Accelerating the simulation of turbulence to stationarity is a critical challenge in various engineering applications. This study presents an innovative equation-free multiscale approach combined with a machine learning technique to address this challenge in the context of the one-dimensional stochastic Burgers' equation, a widely used toy model for turbulence. We employ an encoder-decoder recurrent neural network to perform super-resolution reconstruction of the velocity field from lower-dimensional energy spectrum data, enabling seamless transitions between fine and coarse levels of description. The proposed multiscale-machine learning framework significantly accelerates the computation of the statistically stationary turbulent Burgers' velocity field, achieving up to 442 times faster wall clock time compared to direct numerical simulation, while maintaining three-digit accuracy in the velocity field. Our findings demonstrate the potential of integrating equation-free multiscale methods with machine learning methods to efficiently simulate stochastic partial differential equations and highlight the possibility of using this approach to simulate stochastic systems in other engineering domains. / Master of Science / In many practical engineering problems, simulating turbulence can be computationally expensive and time-consuming. This research explores an innovative method to accelerate these simulations using a combination of equation-free multiscale techniques and deep learning. Multiscale methods allow researchers to simulate the behavior of a system at a coarser scale, even when the specific equations describing its evolution are only available for a finer scale. This can be particularly helpful when there is a notable difference in the time scales between the coarser and finer scales of a system. The ``equation-free approach multiscale method coarse projective integration" can then be used to speed up the simulations of the system's evolution. Turbulence is an ideal candidate for this approach since it can be argued that it evolves to a statistically steady state on two different time scales. Over the course of evolution, the shape of the energy spectrum (the coarse scale) changes slowly, while the velocity field (the fine scale) fluctuates rapidly. However, applying this multiscale framework to turbulence simulations has been challenging due to the lack of a method for reconstructing the velocity field from the lower-dimensional energy spectrum data. This is necessary for moving between the two levels of description in the multiscale simulation framework. In this study, we tackled this challenge by employing a deep neural network model called an encoder-decoder sequence-to-sequence architecture. The model was used to capture and learn the conversions between the structure of the velocity field and the energy spectrum for the one-dimensional stochastic Burgers' equation, a simplified model of turbulence. By combining multiscale techniques with deep learning, we were able to achieve a much faster and more efficient simulation of the turbulent Burgers' velocity field. The findings of this study demonstrated that this novel approach could recover the final steady-state turbulent Burgers' velocity field up to 442 times faster than the traditional direct numerical simulations, while maintaining a high level of accuracy. This breakthrough has the potential to significantly improve the efficiency of turbulence simulations in a variety of engineering applications, making it easier to study and understand these complex phenomena.

Les systèmes complexes sont présents partout dans notre environnement : internet, réseaux de distribution d'électricité, réseaux de transport. Ces systèmes ont pour caractéristiques d'avoir un grand nombre d'entités autonomes, des structures dynamiques, des échelles de temps et d'espace différentes, ainsi que l'émergence de phénomènes. Ce travail de thèse se focalise sur la problématique du contrôle de tels systèmes. Il s'agit de déterminer, à partir d'une perception partielle de l'état du système, quelle(s) actions(s) effectuer pour éviter ou au contraire favoriser certains états globaux du système. Cette problématique pose plusieurs questions difficiles : pouvoir évaluer l'impact au niveau collectif d'actions appliqués au niveau individuel, modéliser la dynamique d'un système hétérogène (plusieurs comportements différents en interaction), évaluer la qualité des estimations issues de la modélisation de la dynamique du système. Nous proposons une architecture de contrôle selon une approche " equation-free ". Nous utilisons un modèle multi-agents pour évaluer l'impact global d'actions de contrôle locales avant d'appliquer la plus pertinente. Associée à cette architecture, une plateforme a été développée pour confronter ces idées à l'expérimentation dans le cadre d'un phénomène simulé de " free-riding " dans les réseaux d'échanges de fichiers pair à pair. Nous avons montré que cette approche permettait d'amener le système dans un état où une majorité de pairs partagent alors que les conditions initiales (sans intervention) feraient évoluer le système vers un état où aucun pair ne partage. Nous avons également expérimenté avec différentes configurations de l'architecture pour identifier les différents moyens d'améliorer ses performances.

Distributed parameter systems (DPS) comprise an important class of engineering systems ranging from "traditional" such as tubular reactors, to cutting edge processes such as nano-scale coatings. DPS have been studied extensively and significant advances have been noted, enabling their accurate simulation. To this end a variety of tools have been developed. However, extending these advances for systems design is not a trivial task . Rigorous design and operation policies entail systematic procedures for optimisation and control. These tasks are "upper-level" and utilize existing models and simulators. The higher the accuracy of the underlying models, the more the design procedure benefits. However, employing such models in the context of conventional algorithms may lead to inefficient formulations. The optimisation and control of DPS is a challenging task. These systems are typically discretised over a computational mesh, leading to large-scale problems. Handling the resulting large-scale systems may prove to be an intimidating task and requires special methodologies. Furthermore, it is often the case that the underlying physical phenomena span various temporal and spatial scales, thus complicating the analysis. Stiffness may also potentially be exhibited in the (nonlinear) models of such phenomena. The objective of this work is to design reliable and practical procedures for the optimisation and control of DPS. It has been observed in many systems of engineering interest that although they are described by infinite-dimensional Partial Differential Equations (PDEs) resulting in large discretisation problems, their behaviour has a finite number of significant components , as a result of their dissipative nature. This property has been exploited in various systematic model reduction techniques. Of key importance in this work is the identification of a low-dimensional dominant subspace for the system. This subspace is heuristically found to correspond to part of the eigenspectrum of the system and can therefore be identified efficiently using iterative matrix-free techniques. In this light, only low-dimensional Jacobians and Hessian matrices are involved in the formulation of the proposed algorithms, which are projections of the original matrices onto appropriate low-dimensional subspaces, computed efficiently with directional perturbations.The optimisation algorithm presented employs a 2-step projection scheme, firstly onto the dominant subspace of the system (corresponding to the right-most eigenvalues of the linearised system) and secondly onto the subspace of decision variables. This algorithm is inspired by reduced Hessian Sequential Quadratic Programming methods and therefore locates a local optimum of the nonlinear programming problem given by solving a sequence of reduced quadratic programming (QP) subproblems . This optimisation algorithm is appropriate for systems with a relatively small number of decision variables. Inequality constraints can be accommodated following a penalty-based strategy which aggregates all constraints using an appropriate function , or by employing a partial reduction technique in which only equality constraints are considered for the reduction and the inequalities are linearised and passed on to the QP subproblem . The control algorithm presented is based on the online adaptive construction of low-order linear models used in the context of a linear Model Predictive Control (MPC) algorithm , in which the discrete-time state-space model is recomputed at every sampling time in a receding horizon fashion. Successive linearisation around the current state on the closed-loop trajectory is combined with model reduction, resulting in an efficient procedure for the computation of reduced linearised models, projected onto the dominant subspace of the system. In this case, this subspace corresponds to the eigenvalues of largest magnitude of the discretised dynamical system. Control actions are computed from low-order QP problems solved efficiently online.The optimisation and control algorithms presented may employ input/output simulators (such as commercial packages) extending their use to upper-level tasks. They are also suitable for systems governed by microscopic rules, the equations of which do not exist in closed form. Illustrative case studies are presented, based on tubular reactor models, which exhibit rich parametric behaviour.

Mathematical modeling and simulation has emerged as a fundamental means to understand physical process around us with countless real-world applications in applied science and engineering problems. However, heavy reliance on first principles, symmetry relations, and conservation laws has limited its applicability to a few scientific domains and even few real-world scenarios. Especially in disciplines like biology the underlying living constituents exhibit a myriad of complexities like non-linearities, non-equilibrium physics, self-organization and plasticity that routinely escape mathematical treatment based on governing laws. Meanwhile, recent decades have witnessed rapid advancement in computing hardware, sensing technologies, and algorithmic innovations in machine learning. This progress has helped propel data-driven paradigms to achieve unprecedented practical success in the fields of image processing and computer vision, natural language processing, autonomous transport, and etc. In the current thesis, we explore, apply, and advance statistical and machine learning strategies that help bridge the gap between data and mathematical models, with a view toward modeling and simulation of spatiotemporal processes in biology. As first, we address the problem of learning interpretable mathematical models of biologial process from limited and noisy data. For this, we propose a statistical learning framework called PDE-STRIDE based on the theory of stability selection and ℓ0-based sparse regularization for parsimonious model selection. The PDE-STRIDE framework enables model learning with relaxed dependencies on tuning parameters, sample-size and noise-levels. We demonstrate the practical applicability of our method on real-world data by considering a purely data-driven re-evaluation of the advective triggering hypothesis explaining the embryonic patterning event in the C. elegans zygote. As a next natural step, we extend our PDE-STRIDE framework to leverage prior knowledge from physical principles to learn biologically plausible and physically consistent models rather than models that simply fit the data best. For this, we modify the PDE-STRIDE framework to handle structured sparsity constraints for grouping features which enables us to: 1) enforce conservation laws, 2) extract spatially varying non-observables, 3) encode symmetry relations associated with the underlying biological process. We show several applications from systems biology demonstrating the claim that enforcing priors dramatically enhances the robustness and consistency of the data-driven approaches. In the following part, we apply our statistical learning framework for learning mean-field deterministic equations of active matter systems directly from stochastic self-propelled active particle simulations. We investigate two examples of particle models which differs in the microscopic interaction rules being used. First, we consider a self-propelled particle model endowed with density-dependent motility character. For the chosen hydrodynamic variables, our data-driven framework learns continuum partial differential equations that are in excellent agreement with analytical derived coarse-grain equations from Boltzmann approach. In addition, our structured sparsity framework is able to decode the hidden dependency between particle speed and the local density intrinsic to the self-propelled particle model. As a second example, the learning framework is applied for coarse-graining a popular stochastic particle model employed for studying the collective cell motion in epithelial sheets. The PDE-STRIDE framework is able to infer novel PDE model that quantitatively captures the flow statistics of the particle model in the regime of low density fluctuations. Modern microscopy techniques produce GigaBytes (GB) and TeraBytes (TB) of data while imaging spatiotemporal developmental dynamics of living organisms. However, classical statistical learning based on penalized linear regression models struggle with issues like accurate computation of derivatives in the candidate library and problems with computational scalability for application to “big” and noisy data-sets. For this reason we exploit the rich parameterization of neural networks that can efficiently learn from large data-sets. Specifically, we explore the framework of Physics-Informed Neural Networks (PINN) that allow for seamless integration of physics priors with measurement data. We propose novel strategies for multi-objective optimization that allow for adapting PINN architecture to multi-scale modeling problems arising in biology. We showcase application examples for both forward and inverse modeling of mesoscale active turbulence phenomenon observed in dense bacterial suspensions. Employing our strategies, we demonstrate orders of magnitude gain in accuracy and convergence in comparison with conventional formulation for solving multi-objective optimization in PINNs. In the concluding chapter of the thesis, we skip model interpretability and focus on learning computable models directly from noisy data for the purpose of pure dynamics forecasting. We propose STENCIL-NET, an artificial neural network architecture that learns solution adaptive spatial discretization of an unknown PDE model that can be stably integrated in time with negligible loss in accuracy. To support this claim, we present numerical experiments on long-term forecasting of chaotic PDE solutions on coarse spatio-temporal grids, and also showcase de-noising application that help decompose spatiotemporal dynamics from the noise in an equation-free manner.