Stochastic delay differential equations naturally arise in models of complex natural phenomena, yet continue to resist efforts to find analytical solutions to them: general solutions are limited to linear systems with additive noise and a single delayed term. In this work we solve the case of distributed delays in linear systems with additive noise. Key to our solution is the development of a consistent interpretation for integrals over stochastic variables, obtained by means of a virtual discretization procedure. This procedure makes no assumption on the form of noise, and would likely be useful for a wider variety of cases than those we have considered. We show how it can be used to map the distributed delay equation to a known multivariate system, and obtain expressions for the system's time-dependent mean and autocovariance. These are in the form of series over the system's natural modes and completely define the solution. — An interpretation of the system as an amplitude process is explored. We show that for a wide range of realistic parameters, dynamics are dominated by only a few modes, implying that most of the observed behaviour of stochastic delayed equations is constrained to a low-dimensional subspace. — The expression for the autocovariance is given particular attention. A recurring problem for stochastic delay equations is the description of their temporal structure. We show that the series expression for the autocovariance does converge over a meaningful range of time lags, and therefore provides a means of describing this temporal structure.

Rich dynamics have been demonstrated when a discrete time delay is introduced in a simple predator-prey system. For example, Hopf bifurcations and a sequence of period doubling bifurcations that appear to lead to chaotic dynamics have been observed. In this thesis we consider two different predator-prey models: the classical Gause-type predator-prey model and the chemostat predator-prey model. In both cases, we explore how different ways of modeling the time between the first contact of the predator with the prey and its eventual conversion to predator biomass affects the possible range of dynamics predicted by the models. The models we explore are systems of integro-differential equations with delay kernels from various distributions including the gamma distribution of different orders, the uniform distribution, and the Dirac delta distribution. We study the models using bifurcation theory taking the mean delay as the main bifurcation parameter. We use both an analytical approach and a computational approach using the numerical continuation software XPPAUT and DDE-BIFTOOL. First, general results common to all the models are established. Then, the differences due to the selection of particular delay kernels are considered. In particular, the differences in regions of stability of the coexistence equilibrium are investigated. Finally, the effects on the predicted range of dynamics between the classical Gause-type and the chemostat predator-prey models are compared. / Thesis / Doctor of Philosophy (PhD)

<p>The goal of this work is to develop a practical and comprehensive methodology to study the response and the stability of various delay differential equations (DDEs). The development of these new analysis techniques is motivated by the existence of delays in the governing equations of many physical systems such as turning and milling processes. </p><p>Delay differential equations appear in many models in science in engineering either as an intrinsic component (e.g. machining dynamics) or as a modeling decision (biology related dynamics). However, the infinite dimensionality of DDEs significantly complicates the resulting analysis from both an analytical and numerical perspective. Since the delay results in an infinite dimensional state-space, it is often necessary to use an approximate procedure to study DDEs and ascertain their stability.</p><p>Several approximate techniques appeared in literature to study the stability of DDEs. However, a large number of these techniques---such as D-subdivision, Cluster Treatment of Characteristic Roots and Continuous Time Approximation---are limited to autonomous DDEs. Moreover, the methods that are suitable for non-autonomous DDEs, e.g. the Semi-discretization approach, often result in a very large system of algebraic equations that can cause computational difficulties. Collocation-type methods, such as Chebyshev-collocation approach, have also been used to study DDEs. One major limitation of the conventional Chebyshev collocation approach is that it is strictly applicable to DDEs with continuous coefficients. An alternative approach that can handle DDEs with piecewise continuous coefficients is the Temporal Finite Element Analysis (TFEA). However, TFEA has only linear rates of convergence and is limited to h-convergence schemes. The limited rate of convergence in TFEA has prohibited its application to a wide class of DDEs such as DDEs with complicated coefficients or with distributed and multiple delays. </p><p>In this thesis, I develop a spectral element method for the stability analysis of DDEs. The spectral element method is a Galerkin-type approach that discretizes the infinite dimensional DDE into a finite set of algebraic equations (or a dynamic map). The stability of the system is then studied using the eigenvalues of the map. </p><p>In contrast to TFEA, the spectral element method was shown to have exponential rates of convergence and hp-refinement capabilities. Further, a comparison with the widely-used collocation methods showed that our approach can often yield higher rates of convergence. The higher rates of convergence of the developed approach enabled extending it to DDEs with multiple and distributed delays. I further extended this approach to calculating the periodic orbits of DDEs and their stability. </p><p>As an application of the methods developed in this thesis, I studied the stability of turning and milling models. For example, a distributed force model was proposed to characterize cutting forces in turning. The stability of the resulting delay integro-differential equation was studied using the methods developed in this study and they were shown to agree with practical observations. As another example, the stability of a milling process--- whose model contains piecewise coefficients---was investigated. The effect of multiple-flute engagement, which contributed to the complexity of the coefficients, was also investigated. The resulting stability charts revealed new stability observations in comparison to typical analysis methods. Specifically, I was able to show that unstable regions appear in what was deemed a stable region by prior analysis techniques.</p> / Dissertation

Communication delays are common in many complex systems. It has been shown that these delays cannot be neglected when they are long enough compared to other timescales in the system. In systems of coupled phase oscillators discrete delays in the coupling give rise to effects such as multistability of steady states. However, variability in the communication times inherent to many processes suggests that the description with discrete delays maybe insufficient to capture all effects of delays. An interesting example of the effects of communication delays is found during embryonic development of vertebrates. A clock based on biochemical reactions inside cells provides the periodicity for the successive and robust formation of somites, the embryonic precursors of vertebrae, ribs and some skeletal muscle. Experiments show that these cellular clocks communicate in order to synchronize their behavior. However, in cellular systems, fluctuations and stochastic processes introduce a variability in the communication times. Here we account for such variability by considering the effects of distributed delays. Our approach takes into account entire intervals of past states, and weights them according to a delay distribution. We find that the stability of the fully synchronized steady state with zero phase lag does not depend on the shape of the delay distribution, but the dynamics when responding to small perturbations about this steady state do. Depending on the mean of the delay distribution, a change in its shape can enhance or reduce the ability of these systems to respond to small perturbations about the phase-locked steady state, as compared to a discrete delay with a value equal to this mean. For synchronized steady states with non-zero phase lag we find that the stability of the steady state can be altered by changing the shape of the delay distribution. We conclude that the response to a perturbation in systems of phase oscillators coupled with discrete delays has a sharper functional dependence on the mean delay than in systems with distributed delays in the coupling. The strong dependence of the coupling on the mean delay time is partially averaged out by distributed delays that take into account intervals of the past.

Computer network complexity has increased in the last decades due to the introduction of various concepts, leaving network maintainers in hardship to manage such huge and tangled networks. In this study, we aim to aid service providers to optimize and automate their networks. Currently, network maintainers perform a vast number of explicit measurements, which has a negative effect on the network’s health and stability. Depending on the service’s nature, measurements are either made at service initiation as in the case of server-client selection or continuously done to monitor the quality of service as in the case of quality assurance applications. We intend to apply artificial intelligence to minimize the dependency on such explicit measurements and hence, optimize the network with minimal cost. From the two types of applications, we focus on distributed delay measurements for Esports server-client selection problem as well as network automation and failure mitigation task done by Internet service providers. In large-scale networks, it is impractical to measure the delay between every node explicitly. As a result, we propose an AI-based delay measurement estimator system. The system’s inputs are just the source and destination nodes’ IP-addresses. Network maintainers continuously monitor their network status to detect any sudden change in the network and take suitable action(s) to keep the network in the best conditions. We propose an ML-based action recommender engine that is able to identify the current network status and suggest a set of actions that restore the network to its optimum state.

Stochastic delay differential equations (SDDEs) are systems of differential equations with a time lag in a noisy or random environment. There are many nonlinear SDDEs where a linear growth condition is not satisfied, for example, the stochastic delay Lotka-Volterra model of food chain discussed by Xuerong Mao and Martina John Rassias in 2005. Much research has been done using discrete delay where the dynamics of a process at time t depend on the state of the process in the past after a single fixed time lag \tau. We are researching processes with continuously distributed delay which depend on weighted averages of past states over the entire time lag interval [t-\tau,t].By using martingale concepts, we prove sufficient conditions for the existence of a unique solution, ultimate boundedness, and non-extinction of one-dimensional nonlinear SDDE with continuously distributed delay. We give generalized Khasminskii-type conditions which along with local Lipschitz conditions are sufficient to guarantee the existence of a unique global solution of certain n-dimensional nonlinear SDDEs with continuously distributed delay. Further, we give conditions under which Euler-Maruyama numerical approximations of such nonlinear SDDEs converge in probability to their exact solutions.We give some examples of one-dimensional and 2-dimensional stochastic differential equations with continuously distributed delay which satisfy the sufficient conditions of our theorems. Moreover, we simulate their solutions and analyze the error of approximation using MATLAB to implement the Euler-Maruyama algorithm.

We investigate the linear stability and perform the bifurcation analysis for Hopfield neural networks with a general distribution of delays, where the neurons are identical. We start by analyzing the scalar model and show what kind of information can be gained with only minimal information about the exact distribution of delays. We determine a mean delay and distribution independent stability region. We then illustrate a way of improving on this conservative result by approximating the true region of stability when the actual distribution is not known, but some moments or cumulants of the distribution are. We compare the approximate stability regions with the stability regions in the case of the uniform and gamma distributions. We show that, in general, the approximations improve as more moments or cumulants are used, and that the approximations using cumulants give better results than the ones using moments. Further, we extend these results to a network of n identical neurons, where we examine the stability of a symmetrical equilibrium point via the analysis of the characteristic equation both when the connection matrix is symmetric and when it is not. Finally, for the scalar model, we show under what conditions a Hopf bifurcation occurs and we use the centre manifold technique to determine the criticality of the bifurcation. When the kernel represents the gamma distribution with p=1 and p=2, we transform the delay differential equation into a system of ordinary differential equations and we compare the centre manifold computation to the one we obtain in the ordinary differential case.

A distributed delay is a linear input-output operators and appears in many control problems. We investigate distributed delay and its applications. After introducing the definition and the main properties of the distributed delay, the numerical implementation problem of distributed delays is analyzed and a general method for its approximation is given. Then three applications are focused on where distributed delay appears. The first application is the stable inversion and model matching. A new class of stable inversion and model matching problem for finite dimensional linear time-invariant systems is defined. The stable inversion (resp. model matching) is an approximation of the inverse of a given model (resp. model matching), where exact inversion (resp. exact matching) is reached after a time $t=h$, which is a parameter of our procedure. The second application is concerned with stabilization and finite spectrum assignment for a class of infinite dimensional systems. The last application concerns observer synthesis for estimation or output control. For linear finite dimensional systems. A closed-loop memoryless observer by input injection is introduced. Asymptotic convergence as well as finite time convergence of the estimation are analyzed by output injection and input information via distributed delay. At last, we introduce a new class for approximation of distributed parameter systems. We work on the graph topology, and show that under some weak assumptions, such an approximation can be realized using distributed delay.

We investigate the linear stability and perform the bifurcation analysis for Hopfield neural networks with a general distribution of delays, where the neurons are identical. We start by analyzing the scalar model and show what kind of information can be gained with only minimal information about the exact distribution of delays. We determine a mean delay and distribution independent stability region. We then illustrate a way of improving on this conservative result by approximating the true region of stability when the actual distribution is not known, but some moments or cumulants of the distribution are. We compare the approximate stability regions with the stability regions in the case of the uniform and gamma distributions. We show that, in general, the approximations improve as more moments or cumulants are used, and that the approximations using cumulants give better results than the ones using moments. Further, we extend these results to a network of n identical neurons, where we examine the stability of a symmetrical equilibrium point via the analysis of the characteristic equation both when the connection matrix is symmetric and when it is not. Finally, for the scalar model, we show under what conditions a Hopf bifurcation occurs and we use the centre manifold technique to determine the criticality of the bifurcation. When the kernel represents the gamma distribution with p=1 and p=2, we transform the delay differential equation into a system of ordinary differential equations and we compare the centre manifold computation to the one we obtain in the ordinary differential case.

Communication delays are common in many complex systems. It has been shown that these delays cannot be neglected when they are long enough compared to other timescales in the system. In systems of coupled phase oscillators discrete delays in the coupling give rise to effects such as multistability of steady states. However, variability in the communication times inherent to many processes suggests that the description with discrete delays maybe insufficient to capture all effects of delays. An interesting example of the effects of communication delays is found during embryonic development of vertebrates. A clock based on biochemical reactions inside cells provides the periodicity for the successive and robust formation of somites, the embryonic precursors of vertebrae, ribs and some skeletal muscle. Experiments show that these cellular clocks communicate in order to synchronize their behavior. However, in cellular systems, fluctuations and stochastic processes introduce a variability in the communication times. Here we account for such variability by considering the effects of distributed delays. Our approach takes into account entire intervals of past states, and weights them according to a delay distribution. We find that the stability of the fully synchronized steady state with zero phase lag does not depend on the shape of the delay distribution, but the dynamics when responding to small perturbations about this steady state do. Depending on the mean of the delay distribution, a change in its shape can enhance or reduce the ability of these systems to respond to small perturbations about the phase-locked steady state, as compared to a discrete delay with a value equal to this mean. For synchronized steady states with non-zero phase lag we find that the stability of the steady state can be altered by changing the shape of the delay distribution. We conclude that the response to a perturbation in systems of phase oscillators coupled with discrete delays has a sharper functional dependence on the mean delay than in systems with distributed delays in the coupling. The strong dependence of the coupling on the mean delay time is partially averaged out by distributed delays that take into account intervals of the past.:Abstract i Acknowledgement iii I. INTRODUCTION 1. Coupled Phase Oscillators Enter the Stage 5 1.1. Adjusting rhythms – synchronization . . . . . . . . . . . . . . . . . . 5 1.2. Historical remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3. Reducing variables – phase models . . . . . . . . . . . . . . . . . . . . 9 1.4. The Kuramoto order parameter . . . . . . . . . . . . . . . . . . . . . . 10 1.5. Who talks to whom – coupling topologies . . . . . . . . . . . . . . . . 12 2. Coupled Phase Oscillators with Delay in the Coupling 15 2.1. Communication needs time – coupling delays . . . . . . . . . . . . . . 15 2.1.1. Discrete delays consider one past time . . . . . . . . . . . . . . 16 2.1.2. Distributed delays consider multiple past times . . . . . . . . 17 2.2. Coupled phase oscillators with discrete delay . . . . . . . . . . . . . . 18 2.2.1. Phase locked steady states with no phase lags . . . . . . . . . 18 2.2.2. m-twist solutions: phase-locked steady states with phase lags 21 3. The Vertebrate Segmentation Clock – What Provides the Rhythm? 25 3.1. The clock and wavefront mechanism . . . . . . . . . . . . . . . . . . . 26 3.2. Cyclic gene expression on the cellular and the tissue level . . . . . . 27 3.3. Coupling by Delta-Notch signalling . . . . . . . . . . . . . . . . . . . . 29 3.4. The Delayed Coupling Theory . . . . . . . . . . . . . . . . . . . . . . . 30 3.5. Discrete delay is an approximation – is it sufficient? . . . . . . . . . 32 4. Outline of the Thesis 33 II. DISTRIBUTED DELAYS 5. Setting the Stage for Distributed Delays 37 5.1. Model equations with distributed delays . . . . . . . . . . . . . . . . . 37 5.2. How we include distributed delays . . . . . . . . . . . . . . . . . . . . 38 5.3. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 6. The Phase-Locked Steady State Solution 41 6.1. Global frequency of phase-locked steady states . . . . . . . . . . . . . 41 6.2. Linear stability of the steady state . . . . . . . . . . . . . . . . . . . . 42 6.3. Linear dynamics of the perturbation – the characteristic equation . 43 6.4. Summary and application to the Delayed Coupling Theory . . . . . . 50 7. Dynamics Close to the Phase-Locked Steady State 53 7.1. The response to small perturbations . . . . . . . . . . . . . . . . . . . 53 7.2. Relation between order parameter and perturbation modes . . . . . 54 7.3. Perturbation dynamics in mean-field coupled systems . . . . . . . . 56 7.4. Nearest neighbour coupling with periodic boundary conditions . . . 62 7.4.1. How variance and skewness influence synchrony dynamics . 73 7.4.2. The dependence of synchrony dynamics on the number of oscillators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 7.5. Synchrony dynamics in systems with arbitrary coupling topologies . 88 7.6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 8. The m-twist Steady State Solution on a Ring 95 8.1. Global frequency of m-twist steady states . . . . . . . . . . . . . . . . 95 8.2. Linear stability of m-twist steady states . . . . . . . . . . . . . . . . . 97 8.3. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 9. Dynamics Approaching the m-twist Steady States 105 9.1. Relation between order parameter and perturbation modes . . . . . 105 9.2. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 10.Conclusions and Outlook 111 vi III. APPENDICES A. 119 A.1. Distribution composed of two adjacent boxcar functions . . . . . . . 119 A.2. The gamma distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 124 A.3. Distribution composed of two Dirac delta peaks . . . . . . . . . . . . 125 A.4. Gerschgorin’s circle theorem . . . . . . . . . . . . . . . . . . . . . . . . 127 A.5. The Lambert W function . . . . . . . . . . . . . . . . . . . . . . . . . . 127 A.6. Roots of unity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 B. Simulation methods 129