All fields of physics - be it nuclear, atomic and molecular, solid state, or optical - offer examples of systems which are strongly influenced by the environment of the actual system under investigation. The scope of what is called "the environment" may vary, i.e., how far from the system of interest an interaction between the two does persist. Typically, however, it is much larger than the open system itself. Hence, a fully quantum mechanical treatment of the combined system without approximations and without limitations of the type of system is currently out of reach. With the single assumption of the environment to consist of an internally thermalized set of infinitely many harmonic oscillators, the seminal work of Stockburger and Grabert [Chem. Phys., 268:249-256, 2001] introduced an open system description that captures the environmental influence by means of a stochastic driving of the reduced system. The resulting stochastic Liouville-von Neumann equation describes the full non-Markovian dynamics without explicit memory but instead accounts for it implicitly through the correlations of the complex-valued noise forces. The present thesis provides a first application of the Stockburger-Grabert stochastic Liouville-von Neumann equation to the computation of the dynamics of anharmonic, continuous open systems. In particular, it is demonstrated that trajectory based propagators allow for the construction of a numerically stable propagation scheme. With this approach it becomes possible to achieve the tremendous increase of the noise sample count necessary to stochastically converge the results when investigating such systems with continuous variables. After a test against available analytic results for the dissipative harmonic oscillator, the approach is subsequently applied to the analysis of two different realistic, physical systems. As a first example, the dynamics of a dissipative molecular oscillator is investigated. Long time propagation - until thermalization is reached - is shown to be possible with the presented approach. The properties of the thermalized density are determined and they are ascertained to be independent of the system's initial state. Furthermore, the dependence on the bath's temperature and coupling strength is analyzed and it is demonstrated how a change of the bath parameters can be used to tune the system from the dissociative to the bound regime. A second investigation is conducted for a dissipative tunneling scenario in which a wave packet impinges on a barrier. The dependence of the transmission probability on the initial state's kinetic energy as well as the bath's temperature and coupling strength is computed. For both systems, a comparison with the high-temperature Markovian quantum Brownian limit is performed. The importance of a full non-Markovian treatment is demonstrated as deviations are shown to exist between the two descriptions both in the low temperature cases where they are expected and in some of the high temperature cases where their appearance might not be anticipated as easily.:1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 Theory of Open Quantum Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.1 Influence Functional Formalism . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2 Quantum Brownian Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.3 Stochastic Unraveling of the Influence Functional . . . . . . . . . . . . . . . 20 2.4 Improved Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.4.1 Modified Dynamic Response . . . . . . . . . . . . . . . . . . . . . . . 23 2.4.2 Guide Trajectory Transformation . . . . . . . . . . . . . . . . . . . . 24 2.5 Obtaining Properly Correlated Stochastic Samples from Filtered White Noise 24 3 Unified Stochastic Trajectory Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.1 Semiclassical Brownian Motion . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.1.1 Guide Trajectory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.1.2 Real Coherent State Center Coordinates . . . . . . . . . . . . . . . . 31 3.1.3 Propagation Scheme Including Stochastic Forces . . . . . . . . . . . 32 3.2 Stochastic Bohmian Mechanics with Complex Action . . . . . . . . . . . . . 33 3.2.1 Hydrodynamic Formulation of Bohmian Mechanics . . . . . . . . . . 33 3.2.2 Bohmian Mechanics with Complex Action . . . . . . . . . . . . . . . 34 3.2.3 Stochastic BOMCA Trajectories . . . . . . . . . . . . . . . . . . . . 38 3.3 Noise Distribution Preserving Removal of Adverse Samples . . . . . . . . . . 39 4 Dissipative Harmonic Oscillator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.1 Reservoir Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.2 Harmonic Oscillator Analytic Expectation Values . . . . . . . . . . . . . . . 42 4.2.1 Ohmic Bath . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.2.2 Drude Regularized Bath . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.3 Sampling Strategies and Analytic Comparison . . . . . . . . . . . . . . . . . 44 4.4 Limits of the Markovian Approximation . . . . . . . . . . . . . . . . . . . . 45 5 Dissipative Vibrational Dynamics of Diatomics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.1 Molecular Morse Potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.2 Anharmonic Phenomena . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.3 Transient Non-Markovian Effects . . . . . . . . . . . . . . . . . . . . . . . . 53 5.4 Trapping by Dissipation and Thermalization . . . . . . . . . . . . . . . . . . 53 6 Tunneling with Dissipation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 6.1 Eckart Barrier Scattering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 6.2 Dissipative Tunneling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 6.3 Investigation of Markovianity . . . . . . . . . . . . . . . . . . . . . . . . . . 61 7 Summary and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Appendix A Conventions for Constants, Reservoir Kernels, and Influence Phases 69 Appendix B Stochastic Calculus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 B.1 Stochastic Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . 71 B.2 Position Verlet Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 B.3 Runge-Kutta Fourth Order Scheme . . . . . . . . . . . . . . . . . . . . . . . 73 Appendix CMorse Oscillator Expectation Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Appendix DPrerequisites of a Successful Stochastic Propagation . . . . . . . . . . . . . . 79 D.1 Hubbard-Stratonovich Transformation . . . . . . . . . . . . . . . . . . . . . 79 D.2 Kernels of the Reservoir . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 D.2.1 Quadratic Cutoff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 D.2.2 Quartic Cutoff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 D.2.3 Strictly Ohmic Reservoir . . . . . . . . . . . . . . . . . . . . . . . . . 89 D.3 Guide Trajectory Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 90 D.3.1 Quadratic Cutoff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 D.3.2 Quartic Cutoff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 D.3.3 Strictly Ohmic Reservoir . . . . . . . . . . . . . . . . . . . . . . . . . 92 D.4 Computation of Matrix Elements and Expectation Values . . . . . . . . . . 92 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

En planification théorique de la décision, le cadre des Processus Décisionnels Markoviens Factorisés (Factored Markov Decision Process, FMDP) a produit des algorithmes efficaces de résolution des problèmes de décisions séquentielles dans l'incertain. L'efficacité de ces algorithmes repose sur des structures de données telles que les Arbres de Décision ou les Diagrammes de Décision Algébriques (ADDs). Ces techniques de planification sont utilisées en Apprentissage par Renforcement par l'architecture SDYNA afin de résoudre des problèmes inconnus de grandes tailles. Toutefois, l'état-de-l'art des algorithmes d'apprentissage, de programmation dynamique et d'apprentissage par renforcement utilisés par SDYNA, requière que le problème soit spécifié uniquement à l'aide de variables binaires et/ou utilise des structures améliorables en termes de compacité. Dans ce manuscrit, nous présentons nos travaux de recherche visant à élaborer et à utiliser une structure de donnée plus efficace et moins contraignante, et à l'intégrer dans une nouvelle instance de l'architecture SDYNA. Dans une première partie, nous présentons l'état-de-l'art de la modélisation de problèmes de décisions séquentielles dans l'incertain à l'aide de FMDP. Nous abordons en détail la modélisation à l'aide d'DT et d'ADDs.Puis nous présentons les ORFGs, nouvelle structure de données que nous proposons dans cette thèse pour résoudre les problèmes inhérents aux ADDs. Nous démontrons ainsi que les ORFGs s'avèrent plus efficaces que les ADDs pour modéliser les problèmes de grandes tailles. Dans une seconde partie, nous nous intéressons à la résolution des problèmes de décision dans l'incertain par Programmation Dynamique. Après avoir introduit les principaux algorithmes de résolution, nous nous attardons sur leurs variantes dans le domaine factorisé. Nous précisons les points de ces variantes factorisées qui sont améliorables. Nous décrivons alors une nouvelle version de ces algorithmes qui améliore ces aspects et utilise les ORFGs précédemment introduits. Dans une dernière partie, nous abordons l'utilisation des FMDPs en Apprentissage par Renforcement. Puis nous présentons un nouvel algorithme d'apprentissage dédié à la nouvelle structure que nous proposons. Grâce à ce nouvel algorithme, une nouvelle instance de l'architecture SDYNA est proposée, se basant sur les ORFGs ~:~l'instance SPIMDDI. Nous testons son efficacité sur quelques problèmes standards de la littérature. Enfin nous présentons quelques travaux de recherche autour de cette nouvelle instance. Nous évoquons d'abord un nouvel algorithme de gestion du compromis exploration-exploitation destiné à simplifier l'algorithme F-RMax. Puis nous détaillons une application de l'instance SPIMDDI à la gestion d'unités dans un jeu vidéo de stratégie en temps réel. / In decision theoretic planning, the factored framework (Factored Markovian Decision Process, FMDP) has produced several efficient algorithms in order to resolve large sequential decision making under uncertainty problems. The efficiency of this algorithms relies on data structures such as decision trees or algebraïc decision diagrams (ADDs). These planification technics are exploited in Reinforcement Learning by the architecture SDyna in order to resolve large and unknown problems. However, state-of-the-art learning and planning algorithms used in SDyna require the problem to be specified uniquely using binary variables and/or to use improvable data structure in term of compactness. In this book, we present our research works that seek to elaborate and to use a new data structure more efficient and less restrictive, and to integrate it in a new instance of the SDyna architecture. In a first part, we present the state-of-the-art modeling tools used in the algorithms that tackle large sequential decision making under uncertainty problems. We detail the modeling using decision trees and ADDs. Then we introduce the Ordered and Reduced Graphical Representation of Function, a new data structure that we propose in this thesis to deal with the various problems concerning the ADDs. We demonstrate that ORGRFs improve on ADDs to model large problems. In a second part, we go over the resolution of large sequential decision under uncertainty problems using Dynamic Programming. After the introduction of the main algorithms, we see in details the factored alternative. We indicate the improvable points of these factored versions. We describe our new algorithm that improve on these points and exploit the ORGRFs previously introduced. In a last part, we speak about the use of FMDPs in Reinforcement Learning. Then we introduce a new algorithm to learn the new datastrcture we propose. Thanks to this new algorithm, a new instance of the SDyna architecture is proposed, based on the ORGRFs : the SPIMDDI instance. We test its efficiency on several standard problems from the litterature. Finally, we present some works around this new instance. We detail a new algorithm for efficient exploration-exploitation compromise management, aiming to simplify F-RMax. Then we speak about an application of SPIMDDI to the managements of units in a strategic real time video game.

No description available.

A time-delayed response of individual living organisms to information exchanged within flocks or swarms leads to the emergence of complex collective behaviors. A recent experimental setup by (Khadka et al 2018 Nat. Commun. 9 3864), employing synthetic microswimmers, allows to emulate and study such behavior in a controlled way, in the lab. Motivated by these experiments, we study a system of N Brownian particles interacting via a retarded harmonic interaction. For N 3 , we characterize its collective behavior analytically, by solving the pertinent stochastic delay-differential equations, and for N>3 by Brownian dynamics simulations. The particles form molecule-like non-equilibrium structures which become unstable with increasing number of particles, delay time, and interaction strength. We evaluate the entropy and information fluxes maintaining these structures and, to quantitatively characterize their stability, develop an approximate time-dependent transition-state theory to characterize transitions between different isomers of the molecules. For completeness, we include a comprehensive discussion of the analytical solution procedure for systems of linear stochastic delay differential equations in finite dimension, and new results for covariance and time-correlation matrices

We study the thermodynamic properties induced by non-reciprocal interactions between stochastic degrees of freedom in time- and space-continuous systems. We show that, under fairly general conditions, non-reciprocal coupling alone implies a steady energy flow through the system, i.e., non-equilibrium. Projecting out the non-reciprocally coupled degrees of freedom renders non-Markovian, one-variable Langevin descriptions with complex types of memory, for which we find a generalized second law involving information flow.We demonstrate that non-reciprocal linear interactions can be used to engineer non-monotonic memory, which is typical for, e.g., time-delayed feedback control, and is automatically accompanied with a nonzero information flow through the system. Furthermore, already a single non-reciprocally coupled degree of freedom can extract energy from a single heat bath (at isothermal conditions), and can thus be viewed as a minimal version of a time-continuous, autonomous ‘Maxwell demon’.We also show that for appropriate parameter settings, the non-reciprocal system has characteristic features of active matter, such as a positive energy input on the level of the fluctuating trajectories without global particle transport.

Queueing theory is the mathematical study of queues or waiting lines, which are formed whenever demand for service exceeds the capacity to provide service. A queueing system is composed of customers, packets or calls that need some kind of service. These entities arrive at queueing system, join a queue if service is not immediately available and leave system after receiving service. There are also cases when customers, packets or calls leave system without joining queue or drop out without receiving service even after waiting for some time. Queueing network models with finite capacity have facilitated the analysis of discrete flow systems, such as computer systems, transportation networks, manufacturing systems and telecommunication networks, by providing powerful and realistic tools for performance evaluation and prediction. In wireless cellular systems mobility is the most important feature and continuous service is achieved by supporting handoff from one cell to another. Hand off is the process of changing channel associated with the current connection while a call is in progress. A handoff is required when a mobile terminal moves from one cell to another or the signal quality deteriorates in current cell. Since neighbouring cells use disjoint subset of frequency bands therefore negotiation must take place between mobile terminal, the current base station and next potential base station. A poorly designed handoff scheme significantly decreases quality of service (QOS). Different schemes have been devised and in these schemes handoff calls are prioritize. Also most of the performance evaluation techniques consider the case where the arrival process is poisson and service is exponential i.e. there is single arrival and single departure. Whereas in practice there is burstiness in cellular traffic i.e. there can be bulk arrivals and bulk departures. Other issue is that, assumptions made by stochastic process models are not satisfied. Most of the effort is concentrated on providing different interpretations of M/M queues rather than attempting to provide a new methodology. In this thesis performance evaluation of multi traffic cellular models i.e. non priority, priority and hand off calling scheme for bursty traffic are devised. Moreover extensions are carried out towards the analysis of a multi-traffic M/M queueing system and state probabilities are calculated analytically.

In this project, we are aiming to solve option pricing and hedging problems numerically via Backward Stochastic Differential Equations (BSDEs). We use Markovian BSDEs to formulate nonlinear pricing and hedging problems of both European and American option types. This method of formulation is crucial for pricing financial instruments since it enables consideration of market imperfections and computations in high dimensions. We conduct numerical experiments of the pricing and hedging problems, where there is a higher interest rate for borrowing than lending, using the least squares Monte Carlo and deep neural network methods. Moreover, based on the experiment results, we point out which method to chooseover the other depending on the the problem at hand.

Neste trabalho é apresentada uma abordagem fuzzy para a estabilização de uma classe de sistemas não-lineares com parâmetros descritos por saltos Markovianos. Uma nova modelagem fuzzy de sistemas é formulada para representar esta classe de sistemas na vizinhança de pontos de operação escolhidos. A estrutura deste sistema fuzzy é composta de dois níveis, um para descrição dos saltos Markovianos e outro para descrição das não-linearidades no estado do sistema. Condições suficientes para a estabilização estocástica do sistema fuzzy considerado são derivadas usando uma função de Lyapunov acoplada. O projeto de controle fuzzy é então formulado a partir de um conjunto de desigualdades matriciais lineares. Em adição, um exemplo de aplicação, envolvendo a representação da operação de um sistema elétrico de potência em esquema de co-geração por um sistema com saltos Markovianos, é construído para validação dos resultados. / This work deals with the fuzzy-model-based control design for a class of Markovian jump nonlinear systems. A new fuzzy system modeling is proposed to approximate the dynamics of this class of systems. The structure of the new fuzzy system is composed of two levels, a crisp level which describes the Markovian jumps and a fuzzy level which describes the system nonlinearities. A sufficient condition on the existence of a stochastically stabilizing controller using a Lyapunov function approach is presented. The fuzzy-model-based control design is formulated in terms of a set of linear matrix inequalities. In addition, simulation results for a single-machine infinite-bus power system in cogeneration scheme, whose operation is modeled as an Markovian jump nonlinear system, are presented to illustrate the applicability of the technique.

A stochastic model for the dynamics of a limit order book is evaluated and tested on empirical data. Arrival of limit, market and cancellation orders are described in terms of a Markovian queuing system with exponentially distributed occurrences. In this model, several key quantities can be analytically calculated, such as the distribution of times between price moves, price volatility and the probability of an upward price move, all conditional on the state of the order book. We show that the exponential distribution poorly fits the occurrences of order book events and further show that little resemblance exists between the analytical formulas in this model and the empirical data. The log-normal and Weibull distribution are suggested as replacements as they appear to fit the empirical data better.

Les propriétés magnétiques détaillées des solides peuvent être vu comme le résultat de l'interaction de plusieurs sous-systèmes: celui des spins effectifs, portant l'aimantation, celui des électrons et celui du réseau crystallin. Différents processus permettent à ces sous-systèmes d'échanger de l'énergie. Parmis ceux-ci, les phénomènes de relaxation jouent un rôle prépondérants. Cependant, la complexité de ces processus en rend leur modélisation ardue. Afin de prendre en compte ces interactions de façon abordable aux calculs, l'approche de Langevin est depuis longtemps appliquée à la dynamique d'aimantation, qui peut être vue comme la réponse collective des spins. Elle consiste à modéliser les interactions entre les trois sous-systèmes par des interactions effectives entre le sous-système d'intérêt, les spins, et un bain thermique, dont seulement la densité de probabilité constituerait une quantité pertinente. Après avoir présenté cette approche, nous verrons en quoi elle permet de bâtir une dynamique atomique de spin. Une fois son implémentation détaillée, cette méthodologie sera appliquée à un exemple tiré de la littérature et basé sur le superparamagnétisme de nanoaimants de fer. / Detailed magnetic properties of solids can be regarded as the result of the interaction between three subsystems: the effective spins, that will be our focus in this thesis, the electrons and the crystalline lattice. These three subsystems exchange energy, in many ways, in particular, through relaxation processes. The nature of these processes remains extremely hard to understand, and even harder to simulate. A practical approach, for performing such simulations, involves adapting the description of random processes by Langevin to the collective dynamics of the spins, usually called the magnetization dynamics. It consists in describing the, complicated, interactions between the subsystems, by the effective interactions of the subsystem of interest, the spins, and a thermal bath, whose probability density is only of relevance. This approach allows us to interpret the results of atomistic spin dynamics simulations in appropriate macroscopic terms. After presenting the numerical implementation of this methodology, a typical study of a magnetic device based on superparamagnetic iron monolayers is presented, as an example. The results are compared to experimental data and allow us to validate the atomistic spin dynamics simulations.