This dissertation introduces advances in the Wiener path integral (WPI) technique for determining efficiently and accurately the stochastic response of diverse nonlinear dynamical systems. First, a novel, general, formalism of the WPI technique is developed to account, in a direct manner, also for systems with non-Markovian response processes. Specifically, the probability of a path and the associated transition probability density function (PDF) corresponding to the Wiener excitation process are considered. Next, a functional change of variables is employed, in conjunction with the governing stochastic differential equation, for deriving the system response joint transition PDF as a functional integral over the space of possible paths connecting the initial and final states of the response vector. In comparison to alternative derivations in the literature, the herein-developed formalism does not require the Markovian assumption for the system response process. Overall, the veracity and mathematical legitimacy of the WPI technique to treat also non-Markovian system response processes are demonstrated. In this regard, nonlinear systems with a history-dependent state, such as hysteretic structures or oscillators endowed with fractional derivative elements, can be accounted for in a direct manner—that is, without resorting to any ad hoc modifications of the WPI technique pertaining, typically, to employing additional auxiliary filter equations and state variables. Next, a reduced-order WPI formulation is introduced for efficiently determining the stochastic response of diverse nonlinear systems with fractional derivative elements. This formulation can be also construed as a dimension reduction approach that renders the associated computational cost independent of the total number of stochastic dimensions of the problem. In fact, the proposed technique can determine, directly, any lower-dimensional joint response PDF corresponding to a subset only of the response vector components. This is accomplished by utilizing an appropriate combination of fixed and free boundary conditions in the related variational, functional minimization, problem. Notably, the reduced-order WPI formulation is particularly advantageous for problems where the interest lies in, few only, specific degrees-of-freedom whose stochastic response is critical for the design and optimization of the overall system. Further, an extrapolation approach within the WPI technique is developed that significantly enhances the computational efficiency of the technique without, practically, affecting the associated degree of accuracy. Overall, the WPI technique treats the system response joint transition PDF as a functional integral over the space of all possible paths connecting the initial and the final states of the response vector. Next, the functional integral is evaluated, ordinarily, by considering the contribution only of the most probable path. This corresponds to an extremum of the functional integrand, and is determined by solving a functional minimization problem that takes the form of a deterministic boundary value problem (BVP). This BVP corresponds to a specific grid point of the response PDF domain. Remarkably, the BVPs corresponding to two neighboring grid points not only share the same equations, but also the boundary conditions differ only slightly. This unique aspect of the technique is exploited, and it is shown that solution of a BVP and determination of the response PDF value at a specific grid point can be used for extrapolating and estimating efficiently and accurately the PDF values at neighboring points without the need for considering additional BVPs. Last, a joint time-space extrapolation approach within WPI technique is developed for determining, efficiently and accurately, the non-stationary stochastic response of diverse nonlinear dynamical systems. The approach can be construed as an extension of the above space-domain extrapolation scheme to account also for the temporal dimension. Specifically, it is shown that information inherent in the time-history of an already determined most probable path can be used for evaluating points of the response PDF corresponding to arbitrary time instants, without the need for solving additional BVPs. In a nutshell, relying on the aforementioned unique and advantageous features of the WPI-based BVP, the complete non-stationary response joint PDF is determined, first, by calculating numerically a relatively small number of PDF points, and second, by extrapolating in the joint time-space domain at practically zero additional computational cost. Compared to an alternative brute-force implementation of the WPI technique, and to a standard Monte Carlo simulation (MCS) solution treatment, the developed extrapolation approach reduces the associated computational cost by several orders of magnitude. Several representative numerical examples are considered to demonstrate the reliability of the developed techniques. Juxtapositions with pertinent MCS data are included as well.

No description available.

Recent advancements in timber harvesting systems analysis have been almost exclusively simulation based. A similar degree of effort in developing analytic models has been conspicuously absent. That part of timber harvesting analysis where simulation plays its most vital role is the study of machine interactions. The importance of machine interactions lies in determining the proportions of delay, idle and productive time for the interacting machines. This in turn, is important for balancing productivity so that no single component of the interaction is accumulating excessive amounts of delay or idle time. The objective of this study was to determine the feasibility of applying Markov process theory to the analysis of timber harvesting systems and components. Through modeling the interaction between a fixed location slasher and a grapple skidder, it is shown how a Markov model can be used to obtain proportions of delay, idle and productive time. Unlike the statistical solutions derived from simulation models, the Markov model improves upon this by providing an analytic solution. The Markov model also avoids the problems of correlated output data from simulations by explicitly recognizing that any possible future state is dependent only on the current state of the system and is conditionally independent of the past history of the system. The methodology for building a Markov model requires dealing with only two probability distributions, the Erlang and mixed Erlang, for modeling time based activities (such as cycle times) of the interacting machines. These probability distributions in turn, provide the necessary data for developing a system of algebraic equations for solving the Markov process model. While this is the first step in applying stochastic process theory to timber harvesting analysis, the results of this study indicate that the technique has considerable potential for application in timber harvesting system modeling. / Ph. D.

Learning human driving behaviors is a promising approach to enhance the performance of self-driving vehicles. By understanding and replicating the complex decision-making processes of human drivers, developers are able to program vehicles to navigate real-world scenarios with better safety and reliability. This strategy not only improves the adaptability of autonomous driving systems but also ensures their capability to manage unexpected situations on the road. Traditional Imitation Learning (IL) methods have been a cornerstone in achieving this objective, which typically assume that the expert demonstrations follow Markov Decision Processes (MDPs). However, in reality, this assumption does not always hold true. Spurious correlation may exist through the paths of historical variables because of the existence of unobserved confounders. Additionally, agents may differ in their sensory capabilities, meaning that some of the expert's features might not always be observed by the imitator. Accounting for the latent causal relationships from unobserved variables to outcomes, this dissertation focuses on Causal Imitation Learning for learning driver behaviors. First of all, this dissertation develops a sequential causal template that generalizes the default MDP settings to one with Unobserved Confounders (MDPUC-HD). Based on it, a sufficient graphical criterion is developed to determine when ignoring causality leads to poor performances in MDPUC-HD. Through the framework of Adversarial Imitation Learning (AIL), a procedure is developed to imitate the expert policy by blocking 𝜋-backdoor paths at each time step. The proposed methods are evaluated on a synthetic dataset and a real-world highway driving dataset (NGSIM), both demonstrating that the proposed procedure significantly outperforms non-causal imitation learning methods. Generalizing the findings across various graphical settings, this dissertation further proposes novel graphical conditions that allow the imitator to learn a policy performing as well as the expert's behavior policy, even when the imitator and the expert's state-action space disagree, and unobserved confounders (UCs) are generally present. When provided with parametric knowledge about the unknown reward function, such a policy is able to outperform expert performance. Additionally, our method is easily extensible with the existing IRL algorithms, including the multiplicative-weights algorithm (MWAL) and the generative adversarial imitation learning (GAIL), enhancing their adaptability to diverse conditions. The validity of the framework has been rigorously tested through extensive experiments, covering different dimensions of the causal imitation learning tasks, including: different causal assumptions, parametric families of reward functions, and multiple datasets, and infinite horizons. The results consistently affirm the superiority of the causal imitation learning approach over traditional methods, particularly in environments with unobserved confounders and different input covariate spaces.

In view of the fact that a same complex phenomenon can be approached by different conceptual frameworks, it is natural to inquire on the possibility to find connections between different types of quantities, such as topological, dynamical, statistical or thermodynamical, characterizing the same system. The present work is built on the idea that this line of approach can provide interesting insights on possible universal principles governing complex phenomena. In Chapter I we introduce concepts and tools of dynamical systems and thermodynamics as applied in macroscopic scale description as well as, for a later use, a number of selected representative models. In Chapter II we briefly present the elements of the theory of Markov processes describing a large class of stochastic process and also introduce some important concepts on the probabilistic description of deterministic systems. This chapter ends with a thermodynamic formulation accounting for the evolution of the entropy under the effect of stochastic fluctuations. In Chapter III, after introducing the main concepts and recent advances in network theory, we provide a connection between dynamical systems and network theory, which shows how universal structural properties of evolving networks can arise from deterministic dynamics. More specifically, we show explicitly the relation between the connectivity patterns of these networks and the indicators of the underlying dynamics, such as the local Lyapunov exponents. Our analysis is applied to representative models of chaotic maps, chaotic flows and is finally extended to stochastic processes. In Chapter IV we address the inverse problem, namely, processes whose dynamics is determined, in part, by the structure of the network in which they are embedded. In particular, we focus on systems of particles diffusing on a lattice and reacting instantaneously upon encountering each other. We study the role of the topology, the degree of synchronicity of motion and the reaction mechanism on the efficiency of the process. This lead us to identify a common generic mechanism responsible for the behavior of the efficiency, as a function of the control parameters. Finally, in Chapter V we study the connection between the topology and the thermodynamic properties of reaction networks, with focus on the entropy production and the system’s efficiency at nonequilibrium steady states. We also explore the connection between dynamic and thermodynamic properties of nonlinear feedbacks, as well as the response properties of reaction networks against both deterministic and stochastic external perturbations. We address networks of varying topologies, from regular lattices to complex structures./Le présent travail s’inscrit dans le domaine de recherche sur les systèmes complexes. Différentes approches, basées des systèmes dynamiques, de la thermodynamique des systèmes hors d’équilibre, de la physique statistique et, plus récemment, de la théorie des réseaux, sont combinés afin d’explorer des liens entre différentes types de grandeurs qui caractérisent certaines classes de comportements complexes. Dans le Chapitre I nous introduisons les principaux concepts et outils de systèmes dynamiques et de thermodynamique. Dans le Chapitre II nous présentons premièrement des éléments de la théorie de processus de Markov, ainsi que les concepts à la base de la description probabiliste des systèmes déterministes. Nous finissons le chapitre en proposant une formulation thermodynamique qui décrit l’évolution de l’entropie hors d’équilibre, soumis à l’influence de fluctuations stochastiques. Dans le Chapitre III nous introduisons les concepts de base en théorie des réseaux, ainsi qu’un résumé générale des progrès récents dans le domaine. Nous établissons ensuite une connexion entre la théorie des systèmes dynamiques et la théorie de réseaux. Celle-ci permet d’approfondir la compréhension des mécanismes responsables de l’émergence des propriétés structurelles dans des réseaux crées par des lois dynamiques déterministes. En particulier, nous mettons en évidence la relation entre des motifs de connectivité de ce type de réseaux et des indicateurs de la dynamique sous-jacente, tel que des exposant de Lyapounov locaux. Notre analyse est illustrée par des applications et des flots chaotiques et étendue à des processus stochastiques. Dans le Chapitre IV nous étudions le problème complémentaire, à savoir, celui de processus dont la dynamique est déterminée, en partie, par la structure du réseau dans lequel elle se déroule. Plus précisément, nous nous concentrons sur le cas de systèmes de particules réactives, diffusent au travers d’un réseau et réagissant instantanément lorsqu’un rencontre se produit entre elles. Nous étudions le rôle de la topologie, du degré de synchronicité des mouvements et aussi celui du mécanisme de réaction sur l’efficacité du processus. Dans les différents modèles étudiés, nous identifions un mécanisme générique commun, responsable du comportement de l’efficacité comme fonction des paramètres de contrôle. Enfin, dans le Chapitre V nous abordons la connexion entre la topologie et les propriétés thermodynamiques des réseaux de réactions, en analysant le comportement local et global de la production d’entropie et l’efficacité du système dans des état stationnaires de non-équilibre. Nous explorons aussi la connexion entre la dynamique et les propriétés de boucles de rétroaction non linéaires, ainsi que les propriétés de réponse des réseaux de réaction à des perturbations stochastiques et déterministes externes. Nous considérons le cas de réseaux à caractère régulier aussi bien que celui de réseaux complexes.<p><p> / Doctorat en Sciences / info:eu-repo/semantics/nonPublished

This research investigates existing relationships between the three apparently unrelated subjects: Markov process, Semigroups and Partial difierential equations. Markov processes define semigroups through their transition functions. Conversely particular semigroups determine transition functions and can be regarded as Markov processes. We have exploited these relationships to study some Markov chains. The infnitesimal generator of a Feller semigroup on the closure of a bounded domain of Rn; (n ^ 2), is an integro-diferential operator in the interior of the domain and verifes a boundary condition. The existence of a Feller semigroup defined by a diferential operator and a boundary condition is due to the existence of solution of a bounded value problem. From this result other existence suficient conditions on the existence of Feller semigroups have been obtained and we have applied some of them to construct Feller semigroups on the unity disk of R2. / Decision Sciences / M. Sc. (Operations Research)

Thesis (MSc)--Stellenbosch University, 2014. / ENGLISH ABSTRACT: We study random walks on nite graphs. The reader is introduced to general Markov chains before we move on more specifically to random walks on graphs. A random walk on a graph is just a Markov chain that is time-reversible. The main parameters we study are the hitting time, commute time and cover time. We nd novel formulas for the cover time of the subdivided star graph and broom graph before looking at the trees with extremal cover times. Lastly we look at a connection between random walks on graphs and electrical networks, where the hitting time between two vertices of a graph is expressed in terms of a weighted sum of e ective resistances. This expression in turn proves useful when we study the cover cost, a parameter related to the cover time. / AFRIKAANSE OPSOMMING: Ons bestudeer toevallige wandelings op eindige gra eke in hierdie tesis. Eers word algemene Markov kettings beskou voordat ons meer spesi ek aanbeweeg na toevallige wandelings op gra eke. 'n Toevallige wandeling is net 'n Markov ketting wat tyd herleibaar is. Die hoof paramaters wat ons bestudeer is die treftyd, pendeltyd en dektyd. Ons vind oorspronklike formules vir die dektyd van die verdeelde stergra ek sowel as die besemgra ek en kyk daarna na die twee bome met uiterste dektye. Laastens kyk ons na 'n verband tussen toevallige wandelings op gra eke en elektriese netwerke, waar die treftyd tussen twee punte op 'n gra ek uitgedruk word in terme van 'n geweegde som van e ektiewe weerstande. Hierdie uitdrukking is op sy beurt weer nuttig wanneer ons die dekkoste bestudeer, waar die dekkoste 'n paramater is wat verwant is aan die dektyd.

Thesis (PhD)--Stellenbosch University, 2012. / ENGLISH ABSTRACT: Multi-state models are used in this dissertation to model panel data, also known as longitudinal or cross-sectional time-series data. These are data sets which include units that are observed across two or more points in time. These models have been used extensively in medical studies where the disease states of patients are recorded over time. A theoretical overview of the current multi-state Markov models when applied to panel data is presented and based on this theory, a simulation procedure is developed to generate panel data sets for given Markov models. Through the use of this procedure a simulation study is undertaken to investigate the properties of the standard likelihood approach when fitting Markov models and then to assess its shortcomings. One of the main shortcomings highlighted by the simulation study, is the unstable estimates obtained by the standard likelihood models, especially when fitted to small data sets. A Bayesian approach is introduced to develop multi-state models that can overcome these unstable estimates by incorporating prior knowledge into the modelling process. Two Bayesian techniques are developed and presented, and their properties are assessed through the use of extensive simulation studies. Firstly, Bayesian multi-state models are developed by specifying prior distributions for the transition rates, constructing a likelihood using standard Markov theory and then obtaining the posterior distributions of the transition rates. A selected few priors are used in these models. Secondly, Bayesian multi-state imputation techniques are presented that make use of suitable prior information to impute missing observations in the panel data sets. Once imputed, standard likelihood-based Markov models are fitted to the imputed data sets to estimate the transition rates. Two different Bayesian imputation techniques are presented. The first approach makes use of the Dirichlet distribution and imputes the unknown states at all time points with missing observations. The second approach uses a Dirichlet process to estimate the time at which a transition occurred between two known observations and then a state is imputed at that estimated transition time. The simulation studies show that these Bayesian methods resulted in more stable results, even when small samples are available. / AFRIKAANSE OPSOMMING: Meerstadium-modelle word in hierdie verhandeling gebruik om paneeldata, ook bekend as longitudinale of deursnee tydreeksdata, te modelleer. Hierdie is datastelle wat eenhede insluit wat oor twee of meer punte in tyd waargeneem word. Hierdie tipe modelle word dikwels in mediese studies gebruik indien verskillende stadiums van ’n siekte oor tyd waargeneem word. ’n Teoretiese oorsig van die huidige meerstadium Markov-modelle toegepas op paneeldata word gegee. Gebaseer op hierdie teorie word ’n simulasieprosedure ontwikkel om paneeldatastelle te simuleer vir gegewe Markov-modelle. Hierdie prosedure word dan gebruik in ’n simulasiestudie om die eienskappe van die standaard aanneemlikheidsbenadering tot die pas vanMarkov modelle te ondersoek en dan enige tekortkominge hieruit te beoordeel. Een van die hoof tekortkominge wat uitgewys word deur die simulasiestudie, is die onstabiele beramings wat verkry word indien dit gepas word op veral klein datastelle. ’n Bayes-benadering tot die modellering van meerstadiumpaneeldata word ontwikkel omhierdie onstabiliteit te oorkom deur a priori-inligting in die modelleringsproses te inkorporeer. Twee Bayes-tegnieke word ontwikkel en aangebied, en hulle eienskappe word ondersoek deur ’n omvattende simulasiestudie. Eerstens word Bayes-meerstadium-modelle ontwikkel deur a priori-verdelings vir die oorgangskoerse te spesifiseer en dan die aanneemlikheidsfunksie te konstrueer deur van standaard Markov-teorie gebruik te maak en die a posteriori-verdelings van die oorgangskoerse te bepaal. ’n Gekose aantal a priori-verdelings word gebruik in hierdie modelle. Tweedens word Bayesmeerstadium invul tegnieke voorgestel wat gebruik maak van a priori-inligting om ontbrekende waardes in die paneeldatastelle in te vul of te imputeer. Nadat die waardes ge-imputeer is, word standaard Markov-modelle gepas op die ge-imputeerde datastel om die oorgangskoerse te beraam. Twee verskillende Bayes-meerstadium imputasie tegnieke word bespreek. Die eerste tegniek maak gebruik van ’n Dirichletverdeling om die ontbrekende stadium te imputeer by alle tydspunte met ’n ontbrekende waarneming. Die tweede benadering gebruik ’n Dirichlet-proses om die oorgangstyd tussen twee waarnemings te beraam en dan die ontbrekende stadium te imputeer op daardie beraamde oorgangstyd. Die simulasiestudies toon dat die Bayes-metodes resultate oplewer wat meer stabiel is, selfs wanneer klein datastelle beskikbaar is.

Approved for public release, distribution is unlimited / A Markov Model was used to determine the number of nurses the Navy must gain each year in order to maintain desired end strength. Significant characteristics affecting career progression of individuals in the Navy Nurse Corps were identified. The characteristic of primary concern, accession source, was determined to be significant. Markov models were created to identify personnel flow from ENS through LCDR. The models end-strength projections for 2006-2009 were then compared to Nurse Corps targeted end-strengths for this same period. Several scenarios were run to minimize overages and underages in rank distribution. Optimization was achieved by changing both the distribution of accession sources and the distribution of recruited ranks. Optimal distribution of accession source and rank are dependant upon the degree of accepTable deviation from these targets. As stated above we were not able to acquire this information limiting our ability to accurately forecast optimized distribution of accession source or rank. The Markov Model demonstrated that the Nurse Corps current business practices optimize accessions for two year projections. Increasing variation between the current force structure plan and our models projections suggest that greater efficiency could be obtained in the out-years. This Markov Model provides a tool for improving extended forecasts. / Lieutenant Commander, United States Navy

Approved for public release, distribution is unlimited. / High-resolution combat simulations that model urban combat currently use computationally expensive algorithms to represent urban target acquisition at the entity level. While this may be suitable for small-scale urban combat scenarios, simulation run time can become unacceptably long for larger scenarios. Consequently, there is a need for models that can lend insight into target acquisition in urban terrain for largescale scenarios in an acceptable length of time. This research develops urban target acquisition models that can be substituted for existing physicsbased or computationally expensive combat simulation algorithms and result in faster simulation run time with an acceptable loss of aggregate simulation accuracy. Specifically, this research explores (1) the adaptability of probability of line of sight estimates to urban terrain; (2) how cumulative distribution functions can be used to model the outcomes when a set of sensors is employed against a set of targets; (3) the uses for Markov Chains and Event Graphs to model the transition of a target among acquisition states; and (4) how a system of differential equations may be used to model the aggregate flow of targets from one acquisition state to another. / Captain, United States Marine Corps