On considère une marche aléatoire réelle dont les accroissements sont construits à partir d’une chaîne de Markov définie sur un espace abstrait. Sous des hypothèses de centrage de la marche et de décroissance rapide de la dépendance de la chaîne de Markov par rapport à son passé (de type trou spectral), on se propose d’étudier le premier instant pour lequel une telle marche markovienne passe dans les négatifs. Plus précisément, on établit que le comportement asymptotique de la probabilité de survie est inversement proportionnel à la racine carrée du temps. On étend également à nos modèles markoviens le résultat des marches aléatoires aux accroissements indépendants suivant : la loi asymptotique de la marche aléatoire renormalisée et conditionnée à rester positive est la loi de Rayleigh. Dans un deuxième temps, on restreint notre modèle aux cas où la chaîne de Markov définissant les accroissements de la marche aléatoire est à valeurs dans un espace d’états fini. Sous cette hypothèse et lorsque que la marche est dite non-lattice, on complète nos résultats par des théorèmes locaux pour la marche aléatoire conjointement avec le fait qu’elle soit restée positive. Enfin on applique ces développements aux processus de branchement soumis à un environnement aléatoire, lui-même défini à partir d’une chaîne de Markov à valeurs dans un espace d’états fini. On établit le comportement asymptotique de la probabilité de survie du processus dans le cas critique et les trois cas sous-critiques (fort, intermédiaire et faible) / We consider a real random walk whose increments are constructed by a Markov chain definedon an abstract space. We suppose that the random walk is centred and that the dependence of the Markov walk in its past decreases exponentially fast (due to the spectral gap property). We study the first time when the random walk exits the positive half-line and prove that the asymptotic behaviour of the survey probability is inversely proportional to the square root of the time. We extend also to our Markovian model the following result of random walks with independent increments: the asymptotic law of the random walk renormalized and conditioned to stay positive is the Rayleigh law. Subsequently, we restrict our model to the cases when the Markov chain defining the increments of the random walk takes its values on a finite state space. Under this assumption and the condition that the walk is non-lattice, we complete our results giving local theorems for the random walk conditioned to stay positive. Finally, we apply these developments to branching processes under a random environment defined by a Markov chain taking its values on a finite state space. We give the asymptotic behaviour of the survey probability of the process in the critical case and the three subcritical cases (strongly, intermediate and weakly).

The aim of this work was to investigate if interesting patterns could be found in time series radar data that had been discretized by the algorithm ARAVQ into symbolic representations and if the ARAVQ thus might be suitable for use in the radar domain. An experimental study was performed where the ARAVQ was used to create symbolic representations of data sets with radar data. Two experiments were carried out that used a Markov model to calculate probabilities used for discovering potentially interesting patterns. Some of the most interesting patterns were then investigated further. Results have shown that the ARAVQ was able to create accurate representations for several time series and that it was possible to discover patterns that were interesting and represented higher level concepts. However, the results also showed that the ARAVQ was not able to create accurate representations for some of the time series.

Cette thèse est consacrée à l'étude des objets binaires du Système solaire selon deux axes principaux. Premièrement, nous examinons les paramètres physiques, tels que la taille et l'albédo des binaires transneptuniens, obtenus à partir des mesures de flux thermique en infrarouge par les télescopes spatiaux Herschel et Spitzer. Avec ces paramètres, nous comparons les objets binaires avec les transneptuniens sans satellite. Cette analyse montre que les distributions de tailles dans les deux populations sont différentes. Nous supposons que cette tendance est liée à la prépondérance des petits binaires dans le groupe des objets \og froide \fg{}, qui est plus favorable à la survie des binaires, parmi les autre groupes.De plus, nous étudions les corrélations entre la taille et l'albédo et d'autres paramètres physiques et orbitaux pour la population des binaires. Cette étude montre les fortes corrélations suivantes: entre la taille et la masse, la taille et l'inclinaison héliocentrique, la taille et la différence de magnitudes des composantes. L'étude trouve également deux corrélations moins significatives -- la densité avec la taille et la densité avec l'albédo -- qui nécessitent des vérifications ultérieures avec des données complémentaires. Nous donnons une interprétation possible des résultats du point de vue des différents modèles de formation de tels objets.Deuxièmement, nous présentons une nouvelle méthode de détermination d'orbite mutuelle d'un système binaire. Cette méthode est basée sur la technique de Monte-Carlo par chaînes de Markov avec une approche bayésienne. L'algorithme, développé dans cette thèse, permet de déterminer où d'ajuster les paramètres d'une orbite képlérienne ou d'une orbite perturbée à partir des observations simulées et réelles. Nous montrons que la méthode peut être efficace même pour un petit nombre d'observations et sans condition initiale particulière. / This thesis is devoted to the study of binary objects in the Solar System and explores two main themes. First, we examined physical parameters, such as size and albedo of trans-Neptunian binaries, obtained from thermal flux measurements by the Herschel and Spitzer space telescopes. Within these parameters, we compared binary objects with simple trans-Neptunian objects without satellites. This analysis showed that the size distributions of two populations are different. We assume that this trend is related to the predominance of small binaries of cold classical group, which may be more favourable for the survival of binaries, among other groups.In addition, we studied the correlations between the size and albedo and other physical and orbital parameters of the binaries population. This study obtained the following strong correlation: size vs. system mass, size vs. heliocentric inclination, size vs. magnitude difference of components. We also found two less significant correlations -- bulk density vs. size and bulk density vs. albedo -- which require further verification with additional data. We then set out a possible interpretations of the results from the perspective of different formation models of such objects. Secondly, we have presented a new method of binary system mutual orbit determination. This method is based on the Monte Carlo Markov chain techniques with a Bayesian approach. The algorithm developed in this thesis is used for Keplerian or perturbed orbit fitting to simulated or real observations. We show that the method can be effective even for a small number of observations and without regard to particular initial conditions.

Abstract:! The aim of this thesis is to find the optimal control of Markov chain with discounted evaluation of transitions in discrete and also in continuous time. We present Howard's iterative algorithm, the algorithm for finding the optimal control. Then the strategy is applied to the problem of optimal trading, where the goal is to maximize market price of the portfolio in infinite time horizont, given the existence of the proportional transaction costs. Market price is simulated with Brownian motion.

The global importance of and need for sustainable development demand an informed decision-making process from the built environment to ensure optimum service life, which depends on the ability to quantify changes in condition of building materials over time. The objective of this thesis is to develop a model, which translates expert knowledge and reasoning into probability values through the application of Fuzzy Logic Artificial Intelligence to supplement limited historical performance data on degradation of building materials for the development of Markov Chain transitional probability matrices to predict service life, condition changes over time, and consequences of maintenance levels on service life of buildings. The Markov Chain methodology, a stochastic approach used for simulating the transition from one condition to another over time, has been identified as the preferred method for service life prediction by a number of studies. Limited availability of historic performance data on degradation and durability of building materials, required to populate the Markovian transition probability matrices, however restricts the application of the Markov Chain methodology. The durability and degradation factors, defined as design and maintenance levels, material and workmanship quality, external and internal climate, and operational environment, similar to the factors identified in the state-of-the–art ‘Factor Method’ for service life prediction, and current condition are rated on a uniform colour-coded five-point rating system and used to develop “IF-THEN” rules based on expert knowledge and reasoning. Fuzzy logic artificial intelligence is then used to translate these rules into crisp probability values to populate the Markovian transitional probability matrices. Historic performance data from previous condition assessments of six academic hospitals are used to calibrate and test the model. There is good correlation between the transitional probability matrices developed for the proposed model and other Markov applications in concrete bridge deck deterioration and roof maintenance models, based on historic performance data collected over extended periods, which makes the correlation more significant. Proof is presented that the Markov Chain can be used to calculate the estimated service life of a building or component, quantify changes in condition over time and determine the effect of maintenance levels on service life. It is also illustrated that the limited availability of historic performance data on degradation of building materials can be supplemented with expert knowledge, translated into probability values through the application of Fuzzy Logic Artificial Intelligence, to develop transition probability matrices for the Markov Chain. The proposed model can also be used to determine the estimated loss of or gain in service life of a building or component for various levels of maintenance. / Thesis (PhD(Civil Engineering))--University of Pretoria, 2007. / Civil Engineering / unrestricted

No description available.

The objective of the study is to apply Markov chain analysis to analyze student flow through King Abdul Aziz University (KAU) in Saudi Arabia, and to predict important metrics such as graduation and dropout rates. This objective arises from examination of the policies of KAU University. We begin with background information detailing the subject of study, then move into a general outline of stochastic processes. We then use these methods to construct a specific matrix of transition probabilities with data from the university student population. Finally, we discuss the calculation of the possibilities of transition between each level of study and the average time a student takes to complete each stage. The study uses Markov chains with these outcomes to analyze student retention data from the Department of Mathematics at KAU. From this analysis, the study will provide university policy recommendations that can be generalized to examine other universities.

Estudamos o comportamento da familia aresta-triangulo de grafos aleatorios exponenciais (ERG) usando metodos de Monte Carlo baseados em Cadeias de Markov. Comparamos contagens de subgrafos e correlacoes entre arestas de ergs as de Grafos Aleatorios Binomiais (BRG, tambem chamados de Erdos-Renyi). E um resultado teorico conhecido que para algumas parametrizacoes os limites das contagens de subgrafos de ERGs convergem para os de BRGs, assintoticamente no numero de vertices [BBS11, CD11]. Observamos esse fenomeno em grafos com poucos (20) vertices em nossas simulacoes. / We study the behavior of the edge-triangle family of exponential random graphs (ERG) using the Markov Chain Monte Carlo method. We compare ERG subgraph counts and edge correlations to those of the classic Binomial Random Graph (BRG, also called Erdos-Renyi model). It is a known theoretical result that for some parameterizations the limit ERG subgraph counts converge to those of BRGs, as the number of vertices grows [BBS11, CD11]. We observe this phenomenon on graphs with few (20) vertices in our simulations.

Species conservation initiatives play an important role in ecological studies. Occupancy models have been a useful tool for ecologists to make inference about species distribution and occurrence. Bayesian methodology is a popular framework used to model the relationship between species and environmental variables. In this dissertation we develop a Gibbs sampling method using a logit link function in order to model posterior parameters of the single-season spatial occupancy model. We incorporate the widely used Intrinsic Conditional Autoregressive (ICAR) prior model to specify the spatial random effect in our sampler. We also develop OccuSpytial, a statistical package implementing our Gibbs sampler in the Python programming language. The aim of this study is to highlight the computational efficiency that can be obtained by employing several techniques, which include exploiting the sparsity of the precision matrix of the ICAR model and also making use of Polya-Gamma latent variables to obtain closed form expressions for the posterior conditional distributions of the parameters of interest. An algorithm for efficiently sampling from the posterior conditional distribution of the spatial random effects parameter is also developed and presented. To illustrate the sampler's performance a number of simulation experiments are considered, and the results are compared to those obtained by using a Gibbs sampler incorporating Restricted Spatial Regression (RSR) to specify the spatial random effect. Furthermore, we fit our model to the Helmeted guineafowl (Numida meleagris) dataset obtained from the 2nd South African Bird Atlas Project database in order to obtain a distribution map of the species. We compare our results with those obtained from the RSR variant of our sampler, those obtained by using the stocc statistical package (written using the R programming language), and those obtained from not specifying any spatial information about the sites in the data. It was found that using RSR to specify spatial random effects is both statistically and computationally more efficient that specifying them using ICAR. The OccuSpytial implementations of both ICAR and RSR Gibbs samplers has significantly less runtime compared to other implementations it was compared to.

Speech is intuitive, fast and easy to generate, but it is hard to index and easy to forget. What is more, listening to speech is slow. Text is easier to store, process and consume, both for computers and for humans, but writing text is slow and requires some intention. In this thesis, we study speech recognition which allows converting speech into text, making it easier both to create and to use information. Our tool of study is Hidden Markov Models which is one of the most important machine learning models in speech and language processing. The aim of this thesis is to do a rst study in Hidden Markov Models and understand their importance, particularly in speech recognition. We will go through three fundamental problems that come up naturally with Hidden Markov Models: to compute a likelihood of an observation sequence, to nd an optimal state sequence given an observation sequence and the model, and to adjust the model parameters. A solution to each problem will be given together with an example and the corresponding simulations using MatLab. The main importance lies in the last example, in which a rst approach to speech recognition will be done.