<p>I denna avhandling studerar vi nödvändiga och tillräckliga villkor för svag konvergens av första-sällan-händelsetider för semi-Markovska processer.</p><p>I introduktionen ger vi nödvändiga grundläggande definitioner och beskrivningar av modeller som betraktas i avhandlingen, samt ger några exempel på situationer i vilka metoder av första-sällan-händelsetider kan vara lämpliga att använda. Dessutom analyserar vi publicerade resultat om asymptotiska problem för stokastiska funktionaler som definieras på semi-Markovska processer.</p><p>I artikel A betraktar vi första-sällan-händelsetider för semi-Markovska processer med en ändlig mängd av lägen. Vi ger också en sammanfattning av våra resultat om nödvändiga och tillräckliga villkor för svag konvergens, samt diskuterar möjliga tillämpningar inom aktuarie-området.</p><p>I artikel B redovisar vi i detalj de resultat som annonseras i artikel A och bevisen för dem. Vi ger också nödvändiga och tillräckliga villkor för svag konvergens av första-sällan-händelsetider för semi-Markovska processer med en ändlig mängd av lägen i ett icke-triangulärt tillstånd. Dessutom beskriver vi med hjälp av Laplacetransformationen klassen av alla möjliga gränsfördelningar.</p><p>I artikel C studerar vi villkor av svag konvergens av flöden av sällan-händelser i ett icke-triangulärt tillstånd. Vi formulerar nödvändiga och tillräckliga villkor för konvergens, och beskriver klassen av alla möjliga gränsflöden. Vi tillämpar också våra resultat i asymptotisk analys av icke-ruin-sannolikheten för störda riskprocesser.</p><p>I artikel D ger vi nödvändiga och tillräckliga villkor för svag konvergens av första-sällan-händelsetider för semi-Markovska rocesser med en ändlig mängd av lägen i ett triangulärt tillstånd, samt beskriver klassen av alla möjliga gränsfördelningar. Resultaten utvidgar slutsatser från artikel B till att gälla för ett allmänt triangulärt tillstånd.</p><p>I artikel E ger vi nödvändiga och tillräckliga villkor för svag konvergens av flöden av sällan-händelser för semi-Markovska processer i ett triangulärt tillstånd. Detta generaliserar resultaten från artikel C till att beskriva ett allmänt triangulärt tillstånd. Vidare ger vi tillämpningar av våra resultat på asymptotiska problem av störda riskprocesser och till kösystemen med snabb service.</p> / <p>In this thesis we study necessary and sufficient conditions for weak convergence of first-rare-event times for semi-Markov processes, we describe the class of all possible limit distributions, and give the applications of the results to risk theory and queueing systems.</p><p>In paper <b>A</b>, we consider first-rare-event times for semi-Markov processes with a finite set of states, and give a summary of our results concerning necessary and sufficient conditions for weak convergence of first-rare-event times and their actuarial applications.</p><p>In paper <b>B</b>, we present in detail results announced in paper <b>A</b> as well as their proofs. We give necessary and sufficient conditions for weak convergence of first-rare-event times for semi-Markov processes with a finite set of states in non-triangular-array mode and describe the class of all possible limit distributions in terms of their Laplace transforms.</p><p>In paper <b>C</b>, we study the conditions for weak convergence for flows of rare events for semi-Markov processes with a finite set of states in non-triangular array mode. We formulate necessary and sufficient conditions of convergence and describe the class of all possible limit stochastic flows. In the second part of the paper, we apply our results to the asymptotical analysis of non-ruin probabilities for perturbed risk processes.</p><p>In paper <b>D</b>, we give necessary and sufficient conditions for the weak convergence of first-rare-event times for semi-Markov processes with a finite set of states in triangular array mode as well as describing the class of all possible limit distributions. The results of paper <b>D</b> extend results obtained in paper <b>B</b> to a general triangular array mode.</p><p>In paper <b>E</b>, we give the necessary and sufficient conditions for weak convergence for the flows of rare events for semi-Markov processes with a finite set of states in triangular array case. This paper generalizes results obtained in paper <b>C</b> to a general triangular array mode. In the second part of the paper, we present applications of our results to asymptotical problems of perturbed risk processes and to queueing systems with quick service</p>

I denna avhandling studerar vi nödvändiga och tillräckliga villkor för svag konvergens av första-sällan-händelsetider för semi-Markovska processer. I introduktionen ger vi nödvändiga grundläggande definitioner och beskrivningar av modeller som betraktas i avhandlingen, samt ger några exempel på situationer i vilka metoder av första-sällan-händelsetider kan vara lämpliga att använda. Dessutom analyserar vi publicerade resultat om asymptotiska problem för stokastiska funktionaler som definieras på semi-Markovska processer. I artikel A betraktar vi första-sällan-händelsetider för semi-Markovska processer med en ändlig mängd av lägen. Vi ger också en sammanfattning av våra resultat om nödvändiga och tillräckliga villkor för svag konvergens, samt diskuterar möjliga tillämpningar inom aktuarie-området. I artikel B redovisar vi i detalj de resultat som annonseras i artikel A och bevisen för dem. Vi ger också nödvändiga och tillräckliga villkor för svag konvergens av första-sällan-händelsetider för semi-Markovska processer med en ändlig mängd av lägen i ett icke-triangulärt tillstånd. Dessutom beskriver vi med hjälp av Laplacetransformationen klassen av alla möjliga gränsfördelningar. I artikel C studerar vi villkor av svag konvergens av flöden av sällan-händelser i ett icke-triangulärt tillstånd. Vi formulerar nödvändiga och tillräckliga villkor för konvergens, och beskriver klassen av alla möjliga gränsflöden. Vi tillämpar också våra resultat i asymptotisk analys av icke-ruin-sannolikheten för störda riskprocesser. I artikel D ger vi nödvändiga och tillräckliga villkor för svag konvergens av första-sällan-händelsetider för semi-Markovska rocesser med en ändlig mängd av lägen i ett triangulärt tillstånd, samt beskriver klassen av alla möjliga gränsfördelningar. Resultaten utvidgar slutsatser från artikel B till att gälla för ett allmänt triangulärt tillstånd. I artikel E ger vi nödvändiga och tillräckliga villkor för svag konvergens av flöden av sällan-händelser för semi-Markovska processer i ett triangulärt tillstånd. Detta generaliserar resultaten från artikel C till att beskriva ett allmänt triangulärt tillstånd. Vidare ger vi tillämpningar av våra resultat på asymptotiska problem av störda riskprocesser och till kösystemen med snabb service. / In this thesis we study necessary and sufficient conditions for weak convergence of first-rare-event times for semi-Markov processes, we describe the class of all possible limit distributions, and give the applications of the results to risk theory and queueing systems. In paper <b>A</b>, we consider first-rare-event times for semi-Markov processes with a finite set of states, and give a summary of our results concerning necessary and sufficient conditions for weak convergence of first-rare-event times and their actuarial applications. In paper <b>B</b>, we present in detail results announced in paper <b>A</b> as well as their proofs. We give necessary and sufficient conditions for weak convergence of first-rare-event times for semi-Markov processes with a finite set of states in non-triangular-array mode and describe the class of all possible limit distributions in terms of their Laplace transforms. In paper <b>C</b>, we study the conditions for weak convergence for flows of rare events for semi-Markov processes with a finite set of states in non-triangular array mode. We formulate necessary and sufficient conditions of convergence and describe the class of all possible limit stochastic flows. In the second part of the paper, we apply our results to the asymptotical analysis of non-ruin probabilities for perturbed risk processes. In paper <b>D</b>, we give necessary and sufficient conditions for the weak convergence of first-rare-event times for semi-Markov processes with a finite set of states in triangular array mode as well as describing the class of all possible limit distributions. The results of paper <b>D</b> extend results obtained in paper <b>B</b> to a general triangular array mode. In paper <b>E</b>, we give the necessary and sufficient conditions for weak convergence for the flows of rare events for semi-Markov processes with a finite set of states in triangular array case. This paper generalizes results obtained in paper <b>C</b> to a general triangular array mode. In the second part of the paper, we present applications of our results to asymptotical problems of perturbed risk processes and to queueing systems with quick service

With the advent of autonomous vehicles, the issue of safety-evaluationhas become key. ISO26262 recommends using Markov chains. However, in their most common form, Markov chains lack the flexibility required to model non- exponential probability distributions and systems displaying parallelism. In these cases, generalized semi-Markov processes arebetter suited. Though, these are significantly more taxing to analyze mathematically.  This thesis instead explores the option of simulating these systemsdirectly via MATLAB’s Simulink and Stateflow. An example system, here called CASE, currently under study by Scania was used as an example. The results showed that direct simulation is indeed possible, but the computational times are significantly greater than those from standard MATLAB-functions. The method should therefore be employed on parallel systems when results with a high level of fidelity are needed, and alternative methods are not available.

Autonomous vehicles is soon a reality in the every-day life. Though before it is used commercially the vehicles need to be proven safe. The current standard for functional safety on roads, ISO 26262, does not include autonomous vehicles at the moment, which is why in this project an approach using semi-Markov models is used to assess safety. A semi-Markov process is a stochastic process modelled by a state space model where the transitions between the states of the model can be arbitrarily distributed. The approach is realized as a MATLAB tool where the user can use a steady-state based analysis called a Loss and Risk based measure of safety to assess safety. The tool works and can assess safety of semi-Markov systems as long as they are irreducible and positive recurrent. For systems that fulfill these properties, it is possible to draw conclusions about the safety of the system through a risk analysis and also about which autonomous driving level the system is in through a sensitivity analysis. The developed tool, or the approach with the semi-Markov model, might be a good complement to ISO 26262.

Le présent travail porte sur l’estimation d’un système en temps discret dont l’évolution est décrite par une chaîne semi-markovienne (CSM) d’espace d’état fini. Nous présentons le principe d’invariance sous forme multidimensionnelle pour le noyau semi-markovien (NSM), ainsi que diverses mesures du processus. Ensuite, nous étudions l’estimation non-paramétrique de la loi stationnaire de la CSM, en considérant deux estimateurs différents, et nous montrons qu’ils ont le même comportement asymptotique. La probabilité de la première entrée est également introduite. Nous proposons un estimateur et nous étudions ses propriétés asymptotiques : la convergence forte et la normalité asymptotique.D’autre part, nous nous concentrons sur l’étude de la fiabilité des systèmes semi-markoviens. Nous définissons la fiabilité sur intervalle d’un système dont la fiabilité et la disponibilité sont des cas particuliers et nous étudions les propriétés asymptotiques d’un estimateur proposé. De plus, nous présentons une comparaison de l’estimation des différentes mesures de fiabilité fondées sur deux estimateurs du NSM, en réalisant une trajectoire unique et des observations multiples indépendantes. Ce travail fournit aussi des résultats dans le cas semi-markovien à temps discret avec espace d’état général. Nous évaluons l’approximation de moyenne et de diffusion des chaînes de renouvellement markovien. Enfin, nous nous sommes aussi intéressés à une autre classe des processus pour laquelle nous obtenons des résultats dans le cadre des files d’attente. Nous étudions l’approximation de moyenne pour le modèle d’Engset en temps continu et nous appliquons ce résultat aux files d’attente avec ré-essais. / The present work concerns the estimation of a discrete-time system whose evolution is governed by a semi-Markov chain (SMC) with finitely many states. We present the invariance principle in a multidimensional form for the semi-Markov kernel (SMK) and some associated measures of the process. Afterwards, we study the nonparametric estimation of the stationary distribution of the SMC, considering two different estimators, and we prove that they hold the same asymptotic behavior. We introduce also the first hitting probability. We propose an estimator and study its asymptotic properties : the strong consistency and the asymptotic normality. On the other hand, we focus on the study of the dependability of semi-Markovsystems. We introduce the interval reliability whose special cases are the reliability and the availability measures and we study the asymptotic properties of a proposed estimator. Moreover, we present a comparison of nonparametric estimation for various reliability measures based on two estimators of the SMK, realizing a unique trajectory and multiple independent observations.Furthermore, this work provides results on the discrete-time semi-Markov case with general state space. We evaluate the average and diffusion approximation of Markov renewal chains. Finally, we are also interested in another class of processes for which we obtain results in the framework of queueing systems. We establish the average approximationfor the Engset model in continuous time and we apply this result to retrial queues.

In dieser Arbeit werden zwei intrazelluläre Signalwege, betreffend den Tumorsuppressor p53 und das Signalmolekül Ca2+ , diskutiert und modelliert. Einzelzellmessungen des Tumorsuppressors p53 zeigen pulsatile Antwor- ten nach Zufügung von DNA Doppelstrangbrüchen (DSBs). Außer für sehr hohe Schadensdosen, ist das zeitliche auftreten dieser Pulse unregelmäßig. Mithilfe eines Wavelet basierten Pulsdetektors werden die einzelzell Trajek- torien untersucht und die inter-Puls Intervall (IPI) Verteilungen extrahiert. Diese weisen auf nicht-oszillatorische Regime in den Daten hin. Die Theorie der anregbaren Systeme angewendet auf regulatorische Netzwerke ermöglicht dieses komplexe Verhalten mathematisch zu beschreiben. Die Kopplung von Schadens-Sensor-Kinase Dynamik mit dem kanonischen p53 negativen feedback loop, ergibt ein anregbares p53 Modell. Detaillier- te Bifurkationsanalysen zeigen ein robustes anregbares Regime, welches durch ein starkes Schadenssignal auch in Oszillationen überführt werden kann. Treibt man das p53 Modell mit einem stochastischen DNA-Schadens-Prozess, kann sowohl das oszillatorische Verhalten nach hohem Schaden, als auch das unregelmäßige pulsatile Verhalten ohne äußere Stimulation reproduziert werden. Intrazelluläre Ca 2+ Spikes entstehen durch eine hierarchische Kaskade stochastischer prozesse. Die Anwendung einer semi-markovschen Beschreibung führt zu praktischen analytischen Lösungen des erstpassagezeiten Problems. Eine hierbei entdeckte Zeitskalenseparation ermöglicht ein neues allgemeines Ca2+ -Modell. Dieses erklärt auf äußerst prägnante Weise viele wesentliche experimentelle Ergebnisse, insbesondere die Momentenbeziehungen der inter-Spike Intervall Verteilungen. Schließlich erlaubt die hier vorgestellte Theorie Berechnungen der Stimulus-Enkodierung, also die Adaption des Ca 2+ Signals auf veränderliche extrazelluläre Stimuli. Die Vorhersage einer fold change Enkodierung kann durch Experimente gestützt werden. / In this work two signaling pathways, involving the tumor suppressor p53 and the second messenger Ca2+ , are to be discussed and modelled. The tumor suppressor p53 shows a pulsatile response in single cells after induction of DNA double strand breaks (DSBs). Except for very high amounts of damage, these pulses appear at irregular times. The concept of excitable systems is employed as a convenient way to model such observed dynamics. An application to biomolecular reaction networks shows the need for a positive feedback within the p53 regulatory network. Exploiting the reported ultrasensitive dynamics of the upstream damage sensor kinases, leads to a simplified excitable kinase-phosphatase model. Coupling that to the canonical negative feedback p53 regulatory loop, is the core idea behind the construction of the excitable p53 model. A detailed bifurcation analysis of the model establishes a robust excitable regime, which can be switched to oscillatory dynamics via a strong DNA damage signal. Driving the p53 model with a stochastic DSB process yields pulsatile dynamics which reflect different experimental scenarios. Intracellular Ca 2+ concentration spikes arise from a hierarchic cascade of stochastic events. An analytical solution strategy, employing a semi-Markovian description and involving Laplace transformations, is devised and successfully applied to a specific Ca2+ model. The new gained insights are then used, to construct a new generic Ca2+ model, which elegantly captures many known features of Ca2+ signaling. In particular the experimentally observed relations between the average and the standard deviation of the inter spike intervals (ISIs) can be explained in a concise way. Finally, the theoretical considerations allow to calculate the stimulus encoding relation, which governs the adaption of the Ca 2+ signals to varying extracellular stimuli. This is predicted to be a fold change response and new experimental results display a strong support of this idea.

La Dominance Temporelle des Sensations (DTS) est une méthode d’analyse sensorielle qui mesure la perception temporelle d’un produit au cours de sa dégustation. Pour un panéliste, la DTS consiste à choisir parmi une liste de descripteurs lequel est dominant à chaque instant. Ce travail a pour but la modélisation des données DTS à l’aide de processus stochastiques et propose d’utiliser les processus semi-markoviens (PSM), une généralisation des chaînes de Markov qui permet de modéliser librement les durées de dominance. Le modèle obtenu peut être utilisé pour comparer des échantillons DTS en réalisant un rapport de vraisemblance. Étant donné que les probabilités de transition entre les descripteurs peuvent dépendre du temps, nous proposons d’utiliser des modèles différents par période et nous proposons un algorithme pour déterminer le nombre et les frontières de ces périodes de manière optimale. Le modèle est représenté sous forme d’un graphe montrant les transitions entre descripteurs les plus observées. Finalement, ce travail introduit les modèles de mélange de processus semi-markoviens afin de segmenter le panel en fonction des différences de perception interindividuelles.Les méthodes développées sont appliquées à des jeux de données DTS variés : chocolats, fromages frais et Goudas. Les résultats montrent que la modélisation par un PSM apporte de nouvelles informations sur la perception temporelle, en particulier sur la variabilité de perception au sein d’un panel, alors que les méthodes classiques se focalisent sur une vision moyenne de la perception du panel. De plus, à notre connaissance, ce travail est le premier à proposer l’identification d’un modèle de mélange de processus semi-markoviens. / Temporal Dominance of Sensations (TDS) is a technique to measure temporal perception of food product during tasting. For a panelist, it consists in choosing in a list of attributes which one is dominant at any time. This work aims to model TDS data with a stochastic process and proposes to use semi-Markov processes (SMP), a generalization of Markov chains which allows dominance durations to be modeled by any type of distribution. The model can then be used to compare TDS samples based on likelihood ratio. Because probabilities of transition from one attribute to another one can also depend on time, we propose to model TDS by period and we propose a method to select optimally the number of periods and the frontiers between periods. Graphs built upon the stochastic pattern can be plotted to represent main chronological transitions between attributes. Finally, this work introduces new statistical models based on finite mixtures of semi-Markov processes in order to derive consumer segmentation based on individual differences in temporal perception of a product.The methods are applied to various TDS datasets: chocolates, fresh cheeses and Gouda cheeses. Results show that SMP modeling gives new information about temporal perception compared to classical methods. It particularly emphasizes the existence of several perceptions for a same product in a panel, whereas classical methods only provide a mean panel overview. Furthermore, as far as we know, this work is the first one that considers mixtures of semi-Markov processes.

No description available.

Two different sets of problems are addressed in this thesis. The first one is on partially observed semi-Markov Games (POSMG) and the second one is on semi-Markov modulated financial market model. In this thesis we study a partially observable semi-Markov game in the infinite time horizon. The study of a partially observable game (POG) involves three major steps: (i) construct an equivalent completely observable game (COG), (ii) establish the equivalence between POG and COG by showing that if COG admits an equilibrium, POG does so, (iii) study the equilibrium of COG and find the corresponding equilibrium of original partially observable problem. In case of infinite time horizon game problem there are two different payoff criteria. These are discounted payoff criterion and average payoff criterion. At first a partially observable semi-Markov decision process on general state space with discounted cost criterion is studied. An optimal policy is shown to exist by considering a Shapley’s equation for the corresponding completely observable model. Next the discounted payoff problem is studied for two-person zero-sum case. A saddle point equilibrium is shown to exist for this case. Then the variable sum game is investigated. For this case the Nash equilibrium strategy is obtained in Markov class under suitable assumption. Next the POSMG problem on countable state space is addressed for average payoff criterion. It is well known that under this criterion the game problem do not have a solution in general. To ensure a solution one needs some kind of ergodicity of the transition kernel. We find an appropriate ergodicity of partially observed model which in turn induces a geometric ergodicity to the equivalent model. Using this we establish a solution of the corresponding average payoff optimality equation (APOE). Thus the value and a saddle point equilibrium is obtained for the original partially observable model. A value iteration scheme is also developed to find out the average value of the game. Next we study the financial market model whose key parameters are modulated by semi-Markov processes. Two different problems are addressed under this market assumption. In the first one we show that this market is incomplete. In such an incomplete market we find the locally risk minimizing prices of exotic options in the Follmer Schweizer framework. In this model the stock prices are no more Markov. Generally stock price process is modeled as Markov process because otherwise one may not get a pde representation of price of a contingent claim. To overcome this difficulty we find an appropriate Markov process which includes the stock price as a component and then find its infinitesimal generator. Using Feynman-Kac formula we obtain a system of non-local partial differential equations satisfied by the option price functions in the mildsense. .Next this system is shown to have a classical solution for given initial or boundary conditions. Then this solution is used to have a F¨ollmer Schweizer decomposition of option price. Thus we obtain the locally risk minimizing prices of different options. Furthermore we obtain an integral equation satisfied by the unique solution of this system. This enable us to compute the price of a contingent claim and find the risk minimizing hedging strategy numerically. Further we develop an efficient and stable numerical method to compute the prices. Beside this work on derivative pricing, the portfolio optimization problem in semi-Markov modulated market is also studied in the thesis. We find the optimal portfolio selections by optimizing expected utility of terminal wealth. We also obtain the optimal portfolio selections under risk sensitive criterion for both finite and infinite time horizon.