Pour faire face à la croissance rapide du trafic mobile, différentes technologies d'accès radio (par exemple, HSPA, LTE, WiFi, et WiMAX) sont intégrées et gérées conjointement. Dans ce contexte, la sélection de TAR est une fonction clé pour améliorer les performances du réseau et l'expérience de l'utilisateur. Elle consiste à décider quelle TAR est la plus appropriée aux mobiles. Quand l'intelligence est poussée à la périphérie du réseau, les mobiles décident de manière autonome de leur meilleur TAR. Ils cherchent à maximiser égoïstement leur utilité. Toutefois, puisque les mobiles ne disposent d'aucune information sur les conditions de charge du réseau, leurs décisions peuvent conduire à une inefficacité de la performance. En outre, déléguer les décisions au réseau optimise la performance globale, mais au prix d'une augmentation de la complexité du réseau, des charges de signalisation et de traitement. Dans cette thèse, au lieu de favoriser une de ces deux approches décisionnelles, nous proposons un cadre de décision hybride: le réseau fournit des informations pour les mobiles pour mieux décider de leur TAR. Plus précisément, les utilisateurs mobiles choisissent leur TAR en fonction de leurs besoins et préférences individuelles, ainsi que des paramètres de coût monétaire et de QoS signalés par le réseau. En ajustant convenablement les informations du réseau, les décisions des utilisateurs répondent globalement aux objectifs de l'opérateur. Nous introduisons d'abord notre cadre de décision hybride. Afin de maximiser l'expérience de l'utilisateur, nous présentons une méthode de décision multicritère (MDMC) basée sur la satisfaction. Outre leurs conditions radio, les utilisateurs mobiles tiennent compte des paramètres de coût et de QoS, signalées par le réseau, pour évaluer les TAR disponibles. En comparaison avec les solutions existantes, notre algorithme répond aux besoins de l'utilisateur (par exemple, les demandes en débit, la tolérance de coût, la classe de trafic), et évite les décisions inadéquates. Une attention particulière est ensuite portée au réseau pour s'assurer qu'il diffuse des informations décisionnelles appropriées, afin de mieux exploiter ses ressources radio alors que les mobiles maximisent leur propre utilité. Nous présentons deux méthodes heuristiques pour dériver dynamiquement quoi signaler aux mobiles. Puisque les paramètres de QoS sont modulées en fonction des conditions de charge, l'exploitation des ressources radio s'est avérée efficace. Aussi, nous nous concentrons sur l'optimisation de l'information du réseau. La dérivation des paramètres de QoS est formulée comme un processus de décision semi-markovien, et les stratégies optimales sont calculées en utilisant l'algorithme de Policy Iteration. En outre, et puisque les paramètres du réseau ne peuvent pas être facilement obtenues, une approche par apprentissage par renforcement est introduite pour dériver quoi signaler aux mobiles. / To cope with the rapid growth of mobile broadband traffic, various radio access technologies (e.g., HSPA, LTE, WiFi, and WiMAX) are being integrated and jointly managed. Radio Access Technology (RAT) selection, devoted to decide to what RAT mobiles should connect, is a key functionality to improve network performance and user experience. When intelligence is pushed to the network edge, mobiles make autonomous decisions regarding selection of their most appropriate RAT. They aim to selfishly maximize their utility. However, because mobiles have no information on network load conditions, their decisions may lead to performance inefficiency. Moreover, delegating decisions to the network optimizes overall performance, but at the cost of increased network complexity, signaling, and processing load. In this thesis, instead of favoring either of these decision-making approaches, we propose a hybrid decision framework: the network provides information for the mobiles to make robust RAT selections. More precisely, mobile users select their RAT depending on their individual needs and preferences, as well as on the monetary cost and QoS parameters signaled by the network. By appropriately tuning network information, user decisions are globally expected to meet operator objectives, avoiding undesirable network states. We first introduce our hybrid decision framework. Decision makings, on the network and user sides, are investigated. To maximize user experience, we present a satisfaction-based Multi-Criteria Decision-Making (MCDM) method. In addition to their radio conditions, mobile users consider the cost and QoS parameters, signaled by the network, to evaluate serving RATs. In comparison with existing MCDM solutions, our algorithm meets user needs (e.g., traffic class, throughput demand, cost tolerance), avoiding inadequate decisions. A particular attention is then addressed to the network to make sure it broadcasts suitable decisional information, so as to better exploit its radio resources while mobiles maximize their own utility. We present two heuristic methods to dynamically derive what to signal to mobiles. While QoS parameters are modulated as a function of the load conditions, radio resources are shown to be efficiently exploited. Moreover, we focus on optimizing network information. Deriving QoS parameters is formulated as a semi-Markov decision process, and optimal policies are computed using the Policy Iteration algorithm. Also, and since network parameters may not be easily obtained, a reinforcement learning approach is introduced to derive what to signal to mobiles. The performances of optimal, learning-based, and heuristic policies are analyzed. When thresholds are pertinently set, our heuristic method provides performance very close to the optimal solution. Moreover, although lower performances are observed, our learning-based algorithm has the crucial advantage of requiring no prior parameterization.

<p>Forest management involves numerous stochastic elements. To sustainably manage forest resources, it is crucial to acknowledge these sources as uncertainty or risk, and incorporate them in adaptive decision-making. Here, I developed several stochastic programming models in the form of passive or active adaptive management for natural mixed-species hardwood forests in Indiana. I demonstrated how to use these tools to deal with time-invariant and time-variant natural disturbances in optimal planning of harvests.</p> <p> Markov decision process (MDP) models were first constructed based upon stochastic simulations of an empirical forest growth model for the forest type of interest. Then, they were optimized to seek the optimal or near-optimal harvesting decisions while considering risk and uncertainty in natural disturbances. In particular, a classic expected-criterion infinite-horizon MDP model was first used as a passive adaptive management tool to determine the optimal action for a specific forest state when the probabilities of forest transition remained constant over time. Next, a two-stage non-stationary MDP model combined with a rolling-horizon heuristic was developed, which allowed information update and then adjustments of decisions accordingly. It was used to determine active adaptive harvesting decisions for a three-decade planning horizon during which natural disturbance probabilities may be altered by climate change.</p> <p> The empirical results can be used to make some useful quantitative management recommendations, and shed light on the impacts of decision-making on the forests and timber yield when some stochastic elements in forest management changed. In general, the increase in the likelihood of damages by natural disturbance to forests would cause more aggressive decisions if timber production was the management objective. When windthrow did not pose a threat to mixed hardwood forests, the average optimal yield of sawtimber was estimated to be 1,376 ft³/ac/acre, while the residual basal area was 88 ft²/ac. Assuming a 10 percent per decade probability of windthrow that would reduce the stand basal area considerably, the optimal sawtimber yield per decade would decline by 17%, but the residual basal area would be lowered only by 5%. Assuming that the frequency of windthrow increased in the magnitude of 5% every decade under climate change, the average sawtimber yield would be reduced by 31%, with an average residual basal area slightly around 76 ft²/ac. For validation purpose, I compared the total sawtimber yield in three decades obtained from the heuristic approach to that of a three-decade MDP model making <i>ex post</i> decisions. The heuristic approach was proved to provide a satisfactory result which was only about 18% lower than the actual optimum.</p> These findings highlight the need for landowners, both private and public, to monitor forests frequently and use flexible planning approaches in order to anticipate for climate change impacts. They also suggest that climate change may considerably lower sawtimber yield, causing a concerning decline in the timber supply in Indiana. Future improvements of the approaches used here are recommended, including addressing the changing stumpage market condition and developing a more flexible rolling-horizon heuristic approach.

Early diagnosis is a promising strategy to reduce premature mortalities and for optimal use of resources. But the absence of mathematical models specific to the data settings in LMIC’s impedes the construction of economic analysis necessary for decision-makers in the development of cancer control programs. This thesis presents a new methodology for parameterizing the natural history model of breast cancer based on data availabilities in low and middle income countries, and formulation of a control optimization problem to find the optimal screening schedule for mammography screening, solved using dynamic programming. As harms and benefits are known to increase with the increase in the number of lifetime screens, the trade-off was modeled by formulating the immediate reward as a function of false positives and life-years saved. The method presented in thesis will provide optimal screening schedules for multiple scenarios of Willingness to Pay (numeric value assigned for each life-year lived), including the resulting total number of lifetime screens per person, which can help decision-makers evaluate current resource availabilities or plan future resource needs for implementation.

In this thesis we focus on new methods for probabilistic model checking (PMC) with linear temporal logic (LTL). The standard approach translates an LTL formula into a deterministic ω-automaton with a double-exponential blow up. There are approaches for Markov chain analysis against LTL with exponential runtime, which motivates the search for non-deterministic automata with restricted forms of non-determinism that make them suitable for PMC. For MDPs, the approach via deterministic automata matches the double-exponential lower bound, but a practical application might benefit from approaches via non-deterministic automata. We first investigate good-for-games (GFG) automata. In GFG automata one can resolve the non-determinism for a finite prefix without knowing the infinite suffix and still obtain an accepting run for an accepted word. We explain that GFG automata are well-suited for MDP analysis on a theoretic level, but our experiments show that GFG automata cannot compete with deterministic automata. We have also researched another form of pseudo-determinism, namely unambiguity, where for every accepted word there is exactly one accepting run. We present a polynomial-time approach for PMC of Markov chains against specifications given by an unambiguous Büchi automaton (UBA). Its two key elements are the identification whether the induced probability is positive, and if so, the identification of a state set inducing probability 1. Additionally, we examine the new symbolic Muller acceptance described in the Hanoi Omega Automata Format, which we call Emerson-Lei acceptance. It is a positive Boolean formula over unconditional fairness constraints. We present a construction of small deterministic automata using Emerson-Lei acceptance. Deciding, whether an MDP has a positive maximal probability to satisfy an Emerson-Lei acceptance, is NP-complete. This fact has triggered a DPLL-based algorithm for deciding positiveness.

Probabilistic model checking mainly concentrates on techniques for reasoning about the probabilities of certain path properties or expected values of certain random variables. For the quantitative system analysis, however, there is also another type of interesting performance measure, namely quantiles. A typical quantile query takes as input a lower probability bound p ∈ ]0,1] and a reachability property. The task is then to compute the minimal reward bound r such that with probability at least p the target set will be reached before the accumulated reward exceeds r. Quantiles are well-known from mathematical statistics, but to the best of our knowledge they have not been addressed by the model checking community so far. In this paper, we study the complexity of quantile queries for until properties in discrete-time finite-state Markov decision processes with nonnegative rewards on states. We show that qualitative quantile queries can be evaluated in polynomial time and present an exponential algorithm for the evaluation of quantitative quantile queries. For the special case of Markov chains, we show that quantitative quantile queries can be evaluated in pseudo-polynomial time.

Cette étude s’inscrit dans le domaine de l’intelligence artificielle, plus précisément au croisement des deux domaines que sont la planification autonome en environnement probabiliste et la vérification formelle probabiliste. Dans ce contexte, elle pose la question de la maîtrise de la complexité face à l’intégration de nouvelles technologies dans les systèmes critiques : comment garantir que l’ajout d’une intelligence à un système, sous la forme d’une autonomie, ne se fasse pas au détriment de la sécurité ? Pour répondre à cette problématique, cette étude a pour enjeu de développer un processus outillé, permettant de concevoir des systèmes auto-adaptatifs critiques, ce qui met en œuvre à la fois des méthodes de modélisation formelle des connaissances d’ingénierie, ainsi que des algorithmes de planification sûre et optimale des décisions du système. / This study takes place in the broad field of Artificial Intelligence, specifically at the intersection of two domains : Automated Planning and Formal Verification in probabilistic environment. In this context, it raises the question of the integration of new technologies in critical systems, and the complexity it entails : How to ensure that adding intelligence to a system, in the form of autonomy, is not done at the expense of safety ? To address this issue, this study aims to develop a tool-supported process for designing critical, self-adaptive systems. Throughout this document, innovations are therefore proposed in methods of formal modeling and in algorithms for safe and optimal planning.

Cette thèse aborde différents aspects de la modélisation de la microstructure du marché et des problèmes de Market Making, avec un accent particulier du point de vue du praticien. Le carnet d’ordres, au cœur du marché financier, est un système de files d’attente complexe à haute dimension. Nous souhaitons améliorer la connaissance du LOB pour la communauté de la recherche, proposer de nouvelles idées de modélisation et développer des applications pour les Market Makers. Nous remercions en particuler l’équipe Automated Market Making d’avoir fourni la base de données haute-fréquence de très bonne qualité et une grille de calculs puissante, sans laquelle ces recherches n’auraient pas été possible. Le Chapitre 1 présente la motivation de cette recherche et reprend les principaux résultats des différents travaux. Le Chapitre 2 se concentre entièrement sur le LOB et vise à proposer un nouveau modèle qui reproduit mieux certains faits stylisés. A travers cette recherche, non seulement nous confirmons l’influence des flux d’ordres historiques sur l’arrivée de nouveaux, mais un nouveau modèle est également fourni qui réplique beaucoup mieux la dynamique du LOB, notamment la volatilité réalisée en haute et basse fréquence. Dans le Chapitre 3, l’objectif est d’étudier les stratégies de Market Making dans un contexte plus réaliste. Cette recherche contribueà deux aspects : d’une part le nouveau modèle proposé est plus réaliste mais reste simple à appliquer pour la conception de stratégies, d’autre part la stratégie pratique de Market Making est beaucoup améliorée par rapport à une stratégie naive et est prometteuse pour l’application pratique. La prédiction à haute fréquence avec la méthode d’apprentissage profond est étudiée dans le Chapitre 4. De nombreux résultats de la prédiction en 1- étape et en plusieurs étapes ont retrouvé la non-linéarité, stationarité et universalité de la relation entre les indicateurs microstructure et le changement du prix, ainsi que la limitation de cette approche en pratique. / This thesis addresses different aspects around the market microstructure modelling and market making problems, with a special accent from the practitioner’s viewpoint. The limit order book (LOB), at the heart of financial market, is a complex continuous high-dimensional queueing system. We wish to improve the knowledge of LOB for the research community, propose new modelling ideas and develop concrete applications to the interest of Market Makers. We would like to specifically thank the Automated Market Making team for providing a large high frequency database of very high quality as well as a powerful computational grid, without whom these researches would not have been possible. The first chapter introduces the incentive of this research and resumes the main results of the different works. Chapter 2 fully focuses on the LOB and aims to propose a new model that better reproduces some stylized facts. Through this research, not only do we confirm the influence of historical order flows to the arrival of new ones, but a new model is also provided that captures much better the LOB dynamic, notably the realized volatility in high and low frequency. In chapter 3, the objective is to study Market Making strategies in a more realistic context. This research contributes in two aspects : from one hand the newly proposed model is more realistic but still simple enough to be applied for strategy design, on the other hand the practical Market Making strategy is of large improvement compared to the naive one and is promising for practical use. High-frequency prediction with deep learning method is studied in chapter 4. Many results of the 1-step and multi-step prediction have found the non-linearity, stationarity and universality of the relationship between microstructural indicators and price change, as well as the limitation of this approach in practice.

Energy management systems (EMSs), implemented in the electronic control unit (ECU) of an actual vehicle with electri ed powertrain, is a much simpler version of the theoretically developed EMS. Such simpli cation is done to accommodate the EMS within the given memory constraint and computational capacity of the ECU. The simpli cation should ensure reasonable performance compared to theoretical EMS under real-life driving scenarios. The process of simpli cation must be effective to create a versatile and utilitarian EMS. The reinforcement learning-based controllers feature pro table characteristics in optimizing the performance of controllable physical systems as they do not mandatorily require a mathematical model of system dynamics (i.e. they are model-free). Quite naturally, it can aspired to testify such prowess of reinforcement learning-based controllers in achieving near-global optimal performance for energy management system (supervisory) of electri ed powertrains. Before deployment of any supervisory controller as a mainstream controller, they should be essentially scrutinized through various levels of virtual simulation platforms with an ascending order of physical system emulating-capability. The controller evolves from a mathematical concept to an utilitarian embedded system through a series of these levels where it undergoes gradual transformation to finally become apposite for a real physical system. Implementation of the control strategy in a Simulink-based forward simulation model could be the first stage of the aforementioned evolution process. This brief will delineate all the steps required for implementing an reinforcement learning-based supervisory controller in a forward simulation model of a hybrid electric vehicle. A novel framework of loss-minimization based instantaneous optimal strategy is introduced for the energy management system of a multi-mode hybrid electric powertrain in this brief. The loss-minimization strategy is flexible enough to be implemented in any architecture of electrified powertrains. It is mathematically proven that the overall system loss minimization is equivalent to the minimization of fuel consumption. An online simulation framework is developed in this article to evaluate the performance of a multi-mode electrified powertrain equipped with more than one power source. An electrically variable transmission with two planetary gear-set has been chosen as the centerpiece of the powertrain considering the versatility and future prospects of such transmissions. It is noteworthy to mention that a novel architecture topology selected for this dissertation is engendered through a series of rigorous screening process whose workflow is presented here with brevity. One of the legitimate concern of multi-mode transmission is it's proclivity to contribute discontinuity of power-flow in the downstream of the powertrain. Mode-shift events can be predominantly held responsible for engendering such discontinuity. Advent of dynamic coordinated control as a technique for ameliorating such discontinuity has been substantiated by many scholars in literature. Hence, a system-level coordinated control is employed within the energy management system which governs the mode schedule of the multi-mode powertrain in real-time simulation. / Thesis / Doctor of Philosophy (PhD)

No description available.

No description available.