Controlling a high-dimensional dynamical system with continuous state and action spaces in a partially unknown environment like a gas turbine is a challenging problem. So far often hard coded rules based on experts´ knowledge and experience are used. Machine learning techniques, which comprise the field of reinforcement learning, are generally only applied to sub-problems. A reason for this is that most standard RL approaches still fail to produce satisfactory results in those complex environments. Besides, they are rarely data-efficient, a fact which is crucial for most real-world applications, where the available amount of data is limited. In this thesis recurrent neural reinforcement learning approaches to identify and control dynamical systems in discrete time are presented. They form a novel connection between recurrent neural networks (RNN) and reinforcement learning (RL) techniques. RNN are used as they allow for the identification of dynamical systems in form of high-dimensional, non-linear state space models. Also, they have shown to be very data-efficient. In addition, a proof is given for their universal approximation capability of open dynamical systems. Moreover, it is pointed out that they are, in contrast to an often cited statement, well able to capture long-term dependencies. As a first step towards reinforcement learning, it is shown that RNN can well map and reconstruct (partially observable) MDP. In the so-called hybrid RNN approach, the resulting inner state of the network is then used as a basis for standard RL algorithms. The further developed recurrent control neural network combines system identification and determination of an optimal policy in one network. In contrast to most RL methods, it determines the optimal policy directly without making use of a value function. The methods are tested on several standard benchmark problems. In addition, they are applied to different kinds of gas turbine simulations of industrial scale.

A question concerning not only modellers but also practitioners is: Under what circumstances can mutual cooperation be established and maintained by a group of people facing a common pool dilemma" A step before this question of institutional influences there is need for a different way of modelling human behaviour that does not draw on the rational actor paradigm, because this kind of modelling needs to be able to integrate various deviations from this theory shown in economic experiments. We have chosen a new approach based on observations in form of laboratory and field observations of actual human behaviour. We model human decision making as using an adaptive toolbox following the notion of Gigerenzer. Humans draw on a number of simple heuristics that are meaningful in a certain situation but may be useless in another. This is incorporated into our agent-based model by having agents perceive their environment, draw on a pool of heuristics to choose an appropriate one and use that heuristic.Behavioural differences can be incorporated in two ways. First, each agent has a number of attributes that differ in values, for example there are more and less cooperative agents. The second behavioural difference lies in the way, in which heuristics are chosen. With this modelling approach we contribute to a new way of modelling human behaviour, which is simple enough to be included into morecomplex models while at the same time realistic enough to cover actual decision making processes of humans. Modellers should be able to use this approach without a need to get deep into psychological, sociological or economic theory. Stakeholders in social dilemmas, who may be confronted with such a model should understand, why an agent decides in the way it does.

La chaîne d’approvisionnement des entreprises industrielles (Supply Chain (SC) en anglais) devient de plus en plus complexe de par le nombre d’interactions et interdépendances entre les différentes entités, acteurs, processus et ressources qui la composent. Ainsi, les entreprises s’intéressent de plus en plus à l’étude de leurs systèmes de performances pour aider à piloter et améliorer leurs processus. Les diverses approches existantes traitent essentiellement de la mesure et de l’évaluation de la performance des processus internes de l’entreprise. Notre approche, par la systémique, nous permet de prendre en compte toutes les parties prenantes intervenantes dans les chaines de valeurs globales, allant du besoin à la satisfaction du besoin. Dans le cas spécifique du secteur aéronautique (européen), le défi est d’autant plus important qu’il a été identifié que 70 à 80% de la valeur ajoutée (VA) est produite chez les fournisseurs de rang 1 et 2. C’est la raison pour laquelle nous nous intéressons dans cette thèse au périmètre de la sous-traitance de la production.Nos travaux de recherche sont organisés en trois parties :1) Une modélisation systémique au sein de la SC afin de considérer son système de performance dans sa globalité partant des objectives stratégiques (en termes d’indicateurs globaux) jusqu’à la re-conception des processus créateurs de valeurs internes. Cette modélisation permet d’identifier un sous-systèmeà étudier, les acteurs concernés et les attentes traduites par des indicateurs de performances qui sont mesurés à travers les processus internes. Nous identifions des indicateurs à prioriser et à améliorer en consensus avec les experts industriels.2) Un traitement de l’indicateur priorisé : « la Livraison à temps des fournisseurs (Supplier On Time Delivery) ». Dans un premier temps, nous analysons la mesure de cet indicateur, ses variables et ses paramètres qui l’impactent et nous approfondissons l’étude en se concentrant sur les dysfonctionnements (destructeurs de valeurs) qui empêchent d’atteindre les objectifs stratégiques fixés. Après cette phase d’analyse, nous proposons une simulation des impacts de dysfonctionnements sur l’indicateur et menons ensuite des analyses de sensibilité sur les variables identifiées afin de quantifier ces impacts et leur évolution.3) Une méthodologie d’établissement de plan d’actions d’amélioration à appliquer dans le périmètre étudié. Nous proposons une analyse qualitative sur l’implémentation des leviers et leur efficacité afin d’améliorer la performance.Des conclusions et limites de notre approche sont discutées et nous donnons des perspectives de recherche sur la re-conception et l’amélioration des performances de systèmes au sein de la chaîne d’approvisionnement. / The Supply Chain of industrial companies is becoming more complex due to the high number of interactions and interdependencies between different entities, actors, processes and resources belonging to the supply process. Thus, companies are increasingly interested in the study of their performance systems to help managing and improving their processes. The various existing approaches deal mainly with the measurement and the evaluation of the performance of internal business processes. Our work, based on the systemic approach, allows us to take into account all stakeholders involved in the global value chain, from the need to the fulfillment of it. In the specific case of the aeronautics (European), the challenge is even more important in as much as it was identified that 70 to 80% of value added (VA) occurred with the first and second-tier suppliers. This is why we focus our work within the scope of the outsourcing of aircraft production.Our research is organized in three parts:1) A systemic modeling within the SC is realized to consider the performance of the overall system, starting from the strategic objectives (in terms of global indicators) to the re-design of theinternal processes creating values. This model identifies asubsystem to be studied, stakeholders involved, and their expectations reflected by the performance indicators measured through internal processes. We identify indicators to be prioritized and improved by consensus with industrial experts.2) A treatment of the prioritized indicator "Supplier On Time Delivery" is proposed: first, we analyze the measurement of this indicator, its variables and parameters that impact it, and we go into the study in depth focusing on the dysfunctions (destroying values) that prevent achieving the strategic goals. After this phase of analysis, we propose a simulation of the dysfunctions’ impacts on the indicator and we then conduct sensitivity analysis of the identified variables of interest in order to quantify these impacts and their evolution.3) A methodology leading to an establishment of an improvement actions plan is presented in the perimeter studied. We provide a qualitative analysis on the implementation and effectiveness of the levers improving the performances.Conclusions and limitations of our approach are discussed and we provide research perspectives on the re-design and the improvement of the system’s performance within the supply chain.

L’habitat sain est le thème central des réflexions contemporaines du domaine du bâtiment élargies à l’environnement. Il comporte des préoccupations notables en matière de santé, de consommation énergétique (la ventilation, le chauffage, la climatisation et l’eau chaude), d’impacts environnementaux et de durabilité des matériaux de construction. Le choix préliminaire des matériaux utilisés pour la construction joue un rôle important dans la réussite d’un projet HQE (Haute Qualité Environnementale). Dans ce contexte, la problématique de prévision des champs de température et d’humidité demeure essentielle à l’intérieur des matériaux poreux de construction, où les matériaux biosourcés font l'objet d'un fort intérêt vu leurs qualités environnementales. Les matériaux biosourcés, étant hygroscopiques, ont tendance à absorber ou à restituer l’humidité, ce qui génère respectivement un gonflement ou un retrait. A l’échelle microscopique, l’humidité prend place soit par l’absorption de l’eau liée par les fibres, soit par l’existence d’eau libre dans les pores. Cette complexité des phénomènes microscopiques dans les matériaux biosourcés mène à une forte interaction entre l’aspect mécanique et les aspects de transferts de masse et de chaleur. L’existence de ce couplage est susceptible de modifier sensiblement les performances thermiques du bâtiment, et même sa durabilité. L’objectif visé par ce travail de thèse est l’étude et l’analyse microscopique du comportement hygrique des matériaux poreux de construction. L’aspect mécanique couplé à l’aspect hygrique est abordé en prenant en considération les déformations locales de gonflement - retrait, et leur impact sur l’hystérésis de teneur en eau. La maîtrise de ce couplage est primordiale tant sur le plan de la prédiction de la qualité des ambiances habitables que sur l’évaluation de la durabilité de ces structures. Le projet de thèse consiste à travailler à la fois sur les aspects modélisation, caractérisation et mesure des transferts hygriques. La quantification de ces phénomènes est réalisée à travers des campagnes de mesures expérimentales basées sur des techniques d’imagerie 3D (micro-tomographie aux rayons X). Le recours à la diffraction aux rayons X (DRX), à la corrélation d’images volumique, ainsi qu’à la résonance magnétique nucléaire (RMN) permet d’avoir une meilleure compréhension des échanges entre la matrice solide et l’eau liée et/ou libre. Tous ces travaux ont mené à une meilleure caractérisation de la morphologie du bois d’épicéa à l’échelle microscopique, ainsi qu’à une meilleure estimation des diverses variations dimensionnelles (gonflement) à l’échelle des parois cellulaires et de leurs constituants chimiques. Les résultats numériques obtenus sur la structure réelle 3D du matériau ont été couplés aux mesures expérimentales à travers la corrélation d’images volumiques (micro-tomographie aux rayons X) afin d’identifier les propriétés intrinsèques des phénomènes et du matériau. Ces travaux de thèse constitueront une base scientifique permettant une meilleure modélisation du couplage mécanique avec les transferts de chaleur et de masse dans les matériaux biosourcés. / Healthy living is a main contemporary concern of the construction field, extended to the environment. It has significant concerns about health, energy consumption, environmental impact and sustainability of building materials. The preliminary selection of materials used for construction plays an important role in the success of high environmental quality projects. In this context, it remains essential to predict the temperature and humidity fields inside porous building materials, where bio-based materials are subject to a strong interest due to their environmental qualities.As bio-based materials are hygroscopic, they tend to absorb or restore moisture, which respectively generates swelling or shrinkage. At the microscopic scale, moisture takes place either by absorption of bound water by the fibers, or by the existence of free water in the pores. The complexity of microscopic phenomena in bio-based materials will lead to strong interactions between the mechanical aspect on one side and heat and mass transfers’ aspects on the other side. The existence of this coupling may significantly alter the building's thermal performance, as well as its durability.The objective of this thesis work is to study the microscopic hygric behavior of porous building materials. The mechanical aspect coupled to the hygric one is studied, taking into consideration the local swelling and shrinkage strains, and their impact on the hysteresis phenomenon. Understanding this coupling is very important in order to improve the quality of habitat and evaluate the durability of these structures.The PhD project consists on working on all aspects, modeling, characterization and measurement of hygric transfers. Quantification of these phenomena is achieved through experimental campaigns based on 3D imaging techniques (X-ray micro-tomography). The use of X-ray diffraction (XRD), digital volume correlation, as well as nuclear magnetic resonance (NMR) allows a better understanding of the interactions between the solid matrix and bound and/or free water. The corresponding results have led to a microscopic morphological characterization of spruce wood, as well as to a better estimation of the various dimensional variations of the cell walls, and their chemical components.The numerical results achieved on the real 3D structure of the material have been coupled to the experimental ones, using digital volume correlation technique (X-ray tomography), in order to identify the intrinsic properties of the material.These thesis works provide a scientific basis allowing the improvement of modeling of the mechanical coupling with heat and mass transfers in bio-based materials.

No description available.

Gas injection into saturated porous media has a high practical relevance. It is applied in groundwater remediation (air sparging), in CO2 sequestration into saline aquifers, and in enhanced oil recovery of petroleum reservoirs. This wide range of application necessitates a comprehensive understanding of gas flow patterns that may develop within the porous media and required modeling of multi-phase flow. There is an ongoing controversy in literature, if continuum models are able to describe the complex flow pattern observed in heterogeneous porous media, especially the channelized stochastic flow pattern. Based on Selker’s stochastic hypothesis, a gas channel is caused by a Brownian-motion process during gas injection. Therefore, the pore-scale heterogeneity will determine the shape of the single stochastic gas channels. On the other hand there are many studies on air sparging, which are based on continuum modeling. Up to date it is not clear under which conditions a continuum model can describe the essential features of the complex gas flow pattern. The aim of this study is to investigate the gas flow pattern on bench-scale and field scale using the continuum model TOUGH2. Based on a comprehensive data set of bench-scale experiments and field-scale experiments, we conduct for the first time a systematic study and evaluate the prediction ability of the continuum model. A second focus of this study is the development of a “real world”-continuum model, since on all scales – pore-scale, bench scale, field scale – heterogeneity is a key driver for the stochastic gas flow pattern. Therefore, we use different geostatistical programs to include stochastic conditioned and unconditioned parameter fields. Our main conclusion from bench-scale experiments is that a continuum model, which is calibrated by different independent measurements, has excellent prediction ability for the average flow behavior (e.g. the gas volume-injection rate relation). Moreover, we investigate the impact of both weak and strong heterogeneous parameter fields (permeability and capillary pressure) on gas flow pattern. The results show that a continuum model with weak stochastic heterogeneity cannot represent the essential features of the experimental gas flow pattern (e.g., the single stochastic gas channels). Contrary, applying a strong heterogeneity the continuum model can represent the channelized flow. This observation supports Stauffer’s statement that a so-called subscale continuum model with strong heterogeneity is able to describe the channelized flow behavior. On the other hand, we compare the theoretical integral gas volumes with our experiments and found that strong heterogeneity always yields too large gas volumes. At field-scale the 3D continuum model is used to design and optimize the direct gas injection technology. The field-scale study is based on the working hypotheses that the key parameters are the same as at bench-scale. Therefore, we assume that grain size and injection rate will determine whether coherent channelized flow or incoherent bubbly flow will develop at field-scale. The results of four different injection regimes were compared with the data of the corresponding field experiments. The main conclusion is that because of the buoyancy driven gas flow the vertical permeability has a crucial impact. Hence, the vertical and horizontal permeability should be implemented independently in numerical modeling by conditioned parameter fields.

<div>Pavement drainage systems are one of the key drivers of pavement function and longevity, and effective drain maintenance can significantly extend a pavement's service life. Maintenance of these drains, however, is often hampered by the challenge of locating the drains. Ground Penetrating Radar (GPR) typically offers a rapid and effective method to detect these underground targets. However, typical detection schema that rely upon the observation of the hyperbolic return from a GPR scan of a buried conduit still tend to miss many of the older drains beneath pavements as they may be partially or fully filled with sediment and/or may be fabricated from clay or other earthen materials, yielding a return signal that is convolved with significant background noise. </div><div><br></div><div>To manage this challenge, this work puts forward an improved background noise and clutter reduction method to enhance the target signals in what amounts to a constructed environment that tends to have more consistent subsurface properties than one might encounter in a general setting. Within this technique, two major algorithms are employed. Algorithm 1 is the core of this method, and plays the role of reducing background noise and clutter. Algorithm 2 is supplementary, and helps eliminate anomalous discontinuous returns generated by the equipment itself, which could otherwise lead to false detection indications in the output of Algorithm 1. Instead of traditional 2-D GPR images, the result of the proposed algorithms is a 1-D plot along the survey line, highlighting a set of “points of interest” that could indicate buried drain locations identified at any given GPR operating frequency. Subsurface exploration using two different operating frequencies, 900 MHz and 400 MHz herein, is then employed to further enhance detection confidence. Points of interest are ultimately coded to define the confidence of the detection. Comparing the final result of proposed algorithms with the original GPR images, the improved algorithm is demonstrated to provide significantly improved detection results, and could potentially be applied to similar problems in other contexts.</div><div><br></div><div>Besides the background reduction methods, a group of simulations performed using GPRMAX2D software are examined to explore the influence of road cross-section designs on sub-pavement drainage conduit GPR signatures, and evaluate the effectiveness of alternate GPR antennae configurations in locating these buried conduits in different ground conditions. Two different models were explored to simulate conduit detection. In addition, different pipe and soil conditions were modeled, such as pipe size, pipe material, soil moisture level, and soil type. Four different quantitative measurements are used to analyze GPR performance based on different key factors. The four measurements are 1) signal to background ratio (SBR) in dB; 2) signal to receiver noise ratio (SNR) in dB; 3) signal energy in Volts; and 4) average signal band power in Watts.</div><div><br></div><div>The water and clay content of subsurface soil can significantly influence the detection results obtained from ground penetrating radar (GPR). Due to the variation of the material properties underground, the center frequency of transmitted GPR signals shifts to a lower range as wave attenuation increases. Examination of wave propagation in the subsurface employing an attenuation filter based on a linear system model shows that received GPR signals will be shifted to lower frequencies than those originally transmitted. The amount of the shift is controlled by a wave attenuation factor, which is determined by the dielectric constant, electric conductivity, and magnetic susceptibility of the transmitted medium. This work introduces a receiver-transmitter-receiver dual-frequency configuration for GPR that employs two operational frequencies for a given test - one higher and one slightly lower - to take advantage of this phenomenon to improve subpavement drain detection results. In this configuration, the original signal is transmitted from the higher frequency transmitter. After traveling through underground materials, the signal is received by two receivers with different frequencies. One of the receivers has the same higher center frequency as the transmitter, and the other receiver has a lower center frequency. This configuration can be expressed as Rx(low-frequency)-Tx(high-frequency)-Rx(high-frequency) and was applied in both laboratory experiments and field tests. Results are analyzed in the frequency domain to evaluate and compare the properties of the signal obtained by both receivers. The laboratory experiment used the configuration of Rx(400MHz)-Tx(900MHz)-Rx(900MHz). The field tests, in addition to the configuration used in the lab tests, employed another configuration of Rx(270MHz)-Tx(400MHz)-Rx(400MHz) to obtain more information about this phenomenon. Both lab and field test results illustrate the frequency-shift phenomenon described by theoretical calculations. Based on the power spectrum for each signal, the lower frequency antenna typically received more energy (higher density values) at its peak frequency than the higher frequency antenna.</div>

Membranes separate uid compartments and can comprise transport structures for selective permeation. In biology, channel proteins are specialized in their atomic structure to allow transport of specific compounds (selectivity). Conformational changes in protein structure allow the control of the permeation abilities by outer stimuli (gating). In polymeric membranes, the selectivity is due to electrostatic or size-exclusion. It can thus be controlled by size variation or electric charges. Controllable permeation can be useful to determine particle-size distributions in continuous ow, e.g. in micro uidics and biomedicine to gain cell diameter profiles in blood. The present approach uses patterned polyethylene terephthalate (PET) membranes with hydrogel surface coating for permeation control by size-exclusion. The thermosensitive hydrogel poly(N-isopropylacrylamide) (PNIPAAm) is structured with a cross-shaped pore geometry. A change in the temperature of the water ow through the membrane leads to a pore shape variation. The temperature dependent behavior of PNIPAAm can be numerically modeled with a temperature expansion model, where the swelling and deswelling is depicted by temperature dependent expansion coefficients. In the present study, the free swelling behavior was implemented to the Finite Element tool ABAQUS for the complex composite structure of the permeation control membrane. Experimental values of the geometry characteristics were derived from microscopy images with the tool ImageJ and compared to simulation results. Numerical simulations using the derived thermomechanical model for different pore geometries (circular, rectangle, cross and triangle) were performed. With this study, we show that the temperature expansion model with values from the free swelling behavior can be used to adequately predict the deformation behavior of the complex membrane system. The predictions can be used to optimize the behavior of the membrane pores and the overall performance of the smart membrane.

Every day, we make many value-based decisions where we weigh the value of options with other properties, e.g. their time of delivery. In the laboratory, such value-based decision-making is usually studied on a trial by trial basis and each decision is assumed to represent an isolated choice process. Real-life decisions however are usually embedded in a rich context of previous choices at different time scales. A fundamental question is therefore how the dynamics of value-based decision processes unfold on a time scale across several decisions. Indeed, findings from perceptual decision making suggest that sequential decisions patterns might also be present for vale-based decision making. Here, we use a neural-inspired attractor model as an instance of dynamic models from perceptual decision making, as such models incorporate inherent activation dynamics across decisions. We use the model to predict sequential patterns, namely oscillatory switching, perseveration and dependence of perseveration on the delay between decisions. Furthermore, we predict RT effects for specific sequences of trials. We validate the predictions in two new studies and a reanalysis of existing data from a novel decision game in which participants have to perform delay discounting decisions. Applying the validated reasoning to a well-established choice questionnaire, we illustrate and discuss that taking sequential choice patterns into account may be necessary to accurately analyse and model value-based decision processes, especially when considering differences between individuals.

No description available.