"This thesis focuses on two key problems in reinforcement learning: How to design reward functions to obtain intended behaviors in autonomous systems using the learning-based control? Given complex mission specification, how to shape the reward function to achieve fast convergence and reduce sample complexity while learning the optimal policy? To answer these questions, the first part of this thesis investigates inverse reinforcement learning (IRL) method with a purpose of learning a reward function from expert demonstrations. However, existing algorithms often assume that the expert demonstrations are generated by the same reward function. Such an assumption may be invalid as one may need to aggregate data from multiple experts to obtain a sufficient set of demonstrations. In the first and the major part of the thesis, we develop a novel method, called Non-parametric Behavior Clustering IRL. This algorithm allows one to simultaneously cluster behaviors while learning their reward functions from demonstrations that are generated from more than one expert/behavior. Our approach is built upon the expectation-maximization formulation and non-parametric clustering in the IRL setting. We apply the algorithm to learn, from driving demonstrations, multiple driver behaviors (e.g., aggressive vs. evasive driving behaviors). In the second task, we study whether reinforcement learning can be used to generate complex behaviors specified in formal logic — Linear Temporal Logic (LTL). Such LTL tasks may specify temporally extended goals, safety, surveillance, and reactive behaviors in a dynamic environment. We introduce reward shaping under LTL constraints to improve the rate of convergence in learning the optimal and probably correct policies. Our approach exploits the relation between reward shaping and actor-critic methods for speeding up the convergence and, as a consequence, reducing training samples. We integrate compositional reasoning in formal methods with actor-critic reinforcement learning algorithms to initialize a heuristic value function for reward shaping. This initialization can direct the agent towards efficient planning subject to more complex behavior specifications in LTL. The investigation takes the initial step to integrate machine learning with formal methods and contributes to building highly autonomous and self-adaptive robots under complex missions."

Cette thèse part de l’étude de l’espace social de deux groupes d’ouvriers métallurgistes de Bhopal. L’un est composé de musulmans habitant dans les quartiers autoconstruits pollués suite à la catastrophe industrielle qui a marqué l’histoire contemporaine de la ville et travaillant dans des ateliers de métallurgie au sein de la vieille ville. L’autre est formé majoritairement d’hindous venant de villages parfois éloignés employés sur des chantiers de viaduc dans Bhopal et ses environs. Tous deux travaillent dans le secteur informel, dans un rapport à l’emploi incertain. En explorant leurs relations hors travail, elle décrit la manière dont se construisent les rapports sociaux et les représentations collectives. Elle montre également comment la confrontation à l’incertitude marquant de nombreux aspects de leur quotidien ainsi que le rapport au travail interagissent avec ces constructions. En s’intéressant à la nature des relations au travail et à celle des rapports de domination, elle montre que les travailleurs ont de nombreuses marges de négociation, malgré une importante résurgence du paternalisme combinée à une faiblesse globale des protections concrètes. En s’appuyant sur une ethnographie des techniques et du rapport au corps engagé dans le labeur, elle montre que les idéologies en découlant, trop rarement étudiées, constituent le cœur d’un système de valeurs qui permet de légitimer les hiérarchies, mais aussi de les stabiliser, de les remettre en cause et de rendre possible une mobilité sociale grâce au talent. Mais il est également menacé par l’incertitude qui pèse sur ces cultures de la mètis dans un environnement technologique en profonde mutation et une configuration sociologique dans laquelle la valorisation de l’enseignement supérieur est toujours plus hégémonique. / This doctoral thesis starts with the study of the social space of two groups of metal workers in Bhopal. The first one is made of Muslim inhabitants of the polluted neighborhoods which have been contaminated following the 1984 industrial disaster who work in the Old City’s metal workshops. The other one in made of a majority of Hindus coming from the rural hinterland, sometimes from distant villages, and hired in the flyover construction yards in and around Bhopal. Both are working in the informal sector, and experiment uncertain conditions of employment. By exploring their relationships outside of work it describes the way their social relations and their collective representations are constructing themselves. It also shows how the confronting with uncertainty and their relationship to work are interacting with these social constructs. It shifts then its focus to the relationships on the shop floor, the nature of the labour and domination relationships and it shows that the workers have many margins of negotiation, in spite of an important resurgence of paternalistic structures combined with weak empirical protections. Then, the thesis makes an ethnography of the techniques and the body commitment involved in the labour process in order to unveil ideologies of labour which constitute the core of a value system which allows to legitimate the hierarchical positions but also to contest it, and allows a social mobility based on skills. But this system is also threatened by the uncertainty of these cultures of mètis in a technological environment which experiments a deep technological mutation and a social context where the valorization of the academic education becomes more and more hegemonic.

In this thesis we study the problem of learning under uncertainty using the statistical learning paradigm. We rst propose a linear maximum margin classi er that deals with uncertainty in data input. More speci cally, we reformulate the standard Support Vector Machine (SVM) framework such that each training example can be modeled by a multi-dimensional Gaussian distribution described by its mean vector and its covariance matrix { the latter modeling the uncertainty. We address the classi cation problem and de ne a cost function that is the expected value of the classical SVM cost when data samples are drawn from the multi-dimensional Gaussian distributions that form the set of the training examples. Our formulation approximates the classical SVM formulation when the training examples are isotropic Gaussians with variance tending to zero. We arrive at a convex optimization problem, which we solve e - ciently in the primal form using a stochastic gradient descent approach. The resulting classi er, which we name SVM with Gaussian Sample Uncertainty (SVM-GSU), is tested on synthetic data and ve publicly available and popular datasets; namely, the MNIST, WDBC, DEAP, TV News Channel Commercial Detection, and TRECVID MED datasets. Experimental results verify the e ectiveness of the proposed method. Next, we extended the aforementioned linear classi er so as to lead to non-linear decision boundaries, using the RBF kernel. This extension, where we use isotropic input uncertainty and we name Kernel SVM with Isotropic Gaussian Sample Uncertainty (KSVM-iGSU), is used in the problems of video event detection and video aesthetic quality assessment. The experimental results show that exploiting input uncertainty, especially in problems where only a limited number of positive training examples are provided, can lead to better classi cation, detection, or retrieval performance. Finally, we present a preliminary study on how the above ideas can be used under the deep convolutional neural networks learning paradigm so as to exploit inherent sources of uncertainty, such as spatial pooling operations, that are usually used in deep networks.

The main goal of this thesis is to investigate the frequentist asymptotic properties of nonparametric Bayesian procedures in inverse problems and the Gaussian white noise model. In the first part, we study the frequentist posterior contraction rate of nonparametric Bayesian procedures in linear inverse problems in both the mildly and severely ill-posed cases. This rate provides a quantitative measure of the quality of statistical estimation of the procedure. A theorem is proved in a general Hilbert space setting under approximation-theoretic assumptions on the prior. The result is applied to non-conjugate priors, notably sieve and wavelet series priors, as well as in the conjugate setting. In the mildly ill-posed setting, minimax optimal rates are obtained, with sieve priors being rate adaptive over Sobolev classes. In the severely ill-posed setting, oversmoothing the prior yields minimax rates. Previously established results in the conjugate setting are obtained using this method. Examples of applications include deconvolution, recovering the initial condition in the heat equation and the Radon transform. In the second part of this thesis, we investigate Bernstein--von Mises type results for adaptive nonparametric Bayesian procedures in both the Gaussian white noise model and the mildly ill-posed inverse setting. The Bernstein--von Mises theorem details the asymptotic behaviour of the posterior distribution and provides a frequentist justification for the Bayesian approach to uncertainty quantification. We establish weak Bernstein--von Mises theorems in both a Hilbert space and multiscale setting, which have applications in $L^2$ and $L^\infty$ respectively. This provides a theoretical justification for plug-in procedures, for example the use of certain credible sets for sufficiently smooth linear functionals. We use this general approach to construct optimal frequentist confidence sets using a Bayesian approach. We also provide simulations to numerically illustrate our approach and obtain a visual representation of the different geometries involved.

This dissertation focuses the corporate behaviors in a dynamic world with uncertainty. Especially, I am interested in how firms tradeoff their investment and cash savings when external financing is costly. The first two chapters fit into this theme. One considers optimal investment and financing policies when uncertainty itself is time-varying, the second investigates how firms prepare themselves against devaluation risks. Both chapters build dynamic corporate theories and test them empirically. The third chapter steps back by asking why aggregate volatility is time-varying and why is it persistent in a dynamic general equilibrium with endogenous growth. I show that endogenous asset allocation between different assets can be the reason. In the first chapter I study how firms manage their cash savings, financing, and investment when aggregate uncertainty is time-varying. I develop and estimate a dynamic model featuring aggregate uncertainty shocks, costly external financing, investment irreversibility, and time-varying risk premia. In my model, firms have a precautionary-savings motive and real options to wait, both of which interact with time-varying uncertainty and are reinforced by state-dependent risk premia. My model confirms previous findings that firms save more in cash and invest less when aggregate uncertainty is high. In addition, I show that in the high uncertainty states, (1) firms with high profitability and low cash are more likely to delay equity issuance, (2) firms with low profitability and high cash are more likely to delay payout, and (3) aggregate equity issuance and payout are both lower. Finally, counterfactual experiments show that (1) a model without dynamic uncertainties cannot explain the observed firm behaviors in high uncertainty states, and (2) time-varying risk premia amplify the impact of the aggregate uncertainty shocks. In the second chapter, I investigate the relationship between investment and cash savings in a special setting: devaluation episodes in emerging markets. Devaluation events are typically anticipated by the economy but affect local firms in the tradable versus nontradable sectors differently. Tradable firms expect higher cash flows but nontradable firms expect lower ones, even their current cash flows are stable because of the currency-pegging. I build a model to show that, investment and cash savings are both complementarity, because of future prospects, and substitution because of limited current cash balance. Before devaluation, tradable firms invest more due to better expectation of the future but have to substitute for a lower cash savings tomorrow. Empirically, I use difference-in-difference approach and two devaluation episodes in Mexico and Argentina to test these predictions. I find strong evidence in Mexico that tradable firms invested more than nontradable firms and save less, as the devaluation was approaching. Evidence in Argentina is not strong. We discuss the potential remedies and future works to do. The final chapter explores asset allocation decisions and endogenous growth volatilities in an economy with endogenous growth. Firms have two produced inputs, capital and technology. When a representative firm optimally allocates the investment between the two inputs, both the consumption growth and its volatility are functions of the economy's technology-to-capital ratio. As a result, not only the long run consumption growth is volatile, but also its volatility is endogenously stochastic. Moreover, after a large negative or positive shock, the economy is away from its optimal allocation. This takes time for the economy to travel back to the optimal allocation because of the convex adjustment costs. As a result, both the consumption growth and its stochastic volatility are persistent. Finally, we discuss the asset pricing implication of the model and show that it microfounds Bansal and Yaron (2004) long-run risk model with time-varying volatilities.

Advances in multiscale methods are presented from two perspectives which address the issue of computational complexity of optimizing and inverse analyzing nonlinear composite materials and structures at multiple scales. The optimization algorithm provides several solutions to meet the enormous computational challenge of optimizing nonlinear structures at multiple scales including: (i) enhanced sampling procedure that provides superior performance of the well-known ant colony optimization algorithm, (ii) a mapping-based meshing of a representative volume element that unlike unstructured meshing permits sensitivity analysis on coarse meshes, and (iii) a multilevel optimization procedure that takes advantage of possible weak coupling of certain scales. We demonstrate the proposed optimization procedure on elastic and inelastic laminated plates involving three scales. We also present an adaptive variant of the measure-theoretic approach (MTA) for stochastic characterization of micromechanical properties based on the observations of quantities of interest at the coarse (macro) scale. The salient features of the proposed nonintrusive stochastic inverse solver are: identification of a nearly optimal sampling domain using enhanced ant colony optimization algorithm for multiscale problems, incremental Latin-hypercube sampling method, adaptive discretization of the parameter and observation spaces, and adaptive selection of number of samples. A complete test data of the TORAY T700GC-12K-31E and epoxy #2510 material system from the NIAR report is employed to characterize and validate the proposed adaptive nonintrusive stochastic inverse algorithm for various unnotched and open-hole laminates. Advances in Multiscale methods also provides us a unique tool to study and analyze human bones, which can be seen as a composite material, too. We used two multiscale approaches for fracture analysis of full scale femur. The two approaches are the reduced order homogenization (ROH) and the novel accelerated reduced order homogenization (AROH). The AROH is based on utilizing ROH calibrated to limited data as a training tool to calibrate a simpler, single-scale anisotropic damage model. For bone tissue orientation, we take advantage of so-called Wolff’s law. The meso-phase properties are identified from the least square minimization of error between the overall cortical and trabecular bone properties and those predicted from the homogenization. The overall elastic and inelastic properties of the cortical and trabecular bone microstructure are derived from bone density that can be estimated from the Hounsfield units (HU). For model validation, we conduct ROH and AROH simulations of full scale finite element model of femur created from the QCT and compare the simulation results with available experimental data.

Os métodos determinísticos utilizados para avaliação econômico-financeira de projetos de investimentos, como o Valor Presente Líquido (VPL) e a Taxa Interna de Retorno (TIR), contemplam exatidão do comportamento futuro das variáveis inerentes ao projeto. Porém, as imprevisibilidades futuras acrescidas da alta volatilidade da economia e tecnologia mundial tornam as análises determinísticas frágeis em situações onde existam incertezas, o que pode levar gestores e investidores a tomar decisões equivocadas quanto à alocação de capital. O presente trabalho tem como objetivo geral a comparação entre dois métodos que podem ser utilizados para avaliação de investimentos que abordam a condição de incerteza. O método de Monte Carlo, em seu caráter estatístico, permite que as variáveis presentes sejam consideradas por meio de distribuições de probabilidade, as quais associadas a geração de números aleatórios fornecem uma resposta que considera as incertezas presentes. O Valor Presente Líquido fuzzy constitui-se em um método alternativo para análise, o qual considera as variáveis incertas como números nebulosos, ou seja, concepções matemáticas que não apresentam fronteiras rígidas. Por meio da aplicação dos métodos em uma situação real de investimento, buscou-se realizar uma análise comparativa, que levasse em conta os resultados numéricos obtidos e a conceituação teórica envolvida. / The deterministic methods used for economical and financial evaluation of investments projects, such as Net Present Value (NPV) and Internal Rate Return (IRR) consider the future comportment of the project variables as exact values. Nevertheless, the future unpredictability and the high volatility of world economy and technology make fragile the deterministic analysis under situations that uncertainty is present, what may lead managers and investors to take bad decisions about capital allocation. The main objective of this assignment is two compare two different methods used to evaluate investments under uncertainty. The Monte Carlo method is its statistical character, allows to associate probability distributions with random numbers and this application provides that includes uncertainty. The fuzzy Net Present Value is an alternative method to analyze investments, which consider uncertainty variable as fuzzy numbers, i.e., mathematics conceptions that do not present absolute borders. Two different kinds of comparisons were produced by applications of both methods in a real investment situation: the first was developed based on numeric analysis; the second is based on a theory that involves those methods.

Toda medida, desde que feita por um instrumento real, tem uma imprecisão associada. Neste trabalho, abordamos a questão das imprecisões em experimentos de microarranjos de cDNA de dois canais, uma tecnologia que tem sido muito explorada nos últimos anos e que ainda é um importante auxiliar nos estudos de expressões gênicas. Dezenas de milhares de representantes de genes são impressos em uma lâmina de vidro e hibridizados simultaneamente com RNA mensageiro de duas amostras diferentes de células. Essas amostras são marcadas com corantes fluorescentes diferentes e a lâmina, após a hibridização, é digitalizada, obtendo-se duas imagens. As imagens são analisadas com programas especiais que segmentam os locais que estavam os genes e extraem estatísticas dos píxeis de cada local. Por exemplo, a média, a mediana e a variância das intensidades do conjunto de píxeis de cada local (o mesmo é feito normalmente para uma área em volta de cada local, chamada de fundo). Estimadores estatísticos como o da variância nos dão uma estimativa de quão precisa é uma certa medida. Uma vez de posse das estimativas das intensidades de cada local, para se obter a efetiva expressão de um gene, algumas transformações são feitas nos dados de forma a eliminar variabilidades sistemáticas. Neste trabalho, mostramos como podem ser feitas as análises a partir de uma medida de expressão gênica com um erro estimado. Mostramos como estimar essa imprecisão e estudamos, em termos de propagação da imprecisão, os efeitos de algumas transformações realizadas nos dados, por exemplo, a remoção do viés estimado pelo método de regressão local robusta, mais conhecido como \\textit{lowess}. Uma vez obtidas as estimativas das imprecisões propagadas, mostramos também como utilizá-las na determinação dos genes diferencialmente expressos entre as amostras estudadas. Por fim, comparamos os resultados com os obtidos por formas clássicas de análise, em que são desconsideradas as imprecisões das medidas. Concluímos que a modelagem das imprecisões das medidas pode favorecer as análises, já que os resultados obtidos em uma aplicação com dados reais de expressões gênicas foram condizentes com os que encontramos na literatura. / Any measurement, since it is made for a real instrument, has an uncertainty associated with it. In the present paper, we address this issue of uncertainty in two-channel cDNA Microarray experiments, a technology that has been widely used in recent years and is still an important tool for gene expression studies. Tens of thousands of gene representatives are printed onto a glass slide and hybridized simultaneously with mRNA from two different cell samples. Different fluorescent dyes are used for labeling both samples. After hybridization, the glass slide is scanned yielding two images. Image processing and analysis programs are used for spot segmentation and pixel statistics computation, for instance, the mean, median and variance of pixel intensities for each spot. The same statistics are computed for the pixel intensities in the background region. Statistical estimators such as the variance gives us an estimate of the accuracy of a measurement. Based on the intensity estimates for each spot, some data transformations are applied in order to eliminate systematic variability so we can obtain the effective gene expression. This paper shows how to analyze gene expression measurements with an estimated error. We presented an estimate of this uncertainty and we studied, in terms of error propagation, the effects of some data transformations. An example of data transformation is the correction of the bias estimated by a robust local regression method, also known as \\textit{lowess}. With the propagated errors obtained, we also showed how to use them for detecting differentially expressed genes between different conditions. Finally, we compared the results with those obtained by classical analysis methods, in which the measurement errors are disregarded. We conclude that modeling the measurements uncertainties can improve the analysis, since the results obtained in a real gene expressions data base were consistent with the literature.

Uncertainty propagation in engineering mechanics and dynamics is a highly challenging problem that requires development of analytical/numerical techniques for determining the stochastic response of complex engineering systems. In this regard, although Monte Carlo simulation (MCS) has been the most versatile technique for addressing the above problem, it can become computationally daunting when faced with high-dimensional systems or with computing very low probability events. Thus, there is a demand for pursuing more computationally efficient methodologies. Further, most structural systems are likely to exhibit nonlinear and time-varying behavior when subjected to extreme events such as severe earthquake, wind and sea wave excitations. In such cases, a reliable identification approach is behavior and for assessing its reliability. Current work addresses two research themes in the field of stochastic engineering dynamics related to the above challenges. In the first part of the dissertation, the recently developedWiener Path Integral (WPI) technique for determining the joint response probability density function (PDF) of nonlinear systems subject to Gaussian white noise excitation is generalized herein to account for non-white, non-Gaussian, and non-stationary excitation processes. Specifically, modeling the excitation process as the output of a filter equation with Gaussian white noise as its input, it is possible to define an augmented response vector process to be considered in the WPI solution technique. A significant advantage relates to the fact that the technique is still applicable even for arbitrary excitation power spectrum forms. In such cases, it is shown that the use of a filter approximation facilitates the implementation of the WPI technique in a straightforward manner, without compromising its accuracy necessarily. Further, in addition to dynamical systems subject to stochastic excitation, the technique can also account for a special class of engineering mechanics problems where the media properties are modeled as non-Gaussian and non-homogeneous stochastic fields. Several numerical examples pertaining to both single- and multi-degree-of freedom systems are considered, including a marine structural system exposed to flow-induced non-white excitation, as well as a beam with a non-Gaussian and non-homogeneous Young’s modulus. Comparisons with MCS data demonstrate the accuracy of the technique. In the second part of the dissertation, a novel multiple-input/single-output (MISO) system identification technique is developed for parameter identification of nonlinear time-variant multi-degree-of-freedom oscillators with fractional derivative terms subject to incomplete non-stationary data. The technique utilizes a representation of the nonlinear restoring forces as a set of parallel linear subsystems. In this regard, the oscillator is transformed into an equivalent MISO system in the wavelet domain. Next, a recently developed L1-norm minimization procedure based on compressive sampling theory is applied for determining the wavelet coefficients of the available incomplete non-stationary input-output (excitation-response) data. Finally, these wavelet coefficients are utilized to determine appropriately defined time- and frequency-dependent wavelet based frequency response functions and related oscillator parameters. A nonlinear time-variant system with fractional derivative elements is used as a numerical example to demonstrate the reliability of the technique even in cases of noise corrupted and incomplete data.

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / A tese versa sobre as grandes questões relativas à contracepção no Brasil. Integra um esforço por analisar condutas referentes à contracepção, segundo lógicas que priorizam a situacionalidade e a relacionalidade de tais fenômenos. As estratégias para gerir a fecundidade são constitutivas da sexualidade heterossexual. Mulheres e homens podem usar ou não contracepção; as razões dessa conduta extrapolam aspectos concernentes a informação e acesso. Busca-se compreender as práticas contraceptivas a partir do processo do aprendizado das lógicas relacionais e de gênero, em diferentes momentos dos percursos biográficos: o início da trajetória afetivo-sexual, os contextos de irrupção de uma gravidez e o encerramento da potencialidade reprodutiva, por meio da esterilização contraceptiva. Este compósito demandou a utilização de materiais empíricos distintos para a construção e análise das etapas eleitas dos percursos biográficos. Enfoca-se, primeiramente, o momento de passagem à sexualidade com parceiro. Problematiza-se a ideia de relaxamento das práticas contraceptivas, a partir da iniciação sexual, concepção corrente na literatura nacional em função do decréscimo de uso de preservativo em relações sexuais posteriores. Aborda-se, em seguida, as atitudes e as questões presentes no processo de construção da prática contraceptiva, no momento em que a vida sexual se torna regular. A proposição da perspectiva da gestão contraceptiva sublinha as posições dos protagonistas, marcadas pelo gênero. Por último, analisa-se as circunstâncias biográficas e os cenários relacionais da esterilização contraceptiva, a qual emerge como uma estratégia de estabilização ou de consolidação de um percurso contraceptivo/reprodutivo. O debate em torno da contracepção no Brasil apresenta a tendência a enfatizar a determinação social para explicar as gestações imprevistas. Contudo, salienta-se, com base em uma literatura crítica, as dimensões de agência individual, ainda que circunscritas por um campo delimitado de possibilidades. / The thesis deals with the major issues relating to contraception in Brazil. Integrates an effort to analyze behaviors related to contraception, according to logics that prioritize the situatedness and relationality of such phenomena. Strategies to manage fertility are constitutive of heterosexual sexuality. Women and men can use contraception or not; the reasons for that conduct extrapolate aspects concerning information and access. We seek to understand contraceptive practices from the process of learning of relational logic and gender, at different times of biographies: the beginning of the affective-sexual trajectory, the contexts of eruption of pregnancy and termination of reproductive potential by means contraceptive sterilization. This composite require use of different empirical materials for the construction and analysis of the stages of elected biographies. It focuses, first, the passing moment sexuality with a partner. Problematizes the idea of relaxation of contraceptive practices, from the sexual initiation, current design in the national literature due to the decrease in condom use in sex later. We discuss then the attitudes and issues present in the construction process of contraceptive practice, at the time the sexual life becomes regular. The proposition from the perspective of contraceptive management emphasizes the positions of the protagonists, marked by gender. Finally, we analyze the biographical circumstances and relational scenarios of contraceptive sterilization, which emerges as a strategy of stabilization or consolidation of a contraceptive / reproductive route. The debate around the contraception in Brazil has a tendency to emphasize the social determining to explain unexpected pregnancies. However, it is noted based on a critical literature, the dimensions of individual agency, although circumscribed by a delimited range of possibilities.