This thesis proposes new analysis tools for simulation models in the presence of data. To achieve a representation close to reality, simulation models are typically endowed with a set of inputs, termed parameters, that represent several controllable, stochastic or unknown components of the system. Because these models often utilize computationally expensive procedures, even modern supercomputers require a nontrivial amount of time, money, and energy to run for complex systems. Existing statistical frameworks avoid repeated evaluations of deterministic models through an emulator, constructed by conducting an experiment on the code. In high dimensional scenarios, the traditional framework for emulator-based analysis can fail due to the computational burden of inference. This thesis proposes a new class of experiments where inference from half a million observations is possible in seconds versus the days required for the traditional technique. In a case study presented in this thesis, the parameter of interest is a function as opposed to a scalar or a set of scalars, meaning the problem exists in the high dimensional regime. This work develops a new modeling strategy to nonparametrically study the functional parameter using Bayesian inference. Stochastic simulations are also investigated in the thesis. I describe the development of emulators through a framework termed quantile kriging, which allows for non-parametric representations of the stochastic behavior of the output whereas previous work has focused on normally distributed outputs. Furthermore, this work studied asymptotic properties of this methodology that yielded practical insights. Under certain regulatory conditions, there is the following result: By using an experiment that has the appropriate ratio of replications to sets of different inputs, we can achieve an optimal rate of convergence. Additionally, this method provided the basic tool for the study of defect patterns and a case study is explored.

Let $\Omega\subset\mathbb{R}^n$ be an open and connected subset of $\mathbb{R}^n$. We say that a function $F\colon \Omega-\Omega\to\mathbb{C}$, where $\Omega-\Omega=\{x-y\colon x,y\in\Omega\}$, is positive definite if for any $x_1,\ldots,x_m\in\Omega$ and any $c_1,\ldots,c_m\in \mathbb{C}$ we have that $\sum_{j,k=1}^m F(x_j-x_k)c_j\overline{c_k}\geq 0$. Let $F\colon\Omega-\Omega\to\mathbb{C}$ be a continuous positive definite function. We give necessary and sufficient conditions for $F$ to have an extension to a continuous and positive definite function defined on the entire Euclidean space $\mathbb{R}^n$. The conditions are formulated in terms of strong commutativity of some certain selfadjoint operators defined on a Hilbert space associated to our positive definite function.

In the first part of this paper we show a similarity between the principle of Structural Risk Minimization Principle (SRM) (Vapnik, 1982) and the idea of Sparse Approximation, as defined in (Chen, Donoho and Saunders, 1995) and Olshausen and Field (1996). Then we focus on two specific (approximate) implementations of SRM and Sparse Approximation, which have been used to solve the problem of function approximation. For SRM we consider the Support Vector Machine technique proposed by V. Vapnik and his team at AT&T Bell Labs, and for Sparse Approximation we consider a modification of the Basis Pursuit De-Noising algorithm proposed by Chen, Donoho and Saunders (1995). We show that, under certain conditions, these two techniques are equivalent: they give the same solution and they require the solution of the same quadratic programming problem.

No description available.

We present important results from Hilbert space and functional analysis for understanding the subject ofReproducing kernel Hilbert spaces. We then showcase the underlying theory and properties of Reproducingkernel Hilbert Spaces. Finally, we show how the theory of reproducing kernel Hilbert spaces is applicable inboth interpolation and machine learning.

This thesis aims to contribute to the area of kernel methods, which are a class of machine learning methods known for their wide applicability and state-of-the-art performance, but which suffer from high training and evaluation complexity. The work in this thesis utilizes the notion of reduced-set models to alleviate the training and testing complexities of these methods in a unified manner. In the first part of the thesis, we use recent results in kernel smoothing and integral-operator learning to design a generic strategy to speed up various kernel methods. In Chapter 3, we present a method to speed up kernel PCA (KPCA), which is one of the fundamental kernel methods for manifold learning, by using reduced-set density estimates (RSDE) of the data. The proposed method induces an integral operator that is an approximation of the ideal integral operator associated to KPCA. It is shown that the error between the ideal and approximate integral operators is related to the error between the ideal and approximate kernel density estimates of the data. In Chapter 4, we derive similar approximation algorithms for Gaussian process regression, diffusion maps, and kernel embeddings of conditional distributions. In the second part of the thesis, we use reduced-set models for kernel methods to tackle online learning in model-reference adaptive control (MRAC). In Chapter 5, we relate the properties of the feature spaces induced by Mercer kernels to make a connection between persistency-of-excitation and the budgeted placement of kernels to minimize tracking and modeling error. In Chapter 6, we use a Gaussian process (GP) formulation of the modeling error to accommodate a larger class of errors, and design a reduced-set algorithm to learn a GP model of the modeling error. Proofs of stability for all the algorithms are presented, and simulation results on a challenging control problem validate the methods.

Este trabalho é dedicado ao estudo de propriedades teóricas dos operadores integrais positivos em \'L POT. 2\' (X; u), quando X é um espaço topológico localmente compacto ou primeiro enumerável e u é uma medida estritamente positiva. Damos ênfase à análise de propriedades espectrais relacionadas com extensões do Teorema de Mercer e ao estudo dos espaços de Hilbert de reprodução relacionados. Como aplicação, estudamos o decaimento dos autovalores destes operadores, em um contexto especial. Finalizamos o trabalho com a análise de propriedades de suavidade das funções do espaço de Hilbert de reprodução, quando X é um subconjunto do espaço euclidiano usual e u é a medida de Lebesgue usual de X / In this work we study theoretical properties of positive integral operators on \'L POT. 2\'(X; u), in the case when X is a topological space, either locally compact or first countable, and u is a strictly positive measure. The analysis is directed to spectral properties of the operator which are related to some extensions of Mercer\'s Theorem and to the study of the reproducing kernel Hilbert spaces involved. As applications, we deduce decay rates for the eigenvalues of the operators in a special but relevant case. We also consider smoothness properties for functions in the reproducing kernel Hilbert spaces when X is a subset of the Euclidean space and u is the Lebesgue measure of the space

Neste trabalho analisamos o papel das funções layout de um núcleo positivo definido K sobre um espaço topológico de Hausdor E com relação a duas propriedades específicas: a universalidade de K e a ortogonalidade no espaço de Hilbert de reprodução de K a partir de suportes disjuntos. As funções layout sempre existem mas podem não ser únicas. De uma maneira geral, a função layout e uma aplicação que transfere, convenientemente, informações do espaço E para um espaço com produto interno de dimensão alta, onde métodos lineares podem ser usados. Tanto a universalidade quanto a ortogonalidade pressupõem a continuidade do núcleo. O primeiro conceito exige que para cada compacto não vazio X de E, o conjunto de \"seções\" {K(., y) : y \'PERTENCE\' X} seja total no espaço de todas as funções contínuas com domínio X, munido da topologia da convergência uniforme. Um dos resultados principais do trabalho caracteriza a universalidade de um núcleo K através de uma propriedade de universalidade semelhante da função layout. A ortogonalidade a partir de suportes disjuntos almeja então a ortogonalidade de quaisquer duas funções do espaço de Hilbert de reprodução de K quando seus suportes não se intersectam / We analyze the role of feature maps of a positive denite kernel K acting on a Hausdorff topological space E in two specific properties: the universality of K and the orthogonality in the reproducing kernel Hilbert space of K from disjoint supports. Feature maps always exist but may not be unique. A feature map may be interpreted as a kernel based procedure that maps the data from the original input space E into a potentially higher dimensional \"feature space\" in which linear methods may then be used. Both properties, universality and orthogonality from disjoint supports, make sense under continuity of the kernel. Universality of K is equivalent to the fundamentality of {K(. ; y) : y \'IT BELONGS\' X} in the space of all continuous functions on X, with the topology of uniform convergence, for all nonempty compact subsets X of E. One of the main results in this work is a characterization of the universality of K from a similar concept for the feature map. Orthogonality from disjoint supports seeks the orthogonality of any two functions in the reproducing kernel Hilbert space of K when the functions have disjoint supports

Este trabalho é dedicado ao estudo de propriedades teóricas dos operadores integrais positivos em \'L POT. 2\' (X; u), quando X é um espaço topológico localmente compacto ou primeiro enumerável e u é uma medida estritamente positiva. Damos ênfase à análise de propriedades espectrais relacionadas com extensões do Teorema de Mercer e ao estudo dos espaços de Hilbert de reprodução relacionados. Como aplicação, estudamos o decaimento dos autovalores destes operadores, em um contexto especial. Finalizamos o trabalho com a análise de propriedades de suavidade das funções do espaço de Hilbert de reprodução, quando X é um subconjunto do espaço euclidiano usual e u é a medida de Lebesgue usual de X / In this work we study theoretical properties of positive integral operators on \'L POT. 2\'(X; u), in the case when X is a topological space, either locally compact or first countable, and u is a strictly positive measure. The analysis is directed to spectral properties of the operator which are related to some extensions of Mercer\'s Theorem and to the study of the reproducing kernel Hilbert spaces involved. As applications, we deduce decay rates for the eigenvalues of the operators in a special but relevant case. We also consider smoothness properties for functions in the reproducing kernel Hilbert spaces when X is a subset of the Euclidean space and u is the Lebesgue measure of the space

Neste trabalho analisamos o papel das funções layout de um núcleo positivo definido K sobre um espaço topológico de Hausdor E com relação a duas propriedades específicas: a universalidade de K e a ortogonalidade no espaço de Hilbert de reprodução de K a partir de suportes disjuntos. As funções layout sempre existem mas podem não ser únicas. De uma maneira geral, a função layout e uma aplicação que transfere, convenientemente, informações do espaço E para um espaço com produto interno de dimensão alta, onde métodos lineares podem ser usados. Tanto a universalidade quanto a ortogonalidade pressupõem a continuidade do núcleo. O primeiro conceito exige que para cada compacto não vazio X de E, o conjunto de \"seções\" {K(., y) : y \'PERTENCE\' X} seja total no espaço de todas as funções contínuas com domínio X, munido da topologia da convergência uniforme. Um dos resultados principais do trabalho caracteriza a universalidade de um núcleo K através de uma propriedade de universalidade semelhante da função layout. A ortogonalidade a partir de suportes disjuntos almeja então a ortogonalidade de quaisquer duas funções do espaço de Hilbert de reprodução de K quando seus suportes não se intersectam / We analyze the role of feature maps of a positive denite kernel K acting on a Hausdorff topological space E in two specific properties: the universality of K and the orthogonality in the reproducing kernel Hilbert space of K from disjoint supports. Feature maps always exist but may not be unique. A feature map may be interpreted as a kernel based procedure that maps the data from the original input space E into a potentially higher dimensional \"feature space\" in which linear methods may then be used. Both properties, universality and orthogonality from disjoint supports, make sense under continuity of the kernel. Universality of K is equivalent to the fundamentality of {K(. ; y) : y \'IT BELONGS\' X} in the space of all continuous functions on X, with the topology of uniform convergence, for all nonempty compact subsets X of E. One of the main results in this work is a characterization of the universality of K from a similar concept for the feature map. Orthogonality from disjoint supports seeks the orthogonality of any two functions in the reproducing kernel Hilbert space of K when the functions have disjoint supports