Thesis (M.S.)--Montana State University--Bozeman, 2006. / Typescript. Chairperson, Graduate Committee: Ross K. Snider. Includes bibliographical references (leaves 101-102).

A wide range of statistical and machine learning problems involve learning one or multiple latent functions, or properties thereof, from datasets. Examples include regression, classification, principal component analysis, optimisation, learning intensity functions of point processes and reinforcement learning to name but a few. For all these problems, positive semi-definite kernels (or simply kernels) provide a powerful tool for postulating flexible nonparametric hypothesis spaces over functions. Despite recent work on such kernel methods, parametric alternatives, such as deep neural networks, have been at the core of most artificial intelligence breakthroughs in recent years. In this thesis, both theoretical and methodological foundations are presented for constructing fully automated, scalable, and general-purpose kernel machines that perform very well over a wide range of input dimensions and sample sizes. This thesis aims to contribute towards bridging the gap between kernel methods and deep learning and to propose methods that have the advantage over deep learning in performing well on both small and large scale problems. In Part I we provide a gentle introduction to kernel methods, review recent work, identify remaining gaps and outline our contributions. In Part II we develop flexible and scalable Bayesian kernel methods in order to address gaps in methods capable of dealing with the special case of datasets exhibiting locally homogeneous patterns. We begin with two motivating applications. First we consider inferring the intensity function of an inhomogeneous point process in Chapter 2. This application is used to illustrate that often, by carefully adding some mild asymmetry in the dependency structure in Bayesian kernel methods, one may considerably scale-up inference while improving flexibility and accuracy. In Chapter 3 we propose a scalable scheme for online forecasting of time series and fully-online learning of related model parameters, under a kernel-based generative model that is provably sufficiently flexible. This application illustrates that, for one-dimensional input spaces, restricting the degree of differentiability of the latent function of interest may considerably speed-up inference without resorting to approximations and without any adverse effect on flexibility or accuracy. Chapter 4 generalizes these approaches and proposes a novel class of stochastic processes we refer to as string Gaussian processes (string GPs) that, when used as functional prior in a Bayesian nonparametric framework, allow for inference in linear time complexity and linear memory requirement, without resorting to approximations. More importantly, the corresponding inference scheme, which we derive in Chapter 5, also allows flexible learning of locally homogeneous patterns and automated learning of model complexity - that is automated learning of whether there are local patterns in the data in the first place, how much local patterns are present, and where they are located. In Part III we provide a broader discussion covering all types of patterns (homogeneous, locally homogeneous or heterogeneous patterns) and both Bayesian or frequentist kernel methods. In Chapter 6 we begin by discussing what properties a family of kernels should possess to enable fully automated kernel methods that are applicable to any type of datasets. In this chapter, we discuss a novel mathematical formalism for the notion of ‘general-purpose' families of kernels, and we argue that existing families of kernels are not general-purpose. In Chapter 7 we derive weak sufficient conditions for families of kernels to be general-purpose, and we exhibit tractable such families that enjoy a suitable parametrisation, that we refer to as generalized spectral kernels (GSKs). In Chapter 8 we provide a scalable inference scheme for automated kernel learning using general-purpose families of kernels. The proposed inference scheme scales linearly with the sample size and enables automated learning of nonstationarity and model complexity from the data, in virtually any kernel method. Finally, we conclude with a discussion in Chapter 9 where we show that deep learning can be regarded as a particular type of kernel learning method, and we discuss possible extensions in Chapter 10.

Submitted by Diogo Barreiros (diogo.barreiros@ufba.br) on 2017-02-06T16:59:36Z No. of bitstreams: 1 dissertacao_mestrado_jose-grimaldo.pdf: 19406681 bytes, checksum: d108855fa0fb0d44ee5d1cb59579a04c (MD5) / Approved for entry into archive by Vanessa Reis (vanessa.jamile@ufba.br) on 2017-02-07T11:51:58Z (GMT) No. of bitstreams: 1 dissertacao_mestrado_jose-grimaldo.pdf: 19406681 bytes, checksum: d108855fa0fb0d44ee5d1cb59579a04c (MD5) / Made available in DSpace on 2017-02-07T11:51:58Z (GMT). No. of bitstreams: 1 dissertacao_mestrado_jose-grimaldo.pdf: 19406681 bytes, checksum: d108855fa0fb0d44ee5d1cb59579a04c (MD5) / Accuracy in image object detection has been usually achieved at the expense of much computational load. Therefore a trade-o between detection performance and fast execution commonly represents the ultimate goal of an object detector in real life applications. Most images are composed of non-trivial amounts of background information, such as sky, ground and water. In this sense, using an object detector against a recurring background pattern can require a signi cant amount of the total processing time. To alleviate this problem, search space reduction methods can help focusing the detection procedure on more distinctive image regions. / Among the several approaches for search space reduction, we explored saliency information to organize regions based on their probability of containing objects. Saliency detectors are capable of pinpointing regions which generate stronger visual stimuli based solely on information extracted from the image. The fact that saliency methods do not require prior training is an important benefit, which allows application of these techniques in a broad range of machine vision domains. We propose a novel method toward the goal of faster object detectors. The proposed method was grounded on a multi-scale spectral residue (MSR) analysis using saliency detection. For better search space reduction, our method enables fine control of search scale, more robustness to variations on saliency intensity along an object length and also a direct way to control the balance between search space reduction and false negatives caused by region selection. Compared to a regular sliding window search over the images, in our experiments, MSR was able to reduce by 75% (in average) the number of windows to be evaluated by an object detector while improving or at least maintaining detector ROC performance. The proposed method was thoroughly evaluated over a subset of LabelMe dataset (person images), improving detection performance in most cases. This evaluation was done comparing object detection performance against different object detectors, with and without MSR. Additionally, we also provide evaluation of how different object classes interact with MSR, which was done using Pascal VOC 2007 dataset. Finally, tests made showed that window selection performance of MSR has a good scalability with regard to image size. From the obtained data, our conclusion is that MSR can provide substantial benefits to existing sliding window detectors

Spectral moment formulae of various shapes have proven to be very successful in studying the statistics of central 𝐿-values. In this article, we establish, in a completely explicit fashion, such formulae for the family of 𝐺𝐿(3) × 𝐺𝐿(2) Rankin-Selberg 𝐿-functions using the period integral method. The Kuznetsov and the Voronoi formulae are not needed in our argument. We also prove the essential analytic properties and explicit formulae for the integral transform of our moment formulae. It is hoped that our method will provide insights into moments of 𝐿-functions for higher-rank groups.

Under weak turbulence theory, we will use the random thin phase screen model and the Kolmogorov power-law spectrum to derive approximate models for the scintillation index, covariance function of irradiance fluctuations, and temporal spectrum of irradiance fluctuations for collimated beams. In addition, we will provide an expression for the quasi-frequency of a collimated beam and investigate the relationship between the quasi-frequency and the maximum width of the normalized temporal spectrum of irradiance for a collimated beam.

The objective of this dissertation is to investigate all the necessary components in spectral mixture analysis (SMA) for hyperspectral imagery under an unsupervised circumstance. When SMA is linear, referred to as linear spectral mixture analysis (LSMA), these components include estimation of the number of endmembers, extraction of endmember signatures, and calculation of endmember abundances that can automatically satisfy the sum-to-one and non-negativity constraints. A simple approach for nonlinear spectral mixture analysis (NLSMA) is also investigated. After SMA is completed, a color display is generated to present endmember distribution in the image scene. It is expected that this research will result in an analytic system that can yield optimal or suboptimal SMA output without prior information. Specifically, the uniqueness in each component is described as follow. 1)A new signal subspace-based approach is developed to determine the number of endmembers with relatively robust performance and the least parameter requirement. 2)The best implementation strategy is determined for endmember extraction algorithms using simplex volume maximization and pixel spectral similarity; and algorithm with the special consideration for anomalous pixels is developed to improve the quality of extracted endmembers. 3)A new linear mixture model (LMM) is deployed where the number of endmembers and their types can be changed from pixel to pixel such that the resulting endmember abundances are sum-to-one and nonnegative as required; and fast algorithms are developed to search for a sub-optimal endmember set for each pixel. 4)A simple approach for NLSMA based on LMM is investigated and an automated approach is developed to determine either linear or nonlinear mixing is actually experienced. 5)A color display strategy is developed to present SMA results with high class/endmember separability.

No description available.

No description available.

No description available.

The autocorrelation function is a commonly used tool in statistical time series analysis. Under the assumption of Gaussianity, the sample autocorrelation function is the standard method used to estimate this function given a finite number of observations. Non-Gaussian, impulsive observation noise following probability density functions with thick tails, which often occurs in practice, can bias this estimator, rendering classical time series analysis methods ineffective. This work examines the robustness of two estimators of correlation based on memoryless nonlinear functions of observations, the Phase-Phase Correlator (PPC) and the Median- of-Ratios Estimator (MRE), which are applicable to complex-valued Gaussian random pro- cesses. These estimators are very fast and easy to implement in current processors. We show that these estimators are robust from a bias perspective when complex-valued Gaussian pro- cesses are contaminated with impulsive noise at the expense of statistical efficiency at the assumed Gaussian distribution. Additionally, iterative versions of these estimators named the IMRE and IPPC are developed, realizing an improved bias performance over their non- iterative counterparts and the well-known robust Schweppe-type Generalized M-estimator utilizing a Huber cost function (SHGM). An impulsive noise suppression technique is developed using basis pursuit and a priori atom weighting derived from the newly developed iterative estimators. This new technique is proposed as an alternative to the robust filter cleaner, a Kalman filter-like approach that relies on linear prediction residuals to identity and replace corrupted observations. It does not have the same initialization issues as the robust filter cleaner. Robust spectral estimation methods are developed using these new estimators and impulsive noise suppression techniques. Results are obtained for synthetic complex-valued Guassian processes and real-world digital television signals collected using a software defined radio. / Ph. D.