Thesis (Ph. D.)--Virginia Polytechnic Institute and State University, 1991. / Vita. Abstract. Includes bibliographical references (leaves 132-137). Also available via the Internet.

No description available.

The entire thesis text is included in the research.pdf file; the official abstract appears in the short.pdf file; a non-technical public abstract appears in the public.pdf file. Title from PDF of title page (University of Missouri--Columbia, viewed on March 29, 2010). Thesis advisor: Dr. Alexander Koldobsky. Vita. Includes bibliographical references.

作為機器學習領域中的一個重要課題，模型選擇旨在給定有限樣本的情況下、恰當地確定模型的複雜度。自動模型選擇是指一類快速有效的模型選擇方法，它們以一個足夠大的模型複雜度作為初始，在學習過程中有一種內在機制能夠驅使冗餘結構自動地變為不起作用、從而可以剔除。爲了輔助自動模型選擇的進行，模型的參數通常被假設帶有先驗。對於考慮先驗的各種自動模型選擇方法，已有工作中尚缺乏系統性的比較研究。本篇論文著眼於具有局部高斯結構的模型，進行了系統性的比較分析。 / 具體而言，本文比較了三種典型的自動模型選擇方法的優劣勢，它們分別為變分貝葉斯（Variational Bayesian），最小信息長度（Minimum Message Length），以及貝葉斯陰陽和諧學習（Bayesian Ying‐Yang harmony learning）。首先，我們研究針對高斯混合模型（Gaussian Mixture Model）的模型選擇，即確定該模型中高斯成份的個數。進而，我们假設每個高斯成份都有子空間結構、并研究混合因子分析模型（Mixture of Factor Analyzers）及局部因子分析模型（Local Factor Analysis）下的模型選擇問題，即確定模型中混合成份的個數及各個局部子空間的維度。 / 本篇論文考慮以上各模型的參數的兩類先驗，分別為共軛型先驗及Jeffreys 先驗。其中，共軛型先驗在高斯混合模型上為DNW（Dirichlet‐Normal‐Wishart）先驗，在混合因子分析模型及局部因子分析模型上均為DNG（Dirichlet‐Normal‐Gamma）先驗。由於推導對應Fisher 信息矩陣的解析表達非常困難，在混合因子分析模型及局部因子分析模型上，我們不考慮Jeffreys 先驗以及最小信息長度方法。 / 通過一系列的仿真實驗及應用分析，本文比較了幾種自動模型選擇算法（包括基於高斯混合模型的6 個算法，基於混合因子分析模型及局部因子分析模型的4 個算法），并得到了如下主要發現：1. 對於各種自動模型選擇方法，在所有參數上加先驗都比僅在混合權重上加先驗的效果好。2. 在高斯混合模型上，考慮 DNW 先驗的效果比考慮Jeffreys 先驗的效果好。其中，考慮Jeffreys 先驗時，最小信息長度比變分貝葉斯的效果略好；而考慮DNW 先驗時，變分貝葉斯比最小信息長度的效果好。3. 在高斯混合模型上，當DNW 先驗的超參數（hyper‐parameters）由保持固定變為根據各自學習準則進行優化時，貝葉斯陰陽和諧學習的效果得到了提高，而變分貝葉斯及最小信息長度的結果都會變差。在基於帶DNG 先驗的混合因子分析模型及局部因子分析模型的比較中，以上觀察結果同樣維持。事實上，變分貝葉斯及最小信息長度都缺乏一種引導先驗超參數優化的良好機制。4. 對以上各種模型、無論考慮哪種先驗、以及無論先驗超參數是否在學習過程中進行優化，貝葉斯陰陽和諧學習的效果都明顯地優於變分貝葉斯和最小信息長度。與后兩者相比，貝葉斯陰陽和諧學習對於先驗的依賴程度不高，它的結果在不考慮先驗的情況下已較好，並在考慮Jeffreys 或共軛型先驗時有進一步提高。5. 儘管混合因子分析模型及局部因子分析模型在最大似然準則的參數估計中等價，它們在變分貝葉斯及貝葉斯陰陽和諧學習下的自動模型選擇中卻表现不同。在這兩種方法下，局部因子分析模型皆以明顯的優勢優於混合因子分析模型。 / 爲進行以上比較分析，除了直接使用已有算法或做少許修改之外，本篇論文還提出了五個新的算法來填補空白。針對高斯混合模型，我們提出了帶Jeffreys 先驗的變分貝葉斯算法；通過邊際化（marginalization），我們得到了有多變量學生分佈（Student’s T‐distribution）形式的后驗，并提出了帶DNW 先驗的貝葉斯陰陽和諧學習算法。針對混合因子分析模型及局部因子分析模型，我們通過一系列的近似邊際化過程，得到了有多個學生分佈乘積形式的后驗，并提出了帶DNG 先驗的貝葉斯陰陽和諧學習算法。對應於已有的基於混合因子分析模型的變分貝葉斯算法，我們還提出了基於局部因子分析模型的變分貝葉斯算法，作為一種更有效的可替代選擇。 / Model selection aims to determine an appropriate model scale given a small size of samples, which is an important topic in machine learning. As one type of efficient solution, an automatic model selection starts from a large enough model scale, and has an intrinsic mechanism to push redundant structures to be ineffective and thus discarded automatically during learning. Priors are usually imposed on parameters to facilitate an automatic model selection. There still lack systematic comparisons on automatic model selection approaches with priors, and this thesis is motivated for such a study based on models with local Gaussian structures. / Particularly, we compare the relative strength and weakness of three typical automatic model selection approaches, namely Variational Bayesian (VB), Minimum Message Length (MML) and Bayesian Ying-Yang (BYY) harmony learning, on models with local Gaussian structures. First, we consider Gaussian Mixture Model (GMM), for which the number of Gaussian components is to be determined. Further assuming each Gaussian component has a subspace structure, we extend to consider two models namely Mixture of Factor Analyzers (MFA) and Local Factor Analysis (LFA), for both of which the component number and local subspace dimensionalities are to be determined. / Two types of priors are imposed on parameters, namely a conjugate form prior and a Jeffreys prior. The conjugate form prior is chosen as a Dirichlet-Normal- Wishart (DNW) prior for GMM, and as a Dirichlet-Normal-Gamma (DNG) prior for both MFA and LFA. The Jeffreys prior and the MML approach are not considered on MFA/LFA due to the difficulty in deriving the corresponding Fisher information matrix. Via extensive simulations and applications, comparisons on the automatic model selection algorithms (six for GMM and four for MFA/LFA), we get following main findings:1. Considering priors on all parameters makes each approach perform better than considering priors merely on the mixing weights.2. For all the three approaches on GMM, the performance with the DNW prior is better than with the Jeffreys prior. Moreover, Jeffreys prior makes MML slightly better than VB, while the DNW prior makes VB better than MML.3. As the DNW prior hyper-parameters on GMM are changed from fixed to freely optimized by each of its own learning principle, BYY improves its performance, while VB and MML deteriorate their performances. This observation remains the same when we compare BYY and VB on either MFA or LFA with the DNG prior. Actually, VB and MML lack a good guide for optimizing prior hyper-parameters.4. For bothGMMand MFA/LFA, BYY considerably outperforms both VB and MML, for any type of priors and whether hyper-parameters are optimized. Being different from VB and MML that rely on appropriate priors, BYY does not highly depend on the type of priors. It performs already well without priors and improves by imposing a Jeffreys or a conjugate form prior. 5. Despite the equivalence in maximum likelihood parameter learning, MFA and LFA affect the performances by VB and BYY in automatic model selection. Particularly, both BYY and VB perform better on LFA than on MFA, and the superiority of LFA is reliable and robust. / In addition to adopting the existing algorithms either directly or with some modifications, this thesis develops five new algorithms to fill the missing gap. Particularly on GMM, the VB algorithm with Jeffreys prior and the BYY algorithm with DNW prior are developed, in the latter of which a multivariate Student’s Tdistribution is obtained as the posterior via marginalization. On MFA and LFA, BYY algorithms with DNG priors are developed, where products of multiple Student’s T-distributions are obtained in posteriors via approximated marginalization. Moreover, a VB algorithm on LFA is developed as an alternative choice to the existing VB algorithm on MFA. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Shi, Lei. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2012. / Includes bibliographical references (leaves 153-166). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstract also in Chinese. / Abstract --- p.i / Acknowledgement --- p.iv / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Background --- p.3 / Chapter 1.2 --- Main Contributions of the Thesis --- p.11 / Chapter 1.3 --- Outline of the Thesis --- p.14 / Chapter 2 --- Automatic Model Selection on GMM --- p.16 / Chapter 2.1 --- Introduction --- p.17 / Chapter 2.2 --- Gaussian Mixture, Model Selection, and Priors --- p.21 / Chapter 2.2.1 --- Gaussian Mixture Model and EM algorithm --- p.21 / Chapter 2.2.2 --- Three automatic model selection approaches --- p.22 / Chapter 2.2.3 --- Jeffreys prior and Dirichlet-Normal-Wishart prior --- p.24 / Chapter 2.3 --- Algorithms with Jeffreys Priors --- p.25 / Chapter 2.3.1 --- Bayesian Ying-Yang learning and BYY-Jef algorithms --- p.25 / Chapter 2.3.2 --- Variational Bayesian and VB-Jef algorithms --- p.29 / Chapter 2.3.3 --- Minimum Message Length and MML-Jef algorithms --- p.33 / Chapter 2.4 --- Algorithms with Dirichlet and DNW Priors --- p.35 / Chapter 2.4.1 --- Algorithms BYY-Dir(α), VB-Dir(α) and MML-Dir(α) --- p.35 / Chapter 2.4.2 --- Algorithms with DNW priors --- p.40 / Chapter 2.5 --- Empirical Analysis on Simulated Data --- p.44 / Chapter 2.5.1 --- With priors on mixing weights: a quick look --- p.44 / Chapter 2.5.2 --- With full priors: extensive comparisons --- p.51 / Chapter 2.6 --- Concluding Remarks --- p.55 / Chapter 3 --- Applications of GMM Algorithms --- p.57 / Chapter 3.1 --- Face and Handwritten Digit Images Clustering --- p.58 / Chapter 3.2 --- Unsupervised Image Segmentation --- p.59 / Chapter 3.3 --- Image Foreground Extraction --- p.62 / Chapter 3.4 --- Texture Classification --- p.68 / Chapter 3.5 --- Concluding Remarks --- p.71 / Chapter 4 --- Automatic Model Selection on MFA/LFA --- p.73 / Chapter 4.1 --- Introduction --- p.74 / Chapter 4.2 --- MFA/LFA Models and the Priors --- p.78 / Chapter 4.2.1 --- MFA and LFA models --- p.78 / Chapter 4.2.2 --- The Dirichlet-Normal-Gamma priors --- p.79 / Chapter 4.3 --- Algorithms on MFA/LFA with DNG Priors --- p.82 / Chapter 4.3.1 --- BYY algorithm on MFA with DNG prior --- p.83 / Chapter 4.3.2 --- BYY algorithm on LFA with DNG prior --- p.86 / Chapter 4.3.3 --- VB algorithm on MFA with DNG prior --- p.89 / Chapter 4.3.4 --- VB algorithm on LFA with DNG prior --- p.91 / Chapter 4.4 --- Empirical Analysis on Simulated Data --- p.93 / Chapter 4.4.1 --- On the “chair data: a quick look --- p.94 / Chapter 4.4.2 --- Extensive comparisons on four series of simulations --- p.97 / Chapter 4.5 --- Concluding Remarks --- p.101 / Chapter 5 --- Applications of MFA/LFA Algorithms --- p.102 / Chapter 5.1 --- Face and Handwritten Digit Images Clustering --- p.103 / Chapter 5.2 --- Unsupervised Image Segmentation --- p.105 / Chapter 5.3 --- Radar HRRP based Airplane Recognition --- p.106 / Chapter 5.3.1 --- Background of HRRP radar target recognition --- p.106 / Chapter 5.3.2 --- Data description --- p.109 / Chapter 5.3.3 --- Experimental results --- p.111 / Chapter 5.4 --- Concluding Remarks --- p.113 / Chapter 6 --- Conclusions and FutureWorks --- p.114 / Chapter A --- Referred Parametric Distributions --- p.117 / Chapter B --- Derivations of GMM Algorithms --- p.119 / Chapter B.1 --- The BYY-DNW Algorithm --- p.119 / Chapter B.2 --- The MML-DNW Algorithm --- p.124 / Chapter B.3 --- The VB-DNW Algorithm --- p.127 / Chapter C --- Derivations of MFA/LFA Algorithms --- p.130 / Chapter C.1 --- The BYY Algorithms with DNG Priors --- p.130 / Chapter C.1.1 --- The BYY-DNG-MFA algorithm --- p.130 / Chapter C.1.2 --- The BYY-DNG-LFA algorithm --- p.137 / Chapter C.2 --- The VB Algorithms with DNG Priors --- p.145 / Chapter C.2.1 --- The VB-DNG-MFA algorithm --- p.145 / Chapter C.2.2 --- The VB-DNG-LFA algorithm --- p.149 / Bibliography --- p.152

This investigation is concerned with effective state estimation of a system driven by an unknown nonGaussian input with additive white Gaussian noise, and observed by measurements containing feedthrough of the same nonGaussian input and corrupted by additional white Gaussian noise. A Gaussian sum (GS) approach has previously been developed [6-8] which can cope with the non Gaussian nature of the input signal. Due to a serious growing memory problem in this approach, a modified Gaussian sum (MGS) estimation technique is developed that avoids the growing memory problem while providing effective state estimation. Several differences between the MGS and GS algorithms are examined. An MGS adaptive filter is derived for a general system and a modal system, with simulation examples performed using a non Gaussian input signal. The modal system simulation results are compared to those produced from an augmented Kalman filter based on an augmented modal system model assuming a narrowband Gaussian input signal. A necessary condition for effective MGS estimation is derived. Alternate estimation procedures are developed to compensate for situations when this condition is not met. Several configurations are simulated and their performance results are analyzed and compared. Two methods of monitoring and updating key parameters of the MGS filter are developed. Simulation results are analyzed to investigate the performance of these methods. / Ph. D.

This thesis considers algorithmic and structural aspects of high-dimensional convex sets with respect to the standard Gaussian measure. Among our contributions, (i) we introduce a notion of "influence" for convex sets that yields the first quantitative strengthening of Royen's celebrated Gaussian correlation inequality; (ii) we investigate the approximability of general convex sets by intersections of halfspaces, where the approximation quality is measured with respect to the standard Gaussian distribution; and (iii) we give the first lower bounds for testing convexity and estimating the distance to convexity of an unknown set in the black-box query model. Our results and techniques are inspired by a number of fundamental ingredients and results---such as the influence of variables, noise sensitivity, and various extremal constructions---from the analysis of Boolean functions in complexity theory.

The goal of the thesis is to study geodesics on surfaces of constant Gaussian curvature. The first three sections of the thesis is dedicated to the definitions and theorems necessary to study surfaces of constant Gaussian curvature. The fourth section contains examples of geodesics on these types of surfaces and discusses their properties. The thesis incorporates the use of Maple, a mathematics software package, in some of its calculations and graphs. The thesis' conclusion is that the Gaussian curvature is a surface invariant and the geodesics of these surfaces will be the so-called best paths.

Thesis (Ph.D.)--University of Wollongong, 2004. / Typescript. Includes bibliographical references: leaf 230-241.

Dans cette thèse, nous examinons les propriétés de régularité locale de certains processus stochastiques multiparamètres définis sur RN + , sur une collection d’ensembles, ou encore sur des fonctions de L2. L’objectif est d’étendre certains outils standards de la théorie des processus stochastiques, en particulier concernant la régularité hölderienne locale, à des ensembles d’indexation qui ne sont pas totalement ordonnés. Le critère de continuité de Kolmogorov donne classiquement une borne inférieure pour la régularité hölderienne d’un processus stochastique indicé par un sous-ensemble de R ou RN . Tirant partie de la structure de treillis des ensembles d’indexations dans la théorie des processus indicés par des ensembles de Ivanoff et Merzbach, nous étendons le critère de Kolmogorov dans ce cadre. Différents accroissements pour les processus indicés par des ensembles sont considérés, et leur sont attachés en conséquence des exposants de Hölder. Pour les processus gaussiens, ces exposants sont, presque surement et uniformément le long des trajectoires, déterministes et calculés en fonction de la loi des accroissements du processus. Ces résultats sont appliqués au mouvement brownien fractionnaire set-indexed, pour lequel la régularité est constante. Afin d’exhiber un processus pour lequel la régularité n’est pas constante, nous utilisons la structure d’espace de Wiener abstrait pour introduire un champ brownien fractionnaire indicé par (0, 1=2]_L2(T,m), relié à une famille de covariances kh, h 2 (0, 1=2]. Ce formalisme permet de décrire un grand nombre de processus gaussiens fractionnaires, suivant le choix de l’espacemétrique (T,m). Il est montré que la loi des accroissements d’un tel champ est majorée par une fonction des accroissements en chacun des deux paramètres. Les techniques développées pour mesurer la régularité locale s’appliquent alors pour prouver qu’il existe dans ce cadre des processus gaussiens indicés par des ensembles ou par L2 ayant une régularité prescrite. La dernière partie est consacrée à l’étude des singularités produites par le processus multiparamètre défini par kh sur L2([0, 1]_,dx). Ce processus est une extension naturelle du mouvement brownien fractionnaire et du drap brownien. Au point origine de RN+, ce mouvement brownien fractionnaire multiparamètre possède une régularité hölderienne différente de celle observée en tout autre point qui ne soit pas sur les axes. Une loi du logarithme itéré de Chung permet d’observer finement cette différence. / In this thesis, local regularity properties of some multiparameter, set-indexed and eventually L2-indexed random fields are investigated. The goal is to extend standard tools of the theory of stochastic processes, in particular local Hölder regularity, to indexing collection which are not totally ordered.The classic Kolmogorov continuity criterion gives a lower estimate of the Hölder regularityof a stochastic process indexed by a subset of R or RN . Using the lattice structure of the indexing collections in the theory of set-indexed processes of Ivanoff and Merzbach, Kolmogorov’scriterion is extended to this framework. Different increments for set-indexed processes are considered,and several Hölder exponents are defined accordingly. For Gaussian processes, these exponents are, almost surely and uniformly along the sample paths, deterministic and related to the law of the increments of the process. This is applied to the set-indexed fractional Brownian motion, for which the regularity is constant. In order to exhibit a process having a variable regularity,we resorted to structures of Abstract Wiener Spaces, and defined a fractional Brownian field indexed by a product space (0, 1=2]_L2(T,m), based on a family of positive definite kernels kh, h 2 (0, 1=2]. This field encompasses a large class of existing multiparameter fractional Brownian processes, which are exhibited by choosing appropriate metric spaces (T,m). It is proven that the law of the increments of such a field is bounded above by a function of the increments in both parameters of the field. Applying the techniques developed to measure the local Hölder regularity, it is proven that this field can lead to a set-indexed, or L2-indexed, Gaussian process with prescribed local regularity.The last part is devoted to the study of the singularities induced by the multiparameter process defined by the covariance kh on L2([0, 1]_,dx). This process is a natural extension of the fractional Brownian motion and of the Brownian sheet. At the origin 0 of RN+, this multiparameter fractional Brownian motion has a different regularity behaviour. A Chung (or lim inf ) law of the iterated logarithm permits to observe this.

The focus of this thesis is on the investigation of the geodesics of ruled surfaces.