Graduation date: 1995

Thesis (Ph.D.) -- McMaster University, 1999. / Includes bibliographical references (leaves 76-79). Also available via World Wide Web.

This dissertation addresses probabilistic inference via relative entropy minimization subject to expectation constraints. A canonical representation of the solution is determined without the requirement for convexity of the constraint set, and is given by members of an exponential family. The use of conjugate priors for relative entropy minimization is proposed, and a class of conjugate prior distributions is introduced. An alternative representation of the solution is provided as members of the prior family when the prior distribution is conjugate. It is shown that the solutions can be found by direct optimization with respect to members of such parametric families. Constrained Bayesian inference is recovered as a special case with a specific choice of constraints induced by observed data. The framework is applied to the development of novel probabilistic models for multitask learning subject to constraints determined by domain expertise. First, a model is developed for multitask learning that jointly learns a low rank weight matrix and the prior covariance structure between different tasks. The multitask learning approach is extended to a class of nonparametric statistical models for transposable data, incorporating side information such as graphs that describe inter-row and inter-column similarity. The resulting model combines a matrix-variate Gaussian process prior with inference subject to nuclear norm expectation constraints. In addition, a novel nonparametric model is proposed for multitask bipartite ranking. The proposed model combines a hierarchical matrix-variate Gaussian process prior with inference subject to ordering constraints and nuclear norm constraints, and is applied to disease gene prioritization. In many of these applications, the solution is found to be unique. Experimental results show substantial performance improvements as compared to strong baseline models. / text

published_or_final_version / Statistics / Doctoral / Doctor of Philosophy

Thesis (Ph. D.)--University of Hong Kong, 1990.

This diploma thesis provides an evaluation of Exponential families of distributions which has a special position in mathematical statistics. Diploma will learn the basic concepts and facts associated with the distribution of exponential type. Especially with focusing on the advantages of exponential families in classical parametric statistics, thus in theory of estimation and hypothesis testing. Emphasis will be placed on one-parameter and multi-parameters systems.

Title: Exponential families in statistical inference Author: Sally Abdel-Maksoud Department: Department of Probability and Mathematical Statistics Supervisor: doc. RNDr. Daniel Hlubinka, Ph.D. Supervisor's e-mail address: Daniel.Hlubinka@mff.cuni.cz Abstract: This diploma thesis provides an evaluation of Exponential families of distributions which has a special position in mathematical statistic including appropriate properties for estimation of population parameters, hypothesis testing and other inference problems. Diploma will introduce the basic concepts and facts associated with the distribution of exponential type especially with focusing on the advantages of exponential families in classical parametric statistics, thus in theory of estimation and hypothesis testing. Emphasis will be placed on one-parameter and multi- parameters systems. It also exposes an important concepts about the curvature of a statistical problem including the curvature in exponential families. We will define a quantity that measure how nearly "exponential" the families are. This quantity is said to be the statistical curvature of the family. We will show that the family with a small curvature enjoy the good properties of exponential families Moreover, the properties of the curvature, hypotheses testing and some...

Word embeddings are a powerful approach for capturing semantic similarity among terms in a vocabulary. Exponential family embeddings extend the idea of word embeddings to other types of high-dimensional data. Exponential family embeddings have three ingredients; embeddings as latent variables, a predefined conditioning set for each observation called the context and a conditional likelihood from the exponential family. The embeddings are inferred with a scalable algorithm. This thesis highlights three advantages of the exponential family embeddings model class: (A) The approximations used for existing methods such as word2vec can be understood as a biased stochastic gradients procedure on a specific type of exponential family embedding model --- the Bernoulli embedding. (B) By choosing different likelihoods from the exponential family we can generalize the task of learning distributed representations to different application domains. For example, we can learn embeddings of grocery items from shopping data, embeddings of movies from click data, or embeddings of neurons from recordings of zebrafish brains. On all three applications, we find exponential family embedding models to be more effective than other types of dimensionality reduction. They better reconstruct held-out data and find interesting qualitative structure. (C) Finally, the probabilistic modeling perspective allows us to incorporate structure and domain knowledge in the embedding space. We develop models for studying how language varies over time, differs between related groups of data, and how word usage differs between languages. Key to the success of these methods is that the embeddings share statistical information through hierarchical priors or neural networks. We demonstrate the benefits of this approach in empirical studies of Senate speeches, scientific abstracts, and shopping baskets.

Wong, Tat Wing. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2011. / Includes bibliographical references (p. 53-55). / Abstracts in English and Chinese. / Chapter 1 --- Introduction --- p.1 / Chapter 2 --- Problem Formulation --- p.5 / Chapter 2.1 --- Phase-type distribution --- p.5 / Chapter 2.1.1 --- A generalization of the exponential distribution --- p.5 / Chapter 2.1.2 --- Properties of the phase-type distribution --- p.6 / Chapter 2.2 --- Phase-type jump diffusion model --- p.8 / Chapter 2.2.1 --- Jump diffusion model --- p.8 / Chapter 2.2.2 --- The stock price model --- p.9 / Chapter 2.3 --- Stock Loans --- p.10 / Chapter 3 --- General Properties of Stock Loans --- p.12 / Chapter 3.1 --- Preliminary results --- p.12 / Chapter 3.2 --- Characterization of the function V(x) --- p.15 / Chapter 4 --- Valuation / Chapter 4.1 --- Hyperexponential jumps --- p.25 / Chapter 4.1.1 --- Solution of the linear system --- p.29 / Chapter 4.1.2 --- Solution of the optimal exercise boundary --- p.30 / Chapter 4.2 --- Phase-type jumps --- p.33 / Chapter 4.3 --- The case for G'(1)≥ 0 --- p.36 / Chapter 5 --- Future Research Direction --- p.38 / Chapter 5.1 --- The fast mean-reverting stochastic volatility model --- p.38 / Chapter 5.2 --- Asymptotic expansion of stock loan --- p.39 / Chapter 5.2.1 --- The zeroth order term --- p.41 / Chapter 5.2.2 --- The first order term --- p.43 / Chapter 6 --- Conclusion --- p.52 / Bibliography --- p.53

In many scientific studies, the response variable bears a generalized nonlinear regression relationship with a certain covariate of interest, which may, however, be confounded by other covariates with unknown functional form. We propose a new class of models, the partly parametric generalized additive model (PPGAM) for doing generalized nonlinear regression with the confounding covariate effects adjusted nonparametrically. To avoid the curse of dimensionality, the PPGAM specifies that, conditional on the covariates, the response distribution belongs to the exponential family with the mean linked to an additive predictor comprising a nonlinear parametric function that is of main interest, plus additive, smooth functions of other covariates. The PPGAM extends both the generalized additive model (GAM) and the generalized nonlinear regression model. We propose to estimate a PPGAM by the method of penalized likelihood. We derive some asymptotic properties of the penalized likelihood estimator, including consistency and asymptotic normality of the parametric estimator of the nonlinear regression component. We propose a model selection criterion for the PPGAM, which resembles the BIC. We illustrate the new methodologies by simulations and real applications. We have developed an R package PPGAM that implements the methodologies expounded herein.