Back to search

University of Sheffield

Learning with structured covariance matrices in linear Gaussian models

Abstract

dc:description.abstract

We study structured covariance matrices in a Gaussian setting for a variety of data analysis scenarios. Despite its simplistic nature, we argue for the broad applicability of the Gaussian family through its second order statistics. We focus on three types of common structures in the machine learning literature: covariance functions, low-rank and sparse inverse covariances. Our contributions boil down to combin- ing these structures and designing algorithms for maximum-likelihood or MAP fitting: for instance, we use covariance functions in Gaus- sian processes to encode the temporal structure in a gene-expression time-series, with any residual structure generating iid noise. More generally, for a low-rank residual structure (correlated residuals) we introduce the residual component analysis framework: based on a generalised eigenvalue problem, it decomposes the residual low-rank term given a partial explanation of the covariance. In this example the explained covariance would be an RBF kernel, but it can be any positive-definite matrix. Another example is the low-rank plus sparse- inverse composition for structure learning of GMRFs in the presence of confounding latent variables. We also study RCA as a novel link between classical low-rank methods and modern probabilistic counter- parts: the geometry of oblique projections shows how PCA, CCA and linear discriminant analysis reduce to RCA. Also inter-battery factor analysis, a precursor of multi-view learning, is reduced to an itera- tive application of RCA. Finally, we touch on structured precisions of matrix-normal models based on the Cartesian factorisation of graphs, with appealing properties for regression problems and interpretabil- ity. In all cases, experimental results and simulations demonstrate the performance of the different methods proposed.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Sheffield
Year dc:date.issued
2013

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Kalaitzis, Alfredo
Advisor dc:contributor.advisor
  • Lawrence, Neil D.

Rights

Language dc:language.dcterms_iso639_2
eng

Identifiers

dc:identifier.*
Identifier
uk.bl.ethos.574078

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Kalaitzis, Alfredo. Learning with structured covariance matrices in linear Gaussian models. doctoral thesis, University of Sheffield, 2013.