Back to search

University of Southampton

Learning from protein data with protein language models

Abstract

dc:description.abstract

Proteins are complex macromolecules responsible for the vast majority of biological processes. Advancements in genome sequencing have resulted in an exponential increase in sequence data, while other efforts to understand proteins have led to significant progress in determining their structures and interactions. In this thesis, we explore the capability of protein language models (PLMs) to decode and learn from the complexities of protein data with a focus on predicting protein interactions and learning general protein embeddings. We begin by using functional annotations from the gene ontology to learn functional embeddings for proteins and predict their interactions. This approach improves upon classic semantic similarity functions and provides a degree of interpretability regarding the importance of specific gene ontology terms by analyzing the attention heads of the transformer architecture. Our model demonstrated high accuracy in predicting protein-protein interactions across multiple datasets, outperforming existing methods. Next, we shift our focus to learning protein embeddings by investigating sequence conservation at the molecular surface and interaction interfaces to understand the evolutionary aspects of proteins. We develop a multitask pre-training strategy that learns general protein embeddings from sequences, structures, and interaction interfaces, thus alleviating some computational requirements for large PLMs. This strategy leads to improved performance in various downstream tasks. Additionally, we introduce a methodology for generating protein sequence augmentations using evolutionary information which further improves the model’s generalization capabilities. Finally, we explore compressing pre-training sequence datasets using reduced amino acid alphabets and their utility in unsupervised learning. Our results indicate that while reduced alphabets can efficiently capture meaningful embeddings, they may not always outperform models using the full amino acid alphabet. We then devise a simple method to improve single-sequence structure prediction models for proteins with low-depth multiple-sequence alignments by translating protein sequences using reduced amino acid alphabets.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D.
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Southampton
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ieremie, Ioan
Advisors dc:contributor.advisor
  • Niranjan, Mahesan
  • Ewing, Rob
ORCID person:identifier.orcid

Rights

Language dc:language
English, eng

Chain of custody

source
Harvested from
University of Southampton
Base URL
eprints.soton.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Ieremie, Ioan. Learning from protein data with protein language models. doctoral thesis, University of Southampton, 2024.