Back to search

University of York

Deep Complex-Valued Neural Networks for Natural Language Processing

Abstract

dc:description.abstract

This thesis presents novel work on complex-valued neural networks applied to Natural Language Processing. We experimentally show the validity of complex-valued neural networks for semantic and phonetic processing of natural languages. We highlight important issues that complex networks have in comparison to their real-valued counter parts. In particular this work considers the tasks of Language Modelling, Semantic Similarity Judgement, Basic Question Answering, Phonetic Transcription and Automatic Speech Recognition. Our contributions are the translation of neural network building blocks to the complex plane and their experimental application in a variety of natural language tasks. We present criteria to compare real-valued and complex-valued neural networks for classification tasks. We present various complex embedding methods for words. These produce position and frequency-based word representations trainable using language models and usable in down-stream tasks. We also compare a real-valued and complex-valued memory network used for Question Answering. We derive a quantum-inspired framework for languages. Additionally, we demonstrate quantum-inspired Semantic Spaces. A general framework of complex-valued attention is presented in this thesis. It is used to derive spectral self-attention with a novel activation function. We also introduce two pooling functions to reduce dimensionality of frequency-based representations. A Spectral Transformer architecture facilitates the spectral self-attention for Speech Recognition. This work also includes a novel dataset for transcription of children's utterances consisting of seven sub tasks each with fixed data splits and baselines for better comparison and reproducibility. Throughout this thesis we find that complex-valued neural networks are suitable for natural language tasks, but require additional care in their design and training.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of York
Year dc:date.issued
2019

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Mönning, Nils
Advisor dc:contributor.advisor
  • Manandhar, Suresh

Rights

Language dc:language.dcterms_iso639_2
eng

Identifiers

dc:identifier.*
Identifier
uk.bl.ethos.789559

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Mönning, Nils. Deep Complex-Valued Neural Networks for Natural Language Processing. doctoral thesis, University of York, 2019.