Aristotle University Of Thessaloniki (AUTH)
Optical neural networks for deep learning applications
Abstract
dc:descriptionDeep Learning (DL) is thriving today due to an unprecedented demand for processing large volumes of data originating from intelligent applications such as image classification, language processing, self-driving vehicles, etc. At the same time, the end of Moore’s law is already a reality that imposes physical limitations for developing energy-efficient and high-performance computing systems targeting the efficient deployment of modern DL applications. In this context, Photonic Neural Network Accelerators (PNNAs) seem to be a promising hardware candidate for the post-Moore era high-performance computing systems, bringing the credentials of extremely high energy efficiency and computational density of optics into the DL computing sector. The main research contributions of this thesis revolved around the development of basic building blocks for developing PNNAs, PNNA architectures, and photonic hardware-aware training techniques for maximizing the efficiency of PNNA-based DL computing systems.As a first step, a new portfolio of photonic building blocks was developed including a Sigmoid activation function compatible with the standard Neural Network (NN) models, a coherent linear neuron that exploits the phase of light to imprint efficiently the sign of the weights and optical Content-Addressable Memories (CAMs) to support the efficient temporary storage of DL workload parameters close to the PNNA. The developed PNNA building blocks have been validated experimentally using 10Gbaud data streams. Furthermore, the coherent linear neuron has been combined with the Sigmoid activation function forming a fully functional neuron that has been evaluated through physical layer simulations on image classification.Moving towards implementing complete PNNA architectures, various Recurrent NNs (RNNs) layouts and systems have been designed and demonstrated experimentally. In particular, a Wavelength-Division Multiplexed (WDM) all-optical RNN was designed and implemented to address the challenges of accelerating recurrent DL applications, where the feedback of each neuron is necessary to ensure the memory capabilities of the PNNA. In this context, a 4-input WDM layout has been demonstrated at 10Gbaud, validating the proper functionality of the all-optical recurrent input. Furthermore, the memory properties of the recurrent layout have been extended through an additional optical gate so as to get selective memory capabilities. This new functionality has been validated experimentally with 10Gbaud data sequences, while the software implementation of a photonic RNN with and without selective memory has been trained and tested successfully on the price prediction of stocks. Afterwards, an all-optical space rotation scheme followed by a photonic RNN layout was developed to eliminate negative weights after properly transforming the input signal, enabling all-optical WDM weighting. The proposed layout has been demonstrated experimentally by classifying time-series at 10Gbaud, achieving the fastest all-optical time-series classification.The next step was to transfer the developed PNNA building blocks on a silicon photonic platform to assess their performance on a mass-manufacturable and Complementary Metal-Oxide Semiconductor (CMOS)-compatible platform such as silicon photonics. In this context, a 4-input Coherent Photonic Neural Network (CPNN) has been designed and fabricated on a silicon photonic platform employing electro-optic and thermo-optic components to realize the input vector and the weights, respectively. The fabricated chip was tested on an image classification task at 5 and 10G Multiply-And-accumulate (MAC)/sec/axon rates, revealing ~99% classification accuracy and 6× higher on-chip compute rates compared to state-of-the-art coherent implementations. In doing so, the design and implementation of interfaces between the analogue photonic chip and the digital world and the appropriate evaluation methodology were also developed, realizing the data resampling, time synchronization, filtering, equalization etc. so as to preserve the high signal quality after the photonic MAC operations.Finally, hardware-aware PNNA training techniques have been developed and validated experimentally in order to unleash the full potential of PNNAs. In this context, the resiliency of PNNAs in non-deterministic noise sources has been enhanced by developing the appropriate noise-aware training method. The experimental demonstration of the noise-aware trained model achieved higher classification accuracy by 2.54%, which can be translated as tolerance to 1.8dB higher insertion losses. Then the effect of deterministic noises in the system has been mitigated by introducing a channel response-aware training model that considers the underlying hardware's frequency response of PNNAs. The experimental validation of this pioneering method revealed 92.1% classification accuracy on hand-written digit images at 32GMAC/sec/axon using photonic components with 7GHz 3dB bandwidth only.
Degree
thesis:*- Grantor dc:publisher
- Aristotle University Of Thessaloniki (AUTH)
- Year dc:date
- 2022
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Mourgias-Alexandris, Georgios
- Μουργιάς-Αλεξανδρής, Γεώργιος
Subjects
dc:subject × 10Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/52687