Knowledge of the channel over which signals are sent is of prime importance in modern wireless communications. Inaccurate or incomplete channel information leads to high error rates and wasted bandwidth and energy. Although active channel measurement is commonly used to gain channel knowledge, it can only accurately represent the channel at the time the measurement was taken, makes energy and bandwidth demands, and adds significant complexity to the radio system. Due to the highly time variant nature of wireless channels, active measurements become invalid almost as soon as they are taken, making alternative approaches to predicting future behaviour highly attractive. Such systems would allow maximum advantage to be taken of the limited bandwidth available and make significant power savings. This thesis investigates a number of complementary technologies, leading towards a channel prediction scheme suitable for mobile devices. As a first step towards channel prediction, anomaly detection is investigated within periodic wireless signals to establish when radical changes in the channel occur. In pre- vious experiments, long monotonic sequences had been observed to coincide with certain anomalies but not others when using Kullback-Leibler Divergence (KLD) analysis, possibly allowing the characterisation of anomaly types. An investigation is described to explain the origin of these features in a rigorous mathematical sense. A proof is given for the causes of the monotonic sequences, followed by a discussion of the types of signal anomaly which would underly such a feature and the value of this information. The second part describes a novel channel characterisation method which uses a class of Recurrent Neural Network (RNN) called an Echo State Network (ESN). Using this tool, a channel characterisation system can be constructed without an explicit statistical or mathematical model of the wireless environment, relying instead on observed data. This approach is much more convenient than existing models which require detailed information about the wireless system's parameters and also allows for new channel classifications to be added easily. It is able to achieve double the correct classification rate of a conventional statistical classifier, and is computationally simple to implement, making it ideal for inclusion on low-power mobile devices. Following their successful use in characterisation, ESNs are used in the final part in an investigation into channel prediction in a number of different scenarios. They were however found to be unable to produce useful predictions for all but the most trivial channel models. An alternative method is described for indoor environments using an approach inspired by ray tracing. It is simple and computationally lightweight to implement, again making it suitable for mobile devices. Simulation results show that it can outperform pilot-assisted methods by a significant margin, while not wasting bandwidth on channel measurement.

This dissertation considers a deep reinforcement learning (DRL) setting under the practical challenges of real-world wireless communication systems. The non-stationary and partially observable wireless environments make the learning and the convergence of the DRL agent challenging. One way to facilitate learning in partially observable environments is to combine recurrent neural network (RNN) and DRL to capture temporal information inherent in the system, which is referred to as deep recurrent Q-network (DRQN). However, training DRQN is known to be challenging requiring a large amount of training data to achieve convergence. In many targeted wireless applications in the 5G and future 6G wireless networks, the available training data is very limited. Therefore, it is important to develop DRL strategies that are capable of capturing the temporal correlation of the dynamic environment that only requires limited training overhead. In this dissertation, we design efficient DRL frameworks by utilizing echo state network (ESN), which is a special type of RNNs where only the output weights are trained. To be specific, we first introduce the deep echo state Q-network (DEQN) by adopting ESN as the kernel of deep Q-networks. Next, we introduce federated ESN-based policy gradient (Fed-EPG) approach that enables multiple agents collaboratively learn a shared policy to achieve the system goal. We designed computationally efficient training algorithms by utilizing the special structure of ESNs, which have the advantage of learning a good policy in a short time with few training data. Theoretical analyses are conducted for DEQN and Fed-EPG approaches to show the convergence properties and to provide a guide to hyperparameter tuning. Furthermore, we evaluate the performance under the dynamic spectrum sharing (DSS) scenario, which is a key enabling technology that aims to utilize the precious spectrum resources more efficiently. Compared to a conventional spectrum management policy that usually grants a fixed spectrum band to a single system for exclusive access, DSS allows the secondary system to dynamically share the spectrum with the primary system. Our work sheds light on the real deployments of DRL techniques in next generation wireless systems. / Doctor of Philosophy / Model-free reinforcement learning (RL) algorithms such as Q-learning are widely used because it can learn the policy directly through interactions with the environment without estimating a model of the environment, which is useful when the underlying system model is complex. Q-learning performs poorly for large-scale models because the training has to updates every element in a large Q-table, which makes training difficult or even impossible. Therefore, deep reinforcement learning (DRL) exploits the powerful deep neural network to approximate the Q-table. Furthermore, a deep recurrent Q-network (DRQN) is introduced to facilitate learning in partially observable environments. However, DRQN training requires a large amount of training data and a long training time to achieve convergence, which is impractical in wireless systems with non-stationary environments and limited training data. Therefore, in this dissertation, we introduce two efficient DRL approaches: deep echo state Q-network (DEQN) and federated ESN-based policy gradient (Fed-EPG) approaches. Theoretical analyses of DEQN and Fed-EPG are conducted to provide the convergence properties and the guideline for designing hyperparameters. We evaluate and demonstrate the performance benefits of the DEQN and Fed-EPG under the dynamic spectrum sharing (DSS) scenario, which is a critical technology to efficiently utilize the precious spectrum resources in 5G and future 6G wireless networks.

Echo state network (ESN) is a specific neural network structure composed of high dimensional nonlinear dynamics and learned readout weights. This thesis considers applying ESN for symbol detection in multiple-input, multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) systems. A new ESN structure, namely, windowed echo state networks (WESN) is introduced to further improve the symbol detection performance. Theoretical analysis justifies WESN has an enhanced short-term memory (STM) compared with the standard ESN such that WESN can offer better computing ability. Additionally, the bandwidth spent as the training set is the same as the demodulation reference signals defined in 3GPP LTE/LTE-Advanced systems for the ESN/WESN based symbol detection. Meanwhile, a unified training framework is developed for both comb and scattered pilot patterns. Complexity analysis demonstrates the advantages of ESN/WESN based symbol detector compared to conventional symbol detectors such as linear minimum mean square error (LMMSE) and sphere decoder when the system is employed with a large number of OFDM sub-carriers. Numerical evaluations show that ESN/WESN has an improvement of symbol detection performance as opposed to conventional methods in both low SNR regime and power amplifier (PA) nonlinear regime. Finally, it demonstrates that WESN can generate a better symbol detection result over ESN. / Artificial neural networks (ANN) are widely used in recognition tasks such as recommendation systems, robotics path planning, self-driving, video tracking, image classifications, etc. To further explore the applications of ANN, this thesis considers using a specific ANN, echo state network (ESN) for a wireless communications task: MIMO-OFDM symbol detection. Furthermore, it proposed an enhanced version of the standard ESN, namely, windowed echo state network (WESN). Theoretical analyses on the short term memory (STM) of ESN and WESN show that the later one has a longer STM. Besides, the training set size of this ESN/WESN based method is chosen the same as the pilot symbols used in conventional communications systems. The algorithm complexity analysis demonstrates the ESN/WESN based method performs with lower complexity compared with conventional methods, such as linear mean square error (LMMSE) and sphere decoding. Comprehensive simulations examine how the symbol detection performance can be improved by using ESN and its variant WESN when the transmission link is non-ideal.

The reservoir computer comprises a reservoir of possibly non-linear, possibly chaotic dynamics. By perturbing and taking outputs from this reservoir, its dynamics may be harnessed to compute complex problems at “the edge of chaos”. One of the first forms of reservoir computer, the Echo State Network (ESN), is a form of artificial neural network that builds its reservoir from a large and sparsely connected recurrent neural network (RNN). The ESN was initially introduced as an innovative solution to train RNNs which, up until that point, was a notoriously difficult task. The innovation of the ESN is that, rather than train the RNN weights, only the output is trained. If this output is assumed to be linear, then linear regression may be used. This work presents an effort to implement the Echo State Network, and an offline linear regression training method based on Tikhonov regularisation. This implementation targeted the general purpose graphics processing unit (GPU or GPGPU). The behaviour of the implementation was examined by comparing it with a central processing unit (CPU) implementation, and by assessing its performance against several studied learning problems. These assessments were performed using all 4 cores of the Intel i7-980 CPU and an Nvidia GTX480. When compared with a CPU implementation, the GPU ESN implementation demonstrated a speed-up starting from a reservoir size of between 512 and 1,024. A maximum speed-up of approximately 6 was observed at the largest reservoir size tested (2,048). The Tikhonov regularisation (TR) implementation was also compared with a CPU implementation. Unlike the ESN execution, the GPU TR implementation was largely slower than the CPU implementation. Speed-ups were observed at the largest reservoir and state history sizes, the largest of which was 2.6813. The learning behaviour of the GPU ESN was tested on three problems, a sinusoid, a Mackey-Glass time-series, and a multiple superimposed oscillator (MSO). The normalised root-mean squared errors of the predictors were compared. The best observed sinusoid predictor outperformed the best MSO predictor by 4 orders of magnitude. In turn, the best observed MSO predictor outperformed the best Mackey-Glass predictor by 2 orders of magnitude.

Le Reservoir Computing (RC) est un paradigme s’inspirant du cerveau humain, apparu récemment au début des années2000. Il s'agit d'un calculateur neuromorphique habituellement décomposé en trois parties dont la plus importanteappelée "réservoir" est très proche d'un réseau de neurones récurrent. Il se démarque des autres réseaux de neuronesartificiels notamment grâce aux traditionnelles phases d'apprentissage et d’entraînement qui ne sont plus appliquées surla totalité du réseau de neurones mais uniquement sur la lecture du réservoir, ce qui simplifie le fonctionnement etfacilite une réalisation physique. C'est précisément dans ce contexte qu’ont été réalisés les travaux de recherche de cettethèse, durant laquelle nous avons réalisé une première implémentation physique opto-électronique de système RC.Notre approche des systèmes physiques RC repose sur l'utilisation de dynamiques non-linéaires à retards multiples dansl'objectif de reproduire le comportement complexe d'un réservoir. L'utilisation d'un système dynamique purementtemporel pour reproduire la dimension spatio-temporelle d'un réseau de neurones traditionnel, nécessite une mise enforme particulière des signaux d'entrée et de sortie, appelée multiplexage temporel ou encore étape de masquage. Troisannées auront été nécessaires pour étudier et construire expérimentalement nos démonstrateurs physiques basés sur desdynamiques non-linéaires à retards multiples opto-électroniques, en longueur d'onde et en intensité. La validationexpérimentale de nos systèmes RC a été réalisée en utilisant deux tests de calcul standards. Le test NARMA10 (test deprédiction de séries temporelles) et la reconnaissance vocale de chiffres prononcés (test de classification de données) ontpermis de quantifier la puissance de calcul de nos systèmes RC et d'atteindre pour certaines configurations l'état del'art. / Reservoir Computing (RC) is a currently emerging new brain-inspired computational paradigm, which appeared in theearly 2000s. It is similar to conventional recurrent neural network (RNN) computing concepts, exhibiting essentiallythree parts: (i) an input layer to inject the information in the computing system; (ii) a central computational layercalled the Reservoir; (iii) and an output layer which is extracting the computed result though a so-called Read-Outprocedure, the latter being determined after a learning and training step. The main originality compared to RNNconsists in the last part, which is the only one concerned by the training step, the input layer and the Reservoir beingoriginally randomly determined and fixed. This specificity brings attractive features to RC compared to RNN, in termsof simplification, efficiency, rapidity, and feasibility of the learning, as well as in terms of dedicated hardwareimplementation of the RC scheme. This thesis is indeed concerned by one of the first a hardware implementation of RC,moreover with an optoelectronic architecture.Our approach to physical RC implementation is based on the use of a sepcial class of complex system for the Reservoir,a nonlinear delay dynamics involving multiple delayed feedback paths. The Reservoir appears thus as a spatio-temporalemulation of a purely temporal dynamics, the delay dynamics. Specific design of the input and output layer are shownto be possible, e.g. through time division multiplexing techniques, and amplitude modulation for the realization of aninput mask to address the virtual nodes in the delay dynamics. Two optoelectronic setups are explored, one involving awavelength nonlinear dynamics with a tunable laser, and another one involving an intensity nonlinear dynamics with anintegrated optics Mach-Zehnder modulator. Experimental validation of the computational efficiency is performedthrough two standard benchmark tasks: the NARMA10 test (prediction task), and a spoken digit recognition test(classification task), the latter showing results very close to state of the art performances, even compared with purenumerical simulation approaches.

Satellite communication system designers are continuously struggling to improve the channel capacity. A critical challenge results from the limited power available aboard the satellite.Because of this constraint, the onboard power amplifier must work with a small power supply which limits its maximum output power. To ensure a sufficient Signal-to-Noise power Ratio (SNR) on the receiver side, the power amplifier must work close to its saturation point. This is power efficient but unfortunately adds non-linear distortions to the communication channel. The latters are very penalizing for high order modulations.In the literature, several equalization algorithms have been proposed to cope with the resulting non-linear communication channel. The most popular solution consists in using baseband Volterra series in order to build non-linear equalization filters. On the other hand, the Recurrent Neural Networks (RNNs), which come from the artificial neural network field, are also interesting candidates to generate such non-linear filters. But they are difficult to implement in practice due to the high complexity of their training. To simplify this task, the Echo State Network (ESN) paradigm has been proposed. It has the advantage of offering performances similar to classical RNNs but with a reduced complexity.The purpose of this work is, first, to compare this solution to the state-of-the-art baseband Volterra filters. We show that the classical ESN is able to reach the same performances, evaluated in terms of Bit Error Rate (BER), and has similar complexity. Secondly, we propose a new design for the ESN which achieves a strong reduction in complexity while conserving a similar BER.To compensate for the channel, the literature proposes to adapt the coefficients of these equalizers with the help of a training sequence in order to recover the transmitted constellation points. We show that, in such a case, the usual symbol detection criterion, based on Euclidean distances, is no longer optimal. For this reason, we first propose a new detection criterion which meets the Maximum Likelihood (ML) criterion. Secondly, we propose a modification of the equalizers training reference points in order to improve their performances and make the detection based on Euclidean distances optimal again. This last solution can offer a significant reduction of the BER without increasing the equalization and detection complexity. Only the new training reference points must be evaluated.In this work, we also explore the field of analog equalizers as different papers showed that the ESN is an interesting candidate for this purpose. It is a promising approach to reduce the equalizer complexity as the digital implementation is very challenging and power-hungry, in particular for high bandwidth communications. We numerically demonstrate that a dedicated analog optoelectronic implementation of the ESN can reach the state-of-the-art performance of digital equalizers. In addition, we show that it can reduce the required resolution of the Analog-to-Digital Converters (ADCs).Finally, a hardware demonstration of the digital solutions is proposed. For this purpose, we build a physical layer test bench which depicts a non-linear communication between two radios. We show that if we drive the transmitter power amplifier close to its saturation point, we can improve the communication range if the non-linear distortions are compensated for at the receiver. The transmitter and the receiver are implemented with Software Defined Radios (SDRs). / Doctorat en Sciences de l'ingénieur et technologie / info:eu-repo/semantics/nonPublished

Echo State Network (ESN) is a recently developed machine-learning paradigm whose processing capabilities rely on the dynamical behavior of recurrent neural networks (RNNs). Its performance metrics outperform traditional RNNs in nonlinear system identification and temporal information processing. In this thesis, we design and implement ESNs through Field-programmable gate array (FPGA) and explore their full capacity of digital signal processors (DSPs) to target low-cost and low-power applications. We propose a cost-optimized and scalable ESN architecture on FPGA, which exploits Xilinx DSP48E1 units to cut down the need of configurable logic blocks (CLBs). The proposed work includes a linear combination processor with negligible deployment of CLBs, as well as a high-accuracy non-linear function approximator, both with the help of only 9 DSP units in each neuron. The architecture is verified with the classical NARMA dataset, and a symbol detection task for an orthogonal frequency division multiplexing (OFDM) system on a wireless communication testbed. In the worst-case scenario, our proposed architecture delivers a matching bit error rate (BER) compares to its corresponding software ESN implementation. The performance difference between the hardware and software approach is less than 6.5%. The testbed system is built on a software-defined radio (SDR) platform, showing that our work is capable of processing the real-world data. / Master of Science / Machine learning is a study of computer algorithms that evolves itself by learning through experiences. Currently, machine learning thrives as it opens up promising opportunities of solving the problems that is difficult to deal with conventional methods. Echo state network (ESN), a recently developed machine-learning paradigm, has shown extraordinary effectiveness on a wide variety of applications, especially in nonlinear system identification and temporal information processing. Despite the fact, ESN is still computationally expensive on battery-driven and cost-sensitive devices. A fast and power-saving computer for ESN is desperately needed. In this thesis, we design and implement an ESN computational architecture through the field-programmablegate array (FPGA). FPGA allows designers to build highly flexible customized hardware with rapid development time. Our design further explores the full capacity of digital signal processors (DSP) on Xilinx FPGA to target low-cost and low-power applications. The proposed cost-optimized and scalable ESN architecture exploits Xilinx DSP48E1 units to cut down the need of configurable logic blocks (CLBs). The work includes a linear combination processor with negligible deployment of CLBs, and a high-accuracy non-linear function approximator, both with the help of only 9 DSP units in each neuron. The architecture is verified with the classical NARMA dataset, and a symbol detection task for an orthogonal frequency division multiplexing (OFDM) system in a wireless communication testbed. In the worst-case scenario, our proposed architecture delivers a matching bit error rate (BER) compares to its corresponding software ESN implementation. The performance difference between the hardware and software approach is less than 6.5%. The testbed system is built on a software-defined radio (SDR) platform, showing that our work is capable of processing the real-world data.

No description available.

This thesis deals with an echo state network and with acceleration of its learning by implementing the echo state network on a graphics processor. The theoretical part consists of the description of neural networks and some selected types of neural networks, on which is based the echo state network. After that, there are some other algorithms described used for time series analysis and last but not least, the tools that were used in the practical part of the thesis were briefly described. The practical part describes the creation of the accelerated version of the echo state network. After that, there is described the creation of input data sets of real financial indexes, on which the echo state network and the other algorithmns were then tested. By analyzing this accelerated version it was found that its learning speed did not reach the theoretical expectations. The accelerated version works slower, but with greater precision. By analyzing the results of the measurement of the other algorithmns it was found that the highest precision is achieved by solutions based on the neural network principle.

In industrial environments, operators are exposed to polluted air which after constant exposure can cause irreversible lethal diseases such as lung cancer. The current air monitoring techniques are carried out sparely in either a single day annually or at few measurement positions for a few days.In this thesis a theory-guided data science (TGDS) model is presented. This hybrid model combines a steady state Computational Fluid Dynamics (CFD) model with a machine learning model. Both the CFD model and the machine learning algorithm was developed in Matlab. The CFD model serves as a basis for the airflow whereas the machine learning model addresses dynamical features in the foundry. Measurements have previously been made at a foundry where five stationary sensors and one mobile robot were used for data acquisition. An Echo State Network was used as a supervised learning technique for airflow predictions at each robot measurement position and Gaussian Processes (GP) were used as a regression technique to form an Echo State Map (ESM). The stationary sensor data were used as input for the echo state network and the difference between the CFD and robot measurements were used as teacher signal which formed a dynamic correction map that was added to the steady state CFD. The proposed model utilizes the high spatio-temporal resolution of the echo state map whilst making use of the physical consistency of the CFD. The initial applications of the novel hybrid model proves that the best qualities of these two models could come together in symbiosis to give enhanced characterizations.The proposed model could have an important role for future characterization of airflow and more research on this and similar topics are encouraged to make sure we properly understand the potential of this novel model. / Industriarbetare utsätts för skadliga luftburna ämnen vilket över tid leder till högre prevalens för lungsjukdomar så som kronisk obstruktiv lungsjukdom, stendammslunga och lungcancer. De nuvarande luftmätningsmetoderna genomförs årligen under korta sessioner och ofta vid få selekterade platser i industrilokalen. I denna masteruppsats presenteras en teorivägledd datavetenskapsmodell (TGDS) som kombinerar en stationär beräkningsströmningsdynamik (CFD) modell med en dynamisk maskininlärningsmodell. Både CFD-modellen och maskininlärningsalgoritmen utvecklades i Matlab. Echo State Network (ESN) användes för att träna maskininlärningsmodellen och Gaussiska Processer (GP) används som regressionsteknik för att kartlägga luftflödet över hela industrilokalen. Att kombinera ESN med GP för att uppskatta luftflöden i stålverk genomfördes första gången 2016 och denna modell benämns Echo State Map (ESM). Nätverket använder data från fem stationära sensorer och tränades på differensen mellan CFD-modellen och mätningar genomfördes med en mobil robot på olika platser i industriområdet. Maskininlärningsmodellen modellerar således de dynamiska effekterna i industrilokalen som den stationära CFD-modellen inte tar hänsyn till. Den presenterade modellen uppvisar lika hög temporal och rumslig upplösning som echo state map medan den också återger fysikalisk konsistens som CFD-modellen. De initiala applikationerna för denna model påvisar att de främsta egenskaperna hos echo state map och CFD används i symbios för att ge förbättrad karakteriseringsförmåga. Den presenterade modellen kan spela en viktig roll för framtida karakterisering av luftflöden i industrilokaler och fler studier är nödvändiga innan full förståelse av denna model uppnås.