Knowledge of the channel over which signals are sent is of prime importance in modern wireless communications. Inaccurate or incomplete channel information leads to high error rates and wasted bandwidth and energy. Although active channel measurement is commonly used to gain channel knowledge, it can only accurately represent the channel at the time the measurement was taken, makes energy and bandwidth demands, and adds significant complexity to the radio system. Due to the highly time variant nature of wireless channels, active measurements become invalid almost as soon as they are taken, making alternative approaches to predicting future behaviour highly attractive. Such systems would allow maximum advantage to be taken of the limited bandwidth available and make significant power savings. This thesis investigates a number of complementary technologies, leading towards a channel prediction scheme suitable for mobile devices. As a first step towards channel prediction, anomaly detection is investigated within periodic wireless signals to establish when radical changes in the channel occur. In pre- vious experiments, long monotonic sequences had been observed to coincide with certain anomalies but not others when using Kullback-Leibler Divergence (KLD) analysis, possibly allowing the characterisation of anomaly types. An investigation is described to explain the origin of these features in a rigorous mathematical sense. A proof is given for the causes of the monotonic sequences, followed by a discussion of the types of signal anomaly which would underly such a feature and the value of this information. The second part describes a novel channel characterisation method which uses a class of Recurrent Neural Network (RNN) called an Echo State Network (ESN). Using this tool, a channel characterisation system can be constructed without an explicit statistical or mathematical model of the wireless environment, relying instead on observed data. This approach is much more convenient than existing models which require detailed information about the wireless system's parameters and also allows for new channel classifications to be added easily. It is able to achieve double the correct classification rate of a conventional statistical classifier, and is computationally simple to implement, making it ideal for inclusion on low-power mobile devices. Following their successful use in characterisation, ESNs are used in the final part in an investigation into channel prediction in a number of different scenarios. They were however found to be unable to produce useful predictions for all but the most trivial channel models. An alternative method is described for indoor environments using an approach inspired by ray tracing. It is simple and computationally lightweight to implement, again making it suitable for mobile devices. Simulation results show that it can outperform pilot-assisted methods by a significant margin, while not wasting bandwidth on channel measurement.

This dissertation considers a deep reinforcement learning (DRL) setting under the practical challenges of real-world wireless communication systems. The non-stationary and partially observable wireless environments make the learning and the convergence of the DRL agent challenging. One way to facilitate learning in partially observable environments is to combine recurrent neural network (RNN) and DRL to capture temporal information inherent in the system, which is referred to as deep recurrent Q-network (DRQN). However, training DRQN is known to be challenging requiring a large amount of training data to achieve convergence. In many targeted wireless applications in the 5G and future 6G wireless networks, the available training data is very limited. Therefore, it is important to develop DRL strategies that are capable of capturing the temporal correlation of the dynamic environment that only requires limited training overhead. In this dissertation, we design efficient DRL frameworks by utilizing echo state network (ESN), which is a special type of RNNs where only the output weights are trained. To be specific, we first introduce the deep echo state Q-network (DEQN) by adopting ESN as the kernel of deep Q-networks. Next, we introduce federated ESN-based policy gradient (Fed-EPG) approach that enables multiple agents collaboratively learn a shared policy to achieve the system goal. We designed computationally efficient training algorithms by utilizing the special structure of ESNs, which have the advantage of learning a good policy in a short time with few training data. Theoretical analyses are conducted for DEQN and Fed-EPG approaches to show the convergence properties and to provide a guide to hyperparameter tuning. Furthermore, we evaluate the performance under the dynamic spectrum sharing (DSS) scenario, which is a key enabling technology that aims to utilize the precious spectrum resources more efficiently. Compared to a conventional spectrum management policy that usually grants a fixed spectrum band to a single system for exclusive access, DSS allows the secondary system to dynamically share the spectrum with the primary system. Our work sheds light on the real deployments of DRL techniques in next generation wireless systems. / Doctor of Philosophy / Model-free reinforcement learning (RL) algorithms such as Q-learning are widely used because it can learn the policy directly through interactions with the environment without estimating a model of the environment, which is useful when the underlying system model is complex. Q-learning performs poorly for large-scale models because the training has to updates every element in a large Q-table, which makes training difficult or even impossible. Therefore, deep reinforcement learning (DRL) exploits the powerful deep neural network to approximate the Q-table. Furthermore, a deep recurrent Q-network (DRQN) is introduced to facilitate learning in partially observable environments. However, DRQN training requires a large amount of training data and a long training time to achieve convergence, which is impractical in wireless systems with non-stationary environments and limited training data. Therefore, in this dissertation, we introduce two efficient DRL approaches: deep echo state Q-network (DEQN) and federated ESN-based policy gradient (Fed-EPG) approaches. Theoretical analyses of DEQN and Fed-EPG are conducted to provide the convergence properties and the guideline for designing hyperparameters. We evaluate and demonstrate the performance benefits of the DEQN and Fed-EPG under the dynamic spectrum sharing (DSS) scenario, which is a critical technology to efficiently utilize the precious spectrum resources in 5G and future 6G wireless networks.

Echo state network (ESN) is a specific neural network structure composed of high dimensional nonlinear dynamics and learned readout weights. This thesis considers applying ESN for symbol detection in multiple-input, multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) systems. A new ESN structure, namely, windowed echo state networks (WESN) is introduced to further improve the symbol detection performance. Theoretical analysis justifies WESN has an enhanced short-term memory (STM) compared with the standard ESN such that WESN can offer better computing ability. Additionally, the bandwidth spent as the training set is the same as the demodulation reference signals defined in 3GPP LTE/LTE-Advanced systems for the ESN/WESN based symbol detection. Meanwhile, a unified training framework is developed for both comb and scattered pilot patterns. Complexity analysis demonstrates the advantages of ESN/WESN based symbol detector compared to conventional symbol detectors such as linear minimum mean square error (LMMSE) and sphere decoder when the system is employed with a large number of OFDM sub-carriers. Numerical evaluations show that ESN/WESN has an improvement of symbol detection performance as opposed to conventional methods in both low SNR regime and power amplifier (PA) nonlinear regime. Finally, it demonstrates that WESN can generate a better symbol detection result over ESN. / Artificial neural networks (ANN) are widely used in recognition tasks such as recommendation systems, robotics path planning, self-driving, video tracking, image classifications, etc. To further explore the applications of ANN, this thesis considers using a specific ANN, echo state network (ESN) for a wireless communications task: MIMO-OFDM symbol detection. Furthermore, it proposed an enhanced version of the standard ESN, namely, windowed echo state network (WESN). Theoretical analyses on the short term memory (STM) of ESN and WESN show that the later one has a longer STM. Besides, the training set size of this ESN/WESN based method is chosen the same as the pilot symbols used in conventional communications systems. The algorithm complexity analysis demonstrates the ESN/WESN based method performs with lower complexity compared with conventional methods, such as linear mean square error (LMMSE) and sphere decoding. Comprehensive simulations examine how the symbol detection performance can be improved by using ESN and its variant WESN when the transmission link is non-ideal.

Deep Learning hat in den letzten Jahren mit der Entwicklung leistungsfähigerer Hardware und neuer Architekturen wie dem Convolutional Neural Network (CNN), Transformer, und Netzwerken aus Long-Short Term Memory (LSTM)-Zellen ein rasantes Wachstum erlebt. Modelle für viele verschiedene Anwendungsfälle wurden erfolgreich veröffentlicht, und Deep Learning hat Einzug in viele alltägliche Anwendungen gehalten. Einer der größten Nachteile komplexer Modelle wie den CNNs oder LSTMs ist jedoch ihr hoher Energieverbrauch und der Bedarf an großen Mengen annotierter Trainingsdaten. Zumindest letzteres Problem wird teilweise durch die Einführung von neuen Methoden gelöst, die mit nicht-annotierten Daten umgehen können. In dieser Arbeit werden Echo State Networks (ESNs), eine Variante der rekurrenten neuronalen Netze (RNN), betrachtet, da sie eine Möglichkeit bieten, die betrachteten Probleme vieler Deep-Learning Architekturen zu lösen. Einerseits können sie mit linearer Regression trainiert werden, einer relativ einfachen, effizienten und gut etablierten Trainingsmethode. Andererseits sind ESN-Modelle interessante Kandidaten für die Erforschung neuer Trainingsmethoden, insbesondere unüberwachter Lerntechniken, die später in Deep-Learning-Methoden integriert werden können und diese effizienter und leichter trainierbar machen, da sie in ihrer Grundform relativ einfach zu erzeugen sind. Zunächst wird ein allgemeines ESN-Modell in einzelne Bausteine zerlegt, die flexibel zu neuen Architekturen kombiniert werden können. Anhand eines Beispieldatensatzes werden zunächst Basis-ESN-Modelle mit zufällig initialisierten Gewichten vorgestellt, optimiert und evaluiert. Anschließend werden deterministische ESN-Modelle betrachtet, bei denen der Einfluss unterschiedlicher zufälliger Initialisierungen reduziert ist. Es wird gezeigt, dass diese Architekturen recheneffizienter sind, aber dennoch eine vergleichbare Leistungsfähigkeit wie die Basis-ESN-Modelle aufweisen. Es wird auch gezeigt, dass deterministische ESN-Modelle verwendet werden können, um hierarchische ESN-Architekturen zu bilden. Anschließend werden unüberwachte Trainingsmethoden für die verschiedenen Bausteine des ESN-Modells eingeführt, illustriert und in einer vergleichenden Studie mit Basis- und deterministischen ESN-Architekturen als Basis evaluiert. Anhand einer Vielzahl von Benchmark-Datensätzen für die Zeitreihenklassifikation und verschiedene Audioverarbeitungsaufgaben wird gezeigt, dass die in dieser Arbeit entwickelten ESN-Modelle in der Lage sind, ähnliche Ergebnisse wie der Stand der Technik in den jeweiligen Bereichen zu erzielen. Darüber hinaus werden Anwendungsfälle identifiziert, für die bestimmte ESN-Modelle bevorzugt werden sollten, und es werden die Grenzen der verschiedenen Trainingsmethoden diskutiert. Abschließend wird gezeigt, dass zwischen dem übergeordneten Thema Reservoir Computing und Deep Learning eine Forschungslücke existiert, die in Zukunft zu schließen ist.:Statement of authorship vii Abstract ix Zusammenfassung xi Acknowledgments xiii Contents xv Acronyms xix List of Publications xxiii 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Reservoir Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Objective and Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Echo State Network 5 2.1 Artificial neuron model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 The basic Echo State Network . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Advanced Echo State Network structures . . . . . . . . . . . . . . . . . . . . 15 2.4 Hyper-parameter optimization of Echo State Networks . . . . . . . . . . . . . 22 3 Building blocks of Echo State Networks 25 3.1 Toolboxes for Reservoir Computing Networks . . . . . . . . . . . . . . . . . . 25 3.2 Building blocks of Echo State Networks . . . . . . . . . . . . . . . . . . . . . 26 3.3 Define Extreme LearningMachines . . . . . . . . . . . . . . . . . . . . . . . . 30 3.4 Define Echo State Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.5 Sequential hyper-parameter optimization . . . . . . . . . . . . . . . . . . . . . 32 4 Basic, deterministic and hierarchical Echo State Networks 35 4.1 Running example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.2 Performance of a basic Echo State Network . . . . . . . . . . . . . . . . . . . 37 4.3 Performance of hierarchical Echo State Networks . . . . . . . . . . . . . . . . 42 4.4 Performance of deterministic Echo State Network architectures . . . . . . . . 44 4.5 Performance of hierarchical deterministic Echo State Networks . . . . . . . . 50 4.6 Comparison of the considered ESN architectures . . . . . . . . . . . . . . . . 52 4.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5 Unsupervised Training of the Input Weights in Echo State Networks 57 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 5.3 Optimization of the KM-ESN model . . . . . . . . . . . . . . . . . . . . . . . 63 5.4 Performance of the KM-ESN . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 5.5 Combination of the KM-ESN and deterministic architectures . . . . . . . . . 74 5.6 Hierarchical (determinstic) KM-ESN architectures . . . . . . . . . . . . . . . 77 5.7 Comparison of the considered KM-ESN architectures . . . . . . . . . . . . . . 80 5.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 6 Unsupervised Training of the Recurrent Weights in Echo State Networks 85 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 6.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 6.3 Optimization of the pre-trained models . . . . . . . . . . . . . . . . . . . . . . 88 6.4 Performance of the KM-ESN-based models . . . . . . . . . . . . . . . . . . . 93 6.5 Comparison of all considered ESN architectures . . . . . . . . . . . . . . . . . 95 6.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 7 Multivariate time series classification with non-standard Echo State Networks 101 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 7.2 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 7.3 Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 7.4 Optimization of the hyper-parameters . . . . . . . . . . . . . . . . . . . . . . 105 7.5 Comparison of different ESN architectures . . . . . . . . . . . . . . . . . . . . 107 7.6 Overall results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 7.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 8 Application of Echo State Networks to audio signals 123 8.1 Acoustic Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 8.2 Phoneme Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 8.3 Musical Onset Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 8.4 Multipitch tracking in audio signals . . . . . . . . . . . . . . . . . . . . . . . . 157 8.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 9 Conclusion and Future Work 165 9.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 9.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Bibliography 169 / The field of deep learning has experienced rapid growth in recent years with the development of more powerful hardware and new architectures such as the Convolutional Neural Network (CNN), transformer, and Long-Short Term Memory (LSTM) cells. Models for many different use cases have been successfully published, and deep learning has found its way into many everyday applications. However, one of the major drawbacks of complex models based on CNNs or LSTMs is their resource hungry nature such as the need for large amounts of labeled data and excessive energy consumption. This is partially addressed by introducing more and more methods that can deal with unlabeled data. In this thesis, Echo State Network (ESN) models, a variant of a Recurrent Neural Network (RNN), are studied because they offer a way to address the aforementioned problems of many deep learning architectures. On the one hand, they can easily be trained using linear regression, which is a simple, efficient, and well-established training method. On the other hand, since they are relatively easy to generate in their basic form, ESN models are interesting candidates for investigating new training methods, especially unsupervised learning techniques, which can later find their way into deep learning methods, making them more efficient and easier to train. First, a general ESN model is decomposed into building blocks that can be flexibly combined to form new architectures. Using an example dataset, basic ESN models with randomly initialized weights are first introduced, optimized, and evaluated. Then, deterministic ESN models are considered, where the influence of random initialization is reduced. It is shown that these architectures have a lower computational complexity but that they still show a comparable performance to the basic ESN models. It is also shown that deterministic ESN models can be used to build hierarchical ESN architectures. Then, unsupervised training methods for the different building blocks of the ESN model are introduced, illustrated, and evaluated in a comparative study with basic and deterministic ESN architectures as a baseline. Based on a broad variety of benchmark datasets for time-series classification and various audio processing tasks, it is shown that the ESN models proposed in this thesis can achieve results similar to the state-of-the-art approaches in the respective field. Furthermore, use cases are identified, for which specific models should be preferred, and limitations of the different training methods are discussed. It is also shown that there is a research gap between the umbrella topics of Reservoir Computing and Deep Learning that needs to be filled in the future.:Statement of authorship vii Abstract ix Zusammenfassung xi Acknowledgments xiii Contents xv Acronyms xix List of Publications xxiii 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Reservoir Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Objective and Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Echo State Network 5 2.1 Artificial neuron model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 The basic Echo State Network . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Advanced Echo State Network structures . . . . . . . . . . . . . . . . . . . . 15 2.4 Hyper-parameter optimization of Echo State Networks . . . . . . . . . . . . . 22 3 Building blocks of Echo State Networks 25 3.1 Toolboxes for Reservoir Computing Networks . . . . . . . . . . . . . . . . . . 25 3.2 Building blocks of Echo State Networks . . . . . . . . . . . . . . . . . . . . . 26 3.3 Define Extreme LearningMachines . . . . . . . . . . . . . . . . . . . . . . . . 30 3.4 Define Echo State Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.5 Sequential hyper-parameter optimization . . . . . . . . . . . . . . . . . . . . . 32 4 Basic, deterministic and hierarchical Echo State Networks 35 4.1 Running example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.2 Performance of a basic Echo State Network . . . . . . . . . . . . . . . . . . . 37 4.3 Performance of hierarchical Echo State Networks . . . . . . . . . . . . . . . . 42 4.4 Performance of deterministic Echo State Network architectures . . . . . . . . 44 4.5 Performance of hierarchical deterministic Echo State Networks . . . . . . . . 50 4.6 Comparison of the considered ESN architectures . . . . . . . . . . . . . . . . 52 4.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5 Unsupervised Training of the Input Weights in Echo State Networks 57 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 5.3 Optimization of the KM-ESN model . . . . . . . . . . . . . . . . . . . . . . . 63 5.4 Performance of the KM-ESN . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 5.5 Combination of the KM-ESN and deterministic architectures . . . . . . . . . 74 5.6 Hierarchical (determinstic) KM-ESN architectures . . . . . . . . . . . . . . . 77 5.7 Comparison of the considered KM-ESN architectures . . . . . . . . . . . . . . 80 5.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 6 Unsupervised Training of the Recurrent Weights in Echo State Networks 85 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 6.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 6.3 Optimization of the pre-trained models . . . . . . . . . . . . . . . . . . . . . . 88 6.4 Performance of the KM-ESN-based models . . . . . . . . . . . . . . . . . . . 93 6.5 Comparison of all considered ESN architectures . . . . . . . . . . . . . . . . . 95 6.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 7 Multivariate time series classification with non-standard Echo State Networks 101 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 7.2 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 7.3 Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 7.4 Optimization of the hyper-parameters . . . . . . . . . . . . . . . . . . . . . . 105 7.5 Comparison of different ESN architectures . . . . . . . . . . . . . . . . . . . . 107 7.6 Overall results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 7.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 8 Application of Echo State Networks to audio signals 123 8.1 Acoustic Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 8.2 Phoneme Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 8.3 Musical Onset Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 8.4 Multipitch tracking in audio signals . . . . . . . . . . . . . . . . . . . . . . . . 157 8.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 9 Conclusion and Future Work 165 9.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 9.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Bibliography 169

The reservoir computer comprises a reservoir of possibly non-linear, possibly chaotic dynamics. By perturbing and taking outputs from this reservoir, its dynamics may be harnessed to compute complex problems at “the edge of chaos”. One of the first forms of reservoir computer, the Echo State Network (ESN), is a form of artificial neural network that builds its reservoir from a large and sparsely connected recurrent neural network (RNN). The ESN was initially introduced as an innovative solution to train RNNs which, up until that point, was a notoriously difficult task. The innovation of the ESN is that, rather than train the RNN weights, only the output is trained. If this output is assumed to be linear, then linear regression may be used. This work presents an effort to implement the Echo State Network, and an offline linear regression training method based on Tikhonov regularisation. This implementation targeted the general purpose graphics processing unit (GPU or GPGPU). The behaviour of the implementation was examined by comparing it with a central processing unit (CPU) implementation, and by assessing its performance against several studied learning problems. These assessments were performed using all 4 cores of the Intel i7-980 CPU and an Nvidia GTX480. When compared with a CPU implementation, the GPU ESN implementation demonstrated a speed-up starting from a reservoir size of between 512 and 1,024. A maximum speed-up of approximately 6 was observed at the largest reservoir size tested (2,048). The Tikhonov regularisation (TR) implementation was also compared with a CPU implementation. Unlike the ESN execution, the GPU TR implementation was largely slower than the CPU implementation. Speed-ups were observed at the largest reservoir and state history sizes, the largest of which was 2.6813. The learning behaviour of the GPU ESN was tested on three problems, a sinusoid, a Mackey-Glass time-series, and a multiple superimposed oscillator (MSO). The normalised root-mean squared errors of the predictors were compared. The best observed sinusoid predictor outperformed the best MSO predictor by 4 orders of magnitude. In turn, the best observed MSO predictor outperformed the best Mackey-Glass predictor by 2 orders of magnitude.

In industrial environments, operators are exposed to polluted air which after constant exposure can cause irreversible lethal diseases such as lung cancer. The current air monitoring techniques are carried out sparely in either a single day annually or at few measurement positions for a few days.In this thesis a theory-guided data science (TGDS) model is presented. This hybrid model combines a steady state Computational Fluid Dynamics (CFD) model with a machine learning model. Both the CFD model and the machine learning algorithm was developed in Matlab. The CFD model serves as a basis for the airflow whereas the machine learning model addresses dynamical features in the foundry. Measurements have previously been made at a foundry where five stationary sensors and one mobile robot were used for data acquisition. An Echo State Network was used as a supervised learning technique for airflow predictions at each robot measurement position and Gaussian Processes (GP) were used as a regression technique to form an Echo State Map (ESM). The stationary sensor data were used as input for the echo state network and the difference between the CFD and robot measurements were used as teacher signal which formed a dynamic correction map that was added to the steady state CFD. The proposed model utilizes the high spatio-temporal resolution of the echo state map whilst making use of the physical consistency of the CFD. The initial applications of the novel hybrid model proves that the best qualities of these two models could come together in symbiosis to give enhanced characterizations.The proposed model could have an important role for future characterization of airflow and more research on this and similar topics are encouraged to make sure we properly understand the potential of this novel model. / Industriarbetare utsätts för skadliga luftburna ämnen vilket över tid leder till högre prevalens för lungsjukdomar så som kronisk obstruktiv lungsjukdom, stendammslunga och lungcancer. De nuvarande luftmätningsmetoderna genomförs årligen under korta sessioner och ofta vid få selekterade platser i industrilokalen. I denna masteruppsats presenteras en teorivägledd datavetenskapsmodell (TGDS) som kombinerar en stationär beräkningsströmningsdynamik (CFD) modell med en dynamisk maskininlärningsmodell. Både CFD-modellen och maskininlärningsalgoritmen utvecklades i Matlab. Echo State Network (ESN) användes för att träna maskininlärningsmodellen och Gaussiska Processer (GP) används som regressionsteknik för att kartlägga luftflödet över hela industrilokalen. Att kombinera ESN med GP för att uppskatta luftflöden i stålverk genomfördes första gången 2016 och denna modell benämns Echo State Map (ESM). Nätverket använder data från fem stationära sensorer och tränades på differensen mellan CFD-modellen och mätningar genomfördes med en mobil robot på olika platser i industriområdet. Maskininlärningsmodellen modellerar således de dynamiska effekterna i industrilokalen som den stationära CFD-modellen inte tar hänsyn till. Den presenterade modellen uppvisar lika hög temporal och rumslig upplösning som echo state map medan den också återger fysikalisk konsistens som CFD-modellen. De initiala applikationerna för denna model påvisar att de främsta egenskaperna hos echo state map och CFD används i symbios för att ge förbättrad karakteriseringsförmåga. Den presenterade modellen kan spela en viktig roll för framtida karakterisering av luftflöden i industrilokaler och fler studier är nödvändiga innan full förståelse av denna model uppnås.

The objective of this research is to introduce a neural network based solution for the problem of measuring the actual amount of harmonic current injected into a power network by an individual nonlinear load. Harmonic currents from nonlinear loads propagate through the system and cause harmonic pollution. As a result, voltage at the point of common coupling (PCC) is rarely sinusoidal. The IEEE 519 harmonic standard provides customer and utility harmonic limits and many utilities are now requiring their customers to comply with IEEE 519. Measurements of the customer’s current at the PCC are expected to determine the customer’s compliance with IEEE 519. However, results in this research show that the current measurements at the PCC are not always reliable in that determination. In such a case, it may be necessary to determine what the customer’s true current harmonic distortions would be if the PCC voltage could be a pure sinusoidal voltage. However, establishing a pure sinusoidal voltage at the PCC may not be feasible since that would mean performing utility switching to reduce the system impedance. An alternative approach is to use a neural network that is able to learn the customer’s load admittance. Then, it is possible to predict the customer’s true current harmonic distortions based on mathematically applying a pure sinusoidal voltage to the learned load admittance. The proposed method is called load modeling. Load modeling predicts the true harmonic current that can be attributed to a customer regardless of whether a resonant condition exists on the utility power system. If a corrective action is taken by the customer, another important parameter of interest is the change in the voltage distortion level at the PCC due to the corrective action of the customer. This issue is also addressed by using the dual of the load modeling method. Topologies of the neural networks used in this research include multilayer perceptron neural networks and recurrent neural networks. The theory and implementation of a new neural network topology known as an Echo State Networks is also introduced. The proposed methods are verified on a number of different power electronic test circuits as well as field data. The main advantages of the proposed methods are that only waveforms of voltages and currents are required for their operation and they are applicable to both single and three phase systems. The proposed methods can be integrated into any existing power quality instrument or can be fabricated into a commercial standalone instrument that could be installed in substations of large customer loads, or used as a hand-held clip on instrument.

No description available.

Biological systems constantly evolve and adapt in response to changed environment and external stimuli at the molecular and genomic levels. Building statistical models that characterize such dynamic changes in biological systems is one of the key objectives in bioinformatics and computational biology. Recent advances in high-throughput genomic and molecular profiling technologies such as gene expression and and copy number microarrays provide ample opportunities to study cellular activities at the individual gene and network levels. The aim of this dissertation is to formulate mathematically dynamic changes in biological networks and DNA copy numbers, to develop machine learning algorithms to learn these statistical models from high-throughput biological data, and to demonstrate their applications in systems biological studies. The first part (Chapters 2-4) of the dissertation focuses on the dynamic changes taking placing at the biological network level. Biological networks are context-specific and dynamic in nature. Under different conditions, different regulatory components and mechanisms are activated and the topology of the underlying gene regulatory network changes. We report a differential dependency network (DDN) analysis to detect statistically significant topological changes in the transcriptional networks between two biological conditions. Further, we formalize and extend the DDN approach to an effective learning strategy to extract structural changes in graphical models using l1-regularization based convex optimization. We discuss the key properties of this formulation and introduce an efficient implementation by the block coordinate descent algorithm. Another type of dynamic changes in biological networks is the observation that a group of genes involved in certain biological functions or processes coordinate to response to outside stimuli, producing distinct time course patterns. We apply the echo stat network, a new architecture of recurrent neural networks, to model temporal gene expression patterns and analyze the theoretical properties of echo state networks with random matrix theory. The second part (Chapter 5) of the dissertation focuses on the changes at the DNA copy number level, especially in cancer cells. Somatic DNA copy number alterations (CNAs) are key genetic events in the development and progression of human cancers, and frequently contribute to tumorigenesis. We propose a statistically-principled in silico approach, Bayesian Analysis of COpy number Mixtures (BACOM), to accurately detect genomic deletion type, estimate normal tissue contamination, and accordingly recover the true copy number profile in cancer cells. / Ph. D.

Dynamic spectrum access (DSA) is a promising technology to mitigate spectrum shortage and improve spectrum utilization. However, DSA users have to face two fundamental issues, interference coordination between DSA users and protections to primary users (PUs). These two issues are very challenging, since generally there is no powerful infrastructure in DSA networks to support centralized control. As a result, DSA users have to perform spectrum managements, including spectrum access and power allocations, independently without accurate channel state information. In this thesis, a novel spectrum management approach is proposed, in which Q-learning, a type of reinforcement learning, is utilized to enable DSA users to carry out effective spectrum managements individually and intelligently. For more efficient processes, powerful neural networks (NNs) are employed to implement Q-learning processes, so-called deep Q-network (DQN). Furthermore, I also investigate the optimal way to construct DQN considering both the performance of wireless communications and the difficulty of NN training. Finally, extensive simulation studies are conducted to demonstrate the effectiveness of the proposed spectrum management approach. / Generally, in dynamic spectrum access (DSA) networks, co-operations and centralized control are unavailable and DSA users have to carry out wireless transmissions individually. DSA users have to know other users’ behaviors by sensing and analyzing wireless environments, so that DSA users can adjust their parameters properly and carry out effective wireless transmissions. In this thesis, machine learning and deep learning technologies are leveraged in DSA network to enable appropriate and intelligent spectrum managements, including both spectrum access and power allocations. Accordingly, a novel spectrum management framework utilizing deep reinforcement learning is proposed, in which deep reinforcement learning is employed to accurately learn wireless environments and generate optimal spectrum management strategies to adapt to the variations of wireless environments. Due to the model-free nature of reinforcement learning, DSA users only need to directly interact with environments to obtain optimal strategies rather than relying on accurate channel estimations. In this thesis, Q-learning, a type of reinforcement learning, is adopted to design the spectrum management framework. For more efficient and accurate learning, powerful neural networks (NN) is employed to combine Q-learning and deep learning, also referred to as deep Q-network (DQN). The selection of NNs is crucial for the performance of DQN, since different types of NNs possess various properties and are applicable for different application scenarios. Therefore, in this thesis, the optimal way to construct DQN is also analyzed and studied. Finally, the extensive simulation studies demonstrate that the proposed spectrum management framework could enable users to perform proper spectrum managements and achieve better performance.