The reservoir computer comprises a reservoir of possibly non-linear, possibly chaotic dynamics. By perturbing and taking outputs from this reservoir, its dynamics may be harnessed to compute complex problems at “the edge of chaos”. One of the first forms of reservoir computer, the Echo State Network (ESN), is a form of artificial neural network that builds its reservoir from a large and sparsely connected recurrent neural network (RNN). The ESN was initially introduced as an innovative solution to train RNNs which, up until that point, was a notoriously difficult task. The innovation of the ESN is that, rather than train the RNN weights, only the output is trained. If this output is assumed to be linear, then linear regression may be used. This work presents an effort to implement the Echo State Network, and an offline linear regression training method based on Tikhonov regularisation. This implementation targeted the general purpose graphics processing unit (GPU or GPGPU). The behaviour of the implementation was examined by comparing it with a central processing unit (CPU) implementation, and by assessing its performance against several studied learning problems. These assessments were performed using all 4 cores of the Intel i7-980 CPU and an Nvidia GTX480. When compared with a CPU implementation, the GPU ESN implementation demonstrated a speed-up starting from a reservoir size of between 512 and 1,024. A maximum speed-up of approximately 6 was observed at the largest reservoir size tested (2,048). The Tikhonov regularisation (TR) implementation was also compared with a CPU implementation. Unlike the ESN execution, the GPU TR implementation was largely slower than the CPU implementation. Speed-ups were observed at the largest reservoir and state history sizes, the largest of which was 2.6813. The learning behaviour of the GPU ESN was tested on three problems, a sinusoid, a Mackey-Glass time-series, and a multiple superimposed oscillator (MSO). The normalised root-mean squared errors of the predictors were compared. The best observed sinusoid predictor outperformed the best MSO predictor by 4 orders of magnitude. In turn, the best observed MSO predictor outperformed the best Mackey-Glass predictor by 2 orders of magnitude.

For many challenging problems where the mathematical description is not explicitly defined, artificial intelligence methods appear to be much more robust compared to traditional algorithms. Such methods share the common property of learning from examples in order to “explore” the problem to solve. Then, they generalize these examples to new and unseen input signals. The reservoir computing paradigm is a bio-inspired approach drawn from the theory of artificial Recurrent Neural Networks (RNNs) to process time-dependent data. This machine learning method was proposed independently by several research groups in the early 2000s. It has enabled a breakthrough in analog information processing, with several experiments demonstrating state-of-the-art performance for a wide range of hard nonlinear tasks. These tasks include for instance dynamic pattern classification, grammar modeling, speechrecognition, nonlinear channel equalization, detection of epileptic seizures, robot control, timeseries prediction, brain-machine interfacing, power system monitoring, financial forecasting, or handwriting recognition. A Reservoir Computer (RC) is composed of three different layers. There is first the neural network itself, called “reservoir”, which consists of a large number of internal variables (i.e. reservoir states) all interconnected together to exchange information. The internal dynamics of such a system, driven by a function of the inputs and the former reservoir states, is thus extremely rich. Through an input layer, a time-dependent input signal is applied to all the internal variables to disturb the neural network dynamics. Then, in the output layer, all these reservoir states are processed, often by taking a linear combination thereof at each time-step, to compute the output signal. Let us note that the presence of a non-linearity somewhere in the system is essential to reach high performance computing on nonlinear tasks. The principal novelty of the reservoir computing paradigm was to propose an RNN where most of the connection weights are generated randomly, except for the weights adjusted to compute the output signal from a linear combination of the reservoir states. In addition, some global parameters can be tuned to get the best performance, depending on the reservoir architecture and on the task. This simple and easy process considerably decreases the training complexity compared to traditional RNNs, for which all the weights needed to be optimized. RC algorithms can be programmed using modern traditional processors. But these electronic processors are better suited to digital processing for which a lot of transistors continuously need to be switched on and off, leading to higher power consumption. As we can intuitively understand, processors with hardware directly dedicated to RC operations – in otherwords analog bio-inspired processors – could be much more efficient regarding both speed and power consumption. Based on the same idea of high speed and low power consumption, the last few decades have seen an increasing use of coherent optics in the transport of information thanks to its high bandwidth and high power efficiency advantages. In order to address the future challenge of high performance, high speed, and power efficient nontrivial computing, it is thus natural to turn towards optical implementations of RCs using coherent light. Over the last few years, several physical implementations of RCs using optics and (opto)electronics have been successfully demonstrated. In the present PhD thesis, the reservoirs are based on a large coherently driven linear passive fiber cavity. The internal states are encoded by time-multiplexing in the cavity. Each reservoir state is therefore processed sequentially. This reservoir architecture exhibits many qualities that were either absent or not simultaneously present in previous works: we can perform analog optical signal processing; the easy tunability of each key parameter achieves the best operating point for each task; the system is able to reach a strikingly weak noise floor thanks to the absence of active elements in the reservoir itself; a richer dynamics is provided by operating in coherent light, as the reservoir states are encoded in both the amplitude and the phase of the electromagnetic field; high power efficiency is obtained as a result of the passive nature and simplicity of the setup. However, it is important to note that at this stage we have only obtained low optical power consumption for the reservoir itself. We have not tried to minimize the overall power consumption, including all control electronics. The first experiment reported in chapter 4 uses a quadratic non-linearity on each reservoir state in the output layer. This non-linearity is provided by a readout photodiode since it produces a current proportional to the intensity of the light. On a number of benchmark tasks widely used in the reservoir computing community, the error rates demonstrated with this RC architecture – both in simulation and experimentally – are, to our knowledge, the lowest obtained so far. Furthermore, the analytic model describing our experiment is also of interest, asit constitutes a very simple high performance RC algorithm. The setup reported in chapter 4 requires offline digital post-processing to compute its output signal by summing the weighted reservoir states at each time-step. In chapter 5, we numerically study a realistic model of an optoelectronic “analog readout layer” adapted on the setup presented in chapter 4. This readout layer is based on an RLC low-pass filter acting as an integrator over the weighted reservoir states to autonomously generate the RC output signal. On three benchmark tasks, we obtained very good simulation results that need to be confirmed experimentally in the future. These promising simulation results pave the way for standalone high performance physical reservoir computers.The RC architecture presented in chapter 5 is an autonomous optoelectronic implementation able to electrically generate its output signal. In order to contribute to the challenge of all-optical computing, chapter 6 highlights the possibility of processing information autonomously and optically using an RC based on two coherently driven passive linear cavities. The first one constitutes the reservoir itself and pumps the second one, which acts as an optical integrator onthe weighted reservoir states to optically generate the RC output signal after sampling. A sine non-linearity is implemented on the input signal, whereas both the reservoir and the readout layer are kept linear. Let us note that, because the non-linearity in this system is provided by a Mach-Zehnder modulator on the input signal, the input signal of this RC configuration needs to be an electrical signal. On the contrary, the RC implementation presented in chapter 5 processes optical input signals, but its output is electrical. We obtained very good simulation results on a single task and promising experimental results on two tasks. At the end of this chapter, interesting perspectives are pointed out to improve the performance of this challenging experiment. This system constitutes the first autonomous photonic RC able to optically generate its output signal. / Doctorat en Sciences de l'ingénieur et technologie / info:eu-repo/semantics/nonPublished

No description available.