by Evan, Fung-yu Young. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1993. / Includes bibliographical references (leaves 161-166). / List of Figures --- p.vi / List of Tables --- p.vii / List of Graphs --- p.viii / Abstract --- p.ix / Chapter Part I --- Learning Algorithms / Chapter 1 --- Representing Time in Connectionist Models --- p.1 / Chapter 1.1 --- Introduction --- p.1 / Chapter 1.2 --- Temporal Sequences --- p.2 / Chapter 1.2.1 --- Recognition Tasks --- p.2 / Chapter 1.2.2 --- Reproduction Tasks --- p.3 / Chapter 1.2.3 --- Generation Tasks --- p.4 / Chapter 1.3 --- Discrete Time v.s. Continuous Time --- p.4 / Chapter 1.4 --- Time Delay Neural Network (TDNN) --- p.4 / Chapter 1.4.1 --- Delay Elements in the Connections --- p.5 / Chapter 1.4.2 --- NETtalk: An Application of TDNN --- p.7 / Chapter 1.4.3 --- Drawbacks of TDNN --- p.8 / Chapter 1.5 --- Networks with Context Units --- p.8 / Chapter 1.5.1 --- Jordan's Network --- p.9 / Chapter 1.5.2 --- Elman's Network --- p.10 / Chapter 1.5.3 --- Other Architectures --- p.14 / Chapter 1.5.4 --- Drawbacks of Using Context Units --- p.15 / Chapter 1.6 --- Recurrent Neural Networks --- p.16 / Chapter 1.6.1 --- Hopfield Models --- p.17 / Chapter 1.6.2 --- Fully Recurrent Neural Networks --- p.20 / Chapter A. --- EXAMPLES OF USING RECURRENT NETWORKS --- p.22 / Chapter 1.7 --- Our Objective --- p.25 / Chapter 2 --- Learning Algorithms for Recurrent Neural Networks --- p.27 / Chapter 2.1 --- Introduction --- p.27 / Chapter 2.2 --- Gradient Descent Methods --- p.29 / Chapter 2.2.1 --- Backpropagation Through Time (BPTT) --- p.29 / Chapter 2.2.2 --- Real Time Recurrent Learning Rule (RTRL) --- p.30 / Chapter A. --- RTRL WITH TEACHER FORCING --- p.32 / Chapter B. --- TERMINAL TEACHER FORCING --- p.33 / Chapter C. --- CONTINUOUS TIME RTRL --- p.33 / Chapter 2.2.3 --- Variants of RTRL --- p.34 / Chapter A. --- SUB GROUPED RTRL --- p.34 / Chapter B. --- A FIXED SIZE STORAGE 0(n3) TIME COMPLEXITY LEARNGING RULE --- p.35 / Chapter 2.3 --- Non-Gradient Descent Methods --- p.37 / Chapter 2.3.1 --- Neural Bucket Brigade (NBB) --- p.37 / Chapter 2.3.2 --- Temporal Driven Method (TO) --- p.38 / Chapter 2.4 --- Comparison between Different Approaches --- p.39 / Chapter 2.5 --- Conclusion --- p.41 / Chapter 3 --- Locally Connected Recurrent Networks --- p.43 / Chapter 3.1 --- Introduction --- p.43 / Chapter 3.2 --- Locally Connected Recurrent Networks --- p.44 / Chapter 3.2.1 --- Network Topology --- p.44 / Chapter 3.2.2 --- Subgrouping --- p.46 / Chapter 3.2.3 --- Learning Algorithm --- p.47 / Chapter 3.2.4 --- Continuous Time Learning Algorithm --- p.50 / Chapter 3.3 --- Analysis --- p.51 / Chapter 3.3.1 --- Time Complexity --- p.51 / Chapter 3.3.2 --- Space Complexity --- p.51 / Chapter 3.3.3 --- Local Computations in Time and Space --- p.51 / Chapter 3.4 --- Running on Parallel Architectures --- p.52 / Chapter 3.4.1 --- Mapping the Algorithm to Parallel Architectures --- p.52 / Chapter 3.4.2 --- Parallel Learning Algorithm --- p.53 / Chapter 3.4.3 --- Analysis --- p.54 / Chapter 3.5 --- Ring-Structured Recurrent Network (RRN) --- p.55 / Chapter 3.6 --- Comparison between RRN and RTRL in Sequence Recognition --- p.55 / Chapter 3.6.1 --- Training Sets and Testing Sequences --- p.56 / Chapter 3.6.2 --- Comparison in Training Speed --- p.58 / Chapter 3.6.3 --- Comparison in Recalling Power --- p.59 / Chapter 3.7 --- Comparison between RRN and RTRL in Time Series Prediction --- p.59 / Chapter 3.7.1 --- Comparison in Training Speed --- p.62 / Chapter 3.7.2 --- Comparison in Predictive Power --- p.63 / Chapter 3.8 --- Conclusion --- p.65 / Chapter Part II --- Applications / Chapter 4 --- Sequence Recognition by Ring-Structured Recurrent Networks --- p.67 / Chapter 4.1 --- Introduction --- p.67 / Chapter 4.2 --- Related Works --- p.68 / Chapter 4.2.1 --- Feedback Multilayer Perceptron (FMLP) --- p.68 / Chapter 4.2.2 --- Back Propagation Unfolded Recurrent Rule (BURR) --- p.69 / Chapter 4.3 --- Experimental Details --- p.71 / Chapter 4.3.1 --- Network Architecture --- p.71 / Chapter 4.3.2 --- Input/Output Representations --- p.72 / Chapter 4.3.3 --- Training Phase --- p.73 / Chapter 4.3.4 --- Recalling Phase --- p.73 / Chapter 4.4 --- Experimental Results --- p.74 / Chapter 4.4.1 --- Temporal Memorizing Power --- p.74 / Chapter 4.4.2 --- Time Warping Performance --- p.80 / Chapter 4.4.3 --- Fault Tolerance --- p.85 / Chapter 4.4.4 --- Learning Rate --- p.87 / Chapter 4.5 --- Time Delay --- p.88 / Chapter 4.6 --- Conclusion --- p.91 / Chapter 5 --- Time Series Prediction --- p.92 / Chapter 5.1 --- Introduction --- p.92 / Chapter 5.2 --- Modelling in Feedforward Networks --- p.93 / Chapter 5.3 --- Methodology with Recurrent Networks --- p.94 / Chapter 5.3.1 --- Network Structure --- p.94 / Chapter 5.3.2 --- Model Building - Training --- p.95 / Chapter 5.3.3 --- Model Diagnosis - Testing --- p.95 / Chapter 5.4 --- Training Paradigms --- p.96 / Chapter 5.4.1 --- A Quasiperiodic Series with White Noise --- p.96 / Chapter 5.4.2 --- A Chaotic Series --- p.97 / Chapter 5.4.3 --- Sunspots Numbers --- p.98 / Chapter 5.4.4 --- Hang Seng Index --- p.99 / Chapter 5.5 --- Experimental Results and Discussions --- p.99 / Chapter 5.5.1 --- A Quasiperiodic Series with White Noise --- p.101 / Chapter 5.5.2 --- Logistic Map --- p.103 / Chapter 5.5.3 --- Sunspots Numbers --- p.105 / Chapter 5.5.4 --- Hang Seng Index --- p.109 / Chapter 5.6 --- Conclusion --- p.112 / Chapter 6 --- Chaos in Recurrent Networks --- p.114 / Chapter 6.1 --- Introduction --- p.114 / Chapter 6.2 --- Important Features of Chaos --- p.115 / Chapter 6.2.1 --- First Return Map --- p.115 / Chapter 6.2.2 --- Long Term Unpredictability --- p.117 / Chapter 6.2.3 --- Sensitivity to Initial Conditions (SIC) --- p.118 / Chapter 6.2.4 --- Strange Attractor --- p.119 / Chapter 6.3 --- Chaotic Behaviour in Recurrent Networks --- p.120 / Chapter 6.3.1 --- Network Structure --- p.121 / Chapter 6.3.2 --- Dynamics in Training --- p.121 / Chapter 6.3.3 --- Dynamics in Testing --- p.122 / Chapter 6.4 --- Experiments and Discussions --- p.123 / Chapter 6.4.1 --- Henon Model --- p.123 / Chapter 6.4.2 --- Lorenz Model --- p.127 / Chapter 6.5 --- Conclusion --- p.134 / Chapter 7 --- Conclusion --- p.135 / Appendix A Series 1 Sine Function with White Noise --- p.137 / Appendix B Series 2 Logistic Map --- p.138 / Appendix C Series 3 Sunspots Numbers from 1700 to 1979 --- p.139 / Appendix D A Quasiperiodic Series with White Noise --- p.141 / Appendix E Hang Seng Daily Closing Index in 1991 --- p.142 / Appendix F Network Model for the Quasiperiodic Series with White Noise --- p.143 / Appendix G Network Model for the Logistic Map --- p.144 / Appendix H Network Model for the Sunspots Numbers --- p.145 / Appendix I Network Model for the Hang Seng Index --- p.146 / Appendix J Henon Model --- p.147 / Appendix K Network Model for the Henon Map --- p.150 / Appendix L Lorenz Model --- p.151 / Appendix M Network Model for the Lorenz Map --- p.159 / Bibliography --- p.161
| Identifer | oai:union.ndltd.org:cuhk.edu.hk/oai:cuhk-dr:cuhk_319221 |
| Date | January 1993 |
| Contributors | Young, Evan Fung-yu., Chinese University of Hong Kong Graduate School. Division of Computer Science. |
| Publisher | Chinese University of Hong Kong |
| Source Sets | The Chinese University of Hong Kong |
| Language | English |
| Detected Language | English |
| Type | Text, bibliography |
| Format | print, x, 166 leaves : ill. ; 30 cm. |
| Rights | Use of this resource is governed by the terms and conditions of the Creative Commons “Attribution-NonCommercial-NoDerivatives 4.0 International” License (http://creativecommons.org/licenses/by-nc-nd/4.0/) |
Page generated in 0.002 seconds