This thesis deals with the methodology of building data driven models of nonlinear systems through the framework of dynamic modeling. More specifically this thesis focuses on sequential optimization of nonlinear dynamic models called recurrent neural networks (RNNs). In particular, the thesis considers fully connected recurrent neural networks with one hidden layer of neurons for modeling of nonlinear dynamical systems. The general objective is to improve sequential training of the RNN through sequential second-order methods and to improve generalization of the RNN by regularization. The total contributions of the proposed thesis can be summarized as follows: 1. First, a sequential Bayesian training and regularization strategy for recurrent neural networks based on an extension of the Evidence Framework is developed. 2. Second, an efficient ensemble method for Sequential Monte Carlo filtering is proposed. The methodology allows for efficient O(H 2 ) sequential training of the RNN. 3. Last, the Expectation Maximization (EM) framework is proposed for training RNNs sequentially.
| Identifer | oai:union.ndltd.org:bl.uk/oai:ethos.bl.uk:524919 |
| Date | January 2010 |
| Creators | Mirikitani, Derrick Takeshi |
| Publisher | Goldsmiths College (University of London) |
| Source Sets | Ethos UK |
| Detected Language | English |
| Type | Electronic Thesis or Dissertation |
| Source | http://research.gold.ac.uk/3239/ |
Page generated in 0.0017 seconds