This work presents three techniques for parameter identification for nonlinear systems. The methods presented are expanded from those presented in Adetola and Guay [3, 4, 5] and are intended to improve the performance of existing adaptive control systems. The first two methods exactly recover open-loop system parameters once a defined convergence condition is met. In either case, the true parameters are identified when the regressor matrix is of full rank and can be inverted. The third case uses a novel method developed in Adetola and Guay [5] to define a parameter uncertainty set. The uncertainty set is periodically updated to shrink around the true value of the parameters. Each method is shown to be applicable to a large class of linearly parameterized nonlinear discrete-time system. In each case, parameter convergence is guaranteed subject to an appropriate convergence condition, which has been related to a classical persistence of excitation condition. The effectiveness of the methods is demonstrated using a simulation example. The application of the uncertainty set technique to nonlinearly parameterized systems constitutes the main contribution of the thesis. The parameter uncertainty set method is generalized to the problem of adaptive estimation in nonlinearly parameterized systems, for both continuous-time and discrete-time cases. The method is demonstrated to perform well in simulation for a simplified model of a bioreactor operating under Monod kinetics. / Thesis (Master, Chemical Engineering) -- Queen's University, 2010-10-19 10:58:24.888

A new system identification algorithm, multiple autoregressive exogenous (ARX) inputs-based Takagi-Sugeno (TS) fuzzy model, is developed to identify nonlinear behavior of structure-magnetorheological (MR) damper systems. It integrates a set of ARX models, clustering algorithms, and weighted least squares algorithm with a TS fuzzy model. Based on a set of input-output data that is generated from building structures equipped with MR dampers, premise parameters of the ARX-TS fuzzy model are determined by clustering algorithms. Once the premise part is constructed, consequent parameters of the ARX-TS fuzzy model are optimized by the weighted least squares algorithm. To demonstrate the effectiveness of the proposed ARX-TS fuzzy model, it is applied to a three-, an eight-, a twenty-story building structures. It is demonstrated from the numerical simulation that the proposed ARX-TS fuzzy algorithm is effective to identify nonlinear behavior of seismically excited building structures equipped with MR dampers. A new semiactive nonlinear fuzzy control (SNFC) algorithm is developed through integration of multiple Lyapunov-based state feedback gains, a Kalman filter, and a converting algorithm with TS fuzzy interpolation method. First, the nonlinear ARX-TS fuzzy model is decomposed into a set of linear dynamic models that are operated in only a local linear operating region. Based on the decomposed models, multiple Lyapunov-based state feedback controllers are formulated in terms of linear matrix inequalities (LMIs) such that the structure-MR damper system is globally asymptotically stable and the performance on transient responses is guaranteed. Then, the state feedback controllers are integrated with a Kalman filter and a converting algorithm using a TS fuzzy interpolation method to construct semiactive output feedback controllers. To demonstrate the effectiveness of the proposed SNFC algorithm, it is applied to a three-, an eight-, and a twenty-story building structures. It is demonstrated from the numerical simulation that the proposed SNFC algorithm is effective to control responses of seismically excited building structures equipped with MR dampers. In addition, it is shown that the proposed SNFC system is better than a traditional optimal algorithm, H2/linear quadratic Gaussian-based semiactive control strategy.

This thesis proposes nonlinear system identification techniques for the mathematical modeling of the human smooth pursuit system (SPS) with application to motor symptom quantification in Parkinson's disease (PD). The SPS refers to the complex neuromuscular system in humans that governs the smooth pursuit eye movements (SPEM). Insight into the SPS and its operation is of importance in a wide and steadily expanding array of application areas and research fields. The ultimate purpose of the work in this thesis is to attain a deeper understanding and quantification of the SPS dynamics and thus facilitate the continued development of novel commercial products and medical devices. The main contribution of this thesis is in the derivation and evaluation of several techniques for SPS characterization. While attempts to mathematically model the SPS have been made in the literature before, several key aspects of the problem have been previously overlooked.This work is the first one to devise dynamical models intended for extended-time experiments and also to consider systematic visual stimuli design in the context of SPS modeling. The result is a handful of parametric mathematical models outperforming current State-of-the-Art models in terms of prediction accuracy for rich input signals. As a complement to the parametric dynamical models, a non-parametric technique involving the construction of individual statistical models pertaining to specific gaze trajectories is suggested. Both the parametric and non-parametric models are demonstrated to successfully distinguish between individuals or groups of individuals based on eye movements.Furthermore, a novel approach to Wiener system identification using Volterra series is proposed and analyzed. It is exploited to confirm that the SPS in healthy individuals is indeed nonlinear, but that the nonlinearity of the system is significantly stronger in PD subjects. The nonlinearity in healthy individuals appears to be well-modeled by a static output function, whereas the nonlinear behavior introduced to the SPS by PD is dynamical.

Single Particle Tracking (SPT) plays an important role in the study of physical and dynamic properties of biomolecules moving in their native environment. To date, many algorithms have been developed for localization and parameter estimation in SPT. Though the performance of these methods is good when the signal level is high and the motion model simple, they begin to fail as the signal level decreases or model complexity increases. In addition, the inputs to the SPT algorithms are sequences of images that are cropped from a large data set and that focus on a single particle. This motivates us to seek machine learning tools to deal with that initial step of extracting data from larger images containing multiple particles. This thesis makes contributions to both data extraction question and to the problem of state and parameter estimation. First, we build upon the Expectation Maximization (EM) algorithm to create a generic framework for joint localization refinement and parameter estimation in SPT. Under the EM-based scheme, two representative methods are considered for generating the filtered and smoothed distributions needed by EM: Sequential Monte Carlo - Expectation Maximization (SMC-EM), and Unscented - Expectation Maximization (U-EM). The selection of filtering and smoothing algorithms is very flexible so long as they provide the necessary distributions for EM. The versatility and reliability of EM based framework have been validated via data-intensive modeling and simulation where we considered a variety of influential factors, such as a wide range of {\color{red}Signal-to-background ratios (SBRs)}, diffusion speeds, motion blur, camera types, image length, etc. Meanwhile, under the EM-based scheme, we make an effort to improve the overall computational efficiency by simplifying the mathematical expression of models, replacing filtering/smoothing algorithms with more efficient ones {\color{purple} (trading some accuracy for reduced computation time)}, and using parallel computation and other computing techniques. In terms of localization refinement and parameter estimation in SPT, we also conduct an overall quantitative comparison among EM based methods and standard two-step methods. Regarding the U-EM, we conduct transformation methods to make it adapted to the nonlinearities and complexities of measurement model. We also extended the application of U-EM to more complicated SPT scenarios, including time-varying parameters and additional observation models that are relevant to the biophysical setting. The second area of contribution is in the particle detection and extraction problem to create data to feed into the EM-based approaches. Here we build Particle Identification Networks (PINs) covering three different network architectures. The first, \PINCNN{}, is based on a standard Convolutional Neural Network (CNN) structure that has previously been successfully applied in particle detection and localization. The second, \PINRES, uses a Residual Neural Network (ResNet) architecture that is significantly deeper than the CNN while the third, \PINFPN{}, is based on a more advanced Feature Pyramid Network (FPN) that can take advantage of multi-scale information in an image. All networks are trained using the same collection of simulated data created with a range of SBRs and fluorescence emitter densities, as well as with three different Point Spread Functions (PSFs): a standard Born-Wolf model, a model for astigmatic imaging to allow localization in three dimensions, and a model of the Double-Helix engineered PSF. All PINs are evaluated and compared through data-intensive simulation and experiments under a variety of settings. In the final contribution, we link all above together to create an algorithm that takes in raw camera data and produces trajectories and parameter estimates for multiple particles in an image sequence.

Mathematical modeling of dynamical systems plays a central roll in science and engineering. This thesis is concerned with the process of finding a mathematical model, and it is divided into two parts - one that concentrates on nonlinear system identification and another one where an impulsive model of testosterone regulation is constructed and analyzed. In the first part of the thesis, a new latent variable framework for identification of a large class of nonlinear models is developed. In this framework, we begin by modeling the errors of a nominal predictor using a flexible stochastic model. The error statistics and the nominal predictor are then identified using the maximum likelihood principle. The resulting optimization problem is tackled using a majorization-minimization approach, resulting in a tuning parameter-free recursive identification method. The proposed method learns parsimonious predictive models. Many popular model structures can be expressed within the framework, and in the thesis it is applied to piecewise ARX models. In the first part, we also derive a recursive prediction error method based on the Hammerstein model structure. The convergence properties of the method are analyzed by application of the associated differential equation method, and conditions ensuring convergence are given. In the second part of the thesis, a previously proposed pulse-modulated feedback model of testosterone regulation is extended with infinite-dimensional dynamics, in order to better explain testosterone profiles observed in clinical data. It is then shown how the analysis of oscillating solutions for the finite-dimensional case can be extended to the infinte-dimensional case. A method for blind state estimation in impulsive systems is introduced, with the purpose estimating hormone concentrations that cannot be measured in a non-invasive way. The unknown parameters in the model are identified from clinical data and, finally, a method of incorporating exogenous signals portraying e.g. medical interventions is studied.

Many high-dimensional time-varying signals can be modeled as a sequence of noisy nonlinear observations of a low-dimensional dynamical process. Given high-dimensional observations and a distribution describing the dynamical process, we present a computationally inexpensive approximate algorithm for estimating the inverse of this mapping. Once this mapping is learned, we can invert it to construct a generative model for the signals. Our algorithm can be thought of as learning a manifold of images by taking into account the dynamics underlying the low-dimensional representation of these images. It also serves as a nonlinear system identification procedure that estimates the inverse of the observation function in nonlinear dynamic system. Our algorithm reduces to a generalized eigenvalue problem, so it does not suffer from the computational or local minimum issues traditionally associated with nonlinear system identification, allowing us to apply it to the problem of learning generative models for video sequences.

Identification of thermoacoustic dynamics that exhibit limit cycle behavior is needed to gain a better intuitive feel of the system, to design complex control strategies, and to validate modeling efforts. Limit cycle oscillations arise in thermoacoustic systems due to the coupling between a nonlinear heat release process and the acoustic dynamics of the combustor. This response arises in lean premixed gaseous power generating turbines and is a concern due to the detrimental effect of the pressure oscillations on the structural integrity of the combustor. Due to the volatile environment intrinsic in the combustor, multiple sensing apparatuses are not available. Therefore, in the current study, an identification approach is assessed considering only a single output from the thermoacoustic system. As a means to further investigate the thermoacoustic limit cycle behavior, a scaled version of the industry-based turbine was constructed. By anchoring a flame halfway from end-to-end of a closed-open tube, a similar nonlinear response is achieved. A harmonic balance technique that linearly incorporates the nonlinearity is developed which uses frequency entrainment to offer sufficient information for the identification. Its validity is assessed on a model, which is based on known dynamics of the thermoacoustic system. The structure of the identification algorithm is based on a two-mode acoustic model with both dynamics and nonlinearity in the feedback loop. The limitations of using only a two-mode identification structure for a system with more than two modes is discussed as well as future efforts that may alleviate this problem. / Master of Science

Although nonlinear model predictive control (NMPC) might be the best choice for a nonlinear plant, it is still not widely used. This is mainly due to the computational burden associated with solving online a set of nonlinear differential equations and a nonlinear dynamic optimization problem in real time. This thesis is concerned with strategies aimed at reducing the computational burden involved in different stages of the NMPC such as optimization problem, state estimation, and nonlinear model identification. A major part of the computational burden comes from function and derivative evaluations required in different parts of the NMPC algorithm. In this work, the problem is tackled using a recently introduced efficient tool, the automatic differentiation (AD). Using the AD tool, a function is evaluated together with all its partial derivative from the code defining the function with machine accuracy. A new NMPC algorithm based on nonlinear least square optimization is proposed. In a first–order method, the sensitivity equations are integrated using a linear formula while the AD tool is applied to get their values accurately. For higher order approximations, more terms of the Taylor expansion are used in the integration for which the AD is effectively used. As a result, the gradient of the cost function against control moves is accurately obtained so that the online nonlinear optimization can be efficiently solved. In many real control cases, the states are not measured and have to be estimated for each instance when a solution of the model equations is needed. A nonlinear extended version of the Kalman filter (EKF) is added to the NMPC algorithm for this purpose. The AD tool is used to calculate the required derivatives in the local linearization step of the filter automatically and accurately. Offset is another problem faced in NMPC. A new nonlinear integration is devised for this case to eliminate the offset from the output response. In this method, an integrated disturbance model is added to the process model input or output to correct the plant/model mismatch. The time response of the controller is also improved as a by–product. The proposed NMPC algorithm has been applied to an evaporation process and a two continuous stirred tank reactor (two–CSTR) process with satisfactory results to cope with large setpoint changes, unmeasured severe disturbances, and process/model mismatches. When the process equations are not known (black–box) or when these are too complicated to be used in the controller, modelling is needed to create an internal model for the controller. In this thesis, a continuous time recurrent neural network (CTRNN) in a state–space form is developed to be used in NMPC context. An efficient training algorithm for the proposed network is developed using AD tool. By automatically generating Taylor coefficients, the algorithm not only solves the differentiation equations of the network but also produces the sensitivity for the training problem. The same approach is also used to solve online the optimization problem of the NMPC. The proposed CTRNN and the predictive controller were tested on an evaporator and two–CSTR case studies. A comparison with other approaches shows that the new algorithm can considerably reduce network training time and improve solution accuracy. For a third case study, the ALSTOM gasifier, a NMPC via linearization algorithm is implemented to control the system. In this work a nonlinear state–space class Wiener model is used to identify the black–box model of the gasifier. A linear model of the plant at zero–load is adopted as a base model for prediction. Then, a feedforward neural network is created as the static gain for a particular output channel, fuel gas pressure, to compensate its strong nonlinear behavior observed in open–loop simulations. By linearizing the neural network at each sampling time, the static nonlinear gain provides certain adaptation to the linear base model. The AD tool is used here to linearize the neural network efficiently. Noticeable performance improvement is observed when compared with pure linear MPC. The controller was able to pass all tests specified in the benchmark problem at all load conditions.

Power amplifiers (PAs) are important components of communication systems and are inherently nonlinear. When a non-constant modulus signal goes through a nonlinear PA, spectral regrowth (broadening) appears in the PA output, which in turn causes adjacent channel interference (ACI). Stringent limits on the ACI are imposed by regulatory bodies, and thus the extent of the PA nonlinearity must be controlled. PA linearization is often necessary to suppress spectral regrowth, contain adjacent channel interference, and reduce bit error rate (BER). This dissertation addresses the following aspects of power amplifier research: modeling, linearization, and spectral regrowth analysis. We explore the passband and baseband PA input/output relationships and show that they manifest differently when the PA exhibits long-term, short-term, or no memory effects. The so-called quasi-memoryless case is especially clarified. Four particular nonlinear models with memory are further investigated. We provide experimental results to support our analysis. The benefits of using the orthogonal polynomials as opposed to the conventional polynomials are explored, in the context of digital baseband PA modeling and predistorter design. A closed-form expression for the orthogonal polynomial basis is derived. We demonstrate the improvement in numerical stability associated with the use of orthogonal polynomials for predistortion. Spectral analysis can help to evaluate the suitability of a given PA for amplifying certain signals or to assist in predistortion linearization algorithm design. With the orthogonal polynomials that we derived, spectral analysis of the nonlinear PA becomes a straightforward task. We carry out nonlinear spectral analysis with digitally modulated signal as input. We demonstrate an analytical approach for evaluating the power spectra of filtered QPSK and OQPSK signals after nonlinear amplification. Many communications devices are nonlinear and have a peak power or peak amplitude constraint. In addition to possibly amplifying the useful signal, the nonlinearity also generates distortions. We focus on signal-to-noise-and-distortion ratio (SNDR) optimization within the family of amplitude limited memoryless nonlinearities. We obtain a link between the capacity of amplitude-limited nonlinear channels with Gaussian noise to the SNDR.

This dissertation introduces novel methods for solving highly challenging model- ing and control problems, motivated by advanced aerospace systems. Adaptable, ro- bust and computationally effcient, multi-resolution approximation algorithms based on Radial Basis Function Network and Global-Local Orthogonal Mapping approaches are developed to address various problems associated with the design of large scale dynamical systems. The main feature of the Radial Basis Function Network approach is the unique direction dependent scaling and rotation of the radial basis function via a novel Directed Connectivity Graph approach. The learning of shaping and rota- tion parameters for the Radial Basis Functions led to a broadly useful approximation approach that leads to global approximations capable of good local approximation for many moderate dimensioned applications. However, even with these refinements, many applications with many high frequency local input/output variations and a high dimensional input space remain a challenge and motivate us to investigate an entirely new approach. The Global-Local Orthogonal Mapping method is based upon a novel averaging process that allows construction of a piecewise continuous global family of local least-squares approximations, while retaining the freedom to vary in a general way the resolution (e.g., degrees of freedom) of the local approximations. These approximation methodologies are compatible with a wide variety of disciplines such as continuous function approximation, dynamic system modeling, nonlinear sig-nal processing and time series prediction. Further, related methods are developed for the modeling of dynamical systems nominally described by nonlinear differential equations and to solve for static and dynamic response of Distributed Parameter Sys- tems in an effcient manner. Finally, a hierarchical control allocation algorithm is presented to solve the control allocation problem for highly over-actuated systems that might arise with the development of embedded systems. The control allocation algorithm makes use of the concept of distribution functions to keep in check the "curse of dimensionality". The studies in the dissertation focus on demonstrating, through analysis, simulation, and design, the applicability and feasibility of these ap- proximation algorithms to a variety of examples. The results from these studies are of direct utility in addressing the "curse of dimensionality" and frequent redundancy of neural network approximation.