A central problem in artificial intelligence is to choose actions to maximize reward in a partially observable, uncertain environment. To do so, we must obtain an accurate environment model, and then plan to maximize reward. However, for complex domains, specifying a model by hand can be a time consuming process. This motivates an alternative approach: learning a model directly from observations. Unfortunately, learning algorithms often recover a model that is too inaccurate to support planning or too large and complex for planning to succeed; or, they require excessive prior domain knowledge or fail to provide guarantees such as statistical consistency. To address this gap, we propose spectral subspace identification algorithms which provably learn compact, accurate, predictive models of partially observable dynamical systems directly from sequences of action-observation pairs. Our research agenda includes several variations of this general approach: spectral methods for classical models like Kalman filters and hidden Markov models, batch algorithms and online algorithms, and kernel-based algorithms for learning models in high- and infinite-dimensional feature spaces. All of these approaches share a common framework: the model’s belief space is represented as predictions of observable quantities and spectral algorithms are applied to learn the model parameters. Unlike the popular EM algorithm, spectral learning algorithms are statistically consistent, computationally efficient, and easy to implement using established matrixalgebra techniques. We evaluate our learning algorithms on a series of prediction and planning tasks involving simulated data and real robotic systems.

The extensive use of a least-squares problem formulation in many fields is partly motivated by the existence of an analytic solution formula which makes the theory comprehensible and readily applicable, but also easily embedded in computer-aided design or analysis tools. While the mathematics behind convex optimization has been studied for about a century, several recent researches have stimulated a new interest in the topic. Convex optimization, being a special class of mathematical optimization problems, can be considered as generalization of both least-squares and linear programming. As in the case of a linear programming problem there is in general no simple analytical formula that can be used to find the solution of a convex optimization problem. There exists however efficient methods or software implementations for solving a large class of convex problems. The challenge and the state of the art in using convex optimization comes from the difficulty in recognizing and formulating the problem. The main goal of this thesis is to investigate the potential advantages and benefits of convex optimization techniques in the field of system identification. The primary work focuses on parametric discrete-time system identification models in which we assume or choose a specific model structure and try to estimate the model parameters for best fit using experimental input-output (IO) data. By developing a working knowledge of convex optimization and treating the system identification problem as a convex optimization problem will allow us to reduce the uncertainties in the parameter estimation. This is achieved by reecting prior knowledge about the system in terms of constraint functions in the least-squares formulation.

By September 2009 the International Atomic Energy Agency reported that the number of commercially operated nuclear reactors in 30 countries across the world is 436, around 50 reactors are currently being constructed, 137 reactors have been ordered or is already planned, and there are around 295 proposed reactors. Pressurised water reactors (PWRs) make up the majority of these numbers. The growing number of carbon emissions and the ongoing fight against fossil fuel power stations might see the number of planned nuclear reactors increase even more to be able to satisfy the world’s need for cleaner energy. To ensure that technology keeps pace with this growing demand, ongoing research is essential. Not only is the research of new reactor technologies (i.e. High Temperature Reactors) important, but improving the current technologies (i.e. PWRs) is critical. With the increased contribution of nuclear generated electricity to our grids, it is becoming more common for nuclear reactors to be operated as load following units, and not base load units as they are more commonly being operated. Therefore a need exists to study and develop new strategies and technologies to improve the automatic load following capabilities of reactors. PWR power plants are multivariable systems. In this study a multivariable, more specifically, a model predictive controller (MPC) is developed for controlling the load following of a nuclear power plant, more specifically a PWR plant. In developing this controller system identification is employed to develop a model of the PWR plant. For the identification of the model, measured data from a computer based PWR simulator is used as the input. The identified plant model is used to develop the MPC controller. The controller is developed and tested on the plant model. The MPC controller is also evaluated against another set of measured data from the simulator. To compare the performance of the MPC controller to that of the conventional controller the ITAE performance index is employed. During the process Matlab ® , the System Identification Toolbox™, the MPC Toolbox™ and Simulink ® are used. The results reveal that MPC is practicable to be used in the control of non-linear systems such as PWR plants. The MPC controller showed good results for controlling the system and also outperformed the conventional controllers. A further result from the dissertation is that system identification can successfully be used to develop models for use in model based controllers like MPC controllers. The results of the research show that a need exists for future research to improve the methods to eventually have a controller that can be applied on a commercial plant. / Thesis (M.Ing. (Nuclear Engineering))--North-West University, Potchefstroom Campus, 2010.

Due to their frictionless operation active magnetic bearings (AMBs) are essential components in high-speed rotating machinery. Active magnetic control of a high speed rotating rotor requires precise knowledge of its position. Self-sensing endeavours to eliminate the required position sensors by deducing the rotor’s position from the voltages and currents with which it is levitated. For self-sensing AMBs to be of practical worth, they have to be robust. Robustness analysis aims to quantify a control system’s tolerance for uncertainty. In this study the stability margin of a two degree-of-freedom self-sensing AMB is estimated by means of μ-analysis. Detailed black-box models are developed for the main subsystems in the AMB by means of discrete-time system identification. Suitable excitation signals are generated for system identification in cognisance of frequency induced nonlinear behaviour of the AMB. Novel graphs that characterize an AMB’s behaviour for input signals of different amplitudes and frequency content are quite useful in this regard. In order to obtain models for dynamic uncertainty in the various subsystems (namely the power amplifier, self-sensing module and AMB plant), the identified models are combined to form a closed-loop model for the self-sensing AMB. The response of this closed-loop model is compared to the original AMB’s response and models for the dynamic uncertainty are empirically deduced. Finally, the system’s stability margin for the modelled uncertainty is estimated by means of μ-analysis. The potentially destabilizing effects of parametric uncertainty in the controller coefficients as well as dynamic uncertainty in the AMB plant and self-sensing module are examined. The resultant μ-analyses show that selfsensing AMBs are much less robust for parametric uncertainty in the controller than AMBs equipped with sensors. The μ-analyses for dynamic uncertainty confirm that self-sensing AMBs are rather sensitive for variations in the plant or the self-sensing algorithm. Validation of the stability margins estimated by μ-analysis reveal that μ-analysis is overoptimistic for parametric uncertainty on the controller and conservative for dynamic uncertainty. (Validation is performed by means of Monte Carlo simulations.) The accuracy of μ-analysis is critically dependent on the accuracy of the uncertainty model and the degree to which the system is linear or not. If either of these conditions are violated, μ-analysis is essentially worthless. / Thesis (Ph.D. (Electronical Engineering))--North-West University, Potchefstroom Campus, 2010

Modern power networks are being driven ever closer to both their physical and operational limits. As a result, control systems are being increasingly relied on to assure satisfactory system performance. Power system stabilizers (PSSs) are one example of such controllers. Their purpose is to increase system damping and they are typically designed using a model of the network that is valid during nominal operating conditions. The limitation of this design approach is that during off-nominal operating conditions, such as those triggered by daily load fluctuations, performance of the controller can degrade. The research presented in this report attempts to evaluate the possibility of employing an adaptive PSS as a means of avoiding the performance degradation precipitated by off-nominal operation. Conceptually, an adaptive PSS would be capable of identifying changes in the network and then adjusting its parameters to ensure suitable damping of the identified network. This work begins with a detailed look at the identification algorithm employed followed by a similarly detailed examination of the control algorithm that was used. The results of these two investigations are then combined to allow for a preliminary assessment of the performance that could be expected from an adaptive PSS. The results of this research suggest that an adaptive PSS is a possibility but further work is needed to confirm this finding. Testing using more complex network models must be carried out, details pertaining to control parameter tuning must be resolved and closed-loop time domain simulations using the adaptive PSS design remain to be performed.

Due to their frictionless operation active magnetic bearings (AMBs) are essential components in high-speed rotating machinery. Active magnetic control of a high speed rotating rotor requires precise knowledge of its position. Self-sensing endeavours to eliminate the required position sensors by deducing the rotor’s position from the voltages and currents with which it is levitated. For self-sensing AMBs to be of practical worth, they have to be robust. Robustness analysis aims to quantify a control system’s tolerance for uncertainty. In this study the stability margin of a two degree-of-freedom self-sensing AMB is estimated by means of μ-analysis. Detailed black-box models are developed for the main subsystems in the AMB by means of discrete-time system identification. Suitable excitation signals are generated for system identification in cognisance of frequency induced nonlinear behaviour of the AMB. Novel graphs that characterize an AMB’s behaviour for input signals of different amplitudes and frequency content are quite useful in this regard. In order to obtain models for dynamic uncertainty in the various subsystems (namely the power amplifier, self-sensing module and AMB plant), the identified models are combined to form a closed-loop model for the self-sensing AMB. The response of this closed-loop model is compared to the original AMB’s response and models for the dynamic uncertainty are empirically deduced. Finally, the system’s stability margin for the modelled uncertainty is estimated by means of μ-analysis. The potentially destabilizing effects of parametric uncertainty in the controller coefficients as well as dynamic uncertainty in the AMB plant and self-sensing module are examined. The resultant μ-analyses show that selfsensing AMBs are much less robust for parametric uncertainty in the controller than AMBs equipped with sensors. The μ-analyses for dynamic uncertainty confirm that self-sensing AMBs are rather sensitive for variations in the plant or the self-sensing algorithm. Validation of the stability margins estimated by μ-analysis reveal that μ-analysis is overoptimistic for parametric uncertainty on the controller and conservative for dynamic uncertainty. (Validation is performed by means of Monte Carlo simulations.) The accuracy of μ-analysis is critically dependent on the accuracy of the uncertainty model and the degree to which the system is linear or not. If either of these conditions are violated, μ-analysis is essentially worthless. / Thesis (Ph.D. (Electronical Engineering))--North-West University, Potchefstroom Campus, 2010

By September 2009 the International Atomic Energy Agency reported that the number of commercially operated nuclear reactors in 30 countries across the world is 436, around 50 reactors are currently being constructed, 137 reactors have been ordered or is already planned, and there are around 295 proposed reactors. Pressurised water reactors (PWRs) make up the majority of these numbers. The growing number of carbon emissions and the ongoing fight against fossil fuel power stations might see the number of planned nuclear reactors increase even more to be able to satisfy the world’s need for cleaner energy. To ensure that technology keeps pace with this growing demand, ongoing research is essential. Not only is the research of new reactor technologies (i.e. High Temperature Reactors) important, but improving the current technologies (i.e. PWRs) is critical. With the increased contribution of nuclear generated electricity to our grids, it is becoming more common for nuclear reactors to be operated as load following units, and not base load units as they are more commonly being operated. Therefore a need exists to study and develop new strategies and technologies to improve the automatic load following capabilities of reactors. PWR power plants are multivariable systems. In this study a multivariable, more specifically, a model predictive controller (MPC) is developed for controlling the load following of a nuclear power plant, more specifically a PWR plant. In developing this controller system identification is employed to develop a model of the PWR plant. For the identification of the model, measured data from a computer based PWR simulator is used as the input. The identified plant model is used to develop the MPC controller. The controller is developed and tested on the plant model. The MPC controller is also evaluated against another set of measured data from the simulator. To compare the performance of the MPC controller to that of the conventional controller the ITAE performance index is employed. During the process Matlab ® , the System Identification Toolbox™, the MPC Toolbox™ and Simulink ® are used. The results reveal that MPC is practicable to be used in the control of non-linear systems such as PWR plants. The MPC controller showed good results for controlling the system and also outperformed the conventional controllers. A further result from the dissertation is that system identification can successfully be used to develop models for use in model based controllers like MPC controllers. The results of the research show that a need exists for future research to improve the methods to eventually have a controller that can be applied on a commercial plant. / Thesis (M.Ing. (Nuclear Engineering))--North-West University, Potchefstroom Campus, 2010.

General anesthesia is a drug-induced reversible state where neuromuscular blockade (NMB), hypnosis, and analgesia (jointly denoted by depth of anesthesia - DoA) are guaranteed. This thesis concerns mathematical modeling and feedback control of the effect of the muscle relaxants atracurium and rocuronium, the hypnotic propofol, and the analgesic remifentanil. It is motivated by the need to reduce incidences of awareness and overdose-related post-operative complications that occur in standard clinical practice. A major challenge for identification in closed-loop is the poor excitation provided by the feedback signal. This applies to the case of drugs administered in closed-loop. As a result, the standard models for the effect of anesthetics appear to be over-parameterized. This deteriorates the result of system identification and prevents individualized control. In the first part of the thesis, minimally parameterized models for the single-input single-output NMB and the multiple-input single-output DoA are developed, using real data. The models have a nonlinear Wiener structure: linear time-invariant dynamics cascaded with a static nonlinearity. The proposed models are shown to improve identifiability as compared to the standard ones. The second part of the thesis presents system identification methods for Wiener systems: a batch prediction error method, and two recursive techniques, one based on the extended Kalman filter, and another based on the particle filter. Algorithms are given for both the NMB and the DoA using the minimally parameterized models. Nonlinear adaptive controllers are proposed in the third part of the thesis. Using the model parameter estimates from the extended Kalman filter, the controller performs an online inversion of the Wiener nonlinearity. A pole-placement controller or a linear quadratic Gaussian controller is used for the linearized system. Results show good reference tracking performance both in simulation and in real trials. Relating to patient safety, the existence of undesirable sustained oscillations as consequence of Andronov-Hopf bifurcations for a NMB PID-controlled system is analyzed. Essentially the same bifurcations are observed in the standard and the minimally parameterized models, confirming the ability of the latter to predict the nonlinear behavior of the closed-loop system. Methods to design oscillation-free controllers are outlined.

In order to provide a novel perspective for videography of high speed sporting events, a highly capable trajectory tracking control methodology is developed for a custom designed Kadet Senior Unmanned Aerial Vehicle (UAV). The accompanying high fidelity system identification ensures that accurate flight models are used to design the control laws. A parallel vision based target tracking technique is also demonstrated and implemented on a Graphical Processing Unit (GPU), to assist in real-time tracking of the target. Nonlinear control techniques like feedback linearization require a detailed and accurate system model. This thesis discusses techniques used for estimating these models using data collected during planned test flights. A class of methods known as the Output Error Methods are discussed with extensions for dealing with wind turbulence. Implementation of these methods, including data acquisition details, on the Kadet Senior are also discussed. Results for this UAV are provided. For comparison, additional results using data from a BAC-221 simulation are also provided as well as typical results from the work done at the Dryden Flight Research Center. The proposed controller combines feedback linearization with linear tracking control using the internal model approach, and relies on a trajectory generating exosystem. Three different aircraft models are presented each with increasing levels of complexity, in an effort to identify the simplest controller that yields acceptable performance. The dynamic inversion and linear tracking control laws are derived for each model, and simulation results are presented for tracking of elliptical and periodic trajectories on the Kadet Senior.

Modern power networks are being driven ever closer to both their physical and operational limits. As a result, control systems are being increasingly relied on to assure satisfactory system performance. Power system stabilizers (PSSs) are one example of such controllers. Their purpose is to increase system damping and they are typically designed using a model of the network that is valid during nominal operating conditions. The limitation of this design approach is that during off-nominal operating conditions, such as those triggered by daily load fluctuations, performance of the controller can degrade. The research presented in this report attempts to evaluate the possibility of employing an adaptive PSS as a means of avoiding the performance degradation precipitated by off-nominal operation. Conceptually, an adaptive PSS would be capable of identifying changes in the network and then adjusting its parameters to ensure suitable damping of the identified network. This work begins with a detailed look at the identification algorithm employed followed by a similarly detailed examination of the control algorithm that was used. The results of these two investigations are then combined to allow for a preliminary assessment of the performance that could be expected from an adaptive PSS. The results of this research suggest that an adaptive PSS is a possibility but further work is needed to confirm this finding. Testing using more complex network models must be carried out, details pertaining to control parameter tuning must be resolved and closed-loop time domain simulations using the adaptive PSS design remain to be performed.