Single Particle Tracking (SPT) plays an important role in the study of physical and dynamic properties of biomolecules moving in their native environment. To date, many algorithms have been developed for localization and parameter estimation in SPT. Though the performance of these methods is good when the signal level is high and the motion model simple, they begin to fail as the signal level decreases or model complexity increases. In addition, the inputs to the SPT algorithms are sequences of images that are cropped from a large data set and that focus on a single particle. This motivates us to seek machine learning tools to deal with that initial step of extracting data from larger images containing multiple particles. This thesis makes contributions to both data extraction question and to the problem of state and parameter estimation. First, we build upon the Expectation Maximization (EM) algorithm to create a generic framework for joint localization refinement and parameter estimation in SPT. Under the EM-based scheme, two representative methods are considered for generating the filtered and smoothed distributions needed by EM: Sequential Monte Carlo - Expectation Maximization (SMC-EM), and Unscented - Expectation Maximization (U-EM). The selection of filtering and smoothing algorithms is very flexible so long as they provide the necessary distributions for EM. The versatility and reliability of EM based framework have been validated via data-intensive modeling and simulation where we considered a variety of influential factors, such as a wide range of {\color{red}Signal-to-background ratios (SBRs)}, diffusion speeds, motion blur, camera types, image length, etc. Meanwhile, under the EM-based scheme, we make an effort to improve the overall computational efficiency by simplifying the mathematical expression of models, replacing filtering/smoothing algorithms with more efficient ones {\color{purple} (trading some accuracy for reduced computation time)}, and using parallel computation and other computing techniques. In terms of localization refinement and parameter estimation in SPT, we also conduct an overall quantitative comparison among EM based methods and standard two-step methods. Regarding the U-EM, we conduct transformation methods to make it adapted to the nonlinearities and complexities of measurement model. We also extended the application of U-EM to more complicated SPT scenarios, including time-varying parameters and additional observation models that are relevant to the biophysical setting. The second area of contribution is in the particle detection and extraction problem to create data to feed into the EM-based approaches. Here we build Particle Identification Networks (PINs) covering three different network architectures. The first, \PINCNN{}, is based on a standard Convolutional Neural Network (CNN) structure that has previously been successfully applied in particle detection and localization. The second, \PINRES, uses a Residual Neural Network (ResNet) architecture that is significantly deeper than the CNN while the third, \PINFPN{}, is based on a more advanced Feature Pyramid Network (FPN) that can take advantage of multi-scale information in an image. All networks are trained using the same collection of simulated data created with a range of SBRs and fluorescence emitter densities, as well as with three different Point Spread Functions (PSFs): a standard Born-Wolf model, a model for astigmatic imaging to allow localization in three dimensions, and a model of the Double-Helix engineered PSF. All PINs are evaluated and compared through data-intensive simulation and experiments under a variety of settings. In the final contribution, we link all above together to create an algorithm that takes in raw camera data and produces trajectories and parameter estimates for multiple particles in an image sequence.

Wind observations that are critical for understanding meteorological processes occurring inside of the Earth's atmospheric boundary layer (ABL) are sparse due to limitations of conventional atmospheric sensors. In this dissertation, dynamic systems and estimation theory are combined with experimental methods to exploit the flight envelope of multirotor UAS for wind sensing. The parameters of three quadrotor motion models, consisting of a kinematic particle, a dynamic particle, and a dynamic rigid body models are developed to measure wind velocity in hovering flight. Wind tunnel and steady level flight tests are used to characterize kinematic and dynamic particle models. System identification stepwise regression and output error algorithms are used to determine the model structure and parameter estimates of rigid body models. The comparison of all three models demonstrates the rigid body model to have higher performance resolving slow-varying winds based on a frequency response analysis and field experiments conducted next to a 3-D sonic anemometer. The dissertation also presents an extension of the rigid body wind estimation framework to profile the horizontal components of wind velocity in vertical steady ascending flight. The extension employed system identification to characterize five rigid body models for steady-ascending flight speeds increasing from 0 to 2 m/s in intervals of 0.5~m/s. State observers for wind profiling were synthesized using all five rigid body models. Performance assessments employing wind observations from in situ and remote sensors demonstrated model-based wind profiling results to be be in close agreement with ground-truth wind observations. Finally, the rigid body wind sensing framework developed in this dissertations for multirotor UAS is employed to support science objectives for the Advanced Lagrangian Predictions for Hazards Assessment Project. Quadrotor wind measurements sampled at 10 m above sea level were used to characterize the leeway of a person in water for search and rescue scenarios. Leeway values determined from quadrotor wind measurements were found to be in close to leeway parameters previous published in the literature. This results demonstrates the utility of model-based wind sensing for multirotor UAS for providing wind velocity observations in complex environments where conventional wind observations are not readily available. / Doctor of Philosophy / Wind observations that are critical for understanding meteorological processes occurring inside of the Earth's atmospheric boundary layer (ABL) are sparse due to limitations of conventional atmospheric sensors. In this dissertation, dynamic systems and estimation theory are combined with experimental methods to exploit the flight envelope of multirotor UAS for wind sensing. The parameters of three quadrotor motion models, consisting of a kinematic particle model, a dynamic particle model, and a dynamic rigid body model, are characterized to measure wind velocity in hovering flight. Parameter characterizations are realized using data from wind tunnel, steady level flight tests and system identification experiments. Model-based wind estimations algorithms are developed using the kinematic particle model directly and by synthesizing state observers for the dynamic particle and rigid body models separately. For comparison purposes, the frequency response characteristic of the dynamic particle and rigid body models is examined to determine the range of wind fluctuations that each model can resolve. Performance comparisons demonstrate that the rigid body model to resolve higher wind fluctuations and yield more accurate wind estimates. The dissertation extends the rigid body wind estimation algorithm to estimate wind velocity profiles of the horizontal wind vector. The rigid body wind estimation algorithms is used to answer science questions about about the drift of a person in water.

The use of rotary winged unmanned aerial vehicles in military and civilian applications is rapidly increasing. The primary objective of this study is to develop an automatic flight control system for a radio controlled (RC) helicopter. There is a need for a simple, easy to use methodology to develop automatic flight controllers for first-flight. In order to make the work accessible to new research groups without physical helicopter platforms, a simulation environment is created for validation. The size 30 RC helicopter in AeroSIMRC is treated as the final target platform. A grey box, timedomain system identification method is used to estimate a linear state space model that operates around hover. Identifying the unknown parameters in the model is highly dependent on the initial guess values and the input data. The model is divided into subsystems to make estimation possible. A cascaded controller approach is followed. The helicopter’s fast angular dynamics are separated from the slower translational dynamics. A linear quadratic regulator is used to control the helicopter’s attitude dynamics. An optimised PID outer-loop generates attitude commands from a given inertial position trajectory. The PID controllers are optimised using a simplex search method. An observer estimates the unmeasured states such as blade flapping. The controller is developed in Simulink®, and a plug-in written for AeroSIMRC enables Simulink® to control the simulator through a UDP interface to validate the model and controller. The identified state space model is able to accurately model the flight data from the simulator. The controllers perform well, keeping the helicopter stable even in the presence of considerable disturbances. The attitude controller’s performance is validated using an aeronautical design standard (ADS-33E-PRF) for handling qualities. The trajectory tracking is validated in a series of simulator flight tests. The linear controller is able to sustain stable flight in constant winds of up to 60% of the helicopter’s maximum airspeed. / MIng (Computer and Electronic Engineering), North-West University, Potchefstroom Campus, 2014

The use of rotary winged unmanned aerial vehicles in military and civilian applications is rapidly increasing. The primary objective of this study is to develop an automatic flight control system for a radio controlled (RC) helicopter. There is a need for a simple, easy to use methodology to develop automatic flight controllers for first-flight. In order to make the work accessible to new research groups without physical helicopter platforms, a simulation environment is created for validation. The size 30 RC helicopter in AeroSIMRC is treated as the final target platform. A grey box, timedomain system identification method is used to estimate a linear state space model that operates around hover. Identifying the unknown parameters in the model is highly dependent on the initial guess values and the input data. The model is divided into subsystems to make estimation possible. A cascaded controller approach is followed. The helicopter’s fast angular dynamics are separated from the slower translational dynamics. A linear quadratic regulator is used to control the helicopter’s attitude dynamics. An optimised PID outer-loop generates attitude commands from a given inertial position trajectory. The PID controllers are optimised using a simplex search method. An observer estimates the unmeasured states such as blade flapping. The controller is developed in Simulink®, and a plug-in written for AeroSIMRC enables Simulink® to control the simulator through a UDP interface to validate the model and controller. The identified state space model is able to accurately model the flight data from the simulator. The controllers perform well, keeping the helicopter stable even in the presence of considerable disturbances. The attitude controller’s performance is validated using an aeronautical design standard (ADS-33E-PRF) for handling qualities. The trajectory tracking is validated in a series of simulator flight tests. The linear controller is able to sustain stable flight in constant winds of up to 60% of the helicopter’s maximum airspeed. / MIng (Computer and Electronic Engineering), North-West University, Potchefstroom Campus, 2014

Crane motion induces payload oscillation that makes accurate positioning of the payload a challenging task. As the payload size increases, it may be necessary to utilize multiple cranes for better control of the payload position and orientation. However, simultaneously maneuvering multiple cranes to transport a single payload increases the complexity and danger of the operation. This thesis investigates the dynamics and control of dual-hoist bridge cranes transporting distributed payloads. Insights from this dynamic analysis were used to design input shapers that reduce payload oscillation originating from various crane motions. Also, studies were conducted to investigate the effect input shaping has on the performance of human operators using a dual-hoist bridge crane to transport distributed payloads through an obstacle course. In each study, input shaping significantly improved the task completion time. Furthermore, input-shaping control greatly decreased operator effort, as measured by the number of interface button pushes needed to complete a task. These results clearly demonstrate the benefit of input-shaping control on dual-hoist bridge cranes. In addition, a new system identification method that utilizes input shaping for determining the modal frequencies and relative amplitude contributions of individual modes was developed to aid in the dynamic analysis of dual-hoist bridge cranes, as well as other multi-mode systems. This method uses a new type of input shaper to suppress all but one mode to a low level. The shaper can also be used to bring a small-amplitude mode to light by modifying one of the vibration constraints.

System identification studies how to construct mathematical models for dynamical systems from the input and output data, which finds applications in many scenarios, such as predicting future output of the system or building model based controllers for regulating the output the system. Among many other methods, convex optimization is becoming an increasingly useful tool for solving system identification problems. The reason is that many identification problems can be formulated as, or transformed into convex optimization problems. This transformation is commonly referred to as the convexification technique. The first theme of the thesis is to understand the efficacy of the convexification idea by examining two specific examples. We first establish that a l1 norm based approach can indeed help in exploiting the sparsity information of the underlying parameter vector under certain persistent excitation assumptions. After that, we analyze how the nuclear norm minimization heuristic performs on a low-rank Hankel matrix completion problem. The underlying key is to construct the dual certificate based on the structure information that is available in the problem setting.         Recursive algorithms are ubiquitous in system identification. The second theme of the thesis is the study of some existing recursive algorithms, by establishing new connections, giving new insights or interpretations to them. We first establish a connection between a basic property of the convolution operator and the score function estimation. Based on this relationship, we show how certain recursive Bayesian algorithms can be exploited to estimate the score function for systems with intractable transition densities. We also provide a new derivation and interpretation of the recursive direct weight optimization method, by exploiting certain structural information that is present in the algorithm. Finally, we study how an improved randomization strategy can be found for the randomized Kaczmarz algorithm, and how the convergence rate of the classical Kaczmarz algorithm can be studied by the stability analysis of a related time varying linear dynamical system.

Designing high performance control of rotorcraft unmanned aerial vehicle (UAV) requires a mathematical model that describes the dynamics of the vehicle. The model is derived from first principle modelling, such as rigid-body dynamics, actuator dynamics and etc. It is found that simplified decoupled model of RUAV has slightly better data fitting compared with the complex model for helicopter attitude dynamics in hover or near hover flight condition. In addition, the simplified modelling approach has made the analysis of system dynamics easy. System identification method is applied to identify the unknown intrinsic parameters in the nominal model, where manual piloted flight experiment is carried out and input-output data about a nominal operating region is recorded for parameters identification process. Integral-based parameter identification algorithm is then used to identify model parameters that give the best matching between the simulation and measured output response. The results obtained show that the dominant dynamics is captured. The advantages of using integral-based method include the fast computation time, insensitive to initial parameter value and fast convergence rate in comparison with other contemporary system identification methods such as prediction error method (PEM), maximum likelihood method, equation error method and output error method. Besides, the integral-based parameter identification method can be readily extended to tackle slow time-varying model parameters and fast varying disturbances. The model prediction is found to be improved significantly when the iterative integral-based parameter identification is employed and thus further validates the minimal modelling approach. From the literature review, many control schemes have been designed and validated in simulation. However, few of them has really been implemented in real flight as well as under windy and severe conditions, where unpredictable large system parameters variations and unexpected disturbances are present. Therefore, the emphasis on this part will be on the control design that would have satisfactory reference sequence tracking or regulation capability in the presence of unmodelled dynamics and external disturbances. Generalised Predictive Controller (GPC) is particularly considered as the helicopter attitude dynamics control due to its insensitivity with respect to model mismatch and its capability to address the control problem of nominal model with deadtime. The robustness analysis shows that the robustness of the basic GPC is significantly improved using the Smith Predictor (SP) in place of optimal predictor in basic GPC. The effectiveness of the proposed robust GPC was well proven with the control of helicopter heading on the test rig in terms of the reference sequence tracking performance and the input disturbance rejection capability. The second motivation is the investigation of adaptive GPC from the perspective of performance improvements for the robust GPC. The promising experimental results prove the feasibility of the adaptive GPC controller, and especially evident when the underlying robust GPC is tuned with low robustness and legitimates the use of simplified model. Another approach of robust model predictive control is considered where disturbance is identified in real‐time using an iterative integral‐based method.

Haptic applications often employ devices with many degrees of freedom in order to allow the user to have natural movement during human-machine interaction. From the development point of view, the complexity in mechanical dynamics imposes a lot of challenges in modelling the behaviour of the device. Traditional system identification methods for nonlinear systems are often computationally expensive. Moreover, current research on using neural network approaches disconnect the physical device dynamics with the identification process. This thesis proposes a different approach to system identification of complex haptic devices when analytical models are formulated. It organizes the unknowns to be identified based on the governing dynamic equations of the device and reduces the cost of computation. All the experimental work is done with the Freedom 6S, a haptic device with input and feedback in positions and velocities for all 6 degrees of freedom . <br /><br /> Once a symbolic model is developed, a subset of the overall dynamic equations describing selected joint(s) of the haptic robot can be obtained. The advantage of being able to describe the selected joint(s) is that when other non-selected joints are physically fixed or locked up, it mathematically simplifies the subset dynamic equation. Hence, a reduced set of unknowns (e. g. mass, centroid location, inertia, friction, etc) resulting from the simplified subset equation describes the dynamic of the selected joint(s) at a given mechanical orientation of the robot. By studying the subset equations describing the joints, a locking sequence of joints can be determined to minimize the number of unknowns to be determined at a time. All the unknowns of the system can be systematically determined by locking selected joint(s) of the device following this locking sequence. Two system identification methods are proposed: Method of Isolated Joint and Method of Coupling Joints. Simulation results confirm that the latter approach is able to successfully identify the system unknowns of Freedom 6S. Both open-loop experimental tests and close-loop verification comparison between the measured and simulated results are presented. <br /><br /> Once the haptic device is modelled, fuzzy logic is used to address chattering phenomenon common to strong virtual effects. In this work, a virtual wall is used to demonstrate this approach. The fuzzy controller design is discussed and experimental comparison between the performance of using a proportional-derivative gain controller and the designed fuzzy controller is presented. The fuzzy controller is able to outperform the traditional controller, eliminating the need for hardware upgrades for improved haptic performance. Summary of results and conclusions are included along with suggested future work to be done.

Mathematical modeling of dynamical systems plays a central roll in science and engineering. This thesis is concerned with the process of finding a mathematical model, and it is divided into two parts - one that concentrates on nonlinear system identification and another one where an impulsive model of testosterone regulation is constructed and analyzed. In the first part of the thesis, a new latent variable framework for identification of a large class of nonlinear models is developed. In this framework, we begin by modeling the errors of a nominal predictor using a flexible stochastic model. The error statistics and the nominal predictor are then identified using the maximum likelihood principle. The resulting optimization problem is tackled using a majorization-minimization approach, resulting in a tuning parameter-free recursive identification method. The proposed method learns parsimonious predictive models. Many popular model structures can be expressed within the framework, and in the thesis it is applied to piecewise ARX models. In the first part, we also derive a recursive prediction error method based on the Hammerstein model structure. The convergence properties of the method are analyzed by application of the associated differential equation method, and conditions ensuring convergence are given. In the second part of the thesis, a previously proposed pulse-modulated feedback model of testosterone regulation is extended with infinite-dimensional dynamics, in order to better explain testosterone profiles observed in clinical data. It is then shown how the analysis of oscillating solutions for the finite-dimensional case can be extended to the infinte-dimensional case. A method for blind state estimation in impulsive systems is introduced, with the purpose estimating hormone concentrations that cannot be measured in a non-invasive way. The unknown parameters in the model are identified from clinical data and, finally, a method of incorporating exogenous signals portraying e.g. medical interventions is studied.

System identification (SI) for constructed structural systems has received a lot of attention with the continuous development of modern technologies. This thesis proposes a new nonlinear time series model for use in system identification (SI) of smart structures. The proposed model is implemented by the integration of a wavelet transform (WT) and nonlinear autoregressive moving average (NARMA) time series model. The approach demonstrates the efficient and accurate nonlinear SI of smart structures subjected to both ambient excitation and high impact load. To demonstrate the effectiveness of the wavelet-based NARMA modeling (WNARMA), smart structures equipped with magnetorheological (MR) dampers are investigated. The simulation results show that the computation of the WNARMA model is faster than that of the NARMA model without sacrificing the modeling accuracy. In addition, the WNARMA model is robust against noise in the data since it inherently has a denoising capacity.