The presence of noise and corruption in real-world data can be inevitably caused by accidental outliers, transmission loss, or even adversarial data attacks. Unlike traditional random noise usually assume a specific distribution with low corruption ratio, the data collected from crowdsourcing or labeled by weak annotators can contain adversarial data corruption. More challenge, the adversarial data corruption can be arbitrary, unbounded and do not follow any specific distribution. In addition, in the era of data explosion, the fast-growing amount of data makes the robust models more difficult to handle large-scale data sets. This thesis focuses on the development of methods for scalable robust models under the adversarial data corruption assumptions. Four methods are proposed, including robust regression via heuristic hard-thresholding, online and distributed robust regression with adversarial noises, self-paced robust learning for leveraging clean labels in noisy data, and robust regression via online feature selection with adversarial noises. Moreover, I extended the self-paced robust learning method to its distributed version for the scalability of the proposed algorithm, named distributed self-paced learning in alternating direction method of multiplier. Last, a robust multi-factor personality prediction model is proposed to hand the correlated data noises. For the first method, existing solutions for robust regression lack rigorous recovery guarantee of regression coefficients under the adversarial data corruption with no prior knowledge of corruption ratio. The proposed contributions of our work include: (1) Propose efficient algorithms to address the robust least-square regression problem; (2) Design effective approaches to estimate the corruption ratio; (3) Provide a rigorous robustness guarantee for regression coefficient recovery; and (4) Conduct extensive experiments for performance evaluation. For the second method, existing robust learning methods typically focus on modeling the entire dataset at once; however, they may meet the bottleneck of memory and computation as more and more datasets are becoming too large to be handled integrally. The proposed contributions of our work for this task include: (1) Formulate a framework for the scalable robust least-squares regression problem; (2) Propose online and distributed algorithms to handle the adversarial corruption; (3) Provide a rigorous robustness guarantee for regression coefficient recovery; and (4) Conduct extensive experiments for performance evaluations. For the third method, leveraging the prior knowledge of clean labels in noisy data is actually a crucial issue in practice, but existing robust learning methods typically focus more on eliminating noisy data. However, the data collected by ``weak annotator" or crowd-sourcing can be too noisy for existing robust methods to train an accurate model. Moreover, existing work that utilize additional clean labels are usually designed for some specific problems such as image classification. These methods typically utilize clean labels in large-scale noisy data based on their additional domain knowledge; however, these approaches are difficult to handle extremely noisy data and relied on their domain knowledge heavily, which makes them difficult be used in more general problems. The proposed contributions of our work for this task include: (1) Formulating a framework to leverage the clean labels in noisy data; (2) Proposing a self-paced robust learning algorithm to train models under the supervision of clean labels; (3) Providing a theoretical analysis for the convergence of the proposed algorithm; and (4) Conducting extensive experiments for performance evaluations. For the fourth method, the presence of data corruption in user-generated streaming data, such as social media, motivates a new fundamental problem that learns reliable regression coefficient when features are not accessible entirely at one time. Until now, several important challenges still cannot be handled concurrently: 1) corrupted data estimation when only partial features are accessible; 2) online feature selection when data contains adversarial corruption; and 3) scaling to a massive dataset. This paper proposes a novel RObust regression algorithm via Online Feature Selection (textit{RoOFS}) that concurrently addresses all the above challenges. Specifically, the algorithm iteratively updates the regression coefficients and the uncorrupted set via a robust online feature substitution method. We also prove that our algorithm has a restricted error bound compared to the optimal solution. Extensive empirical experiments in both synthetic and real-world data sets demonstrated that the effectiveness of our new method is superior to that of existing methods in the recovery of both feature selection and regression coefficients, with very competitive efficiency. For the fifth method, existing self-paced learning approaches typically focus on modeling the entire dataset at once; however, this may introduce a bottleneck in terms of memory and computation, as today's fast-growing datasets are becoming too large to be handled integrally. The proposed contributions of our work for this task include: (1) Reformulate the self-paced problem into a distributed setting.; (2) A distributed self-paced learning algorithm based on consensus ADMM is proposed to solve the textit{SPL} problem in a distributed setting; (3) A theoretical analysis is provided for the convergence of our proposed textit{DSPL} algorithm; and (4) Extensive experiments have been conducted utilizing both synthetic and real-world data based on a robust regression task. For the last method, personality prediction in multiple factors, such as openness and agreeableness, is growing in interest especially in the context of social media, which contains massive online posts or likes that can potentially reveal an individual's personality. However, the data collected from social media inevitably contains massive amounts of noise and corruption. To address it, traditional robust methods still suffer from several important challenges, including 1) existence of correlated corruption among multiple factors, 2) difficulty in estimating the corruption ratio in multi-factor data, and 3) scalability to massive datasets. This paper proposes a novel robust multi-factor personality prediction model that concurrently addresses all the above challenges by developing a distributed robust regression algorithm. Specifically, the algorithm optimizes regression coefficients of each factor in parallel with a heuristically estimated corruption ratio and then consolidates the uncorrupted set from multiple factors in two strategies: global consensus and majority voting. We also prove that our algorithm benefits from strong guarantees in terms of convergence rates and coefficient recovery, which can be utilized as a generic framework for the multi-factor robust regression problem with correlated corruption property. Extensive experiment on synthetic and real dataset demonstrates that our algorithm is superior to those of existing methods in both effectiveness and efficiency. / Doctor of Philosophy / Social media has experienced a rapid growth during the past decade. Millions of users of sites such as Twitter have been generating and sharing a wide variety of content including texts, images, and other metadata. In addition, social media can be treated as a social sensor that reflects different aspects of our society. Event analytics in social media have enormous significance for applications like disease surveillance, business intelligence, and disaster management. Social media data possesses a number of important characteristics including dynamics, heterogeneity, noisiness, timeliness, big volume, and network properties. These characteristics cause various new challenges and hence invoke many interesting research topics, which will be addressed here. This dissertation focuses on the development of five novel methods for social media-based spatiotemporal event detection and forecasting. The first of these is a novel unsupervised approach for detecting the dynamic keywords of spatial events in targeted domains. This method has been deployed in a practical project for monitoring civil unrest events in several Latin American regions. The second builds on this by discovering the underlying development progress of events, jointly considering the structural contexts and spatiotemporal burstiness. The third seeks to forecast future events using social media data. The basic idea here is to search for subtle patterns in specific cities as indicators of ongoing or future events, where each pattern is defined as a burst of context features (keywords) that are relevant to a specific event. For instance, an initial expression of discontent gas price increases could actually be a potential precursor to a more general protest about government policies. Beyond social media data, in the fourth method proposed here, multiple data sources are leveraged to reflect different aspects of the society for event forecasting. This addresses several important problems, including the common phenomena that different sources may come from different geographical levels and have different available time periods. The fifth study is a novel flu forecasting method based on epidemics modeling and social media mining. A new framework is proposed to integrate prior knowledge of disease propagation mechanisms and real-time information from social media.

This dissertation forms part of the ongoing process at UP to model and control the electric arc furniture process. Previous work focused on modelling the furnace process from empirical thermodynamic principles as well as fitting the model to actual plant data. Automation of the process mainly focused on subsystems of the process, for example the electric subsystem and the off-gas subsystem. The modelling effort, especially the model fitting resulted in parameter values that are described with confidence intervals, which gives rise to uncertainty in the model, because the parameters can potentially lie anywhere in the confidence interval space. Robust model predictive control is used in this dissertation, because it can explicityly take the model uncertainty into account as part of the synthesis process. Nominal model predictive control – not taking model uncertainty into account – is also applied in order to determine if robust model predictive control provides any advantages over the nominal model predictive control. This dissertation uses the process model from previous wok together with robust model predictive control to determine the feasibility of automating the process with regards to the primary process variables. Possible hurdles that prevent practical implementation are identified and studied. / Dissertation (MEng (Electronic Engineering))--University of Pretoria, 2007. / Electrical, Electronic and Computer Engineering / MEng / unrestricted

This dissertation forms part of the ongoing process at UP to model and control the electric arc furniture process. Previous work focused on modelling the furnace process from empirical thermodynamic principles as well as fitting the model to actual plant data. Automation of the process mainly focused on subsystems of the process, for example the electric subsystem and the off-gas subsystem. The modelling effort, especially the model fitting resulted in parameter values that are described with confidence intervals, which gives rise to uncertainty in the model, because the parameters can potentially lie anywhere in the confidence interval space. Robust model predictive control is used in this dissertation, because it can explicityly take the model uncertainty into account as part of the synthesis process. Nominal model predictive control – not taking model uncertainty into account – is also applied in order to determine if robust model predictive control provides any advantages over the nominal model predictive control. This dissertation uses the process model from previous wok together with robust model predictive control to determine the feasibility of automating the process with regards to the primary process variables. Possible hurdles that prevent practical implementation are identified and studied. / Dissertation (MEng (Electronic Engineering))--University of Pretoria, 2007. / Electrical, Electronic and Computer Engineering / MEng / unrestricted

The main objective of this thesis is to apply ideas and techniques from modern control theory, especially from robust state estimation and model validation, to various important problems in computer vision. Robust model validation is used in texture recognition where new approaches for classifying texture samples and segmenting textured images are developed. Also, a new model validation approach to motion primitive recognition is demonstrated by considering the motion segmentation problem for a mobile wheeled robot. A new approach to image inpainting based on robust state estimation is proposed where the implementation presented here concerns with recovering corrupted frames in video sequences. Another application addressed in this thesis based on robust state estimation is video-based tracking. A new tracking system is proposed to follow connected regions in video frames representing the objects in consideration. The system accommodates tracking multiple objects and is designed to be robust towards occlusions. To demonstrate the performance of the proposed solutions, examples are provided where the developed methods are applied to various gray-scale images, colored images, gray-scale videos and colored videos. In addition, a new algorithm is introduced for motion estimation via inverse polynomial interpolation. Motion estimation plays a primary role within the video-based tracking system proposed in this thesis. The proposed motion estimation algorithm is also applied to medical image sequences. Motion estimation results presented in this thesis include pairs of images from a echocardiography video and a robot-assisted surgery video.

The main objective of this thesis is to apply ideas and techniques from modern control theory, especially from robust state estimation and model validation, to various important problems in computer vision. Robust model validation is used in texture recognition where new approaches for classifying texture samples and segmenting textured images are developed. Also, a new model validation approach to motion primitive recognition is demonstrated by considering the motion segmentation problem for a mobile wheeled robot. A new approach to image inpainting based on robust state estimation is proposed where the implementation presented here concerns with recovering corrupted frames in video sequences. Another application addressed in this thesis based on robust state estimation is video-based tracking. A new tracking system is proposed to follow connected regions in video frames representing the objects in consideration. The system accommodates tracking multiple objects and is designed to be robust towards occlusions. To demonstrate the performance of the proposed solutions, examples are provided where the developed methods are applied to various gray-scale images, colored images, gray-scale videos and colored videos. In addition, a new algorithm is introduced for motion estimation via inverse polynomial interpolation. Motion estimation plays a primary role within the video-based tracking system proposed in this thesis. The proposed motion estimation algorithm is also applied to medical image sequences. Motion estimation results presented in this thesis include pairs of images from a echocardiography video and a robot-assisted surgery video.

No description available.

In traditional meta-analysis, a random-effects model is used to deal with heterogeneity and the random-effect is assumed to be normally distributed. However, this can be problematic in the presence of outliers. One solution involves using a heavy tailed distribution for the random-effect to more adequately model the excess variation due to the outliers. Failure to consider an alternative approach to the standard in the presence of unusual or outlying points can lead to inaccurate inference. A heavy tailed distribution is favoured because it has the ability to down-weight outlying studies appropriately, therefore the removal of a study does not need to be considered. In this thesis, the performance of the t-distribution and a finite mixture model are assessed as alternatives to the normal distribution through a comprehensive simulation study. The parameters varied are the average mean of the non-outlier studies, the number of studies, the proportion of outliers, the heterogeneity and the outlier shift distance from the average mean. The performance of the distributions is measured using bias, mean squared error, coverage probability, coverage width, Type I error and power. The methods are also compared through an empirical study of meta-analyses from The Cochrane Library (2008). The simulation showed that the performance of the alternative distributions is better than the normal distribution for a number of scenarios, particularly for extreme outliers and high heterogeneity. Generally, the mixture model performed quite well. The empirical study reveals that both alternative distributions are able to reduce the influence of the outlying studies on the overall mean estimate and thus produce more conservative p-values than the normal distribution. It is recommended that a practitioner consider the use of an alternative random-effects distribution in the presence of outliers because they are more likely to provide robust results. / Thesis / Master of Science (MSc)

This thesis develops efficient optimization methods for Model Predictive Control (MPC) to enable its application to constrained systems with fast and uncertain dynamics. The key contribution is an active set method which exploits the parametric nature of the sequential optimization problem and is obtained from a dynamic programming formulation of the MPC problem. This method is first applied to the nominal linear MPC problem and is successively extended to linear systems with additive uncertainty and input constraints or state/input constraints. The thesis discusses both offline (projection-based) and online (active set) methods for the solution of controllability problems for linear systems with additive uncertainty. The active set method uses first-order necessary conditions for optimality to construct parametric programming regions for a particular given active set locally along a line of search in the space of feasible initial conditions. Along this line of search the homotopy of optimal solutions is exploited: a known solution at some given plant state is continuously deformed into the solution at the actual measured current plant state by performing the required active set changes whenever a boundary of a parametric programming region is crossed during the line search operation. The sequence of solutions for the finite horizon optimal control problem is therefore obtained locally for the given plant state. This method overcomes the main limitation of parametric programming methods that have been applied in the MPC context which usually require the offline precomputation of all possible regions. In contrast to this the proposed approach is an online method with very low computational demands which efficiently exploits the parametric nature of the solution and returns exact local DP solutions. The final chapter of this thesis discusses an application of robust tube-based MPC to the nonlinear MPC problem based on successive linearization.

In modern cyber-physical systems (CPSs) where the control signals are generally transmitted via shared communication networks, there is a desire to balance the closed-loop control performance with the communication cost necessary to achieve it. In this context, aperiodic real-time scheduling of control tasks comes into being and has received increasing attention recently. It is well known that model predictive control (MPC) is currently widely utilized in industrial control systems and has greatly increased profits in comparison with the proportional integral-derivative (PID) control. As communication and networks play more and more important roles in modern society, there is a great trend to upgrade and transform traditional industrial systems into CPSs, which naturally requires extending conventional MPC to communication-efficient MPC to save network resources. Motivated by this fact, we in this thesis propose robust MPC and scheduling co-design algorithms to networked CPSs possibly affected by both parameter uncertainties and additive disturbances. In Chapter 2, a dynamic event-triggered robust tube-based MPC for constrained linear systems with additive disturbances is developed, where a time-varying pre-stabilizing gain is obtained by interpolating multiple static state feedbacks and the interpolating coefficient is determined via optimization at the time instants when the MPC-based control is triggered. The original constraints are properly tightened to achieve robust constraint optimization and a sequence of dynamic sets used to test events are derived according to the optimized coefficient. We theoretically show that the proposed algorithm is recursively feasible and the closed-loop system is input-to-state stable (ISS) in the attraction region. Numerical results are presented to verify the design. In Chapter 3, a self-triggered min-max MPC strategy is developed for constrained nonlinear systems subject to both parametric uncertainties and additive disturbances, where the robust constraint satisfaction is achieved by considering the worst case of all possible uncertainty realizations. First, we propose a new cost function that relaxes the penalty on the system state in a time period where the controller will not be invoked. With this cost function, the next triggering time instant can be obtained at current time instant by solving a min-max optimization problem where the maximum triggering period becomes a decision variable. The proposed strategy is proved to be input-to-state practical stable (ISpS) in the attraction region at triggering time instants under some standard assumptions. Extensions are made to linear systems with additive disturbances, for which the conditions reduce to a linear matrix inequality (LMI). Comprehensive numerical experiments are performed to verify the correctness of the theoretical results. / Graduate

Due to its generality and flexibility, the state-space model has become one of the most popular models in modern time domain analysis for the description and prediction of time series data. The model is often used to characterize processes that can be conceptualized as "signal plus noise," where the realized series is viewed as the manifestation of a latent signal that has been corrupted by observation noise. In the state-space framework, parameter estimation is generally accomplished by maximizing the innovations Gaussian log-likelihood. The maximum likelihood estimator (MLE) is efficient when the normality assumption is satisfied. However, in the presence of contamination, the MLE suffers from a lack of robustness. Basu, Harris, Hjort, and Jones (1998) introduced a discrepancy measure (BHHJ) with a non-negative tuning parameter that regulates the trade-off between robustness and efficiency. In this manuscript, we propose a new parameter estimation procedure based on the BHHJ discrepancy for fitting state-space models. As the tuning parameter is increased, the estimation procedure becomes more robust but less efficient. We investigate the performance of the procedure in an illustrative simulation study. In addition, we propose a numerical method to approximate the asymptotic variance of the estimator, and we provide an approach for choosing an appropriate tuning parameter in practice. We justify these procedures theoretically and investigate their efficacy in simulation studies. Based on the proposed parameter estimation procedure, we then develop a new model selection criterion in the state-space framework. The traditional Akaike information criterion (AIC), where the goodness-of-fit is assessed by the empirical log-likelihood, is not robust to outliers. Our new criterion is comprised of a goodness-of-fit term based on the empirical BHHJ discrepancy, and a penalty term based on both the tuning parameter and the dimension of the candidate model. We present a comprehensive simulation study to investigate the performance of the new criterion. In instances where the time series data is contaminated, our proposed model selection criterion is shown to perform favorably relative to AIC. Lastly, using the BHHJ discrepancy based on the chosen tuning parameter, we propose two versions of an influence diagnostic in the state-space framework. Specifically, our diagnostics help to identify cases that influence the recovery of the latent signal, thereby providing initial guidance and insight for further exploration. We illustrate the behavior of these measures in a simulation study.