The ongoing European Refugee Crisis has been one of the most popular trending topics on Twitter for the past 8 months. This paper applies topic modeling on bulks of tweets to discover the hidden patterns within these social media discussions. In particular, we perform topic analysis through solving Non-negative Matrix Factorization (NMF) as an Inexact Alternating Least Squares problem. We accelerate the computation using techniques including tweet sampling and augmented NMF, compare NMF results with different ranks and visualize the outputs through topic representation and frequency plots. We observe that supportive sentiments maintained a strong presence while negative sentiments such as safety concerns have emerged over time.

Dans cette thèse, nous nous concentrons sur le développement des méthodes d'optimisation pour résoudre certaines classes de problèmes d'apprentissage avec la parcimonie et/ou avec l'incertitude des données. Nos méthodes sont basées sur la programmation DC (Difference of Convex functions) et DCA (DC Algorithms) étant reconnues comme des outils puissants d'optimisation. La thèse se compose de deux parties : La première partie concerne la parcimonie tandis que la deuxième partie traite l'incertitude des données. Dans la première partie, une étude approfondie pour la minimisation de la norme zéro a été réalisée tant sur le plan théorique qu'algorithmique. Nous considérons une approximation DC commune de la norme zéro et développons quatre algorithmes basées sur la programmation DC et DCA pour résoudre le problème approché. Nous prouvons que nos algorithmes couvrent tous les algorithmes standards existants dans le domaine. Ensuite, nous étudions le problème de la factorisation en matrices non-négatives (NMF) et fournissons des algorithmes appropriés basés sur la programmation DC et DCA. Nous étudions également le problème de NMF parcimonieuse. Poursuivant cette étude, nous étudions le problème d'apprentissage de dictionnaire où la représentation parcimonieuse joue un rôle crucial. Dans la deuxième partie, nous exploitons la technique d'optimisation robuste pour traiter l'incertitude des données pour les deux problèmes importants dans l'apprentissage : la sélection de variables dans SVM (Support Vector Machines) et le clustering. Différents modèles d'incertitude sont étudiés. Les algorithmes basés sur DCA sont développés pour résoudre ces problèmes. / In this thesis, we focus on developing optimization approaches for solving some classes of optimization problems in sparsity and robust optimization for data uncertainty. Our methods are based on DC (Difference of Convex functions) programming and DCA (DC Algorithms) which are well-known as powerful tools in optimization. This thesis is composed of two parts: the first part concerns with sparsity while the second part deals with uncertainty. In the first part, a unified DC approximation approach to optimization problem involving the zero-norm in objective is thoroughly studied on both theoretical and computational aspects. We consider a common DC approximation of zero-norm that includes all standard sparse inducing penalty functions, and develop general DCA schemes that cover all standard algorithms in the field. Next, the thesis turns to the nonnegative matrix factorization (NMF) problem. We investigate the structure of the considered problem and provide appropriate DCA based algorithms. To enhance the performance of NMF, the sparse NMF formulations are proposed. Continuing this topic, we study the dictionary learning problem where sparse representation plays a crucial role. In the second part, we exploit robust optimization technique to deal with data uncertainty for two important problems in machine learning: feature selection in linear Support Vector Machines and clustering. In this context, individual data point is uncertain but varies in a bounded uncertainty set. Different models (box/spherical/ellipsoidal) related to uncertain data are studied. DCA based algorithms are developed to solve the robust problems

The first part of the thesis studies invariant subspaces of strictly singular operators. By a celebrated result of Aronszajn and Smith, every compact operator has an invariant subspace. There are two classes of operators which are close to compact operators: strictly singular and finitely strictly singular operators. Pelczynski asked whether every strictly singular operator has an invariant subspace. This question was answered by Read in the negative. We answer the same question for finitely strictly singular operators, also in the negative. We also study Schreier singular operators. We show that this subclass of strictly singular operators is closed under multiplication by bounded operators. In addition, we find some sufficient conditions for a product of Schreier singular operators to be compact. The second part studies almost invariant subspaces. A subspace Y of a Banach space is almost invariant under an operator T if TY is a subspace of Y+F for some finite-dimensional subspace F ("error"). Almost invariant subspaces of weighted shift operators are investigated. We also study almost invariant subspaces of algebras of operators. We establish that if an algebra is norm closed then the dimensions of "errors" for the operators in the algebra are uniformly bounded. We obtain that under certain conditions, if an algebra of operators has an almost invariant subspace then it also has an invariant subspace. Also, we study the question of whether an algebra and its closure have the same almost invariant subspaces. The last two parts study collections of positive operators (including positive matrices) and their invariant subspaces. A version of Lomonosov theorem about dual algebras is obtained for collections of positive operators. Properties of indecomposable (i.e., having no common invariant order ideals) semigroups of nonnegative matrices are studied. It is shown that the "smallness" (in various senses) of some entries of matrices in an indecomposable semigroup of positive matrices implies the "smallness" of the entire semigroup. / Mathematics

This thesis considers the application of singular value decomposition (SVD) and semi-nonnegative matrix factorization (SNMF) within feedback control systems, called the SVD System and SNMF System, to control numerous subsystems with a reduced number of control inputs. The subsystems are coupled using a row-column structure to allow mn subsystems to be controlled using m+n inputs. Past techniques for controlling systems in this row-column structure have focused on scheduling procedures that offer limited performance. The SVD and SNMF Systems permit simultaneous control of every subsystem, which increases the convergence rate by an order of magnitude compared with previous methods. In addition to closed loop control, open loop procedures using the SVD and SNMF are compared with previous scheduling procedures, demonstrating significant performance improvements. This thesis presents theoretical results for the controllability of systems using the row-column structure and for the stability and performance of the SVD and SNMF Systems. Practical challenges to the implementation of the SVD and SNMF Systems are also examined. Numerous simulation examples are provided, in particular, a dynamic simulation of a pin array device, called Digital Clay, and two physical demonstrations are used to assess the feasibility of the SVD and SNMF Systems for specific applications.

Raman spectroscopy provides a powerful means of chemical identification in a variety of fields, partly because of its non-contact nature and the speed at which measurements can be taken. The development of powerful, inexpensive lasers and sensitive charge-coupled device (CCD) detectors has led to widespread use of commercial and scientific Raman systems. However, relatively little work has been done developing physics-based probabilistic models for Raman measurement systems and crafting inference algorithms within the framework of statistical estimation and detection theory. The objective of this thesis is to develop algorithms and performance bounds for the identification of chemicals from their Raman spectra. First, a Poisson measurement model based on the physics of a dispersive Raman device is presented. The problem is then expressed as one of deterministic parameter estimation, and several methods are analyzed for computing the maximum-likelihood (ML) estimates of the mixing coefficients under our data model. The performance of these algorithms is compared against the Cramer-Rao lower bound (CRLB). Next, the Raman detection problem is formulated as one of multiple hypothesis detection (MHD), and an approximation to the optimal decision rule is presented. The resulting approximations are related to the minimum description length (MDL) approach to inference. In our simulations, this method is seen to outperform two common general detection approaches, the spectral unmixing approach and the generalized likelihood ratio test (GLRT). The MHD framework is applied naturally to both the detection of individual target chemicals and to the detection of chemicals from a given class. The common, yet vexing, scenario is then considered in which chemicals are present that are not in the known reference library. A novel variation of nonnegative matrix factorization (NMF) is developed to address this problem. Our simulations indicate that this algorithm gives better estimation performance than the standard two-stage NMF approach and the fully supervised approach when there are chemicals present that are not in the library. Finally, estimation algorithms are developed that take into account errors that may be present in the reference library. In particular, an algorithm is presented for ML estimation under a Poisson errors-in-variables (EIV) model. It is shown that this same basic approach can also be applied to the nonnegative total least squares (NNTLS) problem. Most of the techniques developed in this thesis are applicable to other problems in which an object is to be identified by comparing some measurement of it to a library of known constituent signatures.

In light of the complex and highly uncertain nature of dynamical systems requiring controls, it is not surprising that reliable system models for many high performance engineering and life science applications are unavailable. In the face of such high levels of system uncertainty, robust controllers may unnecessarily sacrifice system performance whereas adaptive controllers are clearly appropriate since they can tolerate far greater system uncertainty levels to improve system performance. In this dissertation, we develop a Lyapunov-based direct adaptive and neural adaptive control framework that addresses parametric uncertainty, unstructured uncertainty, disturbance rejection, amplitude and rate saturation constraints, and digital implementation issues. Specifically, we consider the following research topics: direct adaptive control for nonlinear uncertain systems with exogenous disturbances; robust adaptive control for nonlinear uncertain systems; adaptive control for nonlinear uncertain systems with actuator amplitude and rate saturation constraints; adaptive reduced-order dynamic compensation for nonlinear uncertain systems; direct adaptive control for nonlinear matrix second-order dynamical systems with state-dependent uncertainty; adaptive control for nonnegative and compartmental dynamical systems with applications to general anesthesia; direct adaptive control of nonnegative and compartmental dynamical systems with time delay; adaptive control for nonlinear nonnegative and compartmental dynamical systems with applications to clinical pharmacology; neural network adaptive control for nonlinear nonnegative dynamical systems; passivity-based neural network adaptive output feedback control for nonlinear nonnegative dynamical systems; neural network adaptive dynamic output feedback control for nonlinear nonnegative systems using tapped delay memory units; Lyapunov-based adaptive control framework for discrete-time nonlinear systems with exogenous disturbances; direct discrete-time adaptive control with guaranteed parameter error convergence; and hybrid adaptive control for nonlinear uncertain impulsive dynamical systems.

In this paper, we will review two fundamental counting methods: inclusionexclusion and pigeonhole principles. The inclusion-exclusion principle considers the elements of the sets satisfied some conditions, and avoids repeat counting by disjoint sets. We also use the inclusion-exclusion principle to solve the problems of Euler phi function and the number of onto functions in number theory, and derangement and the number of nonnegative integer solutions of equations in combinatorics. We derive the closed-form formula to those problems. For the forbidden positions problems, we use the rook polynomials to simplify the counting process. We also show the form of the inclusion-exclusion principle in probability, and use it to solve some probability problems. The pigeonhole principle is an easy concept. We can establish some sets and use the pigeonhole principle to discuss the extreme value about the number of elements. Choose the pigeons and pigeonholes, properly, and solve problems by the concept of the pigeonhole principle. We also introduce the Ramsey theorem which is an important application of the pigeonhole principle. This theorem provides a method to solve problems by complete graph. Finally, we give some contest problems about the inclusion-exclusion and pigeonhole principles to show how those principles are used.

Supervised learning tasks like building a classifier, estimating the error rate of the predictors, are typically performed with labeled data. In most cases, obtaining labeled data is costly as it requires manual labeling. On the other hand, unlabeled data is available in abundance. In this thesis, we discuss methods to perform supervised learning tasks with no labeled data. We prove consistency of the proposed methods and demonstrate its applicability with synthetic and real world experiments. In some cases, small quantities of labeled data maybe easily available and supplemented with large quantities of unlabeled data (semi-supervised learning). We derive the asymptotic efficiency of generative models for semi-supervised learning and quantify the effect of labeled and unlabeled data on the quality of the estimate. Another independent track of the thesis is efficient computational methods for nonnegative tensor factorization (NTF). NTF provides the user with rich modeling capabilities but it comes with an added computational cost. We provide a fast algorithm for performing NTF using a modified active set method called block principle pivoting method and demonstrate its applicability to social network analysis and text mining.

Reducing interference noise in a noisy speech recording has been a challenging task for many years yet has a variety of applications, for example, in handsfree mobile communications, in speech recognition, and in hearing aids. Traditional single-channel noise reduction schemes, such as Wiener filtering, do not work satisfactorily in the presence of non-stationary background noise. Alternatively, supervised approaches, where the noise type is known in advance, lead to higher-quality enhanced speech signals. This dissertation proposes supervised and unsupervised single-channel noise reduction algorithms. We consider two classes of methods for this purpose: approaches based on nonnegative matrix factorization (NMF) and methods based on hidden Markov models (HMM).  The contributions of this dissertation can be divided into three main (overlapping) parts. First, we propose NMF-based enhancement approaches that use temporal dependencies of the speech signals. In a standard NMF, the important temporal correlations between consecutive short-time frames are ignored. We propose both continuous and discrete state-space nonnegative dynamical models. These approaches are used to describe the dynamics of the NMF coefficients or activations. We derive optimal minimum mean squared error (MMSE) or linear MMSE estimates of the speech signal using the probabilistic formulations of NMF. Our experiments show that using temporal dynamics in the NMF-based denoising systems improves the performance greatly. Additionally, this dissertation proposes an approach to learn the noise basis matrix online from the noisy observations. This relaxes the assumption of an a-priori specified noise type and enables us to use the NMF-based denoising method in an unsupervised manner. Our experiments show that the proposed approach with online noise basis learning considerably outperforms state-of-the-art methods in different noise conditions.  Second, this thesis proposes two methods for NMF-based separation of sources with similar dictionaries. We suggest a nonnegative HMM (NHMM) for babble noise that is derived from a speech HMM. In this approach, speech and babble signals share the same basis vectors, whereas the activation of the basis vectors are different for the two signals over time. We derive an MMSE estimator for the clean speech signal using the proposed NHMM. The objective evaluations and performed subjective listening test show that the proposed babble model and the final noise reduction algorithm outperform the conventional methods noticeably. Moreover, the dissertation proposes another solution to separate a desired source from a mixture with arbitrarily low artifacts.  Third, an HMM-based algorithm to enhance the speech spectra using super-Gaussian priors is proposed. Our experiments show that speech discrete Fourier transform (DFT) coefficients have super-Gaussian rather than Gaussian distributions even if we limit the speech data to come from a specific phoneme. We derive a new MMSE estimator for the speech spectra that uses super-Gaussian priors. The results of our evaluations using the developed noise reduction algorithm support the super-Gaussianity hypothesis. / <p>QC 20130916</p>

No description available.