We consider the constrained convex minimization problem min x2C f(x) we will present gradient projection method which generates a sequence fxkg according to the formula xk+1 = PC(xk

Compressed Sensing (CS) is an emerging paradigm in which signals are recovered from undersampled nonadaptive linear measurements taken at a rate proportional to the signal's true information content as opposed to its ambient dimension. The resulting problem consists in finding a sparse solution to an underdetermined system of linear equations. It has now been established, both theoretically and empirically, that certain optimization algorithms are able to solve such problems. Iterative Hard Thresholding (IHT) (Blumensath and Davies, 2007), which is the focus of this thesis, is an established CS recovery algorithm which is known to be effective in practice, both in terms of recovery performance and computational efficiency. However, theoretical analysis of IHT to date suffers from two drawbacks: state-of-the-art worst-case recovery conditions have not yet been quantified in terms of the sparsity/undersampling trade-off, and also there is a need for average-case analysis in order to understand the behaviour of the algorithm in practice. In this thesis, we present a new recovery analysis of IHT, which considers the fixed points of the algorithm. In the context of arbitrary matrices, we derive a condition guaranteeing convergence of IHT to a fixed point, and a condition guaranteeing that all fixed points are 'close' to the underlying signal. If both conditions are satisfied, signal recovery is therefore guaranteed. Next, we analyse these conditions in the case of Gaussian measurement matrices, exploiting the realistic average-case assumption that the underlying signal and measurement matrix are independent. We obtain asymptotic phase transitions in a proportional-dimensional framework, quantifying the sparsity/undersampling trade-off for which recovery is guaranteed. By generalizing the notion of xed points, we extend our analysis to the variable stepsize Normalised IHT (NIHT) (Blumensath and Davies, 2010). For both stepsize schemes, comparison with previous results within this framework shows a substantial quantitative improvement. We also extend our analysis to a related algorithm which exploits the assumption that the underlying signal exhibits tree-structured sparsity in a wavelet basis (Baraniuk et al., 2010). We obtain recovery conditions for Gaussian matrices in a simplified proportional-dimensional asymptotic, deriving bounds on the oversampling rate relative to the sparsity for which recovery is guaranteed. Our results, which are the first in the phase transition framework for tree-based CS, show a further significant improvement over results for the standard sparsity model. We also propose a dynamic programming algorithm which is guaranteed to compute an exact tree projection in low-order polynomial time.

We consider the constrained convex minimization problem: min_x∈C f(x) we will present gradient projection method which generates a sequence x^k according to the formula x^(k+1) = P_c(x^k − £\_k∇f(x^k)), k= 0, 1, ¡P ¡P ¡P , our ideal is rewritten the formula as a xed point algorithm: x^(k+1) = T_(£\k)x^k, k = 0, 1, ¡P ¡P ¡P is used to solve the minimization problem. In this paper, we present the gradient projection method(GPM) and different choices of the stepsize to discuss the convergence of gradient projection method which converge to a solution of the concerned problem.

Support vector regression has been applied to stock market forecasting problems. However, it is usually needed to tune manually the hyperparameters of the kernel functions. Multiple-kernel learning was developed to deal with this problem, by which the kernel matrix weights and Lagrange multipliers can be simultaneously derived through semidefinite programming. However, the amount of time and space required is very demanding. We develop a two-stage multiple-kernel learning algorithm by incorporating sequential minimal optimization and the gradient projection method. By this algorithm, advantages from different hyperparameter settings can be combined and overall system performance can be improved. Besides, the user need not specify the hyperparameter settings in advance, and trial-and-error for determining appropriate hyperparameter settings can then be avoided. Experimental results, obtained by running on datasets taken from Taiwan Capitalization Weighted Stock Index, show that our method performs better than other methods.

Made available in DSpace on 2014-07-29T16:02:22Z (GMT). No. of bitstreams: 1 Dissertacao kelvin.pdf: 379268 bytes, checksum: 69875c577ac81dd2f77bb73f65c9f683 (MD5) Previous issue date: 2009-12-16 / The Convergence theory of gradient method and gradient projection method, for minimization of continuously differentiable generalized convex functions, that is, pseudoconvex functions and quasiconvex functions is studied in this work. We shall see that under certain conditions the gradient method, as well as gradient projection method, generate a convergent sequence and the limit point is a minimizing, whenever the function has minimizing and is pseudoconvex functions. If the objective function is quasiconvex then the generated sequence converges to a stationary point whenever that point exists. / Neste trabalho trataremos da convergência do método do gradiente para minimizar funções continuamente diferenciáveis e convexas-generalizadas, isto é, pseudo-convexas ou quase-convexas. Veremos que sob certas condições o método do gradiente, assim como o método do gradiente projetado, gera uma sequência que converge para minimizador quando existe um e a função objetivo é pseudo-convexa. Quando a função objetivo é quase-convexa a sequência gerada converge para um ponto estacionário do problema quando existe um tal ponto.

Les robots industriels ont évolué fondamentalement ces dernières années pour répondre aux exigences industrielles de machines et mécanismes toujours plus performants. Ceci se traduit aujourd’hui par de nouveaux robots anthropomorphes plus adaptés laissant entrevoir la réalisation de tâches plus complexes comme la découpe d’objets déformables telle que la découpe de viande ou soumis à de fortes sollicitations comme l’usinage. L’étude du comportement des robots anthropomorphes, à structures parallèles ou hybrides montre une anisotropie aussi bien cinématique, que dynamique, impactant la précision attendue. Ces travaux de thèse étudient l’intégration des redondances cinématiques qui permettent de pallier en partie ce problème en positionnant au mieux la tâche à réaliser dans un espace de travail compatible avec les capacités attendues. Ces travaux ont suivi une démarche en trois étapes : la modélisation analytique de cellules robotiques par équivalent sériel basée sur la méthode TCS, la formalisation des contraintes des processus de découpe de viande et d’usinage et une résolution par optimisation multicritère. Une première originalité de ces travaux réside en le développement d’un modèle à 6 degrés de liberté permettant d’analyser les gestes de l’opérateur qui optimise naturellement le comportement de son bras pour garantir la tâche qu’il réalise. La seconde originalité concerne le placement optimisé des redondances structurales (cellules à 9 ddls) où les paramètres de positionnement sont incorporés comme des variables pilotables (cellule à 11 ddls). Ainsi, ces travaux de thèse apportent des contributions à : - la définition de critères adaptés à la réalisation de tâches complexes et sollicitantes pour la gestion des redondances cinématiques ; - l’identification du comportement des structures sous sollicitations par moyen métrologique (Laser tracker) et l’auto-adaptation des trajectoires par l’utilisation d’une commande en effort industrielle ; - l’optimisation du comportement permettant l’amélioration de la qualité de réalisation des différents processus de coupe (découpe de viande et usinage). / Industrial robots have evolved fundamentally in recent years to reach the industrial requirements. We now find more suitable anthropomorphic robots leading to the realization of more complex tasks like deformable objects cutting such as meat cutting or constrained to high stresses as machining. The behavior study of anthropomorphic robots, parallel or hybrid one highlights a kinematic and dynamic anisotropy, which impacts the expected accuracy. This thesis studied the integration of the kinematic redundancy that can partially overcome this problem by well setting the task to achieve it in a space compatible with the expected capacity. This work followed a three-step approach: analytical modeling of robotic cells by serial equivalent based on the TCS method, formalizing the constraints of meat cutting process and machining process and a multicriteria optimization.The first originality of this work focuses on the development of a 6 DoFs model to analyze the operator actions who naturally optimizes his arm behavior to ensure the task it performs. The second originality concerns the optimized placement of structural redundancy (9 DoFs robotic cell) where positioning parameters are incorporated as controllable variables (11 DoFs robotic cell). Thus, the thesis makes contributions to : - the definition of criteria adapted to the realization of complex and under high stress task for the management of the kinematic redundancy; - the structural behavior identification, under stress, by metrology tools (Laser tracker ) and the self- adaptation paths by using an industrial force control; - the behavior optimization to improve the cutting process quality (meat cutting and machining).

Les robots industriels ont évolué fondamentalement ces dernières années pour répondre aux exigences industrielles de machines et mécanismes toujours plus performants. Ceci se traduit aujourd’hui par de nouveaux robots anthropomorphes plus adaptés laissant entrevoir la réalisation de tâches plus complexes et soumis à de fortes sollicitations comme durant l’usinage. L’étude du comportement des robots anthropomorphes, à structures parallèles ou hybrides montre une anisotropie aussi bien cinématique, que dynamique, impactant la précision attendue. Ces travaux de thèse étudient l’intégration des redondances cinématiques qui permettent de pallier en partie ce problème en positionnant au mieux la tâche à réaliser dans un espace de travail compatible avec les capacités attendues. Ces travaux ont permis d’améliorer notre outil d’optimisation et de le tester à la fois sur un modèle Eléments Finis du robot et sur le robot réel. Ainsi, ces travaux de thèse apportent des contributions à : - la définition de critères adaptés à la réalisation de tâches complexes et sollicitantes pour la gestion des redondances cinématiques ; - l’identification du comportement des structures sous sollicitations par moyen métrologique (Laser tracker) ; - l’optimisation du comportement permettant l’amélioration de la qualité de réalisation des opérations d’usinage ; - la modélisation Eléments Finis des robots prenant en compte l’identification des rigidités des corps et articulaires. / Industrial robots have evolved fundamentally in recent years to reach the industrial requirements. We now find more suitable anthropomorphic robots leading to the realization of more complex tasks like deformable objects cutting such as meat cutting or constrained to high loading like during machining. The behavior study of anthropomorphic robots, parallel or hybrid one highlights a kinematic and dynamic anisotropy, which impacts the expected accuracy.This thesis studied the integration of the kinematic redundancy that can partially overcome this problem by well setting the task to achieve it in a space compatible with the expected capacity.This work helped us to improve our optimization tool and to try it on both FE model of the robot and real robot.Thus, the thesis makes contributions to: - the definition of criteria adapted to the realization of complex and under high loading task for the management of the kinematic redundancy; - the structural behavior identification, under loading, by metrology tools (Laser tracker) ; - the behavior optimization to improve the cutting process quality during machining ; - robots finite elements modeling using stiffness identification for both bodies and joints.

Made available in DSpace on 2014-07-29T16:02:19Z (GMT). No. of bitstreams: 1 Dissertacao Milton Gabriel Garcia dos Santos.pdf: 405990 bytes, checksum: b1b10db3be6011cbbae70bc35ed87950 (MD5) Previous issue date: 2011-10-27 / This work we study the convergence properties of the Gradient Method Designed and Descent Method for Multi-objective optimization. At first, our optimization problem is to minimize a real function of n-variables, continuously differentiable and restricted to a set of simple structure and add on the objective function of the hypothesis of pseudo-convexity or quasi-convexity. Then we consider the problem of unconstrained multi-objective optimization and add some hypotheses about the function vector, such as convexity or quasi-convexity, and is continuously differentiable. It is noteworthy that in both problems will be used to search for inexact Armijo over viable directions. / Neste trabalho faremos um estudo das propriedades de convergência do Método do Gradiente Projetado e do Método de Descida para otimização Multi-objetivo. No primeiro momento, o nosso problema de otimização será o de minimizar uma função real de nvariáveis, continuamente diferenciável e restrita a um conjunto de estrutura simples e acrescentaremos sobre a função objetivo a hipótese de quase-convexidade ou pseudoconvexidade. Em seguida iremos considerar o problema de otimização Multi-Objetivo irrestrito e adicionar algumas hipóteses sobre a função vetorial, como a convexidade ou quase-convexidade, além de ser continuamente diferenciável. É importante salientar que em ambos os problemas será utilizado a busca inexata de armijo ao longo de direções viáveis.

Exploiting signal sparsity has recently received considerable attention in a variety of areas including signal and image processing, compressive sensing, machine learning and so on. Many of these applications involve optimization models that are regularized by certain sparsity-promoting metrics. Two most popular regularizers are based on the l1 norm that approximates sparsity of vectorized signals and the total variation (TV) norm that serves as a measure of gradient sparsity of an image. Nevertheless, the l1 and TV terms are merely two representative measures of sparsity. To explore the matter of sparsity further, in this thesis we investigate relaxations of the regularizers to nonconvex terms such as lp and TVp "norms" with 0 <= p < 1. The contributions of the thesis are two-fold. First, several methods to approach globally optimal solutions of related nonconvex problems for improved signal/image reconstruction quality have been proposed. Most algorithms studied in the thesis fall into the category of iterative reweighting schemes for which nonconvex problems are reduced to a series of convex sub-problems. In this regard, the second main contribution of this thesis has to do with complexity improvement of the l1/TV-regularized methodology for which accelerated algorithms are developed. Along with these investigations, new techniques are proposed to address practical implementation issues. These include the development of an lp-related solver that is easily parallelizable, and a matrix-based analysis that facilitates implementation for TV-related optimizations. Computer simulations are presented to demonstrate merits of the proposed models and algorithms as well as their applications for solving general linear inverse problems in the area of signal and image denoising, signal sparse representation, compressive sensing, and compressive imaging. / Graduate

L’apprentissage adaptatif distribué sur les réseaux permet à un ensemble d’agents de résoudre des problèmes d’estimation de paramètres en ligne en se basant sur des calculs locaux et sur des échanges locaux avec les voisins immédiats. La littérature sur l’estimation distribuée considère essentiellement les problèmes à simple tâche, où les agents disposant de fonctions objectives séparables doivent converger vers un vecteur de paramètres commun. Cependant, dans de nombreuses applications nécessitant des modèles plus complexes et des algorithmes plus flexibles, les agents ont besoin d’estimer et de suivre plusieurs vecteurs de paramètres simultanément. Nous appelons ce type de réseau, où les agents doivent estimer plusieurs vecteurs de paramètres, réseau multitâche. Bien que les agents puissent avoir différentes tâches à résoudre, ils peuvent capitaliser sur le transfert inductif entre eux afin d’améliorer les performances de leurs estimés. Le but de cette thèse est de proposer et d’étudier de nouveaux algorithmes d’estimation distribuée sur les réseaux multitâches. Dans un premier temps, nous présentons l’algorithme diffusion LMS qui est une stratégie efficace pour résoudre les problèmes d’estimation à simple-tâche et nous étudions théoriquement ses performances lorsqu’il est mis en oeuvre dans un environnement multitâche et que les communications entre les noeuds sont bruitées. Ensuite, nous présentons une stratégie de clustering non-supervisé permettant de regrouper les noeuds réalisant une même tâche en clusters, et de restreindre les échanges d’information aux seuls noeuds d’un même cluster / Distributed adaptive learning allows a collection of interconnected agents to perform parameterestimation tasks from streaming data by relying solely on local computations and interactions with immediate neighbors. Most prior literature on distributed inference is concerned with single-task problems, where agents with separable objective functions need to agree on a common parameter vector. However, many network applications require more complex models and flexible algorithms than single-task implementations since their agents involve the need to estimate and track multiple objectives simultaneously. Networks of this kind, where agents need to infer multiple parameter vectors, are referred to as multitask networks. Although agents may generally have distinct though related tasks to perform, they may still be able to capitalize on inductive transfer between them to improve their estimation accuracy. This thesis is intended to bring forth advances on distributed inference over multitask networks. First, we present the well-known diffusion LMS strategies to solve single-task estimation problems and we assess their performance when they are run in multitask environments in the presence of noisy communication links. An improved strategy allowing the agents to adapt their cooperation to neighbors sharing the same objective is presented in order to attain improved learningand estimation over networks. Next, we consider the multitask diffusion LMS strategy which has been proposed to solve multitask estimation problems where the network is decomposed into clusters of agents seeking different