L’apprentissage par transfert de réseaux profonds réduit considérablement les coûts en temps de calcul et en données du processus d’entraînement des réseaux et améliore largement les performances de la tâche cible par rapport à l’apprentissage à partir de zéro. Cependant, l’apprentissage par transfert d’un réseau profond peut provoquer un oubli des connaissances acquises lors de l’apprentissage de la tâche source. Puisque l’efficacité de l’apprentissage par transfert vient des connaissances acquises sur la tâche source, ces connaissances doivent être préservées pendant le transfert. Cette thèse résout ce problème d’oubli en proposant deux schémas de régularisation préservant les connaissances pendant l’apprentissage par transfert. Nous examinons d’abord plusieurs formes de régularisation des paramètres qui favorisent toutes explicitement la similarité de la solution finale avec le modèle initial, par exemple, L1, L2, et Group-Lasso. Nous proposons également les variantes qui utilisent l’information de Fisher comme métrique pour mesurer l’importance des paramètres. Nous validons ces approches de régularisation des paramètres sur différentes tâches de segmentation sémantique d’image ou de calcul de flot optique. Le second schéma de régularisation est basé sur la théorie du transport optimal qui permet d’estimer la dissimilarité entre deux distributions. Nous nous appuyons sur la théorie du transport optimal pour pénaliser les déviations des représentations de haut niveau entre la tâche source et la tâche cible, avec le même objectif de préserver les connaissances pendant l’apprentissage par transfert. Au prix d’une légère augmentation du temps de calcul pendant l’apprentissage, cette nouvelle approche de régularisation améliore les performances des tâches cibles et offre une plus grande précision dans les tâches de classification d’images par rapport aux approches de régularisation des paramètres. / Transfer learning with deep convolutional neural networks significantly reduces the computation and data overhead of the training process and boosts the performance on the target task, compared to training from scratch. However, transfer learning with a deep network may cause the model to forget the knowledge acquired when learning the source task, leading to the so-called catastrophic forgetting. Since the efficiency of transfer learning derives from the knowledge acquired on the source task, this knowledge should be preserved during transfer. This thesis solves this problem of forgetting by proposing two regularization schemes that preserve the knowledge during transfer. First we investigate several forms of parameter regularization, all of which explicitly promote the similarity of the final solution with the initial model, based on the L1, L2, and Group-Lasso penalties. We also propose the variants that use Fisher information as a metric for measuring the importance of parameters. We validate these parameter regularization approaches on various tasks. The second regularization scheme is based on the theory of optimal transport, which enables to estimate the dissimilarity between two distributions. We benefit from optimal transport to penalize the deviations of high-level representations between the source and target task, with the same objective of preserving knowledge during transfer learning. With a mild increase in computation time during training, this novel regularization approach improves the performance of the target tasks, and yields higher accuracy on image classification tasks compared to parameter regularization approaches.

The topic of force reconstruction has been studied quite extensively but most of the existing research work that has been done are in the domain of structural and civil engineering construction like bridges and beams. Considerable work in force reconstruction has also being done in fabrication of machines and structures like aircrafts, gear boxes etc. The topic of force reconstruction of the cutting forces during a machining process like turning or milling machines is a recent line of research to suffice the requirement of proactive monitoring of forces generated during the operation of the machine tool. The forces causing vibrations while machining if detected and monitored can enhance system productivity and efficiency of the process. The objective of this study was to investigate the algorithms available in literature for inverse force reconstruction and apply for reconstruction of cutting forces while machining on a computer numerically controlled (CNC) machine. This study has applied inverse force reconstruction technique algorithms 1) Deconvolution method, 2) Kalman filter recursive least square and 3) augmented Kalman filter for inverse reconstruction of forces for multi degree of freedom systems. Results from experiments conducted as part of this thesis work shows the effectiveness of the methods of force reconstruction to monitor the forces generated during the machining process on machine tools in real time without employing dynamometers which are expensive and complex to set-up. This study for developing a cost-effective method of force reconstruction will be instrumental in applications for improving machining efficiency and proactive preventive maintenance. / Graduate

L’opérateur du p-Laplacien non local, l’équation d’évolution et la régularisation variationnelle associées régies par un noyau donné ont des applications dans divers domaines de la science et de l’ingénierie. En particulier, ils sont devenus des outils modernes pour le traitement massif des données (y compris les signaux, les images, la géométrie) et dans les tâches d’apprentissage automatique telles que la classification. En pratique, cependant, ces modèles sont implémentés sous forme discrète (en espace et en temps, ou en espace pour la régularisation variationnelle) comme approximation numérique d’un problème continu, où le noyau est remplacé par la matrice d’adjacence d’un graphe. Pourtant, peu de résultats sur la consistence de ces discrétisations sont disponibles. En particulier, il est largement ouvert de déterminer quand les solutions de l’équation d’évolution ou du problème variationnel des tâches basées sur des graphes convergent (dans un sens approprié) à mesure que le nombre de sommets augmente, vers un objet bien défini dans le domaine continu, et si oui, à quelle vitesse. Dans ce manuscrit, nous posons les bases pour aborder ces questions.En combinant des outils de la théorie des graphes, de l’analyse convexe, de la théorie des semi- groupes non linéaires et des équations d’évolution, nous interprétons rigoureusement la limite continue du problème d’évolution et du problème variationnel du p-Laplacien discrets sur graphes. Plus précisé- ment, nous considérons une suite de graphes (déterministes) convergeant vers un objet connu sous le nom de graphon. Si les problèmes d’évolution et variationnel associés au p-Laplacien continu non local sont discrétisés de manière appropriée sur cette suite de graphes, nous montrons que la suite des solutions des problèmes discrets converge vers la solution du problème continu régi par le graphon, lorsque le nombre de sommets tend vers l’infini. Ce faisant, nous fournissons des bornes d’erreur/consistance.Cela permet à son tour d’établir les taux de convergence pour différents modèles de graphes. En parti- culier, nous mettons en exergue le rôle de la géométrie/régularité des graphons. Pour les séquences de graphes aléatoires, en utilisant des inégalités de déviation (concentration), nous fournissons des taux de convergence nonasymptotiques en probabilité et présentons les différents régimes en fonction de p, de la régularité du graphon et des données initiales. / The non-local p-Laplacian operator, the associated evolution equation and variational regularization, governed by a given kernel, have applications in various areas of science and engineering. In particular, they are modern tools for massive data processing (including signals, images, geometry), and machine learning tasks such as classification. In practice, however, these models are implemented in discrete form (in space and time, or in space for variational regularization) as a numerical approximation to a continuous problem, where the kernel is replaced by an adjacency matrix of a graph. Yet, few results on the consistency of these discretization are available. In particular it is largely open to determine when do the solutions of either the evolution equation or the variational problem of graph-based tasks converge (in an appropriate sense), as the number of vertices increases, to a well-defined object in the continuum setting, and if yes, at which rate. In this manuscript, we lay the foundations to address these questions.Combining tools from graph theory, convex analysis, nonlinear semigroup theory and evolution equa- tions, we give a rigorous interpretation to the continuous limit of the discrete nonlocal p-Laplacian evolution and variational problems on graphs. More specifically, we consider a sequence of (determin- istic) graphs converging to a so-called limit object known as the graphon. If the continuous p-Laplacian evolution and variational problems are properly discretized on this graph sequence, we prove that the solutions of the sequence of discrete problems converge to the solution of the continuous problem governed by the graphon, as the number of graph vertices grows to infinity. Along the way, we provide a consistency/error bounds. In turn, this allows to establish the convergence rates for different graph models. In particular, we highlight the role of the graphon geometry/regularity. For random graph se- quences, using sharp deviation inequalities, we deliver nonasymptotic convergence rates in probability and exhibit the different regimes depending on p, the regularity of the graphon and the initial data.

No description available.

In this paper, we consider the inverse problem of calibrating a generalization of the bivariate Ornstein-Uhlenbeck model introduced by Lo and Wang. Even though the generalized Black-Scholes option pricing formula still holds, option prices change in comparison to the classical Black-Scholes model. The time-dependent volatility function and the other (real-valued) parameters in the model are calibrated simultaneously from option price data and from some empirical moments of the logarithmic returns. This gives an ill-posed inverse problem, which requires a regularization approach. Applying the theory of Engl, Hanke and Neubauer concerning Tikhonov regularization we show convergence of the regularized solution to the true data and study the form of source conditions which ensure convergence rates.

The aim of this thesis is to investigate applicability of regulariza- tion by Krylov subspace methods to discrete inverse problems arising in single particle analysis (SPA). We start with a smooth model formulation and describe its discretization, yielding an ill-posed inverse problem Ax ≈ b, where A is a lin- ear operator and b represents the measured noisy data. We provide theoretical background and overview of selected methods for the solution of general linear inverse problems. Then we focus on specific properties of inverse problems from SPA, and provide experimental analysis based on synthetically generated SPA datasets (experiments are performed in the Matlab enviroment). Turning to the solution of our inverse problem, we investigate in particular an approach based on iterative Hybrid LSQR with inner Tikhonov regularization. A reliable stopping criterion for the iterative part as well as parameter-choice method for the inner regularization are discussed. Providing a complete implementation of the proposed solver (in Matlab and in C++), its performance is evaluated on various SPA model datasets, considering high levels of noise and realistic distri- bution of orientations of scanning angles. Comparison to other regularization methods, including the ART method traditionally used in SPA,...

The purpose of this thesis is to study how to overcome difficulties one typically encounters when solving non-negative inverse problems by standard Krylov subspace methods. We first give a theoretical background to the non-negative inverse problems. Then we concentrate on selected modifications of Krylov subspace methods known to improve the solution significantly. We describe their properties, provide their implementation and propose an improvement for one of them. After that, numerical experiments are presented giving a comparison of the methods and analyzing the influence of the present parameters on the behavior of the solvers. It is clearly demonstrated, that the methods imposing nonnegativity perform better than the unconstrained methods. Moreover, our improvement leads in some cases to a certain reduction of the number of iterations and consequently to savings of the computational time while preserving a good quality of the approximation.

The Growth Curve Model is a bilinear statistical model which can be used to analyse several groups of repeated measurements. Normally the Growth Curve Model is defined in such a way that the permitted sampling frequency of the repeated measurement is limited by the number of observed individuals in the data set.In this thesis, we examine the possibilities of utilizing highly frequently sampled measurements to increase classification accuracy for real world data. That is, we look at the case where the regular Growth Curve Model is not defined due to the relationship between the sampling frequency and the number of observed individuals. When working with this high frequency data, we develop a new method of basis selection for the regression analysis which yields what we call a Bilinear Gaussian Radial Basis Function Network (BGRBFN), which we then compare to more conventional polynomial and trigonometrical functional bases. Finally, we examine if Tikhonov regularization can be used to further increase the classification accuracy in the high frequency data case.Our findings suggest that the BGRBFN performs better than the conventional methods in both classification accuracy and functional approximability. The results also suggest that both high frequency data and furthermore Tikhonov regularization can be used to increase classification accuracy.

Indiana University-Purdue University Indianapolis (IUPUI) / The Convolutional Neural Network (CNN) is a mathematical model designed to distill input information into a more useful representation. This distillation process removes information over time through a series of dimensionality reductions, which ultimately, grant the model the ability to resist noise, and generalize effectively. However, CNNs often contain elements that are ineffective at contributing towards useful representations. This Thesis aims at providing a remedy for this problem by introducing Absolute Cosine Value Regularization (ACVR). This is a regularization technique hypothesized to increase the representational power of CNNs by using a Gradient Descent Orthogonalization algorithm to force the vectors that constitute their filters at any given convolutional layer to occupy unique positions in in their respective spaces. This method should in theory, lead to a more effective balance between information loss and representational power, ultimately, increasing network performance. The following Thesis proposes and examines the mathematics and intuition behind ACVR, and goes on to propose Dynamic-ACVR (D-ACVR). This Thesis also proposes and examines the effects of ACVR on the filters of a low-dimensional CNN, as well as the effects of ACVR and D-ACVR on traditional Convolutional filters in VGG-19. Finally, this Thesis proposes and examines regularization of the Pointwise filters in MobileNetv1.

[pt] Bandas de cisalhamento ocorrem quando há a localização de deformações inelásticas provenientes de esforços cisalhantes em regiões estreitas de um material. Estas estruturas podem influenciar diretamente nas propriedades dos materiais, além de afetar sua integridade e contribuir para o início de falhas estruturais. Este trabalho apresenta uma metodologia para a caracterização das bandas de cisalhamento na rocha carbonática Indiana Limestone por meio de modelagens numéricas utilizando o método dos elementos finitos (MEF). Ao modelar o fenômeno de localização de deformações, o MEF apresenta algumas limitações como perda da elipticidade das equações governantes, produzindo problemas de convergência e resultados dependentes da discretização de malha. Algumas alternativas para superar estes inconvenientes são apresentadas e discutidas, com especial enfase dada à técnica de regularização viscosa utilizada nas modelagens numericas de ensaios biaxiais e triaxiais. Estudos parametricos e de sensibilidade foram conduzidos para identificar o impacto das propriedades mecânicas na ocorrencia das bandas de cisalhamento. Os resultados mostraram que as propriedades de resistência, o uso de leis de fluxo não associadas e o amolecimento por deformação são os fatores que mais influenciam na iniciação e desenvolvimento das bandas de cisalhamento. / [en] Shear bands occur when inelastic shear deformation localize in narrow regions of the material. These structures can directly influence the properties of materials, in addition to affecting their integrity and contributing to the initiation of structural failures. This study presents a methodology for the characterization of shear bands in Indiana Limestone carbonate rock through numerical modeling using the finite element method (FEM). As it is known, the numerical modeling of strain localization phenomena using FEM has some drawbacks, such as loss of ellipticity of the governing equations, triggering convergence problems and results dependent on the mesh discretization. Some alternatives to overcome these problems are presented and discussed, giving a special emphasis to the viscous regularization technique used in the numerical modeling of biaxial and triaxial tests. Parametric and sensitivity studies were performed to identify the impact of the mechanical properties on the occurrence of shear bands. The results showed that strength properties, non associative flow rules and strain-softening are the factors with larger influence on the initiation and development of shear bands.