Electromagnetic (EM) imaging schemes are inherently non-linear and ill-posed. Albeit there exist remedies to these fundamental problems, more efficient solutions are still being sought. To this end, in this thesis, the non-linearity is tackled in- corporating a multitude of techniques (ranging from Born approximation (linear), inexact Newton (linearized) to complete nonlinear iterative Landweber schemes) that can account for weak to strong scattering problems. The ill-posedness of the EM inverse scattering problem is circumvented by formulating the above methods into a minimization problem with a sparsity constraint. More specifically, four novel in- verse scattering schemes are formulated and implemented. (i) A greedy algorithm is used together with a simple artificial neural network (ANN) for efficient and accu- rate EM imaging of weak scatterers. The ANN is used to predict the sparsity level of the investigation domain which is then used as the L0 - constraint parameter for the greedy algorithm. (ii) An inexact Newton scheme that enforces the sparsity con- straint on the derivative of the unknown material properties (not necessarily sparse) is proposed. The inverse scattering problem is formulated as a nonlinear function of the derivative of the material properties. This approach results in significant spar- sification where any sparsity regularization method could be efficiently applied. (iii) A sparsity regularized nonlinear contrast source (CS) framework is developed to di- rectly solve the nonlinear minimization problem using Landweber iterations where the convergence is accelerated using a self-adaptive projected accelerated steepest descent algorithm. (iv) A 2.5D finite difference frequency domain (FDFD) based in- verse scattering scheme is developed for imaging scatterers embedded in lossy and inhomogeneous media. The FDFD based inversion algorithm does not require the Green’s function of the background medium and appears a promising technique for biomedical and subsurface imaging with a reasonable computational time. Numerical experiments, which are carried out using synthetically generated mea- surements, show that the images recovered by these sparsity-regularized methods are sharper and more accurate than those produced by existing methods. The methods developed in this work have potential application areas ranging from oil/gas reservoir engineering to biological imaging where sparse domains naturally exist.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第22808号 / 情博第738号 / 新制||情||126(附属図書館) / 京都大学大学院情報学研究科システム科学専攻 / (主査)教授 大塚 敏之, 教授 加納 学, 教授 太田 快人 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

京都大学 / 新制・課程博士 / 博士(情報学) / 甲第23310号 / 情博第746号 / 新制||情||127(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)教授 鹿島 久嗣, 教授 田中 利幸, 教授 山下 信雄 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

付記する学位プログラム名: 京都大学卓越大学院プログラム「先端光・電子デバイス創成学」 / 京都大学 / 新制・課程博士 / 博士(情報学) / 甲第23325号 / 情博第761号 / 新制||情||130(附属図書館) / 京都大学大学院情報学研究科通信情報システム専攻 / (主査)教授 小野寺 秀俊, 教授 大木 英司, 教授 佐藤 高史 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

L'imagerie électromagnétique est le problème de la détermination de la distribution de matériaux à partir de champs diffractés mesurés venant du domaine les contenant et sous investigation. Résoudre ce problème inverse est une tâche difficile car il est mal posé en raison de la présence d'opérateurs intégraux (de lissage) utilisés dans la représentation des champs diffractés en terme de propriétés des matériaux, et ces champs sont obtenus à un ensemble fini et non nécessairement optimal de points via des mesures bruitées. En outre, le problème inverse est non linéaire simplement en raison du fait que les champs diffractés sont des fonctions non linéaires des propriétés des matériaux. Le travail décrit traite du caractère mal posé de ce problème d'imagerie électromagnétique en utilisant des techniques de régularisation basées sur la parcimonie, qui supposent que le(s) diffracteurs(s) ne capture(nt) de fait qu'une petite fraction du domaine d'investigation. L'objectif principal est d'étudier de manière approfondie la régularisation de parcimonie pour les problèmes inverses non linéaires. Par conséquent, nous nous concentrons sur la méthode de Tikhonov non linéaire normalisée qui résout directement le problème de minimisation non linéaire en utilisant les itérations de Landweber, où une fonction de seuillage est appliquée à chaque étape pour promouvoir la contrainte de parcimonie. Ce schéma est accéléré à l'aide d'une méthode de descente de plus grande pente projetée et remplace l'opération de seuillage pour faire respecter cette contrainte. Cette approche a également été implémentée dans un domaine d'ondelettes qui permet une représentation précise de la fonction inconnue avec un nombre réduit de coefficients. En outre, nous étudions une méthode corrélée à la parcimonie qui offre de multiples solutions parcimonieuses qui partagent un support commun non nul afin de résoudre le problème non linéaire concerné. / So-called quantitative electromagnetic imaging focused onto here is the problem of determining material properties from scattered fields measured away from the domain under investigation. Solving this inverse problem is a challenging task because it is ill-posed due to the presence of (smoothing) integral operators used in the representation of scattered fields in terms of material properties, and scattered fields are obtained at a finite set of points through noisy measurements. Moreover, the inverse problem is nonlinear simply due the fact that scattered fields are nonlinear functions of the material properties. The work described in this thesis deals with the ill-posedness of the electromagnetic imaging problem using sparsity-based regularization techniques, which assume that the scatterer(s) capture only a small fraction of the investigation domain and/or can be described in sparse fashion on a certain basis. The primary aim of the thesis is to intensively investigate sparsity regularization for nonlinear inverse problems. Therefore, we focus on sparsity-regularized nonlinear Tikhonov method which directly solves the nonlinear minimization problem using Landweber iterations, where a thresholding function is applied at every iteration step to promote the sparsity constraint. This scheme is accelerated using a projected steepest descent method and replaces the thresholding operation to enforce the sparsity constraint. This approach has also been implemented in wavelet domain which allows an accurate representation of the unknown function with a reduced number of coefficients. Additionally, we investigate a method correlated with the joint sparsity which gives multiple sparse solutions that share a common nonzero support in order to solve concerned nonlinear problem.

This dissertation aims at theoretically studying a block of layers that is common in al- most all deep learning models. The block of layers of interest is the composition of an affine layer followed by a nonlinear activation that is followed by another affine layer. We study this block from three perspectives. (i) An Optimization Perspective. Is it possible that the output of the forward pass through this block is an optimal solution to a certain convex optimization problem? We show an equivalency between the forward pass through this block and a single iteration of deterministic and stochastic algorithms solving a ten- sor formulated convex optimization problem. As consequence, we derive for the first time a formula for computing the singular values of convolutional layers surpassing the need for the prohibitive construction of the underlying linear operator. Thereafter, we show that several deep networks can have this block replaced with the corresponding optimiza- tion algorithm predicted by our theory resulting in networks with improved generalization performance. (ii) A Probabilistic Perspective. Is it possible to analytically analyze the output of a deep network upon subjecting the input to Gaussian noise? To that regard, we derive analytical formulas for the first and second moments of this block under Gaussian input noise. We demonstrate that the derived expressions can be used to efficiently analyze the output of an arbitrary deep network in addition to constructing Gaussian adversarial attacks surpassing any need for prohibitive data augmentation procedures. (iii) A Tropi- cal Geometry Perspective. Is it possible to characterize the decision boundaries of this block as a geometric structure representing a solution set to a certain class of polynomials (tropical polynomials)? If so, then, is it possible to utilize this geometric representation of the decision boundaries for novel reformulations to classical computer vision and machine learning tasks on arbitrary deep networks? We show that the decision boundaries of this block are a subset of a tropical hypersurface, which is intimately related to a the polytope that is the convex hull of two zonotopes. We utilize this geometric characterization to shed lights on new perspectives of network pruning.

Maintenance of aging buried infrastructure and reinforced concrete are critical issues in the United States. Inexpensive non-destructive techniques for mapping and imaging infrastructure and defects are an integral component of maintenance. Ground penetrating radar (GPR) is a widely-used non-destructive tool for locating buried infrastructure and for imaging rebar and other features of interest to civil engineers. Conventional acquisition and interpretation of GPR profiles is based on the arrival times of strong reflected/diffracted returns, and qualitative interpretation of return amplitudes. Features are thereby generally well located, but their material properties are only qualitatively assessed. For example, in the typical imaging of buried pipes, the average radar wave velocity through the overlying soil is estimated, but the properties of the pipe itself are not quantitatively resolved. For pipes on the order of the radar wavelength (<5-35 cm), pipe dimensions and infilling material remain ambiguous. Full waveform inversion (FWI) methods exploit the entire radar return rather than the time and peak amplitude. FWI can generate better quantitative estimates of subsurface properties. In recent decades FWI methods, developed for seismic oil exploration, have been adapted and advanced for GPR with encouraging results. To date, however, FWI methods for GPR data have not been specifically tuned and applied on surface collected common offset GPR data, which are the most common type of GPR data for engineering applications. I present an effective FWI method specifically tailored for common-offset GPR data. This method is composed of three main components, the forward modeling, wavelet estimation and inversion tools. For the forward modeling and iterative data inversion I use two open-source software packages, gprMax and PEST. The source wavelet, which is the most challenging component that guarantees the success of the method, is estimated with a novel Sparse Blind Deconvolution (SBD) algorithm that I have developed. The present dissertation indicates that with FWI, GPR can yield better quantitative estimates, for example, of both the diameters of small pipes and rebar and their electromagnetic properties (permittivity, conductivity). Also better estimates of electrical properties of the surrounding media (i.e. soil or concrete) are achieved with FWI.

Recommendation systems have been widely utilized in e-commerce settings to aid users through their shopping experiences. The principal advantage of these systems is their ability to narrow down the purchase options in addition to marketing items to customers. However, a number of challenges remain, notably those related to obtaining a clearer understanding of users, their profiles, and their preferences in terms of purchased items. Specifically, recommender systems based on collaborative filtering recommend items that have been rated by other users with preferences similar to those of the targeted users. Intuitively, the more information and ratings collected about the user, the more accurate are the recommendations such systems suggest. In a typical recommender systems database, the data are sparse. Sparsity occurs when the number of ratings obtained by the users is much lower than the number required to build a prediction model. This usually occurs because of the users’ reluctance to share their reviews, either due to privacy issues or an unwillingness to make the extra effort. Grey-sheep users pose another challenge. These are users who shared their reviews and ratings yet disagree with the majority in the systems. The current state-of-the-art typically treats these users as outliers and removes them from the system. Our goal is to determine whether keeping these users in the system may benefit learning. Thirdly, cold-start problems refer to the scenario whereby a new item or user enters the system and is another area of active research. In this case, the system will have no information about the new user or item, making it problematic to find a correlation with others in the system. This thesis addresses the three above-mentioned research challenges through the development of machine learning methods for use within the recommendation system setting. First, we focus on the label and data sparsity though the development of the Hybrid Cluster analysis and Classification learning (HCC-Learn) framework, combining supervised and unsupervised learning methods. We show that combining classification algorithms such as k-nearest neighbors and ensembles based on feature subspaces with cluster analysis algorithms such as expectation maximization, hierarchical clustering, canopy, k-means, and cascade k-means methods, generally produces high-quality results when applied to benchmark datasets. That is, cluster analysis clearly benefits the learning process, leading to high predictive accuracies for existing users. Second, to address the cold-start problem, we present the Popular Users Personalized Predictions (PUPP-DA) framework. This framework combines cluster analysis and active learning, or so-called user-in-the-loop, to assign new customers to the most appropriate groups in our framework. Based on our findings from the HCC-Learn framework, we employ the expectation maximization soft clustering technique to create our user segmentations in the PUPP-DA framework, and we further incorporate Convolutional Neural Networks into our design. Our results show the benefits of user segmentation based on soft clustering and the use of active learning to improve predictions for new users. Furthermore, our findings show that focusing on frequent or popular users clearly improves classification accuracy. In addition, we demonstrate that deep learning outperforms machine learning techniques, notably resulting in more accurate predictions for individual users. Thirdly, we address the grey-sheep problem in our Grey-sheep One-class Recommendations (GSOR) framework. The existence of grey-sheep users in the system results in a class imbalance whereby the majority of users will belong to one class and a small portion (grey-sheep users) will fall into the minority class. In this framework, we use one-class classification to provide a class structure for the training examples. As a pre-assessment stage, we assess the characteristics of grey-sheep users and study their impact on model accuracy. Next, as mentioned above, we utilize one-class learning, whereby we focus on the majority class to first learn the decision boundary in order to generate prediction lists for the grey-sheep (minority class). Our results indicate that including grey-sheep users in the training step, as opposed to treating them as outliers and removing them prior to learning, has a positive impact on the general predictive accuracy.

No description available.

京都大学 / 新制・課程博士 / 博士(経済学) / 甲第24375号 / 経博第662号 / 新制||経||302(附属図書館) / 京都大学大学院経済学研究科経済学専攻 / (主査)教授 西山 慶彦, 教授 江上 雅彦, 講師 柳 貴英 / 学位規則第4条第1項該当 / Doctor of Economics / Kyoto University / DFAM