Multi-modal image fusion is an active research area with many civilian and military applications. Fusion is defined as strategic combination of information collected by various sensors from different locations or different types in order to obtain a better understanding of an observed scene or situation. Fusion of multi-modal images cannot be completed unless these two modalities are spatially aligned. In this research, I consider two important problems. Multi-modal, multi-perspective image registration and decision level fusion of multi-modal images. In particular, LiDAR and visual imagery. Multi-modal image registration is a difficult task due to the different semantic interpretation of features extracted from each modality. This problem is decoupled into three sub-problems. The first step is identification and extraction of common features. The second step is the determination of corresponding points. The third step consists of determining the registration transformation parameters. Traditional registration methods use low level features such as lines and corners. Using these features require an extensive optimization search in order to determine the corresponding points. Many methods use global positioning systems (GPS), and a calibrated camera in order to obtain an initial estimate of the camera parameters. The advantages of our work over the previous works are the following. First, I used high level-features, which significantly reduce the search space for the optimization process. Second, the determination of corresponding points is modeled as an assignment problem between a small numbers of objects. On the other side, fusing LiDAR and visual images is beneficial, due to the different and rich characteristics of both modalities. LiDAR data contain 3D information, while images contain visual information. Developing a fusion technique that uses the characteristics of both modalities is very important. I establish a decision-level fusion technique using manifold models.

Os seres vivos têm uma impressionante capacidade de lidar com ambientes complexos com grandes quantidades de informações de forma muito autônoma. Isto os torna um modelo ideal para o desenvolvimento de sistemas artificiais bioinspirados. A rede neural artificial auto-organizada de Kohonen é um excelente exemplo de um sistema baseado nos modelos biológicos. Esta tese discutirá ilustrativamente o reconhecimento e a generalização de padrões em alta dimensão nos sistemas biológicos e como eles lidam com redução de dimensionalidade para otimizar o armazenamento e o acesso às informações memorizadas para fins de reconhecimento e categorização de padrões, mas apenas para contextualizar o tema com as propostas desta tese. As novas propostas desenvolvidas nesta tese são úteis para aplicações de extração não supervisionada de conhecimento a partir dos mapas auto-organizados. Trabalha-se sobre o modelo da Rede Neural de Kohonen, mas algumas das metodologias propostas também são aplicáveis com outras abordagens de redes neurais auto-organizadas. Será apresentada uma técnica de reconstrução visual dos neurônios do Mapa de Kohonen gerado pelo método híbrido PCA+SOM. Essa técnica é útil quando se trabalha com banco de dados de imagens. Propõe-se também um método para melhorar a representação dos dados do mapa SOM e discute-se o resultado do mapeamento SOM como uma generalização das informações do espaço de dados. Finalmente, apresenta-se um método de exploração de espaço de dados em alta dimensão de maneira auto-organizada, baseado no manifold dos dados, cuja proposta foi denominada Self Organizing Manifold Mapping (SOMM). São apresentados os resultados computacionais de ensaios realizados com cada uma das propostas acima e eles são avaliados as com métricas de qualidade conhecidas, além de uma nova métrica que está sendo proposta neste trabalho. / Living beings have an amazing capacity to deal with complex environments with large amounts of information autonomously. They are the perfect model for bioinspired artificial system development. The artificial neural network developed by Kohonen is an excellent example of a system based on biological models. In this thesis, we will discuss illustratively pattern recognition and pattern generalization in high dimensional data space by biological system. Then, a brief discussion of how they manage dimensionality reduction to optimize memory space and speed up information access in order to categorize and recognize patterns. The new proposals developed in this thesis are useful for applications of unsupervised knowledge extraction using self-organizing maps. The proposals use Kohonens model. However, any self-organizing neural network in general can also use the proposed techniques. It will be presented a visual reconstruction technique for Kohonens neurons, which was generated by hybrid method PCA+SOM. This technique is useful when working with images database. It is also proposed a method for improving the representation of SOMs map and discussing the result of the SOMs mapping as a generalization of the information data space. Finally, it is proposed a method for exploring high dimension data space in a self-organized way on the data manifold. This new proposal was called Self Organizing Manifold Mapping (SOMM). We present the results of computational experiments on each of the above proposals and evaluate the results using known quality metrics, as well as a new metric that is being proposed in this thesis.

Os seres vivos têm uma impressionante capacidade de lidar com ambientes complexos com grandes quantidades de informações de forma muito autônoma. Isto os torna um modelo ideal para o desenvolvimento de sistemas artificiais bioinspirados. A rede neural artificial auto-organizada de Kohonen é um excelente exemplo de um sistema baseado nos modelos biológicos. Esta tese discutirá ilustrativamente o reconhecimento e a generalização de padrões em alta dimensão nos sistemas biológicos e como eles lidam com redução de dimensionalidade para otimizar o armazenamento e o acesso às informações memorizadas para fins de reconhecimento e categorização de padrões, mas apenas para contextualizar o tema com as propostas desta tese. As novas propostas desenvolvidas nesta tese são úteis para aplicações de extração não supervisionada de conhecimento a partir dos mapas auto-organizados. Trabalha-se sobre o modelo da Rede Neural de Kohonen, mas algumas das metodologias propostas também são aplicáveis com outras abordagens de redes neurais auto-organizadas. Será apresentada uma técnica de reconstrução visual dos neurônios do Mapa de Kohonen gerado pelo método híbrido PCA+SOM. Essa técnica é útil quando se trabalha com banco de dados de imagens. Propõe-se também um método para melhorar a representação dos dados do mapa SOM e discute-se o resultado do mapeamento SOM como uma generalização das informações do espaço de dados. Finalmente, apresenta-se um método de exploração de espaço de dados em alta dimensão de maneira auto-organizada, baseado no manifold dos dados, cuja proposta foi denominada Self Organizing Manifold Mapping (SOMM). São apresentados os resultados computacionais de ensaios realizados com cada uma das propostas acima e eles são avaliados as com métricas de qualidade conhecidas, além de uma nova métrica que está sendo proposta neste trabalho. / Living beings have an amazing capacity to deal with complex environments with large amounts of information autonomously. They are the perfect model for bioinspired artificial system development. The artificial neural network developed by Kohonen is an excellent example of a system based on biological models. In this thesis, we will discuss illustratively pattern recognition and pattern generalization in high dimensional data space by biological system. Then, a brief discussion of how they manage dimensionality reduction to optimize memory space and speed up information access in order to categorize and recognize patterns. The new proposals developed in this thesis are useful for applications of unsupervised knowledge extraction using self-organizing maps. The proposals use Kohonens model. However, any self-organizing neural network in general can also use the proposed techniques. It will be presented a visual reconstruction technique for Kohonens neurons, which was generated by hybrid method PCA+SOM. This technique is useful when working with images database. It is also proposed a method for improving the representation of SOMs map and discussing the result of the SOMs mapping as a generalization of the information data space. Finally, it is proposed a method for exploring high dimension data space in a self-organized way on the data manifold. This new proposal was called Self Organizing Manifold Mapping (SOMM). We present the results of computational experiments on each of the above proposals and evaluate the results using known quality metrics, as well as a new metric that is being proposed in this thesis.

This work aims to improve the capability of accurate information extraction from high-dimensional data, with a specific neural learning paradigm, the Self-Organizing Map (SOM). The SOM is an unsupervised learning algorithm that can faithfully sense the manifold structure and support supervised learning of relevant information from the data. Yet open problems regarding SOM learning exist. We focus on the following two issues. (1) Evaluation of topology preservation. Topology preservation is essential for SOMs in faithful representation of manifold structure. However, in reality, topology violations are not unusual, especially when the data have complicated structure. Measures capable of accurately quantifying and informatively expressing topology violations are lacking. One contribution of this work is a new measure, the Weighted Differential Topographic Function ( WDTF ), which differentiates an existing measure, the Topographic Function ( TF ), and incorporates detailed data distribution as an importance weighting of violations to distinguish severe violations from insignificant ones. Another contribution is an interactive visual tool, TopoView, which facilitates the visual inspection of violations on the SOM lattice. We show the effectiveness of the combined use of the WDTF and TopoView through a simple two-dimensional data set and two hyperspectral images. (2) Learning multiple latent variables from high-dimensional data. We use an existing two-layer SOM-hybrid supervised architecture, which captures the manifold structure in its SOM hidden layer, and then, uses its output layer to perform the supervised learning of latent variables. In the customary way, the output layer only uses the strongest output of the SOM neurons. This severely limits the learning capability. We allow multiple, k , strongest responses of the SOM neurons for the supervised learning. Moreover, the fact that different latent variables can be best learned with different values of k motivates a new neural architecture, the Conjoined Twins, which extends the existing architecture with additional copies of the output layer, for preferential use of different values of k in the learning of different latent variables. We also automate the customization of k for different variables with the statistics derived from the SOM. The Conjoined Twins shows its effectiveness in the inference of two physical parameters from Near-Infrared spectra of planetary ices.

A Riemannian manifold is a mathematical concept that generalizes curved surfaces to higher dimensions, giving a precise meaning to concepts like angle, length, area, volume and curvature. A glimpse of the consequences of a non-flat geometry is given on the sphere, where the shortest path between two points – a geodesic – is along a great circle. Different from Euclidean space, the angle sum of geodesic triangles on the sphere is always larger than 180 degrees. Signals and data found in applied research are sometimes naturally described by such curved spaces. This dissertation presents basic research and tools for the analysis, processing and visualization of such manifold-valued data, with a particular emphasis on future applications in medical imaging and visualization. Two-dimensional manifolds, i.e. surfaces, enter naturally into the geometric modelling of anatomical entities, such as the human brain cortex and the colon. In advanced algorithms for processing of images obtained from computed tomography (CT) and ultrasound imaging (US), images themselves and derived local structure tensor fields may be interpreted as two- or three-dimensional manifolds. In diffusion tensor magnetic resonance imaging (DT-MRI), the natural description of diffusion in the human body is a second-order tensor field, which can be related to the metric of a manifold. A final example is the analysis of shape variations of anatomical entities, e.g. the lateral ventricles in the brain, within a population by describing the set of all possible shapes as a manifold. Work presented in this dissertation include: Probabilistic interpretation of intrinsic and extrinsic means in manifolds. A Bayesian approach to filtering of vector data, removing noise from sampled manifolds and signals. Principles for the storage of tensor field data and learning a natural metric for empirical data. The main contribution is a novel class of algorithms called LogMaps, for the numerical estimation of logp (x) from empirical data sampled from a low-dimensional manifold or geometric model embedded in Euclidean space. The logp (x) function has been used extensively in the literature for processing data in manifolds, including applications in medical imaging such as shape analysis. However, previous approaches have been limited to manifolds where closed form expressions of logp (x) have been known. The introduction of the LogMap framework allows for a generalization of the previous methods. The application of LogMaps to texture mapping, tensor field visualization, medial locus estimation and exploratory data analysis is also presented. / The electronic version is corrected for grammatical and spelling errors.

In many real-world application problems, the availability of data labels for supervised learning is rather limited. Incompletely labeled datasets are common in many of the databases generated in some of the currently most active areas of research. It is often the case that a limited number of labeled cases is accompanied by a larger number of unlabeled ones. This is the setting for semi-supervised learning, in which unsupervised approaches assist the supervised problem and vice versa. A manifold learning model, namely Generative Topographic Mapping (GTM), is the basis of the methods developed in this thesis. The non-linearity of the mapping that GTM generates makes it prone to trustworthiness and continuity errors that would reduce the faithfulness of the data representation, especially for datasets of convoluted geometry. In this thesis, a variant of GTM that uses a graph approximation to the geodesic metric is first defined. This model is capable of representing data of convoluted geometries. The standard GTM is here modified to prioritize neighbourhood relationships along the generated manifold. This is accomplished by penalizing the possible divergences between the Euclidean distances from the data points to the model prototypes and the corresponding geodesic distances along the manifold. The resulting Geodesic GTM (Geo-GTM) model is shown to improve the continuity and trustworthiness of the representation generated by the model, as well as to behave robustly in the presence of noise. The thesis then leads towards the definition and development of semi-supervised versions of GTM for partially-labeled data exploration. As a first step in this direction, a two-stage clustering procedure that uses class information is presented. A class information-enriched variant of GTM, namely class-GTM, yields a first cluster description of the data. The number of clusters defined by GTM is usually large for visualization purposes and does not necessarily correspond to the overall class structure. Consequently, in a second stage, clusters are agglomerated using the K-means algorithm with different novel initialization strategies that benefit from the probabilistic definition of GTM. We evaluate if the use of class information influences cluster-wise class separability. A robust variant of GTM that detects outliers while effectively minimizing their negative impact in the clustering process is also assessed in this context. We then proceed to the definition of a novel semi-supervised model, SS-Geo-GTM, that extends Geo-GTM to deal with semi-supervised problems. In SS-Geo-GTM, the model prototypes are linked by the nearest neighbour to the data manifold constructed by Geo-GTM. The resulting proximity graph is used as the basis for a class label propagation algorithm. The performance of SS-Geo-GTM is experimentally assessed, comparing positively with that of an Euclidean distance-based counterpart and that of the alternative Laplacian Eigenmaps method. Finally, the developed models (the two-stage clustering procedure and the semi-supervised models) are applied to the analysis of a human brain tumour dataset (obtained by Nuclear Magnetic Resonance Spectroscopy), where the tasks are, in turn, data clustering and survival prognostic modeling. / Resum de la tesi (màxim 4000 caràcters. Si se supera aquest límit, el resum es tallarà automàticament al caràcter 4000) En muchos problemas de aplicación del mundo real, la disponibilidad de etiquetas de datos para aprendizaje supervisado es bastante limitada. La existencia de conjuntos de datos etiquetados de manera incompleta es común en muchas de las bases de datos generadas en algunas de las áreas de investigación actualmente más activas. Es frecuente que un número limitado de casos etiquetados venga acompañado de un número mucho mayor de datos no etiquetados. Éste es el contexto en el que opera el aprendizaje semi-supervisado, en el cual enfoques no-supervisados prestan ayuda a problemas supervisados y vice versa. Un modelo de aprendizaje de variaciones (manifold learning, en inglés), llamado Mapeo Topográfico Generativo (GTM, en acrónimo de su nombre en inglés), es la base de los métodos desarrollados en esta tesis. La no-linealidad del mapeo que GTM genera hace que éste sea propenso a errores de fiabilidad y continuidad, los cuales pueden reducir la fidelidad de la representación de los datos, especialmente para conjuntos de datos de geometría intrincada. En esta tesis, una extensión de GTM que utiliza una aproximación vía grafos a la métrica geodésica es definida en primer lugar. Este modelo es capaz de representar datos con geometrías intrincadas. En él, el GTM estándar es modificado para priorizar relaciones de vecindad a lo largo de la variación generada. Esto se logra penalizando las divergencias existentes entre las distancias Euclideanas de los datos a los prototipos del modelo y las correspondientes distancias geodésicas a lo largo de la variación. Se muestra que el modelo Geo-GTM resultante mejora la continuidad y fiabilidad de la representación generada y que se comporta de manera robusta en presencia de ruido. Más adelante, la tesis nos lleva a la definición y desarrollo de versiones semi-supervisadas de GTM para la exploración de conjuntos de datos parcialmente etiquetados. Como un primer paso en esta dirección, se presenta un procedimiento de agrupamiento en dos etapas que utiliza información de pertenencia a clase. Una extensión de GTM enriquecida con información de pertenencia a clase, llamada class-GTM, produce una primera descripción de grupos de los datos. El número de grupos definidos por GTM es normalmente grande para propósitos de visualización y no corresponde necesariamente con la estructura de clases global. Por ello, en una segunda etapa, los grupos son aglomerados usando el algoritmo K-means con diferentes estrategias de inicialización novedosas las cuales se benefician de la definición probabilística de GTM. Evaluamos si el uso de información de clase influye en la separabilidad de clase por grupos. Una extensión robusta de GTM que detecta datos atípicos a un tiempo que minimiza de forma efectiva su impacto negativo en el proceso de agrupamiento es evaluada también en este contexto. Se procede después a la definición de un nuevo modelo semi-supervisado, SS-Geo-GTM, que extiende Geo-GTM para ocuparse de problemas semi-supervisados. En SS-Geo-GTM, los prototipos del modelo son vinculados al vecino más cercano a la variación construída por Geo-GTM. El grafo de proximidad resultante es utilizado como base para un algoritmo de propagación de etiquetas de clase. El rendimiento de SS-Geo-GTM es valorado experimentalmente, comparando positivamente tanto con la contraparte de este modelo basada en la distancia Euclideana como con el método alternativo Laplacian Eigenmaps. Finalmente, los modelos desarrollados (el procedimiento de agrupamiento en dos etapas y los modelos semi-supervisados) son aplicados al análisis de un conjunto de datos de tumores cerebrales humanos (obtenidos mediante Espectroscopia de Resonancia Magnética Nuclear), donde las tareas a realizar son el agrupamiento de datos y el modelado de pronóstico de supervivencia.

Compressive sensing (CS) is a novel imaging technology based on the inherent redundancy of natural scenes. The minimum number of required measurements which defines the maximum image compression rate is lower-bounded by the sparsity of the image but is dependent on the type of acquisition patterns employed. Increased measurements by the Rice single pixel camera (SPC) slows down the acquisition process, which may cause the image recovery to be more susceptible to background noise and thus limit CS's application in imaging, detection, or classifying moving targets. In this study, two methods (hybrid-subspace sparse sampling (HSS) for imaging and secant projection on a manifold for classification are applied to solving this problem. For the HSS method, new image pattern are designed via robust principle component analysis (rPCA) on prior knowledge from a library of images to sense a common structure. After measuring coarse scale commonalities, the residual image becomes sparser, and then fewer measurements are needed. For the secant projection case, patterns that can preserve the pairwise distance between data points based on manifold learning are designed via semi-definite programming. These secant patterns turn out to be better in object classification over those learned from PCA. Both methods considerably decrease the number of required measurements for each task when compared with the purely random patterns of a more universal CS imaging system.

This work aims to improve the capability of accurate information extraction from high-dimensional data, with a specific neural learning paradigm, the Self-Organizing Map (SOM). The SOM is an unsupervised learning algorithm that can faithfully sense the manifold structure and support supervised learning of relevant information from the data. Yet open problems regarding SOM learning exist. We focus on the following two issues. 1. Evaluation of topology preservation. Topology preservation is essential for SOMs in faithful representation of manifold structure. However, in reality, topology violations are not unusual, especially when the data have complicated structure. Measures capable of accurately quantifying and informatively expressing topology violations are lacking. One contribution of this work is a new measure, the Weighted Differential Topographic Function (WDTF), which differentiates an existing measure, the Topographic Function (TF), and incorporates detailed data distribution as an importance weighting of violations to distinguish severe violations from insignificant ones. Another contribution is an interactive visual tool, TopoView, which facilitates the visual inspection of violations on the SOM lattice. We show the effectiveness of the combined use of the WDTF and TopoView through a simple two-dimensional data set and two hyperspectral images. 2. Learning multiple latent variables from high-dimensional data. We use an existing two-layer SOM-hybrid supervised architecture, which captures the manifold structure in its SOM hidden layer, and then, uses its output layer to perform the supervised learning of latent variables. In the customary way, the output layer only uses the strongest output of the SOM neurons. This severely limits the learning capability. We allow multiple, k, strongest responses of the SOM neurons for the supervised learning. Moreover, the fact that different latent variables can be best learned with different values of k motivates a new neural architecture, the Conjoined Twins, which extends the existing architecture with additional copies of the output layer, for preferential use of different values of k in the learning of different latent variables. We also automate the customization of k for different variables with the statistics derived from the SOM. The Conjoined Twins shows its effectiveness in the inference of two physical parameters from Near-Infrared spectra of planetary ices.

Dimension reduction is a vital tool in many areas of applied statistics in which the dimensionality of the predictors can be large. In such cases, many statistical methods will fail or yield unsatisfactory results. However, many data sets of high dimensionality actually contain a much simpler, low-dimensional structure. Classical methods such as principal components analysis are able to detect linear structures very effectively, but fail in the presence of nonlinear structures. In the first part of this thesis, we investigate the asymptotic behavior of two nonlinear dimensionality reduction algorithms, LTSA and HLLE. In particular, we show that both algorithms, under suitable conditions, asymptotically recover the true generating coordinates up to an isometry. We also discuss the relative merits of the two algorithms, and the effects of the underlying probability distributions of the coordinates on their performance. Model selection is a fundamental problem in nearly all areas of applied statistics. In particular, a balance must be achieved between good in-sample performance and out-of-sample prediction. It is typically very easy to achieve good fit in the sample data, but empirically we often find that such models will generalize poorly. In the second part of the thesis, we propose a new procedure for the model selection problem which generalizes traditional methods. Our algorithm allows the combination of existing model selection criteria via a ranking procedure, leading to the creation of new criteria which are able to combine measures of in-sample fit and out-of-sample prediction performance into a single value. We then propose an algorithm which provably finds the optimal combination with a specified probability. We demonstrate through simulations that these new combined criteria can be substantially more powerful than any individual criterion.

The thesis concentrates on a methodological research on categorical structural optimization by means of manifold learning. The main difficulty of handling the categorical optimization problems lies in the description of the design variables: they are presented in a discrete manner and do not have any orders. Thus the treatment of the design space is a key issue. In this thesis, the non-ordinal categorical variables are treated as multi-dimensional discrete variables, thus the dimensionality of corresponding design space becomes high. In order to reduce the dimensionality, the manifold learning techniques are introduced to find the intrinsic dimensionality and map the original design space to a reduced-order space. The mechanisms of both linear and non-linear manifold learning techniques are firstly studied. Then numerical examples are tested to compare the performance of manifold learning techniques. It is found that Principal Component Analysis (PCA) and Multi-dimensional Scaling (MDS) can only deal with linear or globally approximately linear cases. Isomap preserves the geodesic distances for non-linear manifold, however, its time consuming is the most. Locally Linear Embedding (LLE) preserves the neighbour weights and can yield good results in a short time. Kernel Principal Component Analysis (KPCA) works as a non-linear classifier and we proves the reason why it cannot preserve distances or angles in some cases.Based on the reduced-order representation obtained by Isomap, the graph-based evolutionary crossover and mutation operators are proposed to deal with categorical structural optimization problems, including the design of dome, six-story rigid frame and dame-like structures. The results show that the proposed graph-based evolutionary approach constructed on the reduced-order space performs more efficiently than traditional methods including simplex approach or evolutionary approach without reduced-order space.The Locally Linear Embedding is applied to reduce the data dimensionality and a polynomial interpolation helps to construct the responding surface from lower dimensional representation to original data. Then the continuous search method of moving asymptotes is executed and yields a competitively good but inadmissible solution within only a few of iteration numbers. Then in the second stage, a discrete search strategy is proposed to find out better solutions based on a neighbour search. The ten-bar truss and dome structural design problems are tested to show the validity of the method. In the end, this method is compared to the Simulated Annealing algorithm and Covariance Matrix Adaptation Evolutionary Strategy, showing its better optimization efficiency.In order to deal with the case in which the categorical design instances are distributed on several manifolds, we propose a k-manifolds learning method based on the Weighted Principal Component Analysis. The obtained manifolds are integrated in the lower dimensional design space. Then the two-stage search method is applied to solve the ten-bar truss, the dome and the dam-like structural design problems. / Doctorat en Sciences de l'ingénieur et technologie / info:eu-repo/semantics/nonPublished