published_or_final_version / Education / Master / Master of Science in Information Technology in Education

Šiame darbe pateikti ir aprašyti biologinio ir dirbtinio neurono modeliai. Didžiausias dėmesys skiriamas vieno tipo neuroniniams tinklams – saviorganizuojantiems žemėlapiams (SOM). Darbe pateiktas jų apmokymas, taip pat pagrindinių sąvokų (epocha, kaimynystės eilė, unifikuotų atstumų matrica ir kt.), susijusių su SOM neuroniniais tinklais (žemėlapiais), apibrėžimai. Buvo nagrinėtos keturios saviorganizuojančių neuroninių tinklų sistemos: NeNet, SOM-Toolbox, DataBionic ESOM, Viscovery SOMine ir Matlab įrankiai „nntool“, „nctool“, kurie naudojami SOM tinklams sukurti ir apmokyti. Pateikiamos sistemų naudojimosi instrukcijos, norint gauti paprasčiausią SOM žemėlapį. Matlab aplinkoje sukurta ir darbe aprašyta naują vizualizavimo būdą turinti sistema „Somas“, pateiktas jos išskirtinumas ir naudojimosi instrukcija. Sistemoje „Somas“ realizuota kita mokymo funkcija nei kitose minėtose sistemose. Pagrindinis analizuotų sistemų tikslas yra suskirstyti duomenis į klasterius pagal jų panašumą ir pateikti juos SOM žemėlapyje. Sistemos viena nuo kitos skiriasi duomenų pateikimu, mokymo taisyklėmis, vizualizavimo galimybėmis, todėl čia aptariami sistemų panašumai ir skirtumai. Nagrinėti susidarę SOM žemėlapiai ir gautos kvantavimo bei topografinės paklaidos, analizuojant tris duomenų aibes: irisų, stiklo ir vyno. Kvantavimo ir topografinės paklaidos yra kiekybiniai vaizdo kokybės įverčiai. Padarytos išvados apie susidariusius klasterius tiriamuose duomenyse. Naudojant naują sistemą „Somas“... [toliau žr. visą tekstą] / In this master thesis, biologic and artificial neuron models have been described. The focus is selforganizing maps (SOM). The self-organizing maps are one of types of artificial neural networks. SOM training as well as the main concepts which need to explain SOM networks (epochs, neighbourhood size, u-matrix and etc.) have been described. Four systems of self-organizing maps: NeNet, SOMToolbox, DataBionic ESOM, Viscovery SOMine, and Matlab tools “nntool” and “nctool” have been analyzed. In the thesis, a system use guide has been presented to make a simple SOM map. A new system “Somas” that has a new visualisation way has been developed in Matlab. The system has been described, its oneness has been emphasized, and a use guide is presented. The main target of the SOM systems is data clustering and their graphical presentation on the self-organizing map. The SOM systems are different one from other in their interfaces, the data pre-processing, learning rules, visualization manners, etc. Similarities and differences of the systems have been highlighted here. The experiments have been carried out with three data sets: iris, glass and wine. The SOM maps, obtained by each system, have been described and some conclusions on the clusters have been drawn. The quantization and topographic errors have been analyzed to estimate the quality of the maps obtained. An investigation has been carried out in the new system “Somas” and system “NeNet” in order to look how quantization and... [to full text]

Saviorganizuojantys neuroniniai tinklai (SOM) yra susilaukę nemažai populiarumo mokslininkų tarpe klasterizuojant ar vizualizuojant daugiamačius duomenis. Šiame magistro diplominiame darbe smulkiai išnagrinėtas SOM algoritmas bei veikimo principai, pateiktos galimos parametrų reikšmės ir kaimynystės funkcijos. Taip pat nurodyti tinklo kokybės įvertinimo kriterijai ir duomenų vizualizavimo metodai taikant saviorganizuojantį neuroninį tinklą. SOM pagrindinis tikslas yra duomenų klasterizavimas, o ne vizualizavimas, todėl duomenų vaizdavimas SOM'u turi savų trūkumų – žemėlapyje negalima matyti atstumų tarp klasės neuronų ir kaip toli nutolusios viena nuo kitos klasės. Pateikta alternatyva – SOM modifikacija ViSOM. Darbe išnagrinėti ViSOM algoritmo esminiai skirtumai, aprašyti parametrų parinkimo ypatumai. Nagrinėti SOM ir ViSOM vizualizavimo skirtumus sukurta MATLAB sistemoje programa, realizuojanti abu algoritmus bei pateiktas programos galimybių ir scenarijų aprašas. Pasirinkus tris kaimynines funkcijas su šia programa atlikti tyrimai, rodantys, kad kvantavimo ir topografinės paklaidos netinkamos vertinant ViSOM vaizdo kokybę. Pasiūlyti trys nauji vertinimo kriterijai, bei su jais atlikti tyrimai, parodantys jų veiksmingumą. Taip pat darbe vizualiai parodytas ir aprašytas ViSOM žemėlapio kitimas priklausomai nuo rezoliucijos. / A self-organizing map is a type of artificial neural networks that has received substantial popularity among scientists in regards to clustering and visualization of multidimensional data. In this master theses, the learning algorithm and the main principals are examined in detail, the neighbourhood functions and values of various parameters are given. Some criteria of the network evaluation quality and the data visualization methods using the self-organizing maps are given as well. The main goal of the SOM is clustering of data, but not the visualization, so the visual data representation by the SOM has its drawbacks – it is impossible to see the distances between neurons, corresponding the vectors belong to a class, and how far from each other the classes are in a map. The alternative – SOM modification, called ViSOM, has been developed. The main differences of SOM and ViSOM are investigated, the peculiarity of parameter selection is also examined in this work. In order to study the differences of SOM and ViSOM visualization, a system in MATLAB has been developed, both algorithms have been implemented, and the feature and scenario list of the program is presented. Some experiments have been carried out by selecting three neighborhood functions. The experiments have showed that the quantization and topographic errors are not suitable for studying the visualization of ViSOM. Three new evaluation criteria are proposed. The investigation shows their effectiveness. In the work... [to full text]

Future wireless networks are expected to be highly heterogeneous and ultra dense with different types of small cells underlaid with traditional macro cells. In the presence of hundreds of different types of small cells, centralized control and manual intervention in network management will be inefficient and expensive. In this case, self-organization has been proposed as a key feature in future wireless networks. In a self-organizing network, the nodes are expected to take individual decisions on their behavior. Therefore, individual decision making in resource allocation (i.e., Distributed Resource Allocation) is of vital important. The objective of this thesis is to develop a distributed resource allocation framework for self-organizing small cell networks. Game theory is a powerful mathematical tool which can model and analyze interactive decision making problems of the agents with conflicting interests. Therefore, it is a well-appropriate tool for modeling the distributed resource allocation problem of small cell networks. In this thesis, I consider three different scenarios of distributed resource allocation in self-organizing small cell networks i.e., i). Distributed downlink power and spectrum allocation to ensure fairness for a small cell network of base stations with bounded rationality, ii). Distributed downlink power control for an ultra dense small cell network of base stations with energy constraints, iii). Distributed joint uplink-downlink power control for a small cell network of possibly deceitful nodes with full-duplexing capabilities. Specifically, I utilize evolutionary games, mean field games, and repeated games to model and analyze the three aforementioned scenarios. I also use stochastic geometry, which is a very powerful mathematical tool that can model the characteristics of the networks with random topologies, to design the payoff functions for the formulated evolutionary game and the mean field game. / October 2016

Self-organizing linear lists Self-organizing linear lists are data structures for fast search, provided that certain elements stored in them are searched more frequently than others, while the probability of access to individual elements is generally not known in advance. Efficient search is achieved using different permutation rules that keep changing the list structure so that the more frequently searched elements are closer to the beginning. This thesis gives an overview of known algorithms for solving this problem (with the theoretical results about their complexity, if they are known), and experimental study of their behavior (using its own or freely available implementations and software for generating input data, testing algorithms and processing the results of experiments).

Depuis une quinzaine d'années, l'Imagerie par Résonance Magnétique fonctionnelle (IRMf) permet d'extraire de l'information sur le fonctionnement cérébral et particulièrement sur la localisation des processus cognitifs. L'information contenue par les acquisitions en IRMf est extraite à l'aide du modèle linéaire général et du processus d'inférence statistique. Bien que cette méthode dite « classique » ait permis de valider la plupart des modèles lésionnels de manière non invasive, elle souffre de certaines limites. Pour résoudre ce problème, différentes techniques d'analyse ont émergé et proposent une nouvelle façon d'interpréter les données de la neuroimagerie. Nous présentons deux nouvelles méthodes multivariées basées sur les cartes de Kohonen. Nos méthodes analysent les données IRMf avec le moins d'a priori possibles. En parallèle, nous tentons d'extraire de l'information sur les réseaux neuronaux impliqués dans les émotions. La première de ces méthodes s'intéresse à l'information de spécialisation fonctionnelle et la seconde à l'information de connectivité fonctionnelle. Nous présentons les résultats qui en découlent, puis chacune des méthodes est comparée à l'analyse dite classique en termes d'informations extraites. De plus, notre attention s'est focalisée sur la notion de valence émotionnelle et nous tentons d'établir l'existence d'un éventuel réseau partagé entre valence positive et valence négative. La constance de ce réseau est évaluée à la fois entre modalités perceptives et entre catégories de stimuli. Chacune des méthodes proposées permet de corroborer l'information recueillie par la méthode classique, en apportant de nouvelles informations sur les processus étudiés. Du point de vue des émotions, notre travail met en lumière un partage du réseau cérébral pour les va-lences négative et positive ainsi qu'une constance de cette information dans certaines régions cérébrales entre modalités perceptives et entre catégories. / In the last fifteen years, functional magnetic resonance imaging (fMRI) have been used to extract information about cognitive processes location. The information contained in fMRI acquisitions is usually extracted using the general linear model coupled to the statistical inference process. Although this classical method has validated noninvasively most of the lesional models, it suffers from some limitations. To solve this problem, various analysis techniques have emerged and propose a new way of interpreting neuroimaging data. In this thesis, we present two multivariate methods to analyze fMRI data with the least possible a priori. In parallel, we are trying to extract information about brain emotion processing. The first method focuses on the brain functional specialization and the second method on the brain functional connectivity. After results presentation, each method is compared to the so-called classical analysis in terms of extracted information. In addition, emphasis was put on the concept of emotional valence. We try to establish the existence of a possible split between positive and negative valence networks. The consistency of the network is evaluated across both perceptual modalities and stimuli categories. Each of the proposed methods are as accurate as the conventional method and provide new highlights on the studied processes. From the perspective of emotions, our work highlights a shared brain network for positive and negative valences and a consistency of this information in some brain regions across both perceptual modalities and stimuli categories.

Com o advento da tecnologia de microarray, uma grande quantidade de dados de expressão gênica encontra-se disponível. Após a extração das taxas de expressão dos genes, técnicas de formação de clusters são utilizadas para a análise dos dados. Diante da diversidade do conhecimento que pode ser extraído dos dados de expressão gênica, existe a necessidade de diferentes técnicas de formação de clusters. O modelo dinâmico desenvolvido em (Zhao et. al. 2003a) apresenta diversas características interessantes para o problema de formação de clusters, entre as quais podemos citar: a não necessidade de fornecer o número de cluster, a propriedade de multi-escala, serem altamente paralelos e, principalmente, permitirem a inserção de regras e mecanismos mais complexos para a formação dos clusters. Todavia, este modelo apresenta dificuldades em determinar clusters de formato e tamanho arbitrários, além de não realizar a clusterização hierárquica, sendo estas duas características desejáveis para uma técnica de clusterização. Neste trabalho, foram desenvolvidas três técnicas para superar as limitações do modelo dinâmico proposto em (Zhao et. al. 2003a). O Modelo1, o qual é uma simplificação do modelo dinâmico original, porém mais eficiente. O Modelo2, que a partir da inserção de um novo conjunto de elementos no modelo dinâmico, permite a formação de clusters de formato e tamanho arbitrário. E um algoritmo para a clusterização hierárquica que utiliza o Modelo1 como bloco de construção. Os modelos desenvolvidos foram aplicados em dados biológicos, segmentando imagens de microarray e auxiliando na análise do conjunto expressão de genes de St. Jude Leukemia. / With the advent of microarray technology, a large amount of gene expression data is now available. Clustering is the computational technique usually employed to analyze and explore the data produced by microarrays. Due to the variety of information that can be extracted from the expression data, many clustering techniques with different approaches are needed. In the work proposed by (Zhao et. al. 2003a), the dynamical model for data clustering has several interesting features to the clustering task: the number of clusters does not need to be known, the multi-scale property, high parallelism, and it is flexible to use more complex rules while clustering the data. However, two desirable features for clustering techniques are not present: the ability to detect different clusters sizes and shapes, and a hierarchical representation of the clusters. This project presents three techniques, overcoming the restrictions of the dynamical model proposed by (Zhao et. al. 2003a). The first technique, called Model1, is more effective than the original model and was obtained simplifying it. The second technique, called Model2, is capable of detecting different clusters sizes and shapes. The third technique consists in a hierarchical algorithm that uses Model1 as a building block. The techniques here developed were used with biological data. Microarray image segmentation was performed and the St. Jude Leukemia gene expression data was analyzed and explored.

Este estudo versa sobre uma investigação de viabilidade da utilização de redes neurais auto-organizadas na classificação e exploração de dados macroeconômicos. Para tanto, foi elaborado um método no qual foram empregadas topologias neurais auto-organizadas, procurando assim explorar as características de melhor desempenho de cada um dos modelos, sob um enfoque seqüencial e com o intuito de se adquirir conhecimento intermediário em cada uma de suas fases, diminuindo o impacto da complexidade tanto no tempo requerido para realização da tarefa quanto na análise dos resultados. Os próprios resultados obtidos sugerem que a utilização de redes neurais artificiais auto-organizadas na aquisição de conhecimento sobre bases de dados aplicáveis às Ciências Econômicas apresenta desempenho análogo aos modelos paramétricos tradicionalmente empregados na construção de cenários com tais informações. / This study turns on an inquiry of viability of the use of self-organizing neural nets in classification and exploration of macroeconomic data. For this purpose, a method in which had been used self-organized neural topologies was elaborated, looking to explore the better characteristics of performance of each one of the models, under a sequential approach and with objective of acquiring intermediate knowledge in each one of its phases, diminishing the impact of the complexity as in time consuming as in analysis of results. Main results obtained suggest the use of self-organized artificial neural nets in acquisition of knowledge on Economic databases presents analog performance to traditional parametric models in scenarios building.

Os seres vivos têm uma impressionante capacidade de lidar com ambientes complexos com grandes quantidades de informações de forma muito autônoma. Isto os torna um modelo ideal para o desenvolvimento de sistemas artificiais bioinspirados. A rede neural artificial auto-organizada de Kohonen é um excelente exemplo de um sistema baseado nos modelos biológicos. Esta tese discutirá ilustrativamente o reconhecimento e a generalização de padrões em alta dimensão nos sistemas biológicos e como eles lidam com redução de dimensionalidade para otimizar o armazenamento e o acesso às informações memorizadas para fins de reconhecimento e categorização de padrões, mas apenas para contextualizar o tema com as propostas desta tese. As novas propostas desenvolvidas nesta tese são úteis para aplicações de extração não supervisionada de conhecimento a partir dos mapas auto-organizados. Trabalha-se sobre o modelo da Rede Neural de Kohonen, mas algumas das metodologias propostas também são aplicáveis com outras abordagens de redes neurais auto-organizadas. Será apresentada uma técnica de reconstrução visual dos neurônios do Mapa de Kohonen gerado pelo método híbrido PCA+SOM. Essa técnica é útil quando se trabalha com banco de dados de imagens. Propõe-se também um método para melhorar a representação dos dados do mapa SOM e discute-se o resultado do mapeamento SOM como uma generalização das informações do espaço de dados. Finalmente, apresenta-se um método de exploração de espaço de dados em alta dimensão de maneira auto-organizada, baseado no manifold dos dados, cuja proposta foi denominada Self Organizing Manifold Mapping (SOMM). São apresentados os resultados computacionais de ensaios realizados com cada uma das propostas acima e eles são avaliados as com métricas de qualidade conhecidas, além de uma nova métrica que está sendo proposta neste trabalho. / Living beings have an amazing capacity to deal with complex environments with large amounts of information autonomously. They are the perfect model for bioinspired artificial system development. The artificial neural network developed by Kohonen is an excellent example of a system based on biological models. In this thesis, we will discuss illustratively pattern recognition and pattern generalization in high dimensional data space by biological system. Then, a brief discussion of how they manage dimensionality reduction to optimize memory space and speed up information access in order to categorize and recognize patterns. The new proposals developed in this thesis are useful for applications of unsupervised knowledge extraction using self-organizing maps. The proposals use Kohonens model. However, any self-organizing neural network in general can also use the proposed techniques. It will be presented a visual reconstruction technique for Kohonens neurons, which was generated by hybrid method PCA+SOM. This technique is useful when working with images database. It is also proposed a method for improving the representation of SOMs map and discussing the result of the SOMs mapping as a generalization of the information data space. Finally, it is proposed a method for exploring high dimension data space in a self-organized way on the data manifold. This new proposal was called Self Organizing Manifold Mapping (SOMM). We present the results of computational experiments on each of the above proposals and evaluate the results using known quality metrics, as well as a new metric that is being proposed in this thesis.

O avanço tecnológico atual está permitindo que as pessoas recebam cada vez mais informações visuais dos mais diferentes tipos, nas mais variadas mídias. Esse aumento fantástico está obrigando os pesquisadores e as indústrias a imaginar soluções para o armazenamento e recuperação deste tipo de informação, pois nossos computadores ainda utilizam, apesar dos grandes avanços nessa área, um sistema de arquivos imaginado há décadas, quando era natural trabalhar com informações meramente textuais. Agora, nos deparamos com novos problemas: Como encontrar uma paisagem específica em um banco de imagens, em que trecho de um filme aparece um cavalo sobre uma colina, em que parte da fotografia existe um gato, como fazer um robô localizar um objeto em uma cena, entre outras necessidades. O objetivo desse trabalho é propor uma arquitetura de rede neural artificial que permita o reconhecimento de objetos genéricos e de categorias em banco de imagens digitais, de forma que se possa recuperar imagens específicas a partir da descrição da cena fornecida pelo usuário. Para que esse objetivo fosse alcançado, foram utilizadas técnicas de Visão Computacional e Processamento de Imagens na etapa de extração de feições de baixo nível e de Redes Neurais(Mapas Auto-Organizáveis de Kohonen) na etapa de agrupamento de classes de objetos. O resultado final desse trabalho pretende ser um embrião para um sistema de reconhecimento de objetos mais genérico, que possa ser estendido para a criação de indices de forma automática ou semi-automática em grandes bancos de imagens. / The current technological progress allows people to receive more and more visual information of the most different types, in different medias. This huge augmentation of image availability forces researchers and industries to propose efficient solutions for image storage and recovery. Despite the extraordinary advances in computational power, the data files system remain the same for decades, when it was natural to deal only with textual information. Nowadays, new problems are in front of us in this field. For instance, how can we find an specific landscape in a image database, in which place of a movie there is a horse on a hill, in which part of a photographic picture there is a cat, how can a robot find an object in a scene, among other queries. The objective of this work is to propose an Artificial Neural Network (ANN) architecture that performs the recognition of generic objects and object’s categories in a digital image database. With this implementation, it becomes possible to do image retrieval through the user´s scene description. To achieve our goal, we have used Computer Vision and Image Processing techniques in low level features extraction and Neural Networks (namely Kohonen’s Self-Organizing Maps) in the phase of object classes clustering. The main result of this work aims to be a seed for a more generic object recognition system, which can be extended to the automatic or semi-automatic index creation in huge image databases.