Thesis (Ph. D.)--Electrical and Computer Engineering, Georgia Institute of Technology, 2007. / Oliver Brand, Committee Member ; James H. McClellan, Committee Member ; Paul Hasler, Committee Chair ; Mark T. Smith, Committee Member ; David Anderson, Committee Member.

Orientadores: Romis Ribeiro de Faissol Attux, Ricardo Suyama / Tese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de Computação / Made available in DSpace on 2018-08-19T01:01:27Z (GMT). No. of bitstreams: 1 Nadalin_EvertonZaccaria_D.pdf: 2615054 bytes, checksum: 5d288f06df05b7075bf283243319df70 (MD5) Previous issue date: 2011 / Resumo: Neste trabalho, foi estudado o problema de Separação Cega de Fontes (BSS), com ênfase nos casos chamados de subparametrizados, isto é, em que o número de fontes é maior do que o de misturas. A primeira contribuição proposta foi a de um limitante relacionado ao erro de inversão intrínseco ao problema quando é utilizada uma estrutura linear de separação. As outras contribuições estão relacionadas à hipótese de que as fontes são esparsas: i) uma proposta de metodologia híbrida, que se utiliza de conceitos baseados em independência e esparsidade dos sinais de forma simultânea para estimar tanto o sistema misturador quanto o número de fontes existentes em misturas com dois sensores; ii) a utilização de ferramentas de otimização baseadas na operação do sistema imunológico para a estimação do sistema misturador em problemas intrinsecamente multimodais; por fim, iii) uma proposta de utilização de um critério baseado em esparsidade para separação de fontes, sendo derivado um processo de otimização baseado na norma ?1 para este fim / Abstract: In this work, we studied the problem of Blind Source Separation (BSS), with emphasis on cases referred to as underdetermined, which occur when the number of sources is greater than the number of mixtures. The first contribution was a proposal of a bound to the inversion error that is intrinsic to the problem when a linear structure is used to perform separation. The other contributions are related to the hypothesis that the signals of the sources are sparse: i) the proposal of a hybrid methodology that employs concepts based on signal independence and sparsity to simultaneously estimate both the mixing system and the number of existing sources in mixtures with two sensors; ii) the use of optimization tools based on the modus operandi of the immune system to estimate the mixing system in problems that are inherently multimodal; finally, iii) the use of a criterion based on sparsity for source separation, which is derived from an optimization process based on the ?1 norm / Doutorado / Engenharia de Computação / Doutor em Engenharia Elétrica

How can science possibly understand the organ through which the Universe knows itself? The scientific method can be used to study how electro-chemical signals represent information in the brain. However, modelling it by simulating its structures and functions is a computation- and communication-intensive task. Whilst supercomputers offer great computational power, brain-scale models are challenging in terms of communication overheads and power consumption. Dedicated neural hardware can be used to enhance simulation performance, but it is often optimised for specific models. While performance and flexibility are desirable simulation features, there is no perfect modelling platform, and the choice is subordinate to the specific research question being investigated. In this context SpiNNaker constitutes a novel parallel architecture, with communication and memory accesses optimised for spike-based computation, permitting simulation of large spiking neural networks in real time. To exploit SpiNNaker's performance and reconfigurability fully, a neural network model must be translated from its conceptual form into data structures for a parallel system. This thesis presents a flexible approach to distributing and mapping neural models onto SpiNNaker, within the constraints introduced by its specialised architecture. The conceptual map underlying this approach characterizes the interaction between the model and the system: during the build phase the model is placed on SpiNNaker; at runtime, placement information mediates communication with devices and instrumentation for data analysis. Integration within the computational neuroscience community is achieved by interfaces to two domain-specific languages: PyNN and Nengo. The real-time, event-driven nature of the SpiNNaker platform is explored using address-event representation sensors and robots, performing visual processing using a silicon retina, and navigation on a robotic platform based on a cortical, basal ganglia and hippocampal place cells model. The approach has been successfully exploited to run models on all iterations of SpiNNaker chips and development boards to date, and demonstrated live in workshops and conferences.

A Clusterização de dados em grupos oferece uma maneira de entender e extrair informações relevantes de grandes conjuntos de dados. A abordagem em relação a aspectos como a representação dos dados e medida de similaridade entre clusters, e a necessidade de ajuste de parâmetros iniciais são as principais diferenças entre os algoritmos de clusterização, influenciando na qualidade da divisão dos clusters. O uso cada vez mais comum de grandes conjuntos de dados aliado à possibilidade de melhoria das técnicas já existentes tornam a clusterização de dados uma área de pesquisa que permite inovações em diferentes campos. Nesse trabalho é feita uma revisão dos métodos de clusterização já existentes, e é descrito um novo método de clusterização de dados baseado na identificação de comunidades em redes complexas e modelos computacionais inspirados biologicamente. A técnica de clusterização proposta é composta por duas etapas: formação da rede usando os dados de entrada; e particionamento dessa rede para obtenção dos clusters. Nessa última etapa, a técnica de otimização por nuvens de partículas é utilizada a fim de identificar os clusters na rede, resultando em um algoritmo de clusterização hierárquico divisivo. Resultados experimentais revelaram como características do método proposto a capacidade de detecção de clusters de formas arbitrárias e a representação de clusters com diferentes níveis de refinamento. / DAta clustering is an important technique to understand and to extract relevant information in large datasets. Data representation and similarity measure adopted, and the need to adjust initial parameters, are the main differences among clustering algorithms, interfering on clusters quality. The crescent use of large datasets and the possibility to improve existing techniques make data clustering a research area that allows innovation in different fields. In this work is made a review of existing data clustering methods, and it is proposed a new data clustering technique based on community dectection on complex networks and bioinspired models. The proposed technique is composed by two steps: network formation to represent input data; and network partitioning to identify clusters. In the last step, particle swarm optimization technique is used to detect clusters, resulting in an hierarchical clustering algorithm. Experimental results reveal two main features of the algorithm: the ability to detect clusters in arbitrary shapes and the ability to generate clusters with different refinement degrees

One of the approaches to building artificial intelligence (AI) is to decipher the princi- ples of the brain function and to employ similar mechanisms for solving cognitive tasks, such as visual perception or natural language understanding, using machines. The recent breakthrough, named deep learning, demonstrated that large multi-layer networks of arti- ficial neural-like computing units attain remarkable performance on some of these tasks. Nevertheless, such artificial networks remain to be very loosely inspired by the brain, which rich structures and mechanisms may further suggest new algorithms or even new paradigms of computation. In this thesis, we explore brain-inspired probabilistic mechanisms, such as neural and synaptic stochasticity, in the context of generative models. The two questions we ask here are: (i) what kind of models can describe a neural learning system built of stochastic components? and (ii) how can we implement such systems e ̆ciently? To give specific answers, we consider two well known models and the corresponding neural architectures: the Naive Bayes model implemented with a winner-take-all spiking neural network and the Boltzmann machine implemented in a spiking or non-spiking fashion. We propose and analyze an e ̆cient neuromorphic implementation of the stochastic neu- ral firing mechanism and study the e ̄ects of synaptic unreliability on learning generative energy-based models implemented with neural networks.

Au cours de ces dernières années, les réseaux de capteurs sans fils ont connu un intérêt croissant à la fois au sein de la communauté scientifique et industrielle en raison du large potentiel en terme d’applications offertes. Toutefois, les capteurs sont conçus avec d’extrêmes contraintes en ressources, en particulier la limitation de l’énergie. Il est donc nécessaire de concevoir des protocoles efficaces, évolutifs et moins consommateur d’énergie afin de prolonger la durée de vie de ces réseaux. Le clustering est une approche très populaire, utilisée pour l’optimisation de la consommation d’énergie des capteurs. Cette technique permet d’influencer fortement la performance globale du réseau. En outre, dans de tels réseaux, le routage génère un nombre assez élevé d’opérations non négligeables qui affectent considérablement la durée de vie du réseau ainsi que le débit offert. Dans cette thèse, nous nous sommes intéressés d’une part aux problèmes de clustering et de routage en utilisant des méthodes d’optimisation inspirées de certaines sociétés biologiques fournissant des modèles puissants qui conduisent à l’établissement d’une intelligence globale en se basant sur des comportements individuels très simples. Nous avons proposé une approche de clustering distribuée basée sur le processus de sélection des sites de nidification chez les colonies d’abeilles. Nous avons formulé le problème de clustering distribuée comme un processus social de prise de décision dans lequel les capteurs agissent d’une manière collective pour choisir des représentants au sein de leurs clusters respectifs. Le protocole proposé assure une distribution de l’équilibrage de charge entre les membres de chaque cluster afin de prolonger la durée de vie du réseau en faisant un compromis entre la consommation d’énergie et la qualité du canal de communication. D’autre part, nous avons proposé un protocole de routage basé sur des clusters en utilisant un algorithme inspiré du phénomène de butinage des abeilles. Nous avons formulé le problème de clustring comme un problème de programmation linéaire alors que le problème du routage est résolu par une fonction de coûts. L’algorithme de clustering permet la construction efficace des clusters en faisant un compromis entre la consommation d’énergie et la qualité du canal communication au sein des clusters tandis que le routage est réalisé de manière distribuée. Les protocoles proposés ont été intensivement expérimentés sur plusieurs topologies dans différents scénarios de réseaux et comparés avec des protocoles bien connus de clustering et routage. Les résultats obtenus démontrent l’efficacité des protocoles proposés. / During the past few years, wireless sensor networks witnessed an increased interest in both the industrial and the scientific community due to the potential wide area of applications. However, sensors’ components are designed with extreme resource constraints, especially the power supply limitation. It is therefore necessary to design low power, scalable and energy efficient protocols in order to extend the lifetime of such networks. Cluster-based sensor networks are the most popular approach for optimizing the energy consumption of sensor nodes, in order to strongly influence the overall performance of the network. In addition, routing involves non negligible operations that considerably affect the network lifetime and the throughput. In this thesis, we addressed the clustering and routing problems by hiring intelligent optimization methods through biologically inspired computing, which provides the most powerful models that enabled a global intelligence through local and simple behaviors. We proposed a distributed clustering approach based on the nest-sites selection process of a honeybee swarm. We formulated the distributed clustering problem as a social decision-making process in which sensors act in a collective manner to choose their cluster heads. To achieve this choice, we proposed a multi- objective cost-based fitness function. In the design of our proposed algorithm, we focused on the distribution of load balancing among each cluster member in order to extend network lifetime by making a tradeoff between the energy consumption and the quality of the communication link among sensors. Then, we proposed a centralized cluster-based routing protocol for wireless sensor networks by using the fast and efficient searching features of the artificial bee colony algorithm. We formulated the clustering as a linear programming problem and the routing problem is solved by proposing a cost-based function. We designed a multi-objective fitness function that uses the weighted sum approach, in the assignment of sensors to a cluster. The clustering algorithm allows the efficient building of clusters by making a tradeoff between the energy consumption and the quality of the communication link within clusters while the routing is realized in a distributed manner. The proposed protocols have been intensively experimented with a number of topologies in various network scenarios and the results are compared with the well-known cluster-based routing protocols. The results demonstrated the effectiveness of the proposed protocols.

In recent years there has been significant research in the field of computational neuroscience and many of these biologically inspired cognitive models are based on the theory of operation of mammalian visual cortex. One such model of neocortex developed by George & Hawkins, known as Hierarchical Temporal Memories (HTM), is considered for the research discussed here. We propose a simple hierarchical model that is derived from HTM. The aim of this work is to evaluate the hardware cost and performance against software based simulations. This work presents a detailed hardware implementation and analysis of the derived hierarchical model. We show that these networks are inherently parallel in their architecture, similar to the biological computing, and that parallelism can be exploited by massively parallel architectures implemented using reconfigurable devices such as the FPGA. Hardware implementation accelerates the learning process which is useful in many real world problems. We have implemented a complex network node that operates in real time using an FPGA. The current architecture is modular and allows us to estimate the hardware resources and computational units required to realize large scale networks in the future.

Analogy is ubiquitous in human cognition. One of the important questions related to understanding the situated nature of analogy-making is how people retrieve source analogues via their interactions with external environments. This dissertation studies interactive analogical retrieval in the context of biologically inspired design (BID). BID involves creative use of analogies to biological systems to develop solutions for complex design problems (e.g., designing a device for acquiring water in desert environments based on the analogous fog-harvesting abilities of the Namibian Beetle). Finding the right biological analogues is one of the critical first steps in BID. Designers routinely search online in order to find their biological sources of inspiration. But this task of online bio-inspiration seeking represents an instance of interactive analogical retrieval that is extremely time consuming and challenging to accomplish. This dissertation focuses on understanding and supporting the task of online bio-inspiration seeking. Through a series of field studies, this dissertation uncovered the salient characteristics and challenges of online bio-inspiration seeking. An information-processing model of interactive analogical retrieval was developed in order to explain those challenges and to identify the underlying causes. A set of measures were put forth to ameliorate those challenges by targeting the identified causes. These measures were then implemented in an online information-seeking technology designed to specifically support the task of online bio-inspiration seeking. Finally, the validity of the proposed measures was investigated through a series of experimental studies and a deployment study. The trends are encouraging and suggest that the proposed measures has the potential to change the dynamics of online bio-inspiration seeking in favor of ameliorating the identified challenges of online bio-inspiration seeking.

The purpose of this work is to investigate a series of biologically inspired modifications to state-of-the-art Mel- frequency cepstral coefficients (MFCCs) that may improve automatic speech recognition results. We have provided recommendations to improve speech recognition results de- pending on signal-to-noise ratio levels of input signals. This work has been motivated by noise-robust auditory features (NRAF). In the feature extraction technique, after a signal is filtered using bandpass filters, a spatial derivative step is used to sharpen the results, followed by an envelope detector (recti- fication and smoothing) and down-sampling for each filter bank before being compressed. DCT is then applied to the results of all filter banks to produce features. The Hidden- Markov Model Toolkit (HTK) is used as the recognition back-end to perform speech recognition given the features we have extracted. In this work, we investigate the role of filter types, window size, spatial derivative, rectification types, smoothing, down- sampling and compression and compared the final results to state-of-the-art Mel-frequency cepstral coefficients (MFCC). A series of conclusions and insights are provided for each step of the process. The goal of this work has not been to outperform MFCCs; however, we have shown that by changing the compression type from log compression to 0.07 root compression we are able to outperform MFCCs for all noisy conditions.

This thesis focuses on novel approaches to distributed and parallel computing that are inspired by the mechanism and functioning of biological cells. We refer to this concept as cellular distributed and parallel computing which focuses on three important principles: simplicity, parallelism, and locality. We first give a parallel polynomial-time solution to the constraint satisfaction problem (CSP) based on a theoretical model of cellular distributed and parallel computing, which is known as neural-like P systems (or neural-like membrane systems). We then design a class of simple neural-like P systems to solve the fundamental maximal independent set (MIS) selection problem efficiently in a distributed way, by drawing inspiration from the way that developing cells in the fruit fly become specialised. Building on the novel bio-inspired approach to distributed MIS selection, we propose a new simple randomised algorithm for another fundamental distributed computing problem: the distributed greedy colouring (GC) problem. We then propose an improved distributed MIS selection algorithm that incorporates for the first time another important feature of the biological system: adapting the probabilities used at each node based on local feedback from neighbouring nodes. The improved distributed MIS selection algorithm is again extended to solve the distributed greedy colouring problem. Both improved algorithms are simple and robust and work under very restrictive conditions, moreover, they both achieve state-of-the-art performance in terms of their worst-case time complexity and message complexity. Given any n-node graph with maximum degree Delta, the expected time complexity of our improved distributed MIS selection algorithm is O(log n) and the message complexity per node is O(1). The expected time complexity of our improved distributed greedy colouring algorithm is O(Delta + log n) and the message complexity per node is again O(1). Finally, we provide some experimental results to illustrate the time and message complexity of our proposed algorithms in practice. In particular, we show experimentally that the number of colours used by our distributed greedy colouring algorithms turns out to be optimal or near-optimal for many standard graph colouring benchmarks, so they provide effective simple heuristic approaches to computing a colouring with a small number of colours.