Thesis (Ph. D.)--University of Sydney, 2009. / Title from title screen (viewed November 5, 2009). Submitted in fulfilment of the requirements for the degree of Doctor of Philosophy to the School of Information Technologies in the the Faculty of Engineering & Information Technologies. Degree awarded 2009; thesis submitted 2008. Includes bibliographical references. Also available in print form.

This thesis presents a novel two stage multimodal speech enhancement system, making use of both visual and audio information to filter speech, and explores the extension of this system with the use of fuzzy logic to demonstrate proof of concept for an envisaged autonomous, adaptive, and context aware multimodal system. The design of the proposed cognitively inspired framework is scalable, meaning that it is possible for the techniques used in individual parts of the system to be upgraded and there is scope for the initial framework presented here to be expanded. In the proposed system, the concept of single modality two stage filtering is extended to include the visual modality. Noisy speech information received by a microphone array is first pre-processed by visually derived Wiener filtering employing the novel use of the Gaussian Mixture Regression (GMR) technique, making use of associated visual speech information, extracted using a state of the art Semi Adaptive Appearance Models (SAAM) based lip tracking approach. This pre-processed speech is then enhanced further by audio only beamforming using a state of the art Transfer Function Generalised Sidelobe Canceller (TFGSC) approach. This results in a system which is designed to function in challenging noisy speech environments (using speech sentences with different speakers from the GRID corpus and a range of noise recordings), and both objective and subjective test results (employing the widely used Perceptual Evaluation of Speech Quality (PESQ) measure, a composite objective measure, and subjective listening tests), showing that this initial system is capable of delivering very encouraging results with regard to filtering speech mixtures in difficult reverberant speech environments. Some limitations of this initial framework are identified, and the extension of this multimodal system is explored, with the development of a fuzzy logic based framework and a proof of concept demonstration implemented. Results show that this proposed autonomous,adaptive, and context aware multimodal framework is capable of delivering very positive results in difficult noisy speech environments, with cognitively inspired use of audio and visual information, depending on environmental conditions. Finally some concluding remarks are made along with proposals for future work.

Esta investigação tece relações comparativas entre padrões de crescimento de redes vasculares e argumenta que no conceito de morfogênese, analogamente à biologia, há evoluções e agrupamentos gerativos em sistemas estruturais ramificados. Para a compreensão deste agrupamento gerativo, são utilizados conceitos adjacentes à geometria fractal - estudo de estruturas rugosas, porosas ou fragmentadas, que mantém sua irregularidade em grau similar para todas as escalas. Embora os fractais sejam mais conhecidos como objetos resultantes de algoritmos matemáticos em iterações sucessivas no campo das ciências computacionais, nesta pesquisa as estruturas naturais que se aproximam de modelos de geometria fractal irregular, isto é, classificadas como fractais estatísticos, e que são encontradas em vários sistemas orgânicos e minerais, têm predominância. Pesquisam-se e coletam-se in loco imagens de padrões ramificados para início de produção de trabalhos práticos. Elas são em sua maioria fotografias de seres biológicos naturais com algum grau de ramificação, mas incluem-se também estruturas estatisticamente fractais análogas a sistemas biológicos. Esses estudos de casos de ramificações são então categorizados de acordo com padrões superficiais, colorações, estruturação corporal, funcionalidade no sistema em que atuam e interação com os demais sistemas ou seres vivos presentes num dado ecossistema. A partir de seleção de alguns organismos de acordo com sua estrutura corporal e seus movimentos característicos, são produzidas arte-instalações onde estruturas biomiméticas híbridas são criadas para responder cinética e sensorialmente a estímulos humanos, ambientais e climáticos. / This survey weaves comparative relationships among growth patterns of vascular networks and argues that, in the concept of morphogenesis, in analogy to biology, there are generative clustering and evolutions in branched structural systems. To understand this generative clustering, adjacent concepts to fractal geometry are used - the study of rough, porous or fragmented structures that keep their irregularity to a similar degree at all scales. Although fractals are better known as resulting objects of mathematical algorithms in successive iterations in the field of computer science, in this research the natural structures that approximate to models of irregular fractals, i.e., classified as statistical, and that are found in various minerals and organic systems are predominant. Images of branched patterns are researched and collected in loco for a first production of practical work. They are mostly photographs of natural biological beings with some branching level, but they also include statistical fractal structures analogous to biological systems. These branching study cases are then categorized according to surface patterns, colors, body structure, functionality in the system in which they operate and interaction with other systems or living organisms present in a given ecosystem. From the selection of a few organisms according to their characteristic movements and body structure, installation artworks are experimentally designed where hybrid biomimetic structures are created to sensory and kinetically respond to human, environmental and climate stimuli.

Esta investigação tece relações comparativas entre padrões de crescimento de redes vasculares e argumenta que no conceito de morfogênese, analogamente à biologia, há evoluções e agrupamentos gerativos em sistemas estruturais ramificados. Para a compreensão deste agrupamento gerativo, são utilizados conceitos adjacentes à geometria fractal - estudo de estruturas rugosas, porosas ou fragmentadas, que mantém sua irregularidade em grau similar para todas as escalas. Embora os fractais sejam mais conhecidos como objetos resultantes de algoritmos matemáticos em iterações sucessivas no campo das ciências computacionais, nesta pesquisa as estruturas naturais que se aproximam de modelos de geometria fractal irregular, isto é, classificadas como fractais estatísticos, e que são encontradas em vários sistemas orgânicos e minerais, têm predominância. Pesquisam-se e coletam-se in loco imagens de padrões ramificados para início de produção de trabalhos práticos. Elas são em sua maioria fotografias de seres biológicos naturais com algum grau de ramificação, mas incluem-se também estruturas estatisticamente fractais análogas a sistemas biológicos. Esses estudos de casos de ramificações são então categorizados de acordo com padrões superficiais, colorações, estruturação corporal, funcionalidade no sistema em que atuam e interação com os demais sistemas ou seres vivos presentes num dado ecossistema. A partir de seleção de alguns organismos de acordo com sua estrutura corporal e seus movimentos característicos, são produzidas arte-instalações onde estruturas biomiméticas híbridas são criadas para responder cinética e sensorialmente a estímulos humanos, ambientais e climáticos. / This survey weaves comparative relationships among growth patterns of vascular networks and argues that, in the concept of morphogenesis, in analogy to biology, there are generative clustering and evolutions in branched structural systems. To understand this generative clustering, adjacent concepts to fractal geometry are used - the study of rough, porous or fragmented structures that keep their irregularity to a similar degree at all scales. Although fractals are better known as resulting objects of mathematical algorithms in successive iterations in the field of computer science, in this research the natural structures that approximate to models of irregular fractals, i.e., classified as statistical, and that are found in various minerals and organic systems are predominant. Images of branched patterns are researched and collected in loco for a first production of practical work. They are mostly photographs of natural biological beings with some branching level, but they also include statistical fractal structures analogous to biological systems. These branching study cases are then categorized according to surface patterns, colors, body structure, functionality in the system in which they operate and interaction with other systems or living organisms present in a given ecosystem. From the selection of a few organisms according to their characteristic movements and body structure, installation artworks are experimentally designed where hybrid biomimetic structures are created to sensory and kinetically respond to human, environmental and climate stimuli.