Vocal self-perception plays an important role in the learning process as a classical singer, especially to a hearing impaired classical singer like myself. To explore and challenge my vocal self-perception, I used two different enchanced feedback methods to observe how I responded with my singing technique: one based on auditory feedback and one based on visual feedback. I formulated two training sequences with a defined schedule and procedure. I sang excerpts from two contrasting arias and made a total of seventeen audio recordings and eleven video recordings. Those recordings were then evaluated by me in listening sessions, focusing on the quality of tone. I chose six audio recordings to play for a small discussion group and collected the group's perceptual data. The results suggest that these methods did not positively impact my singing technique, but the experience did lead me to some important realizations about certain timbral qualities in my voice, and I learned a great deal about different aspects of vocal self-perception in my singing practice.

Speech perception in crowded acoustic environments is particularly challenging for hearing impaired listeners. While assistive hearing devices can suppress background noises distinct from speech, they struggle to lower interfering speakers without knowing the speaker on which the listener is focusing. The human brain has a remarkable ability to pick out individual voices in a noisy environment like a crowded restaurant or a busy city street. This inspires the brain-controlled hearing technologies. A brain-controlled hearing aid acts as an intelligent filter, reading wearers’ brainwaves and enhancing the voice they want to focus on. Two essential elements form the core of brain-controlled hearing aids: automatic speech separation (SS), which isolates individual speakers from mixed audio in an acoustic scene, and auditory attention decoding (AAD) in which the brainwaves of listeners are compared with separated speakers to determine the attended one, which can then be amplified to facilitate hearing. This dissertation focuses on speech separation and its integration with AAD, aiming to propel the evolution of brain-controlled hearing technologies. The goal is to help users to engage in conversations with people around them seamlessly and efficiently. This dissertation is structured into two parts. The first part focuses on automatic speech separation models, beginning with the introduction of a real-time monaural speech separation model, followed by more advanced real-time binaural speech separation models. The binaural models use both spectral and spatial features to separate speakers and are more robust to noise and reverberation. Beyond performing speech separation, the binaural models preserve the interaural cues of separated sound sources, which is a significant step towards immersive augmented hearing. Additionally, the first part explores using speaker identifications to improve the performance and robustness of models in long-form speech separation. This part also delves into unsupervised learning methods for multi-channel speech separation, aiming to improve the models' ability to generalize to real-world audio. The second part of the dissertation integrates speech separation introduced in the first part with auditory attention decoding (SS-AAD) to develop brain-controlled augmented hearing systems. It is demonstrated that auditory attention decoding with automatically separated speakers is as accurate and fast as using clean speech sounds. Furthermore, to better align the experimental environment of SS-AAD systems with real-life scenarios, the second part introduces a new AAD task that closely simulates real-world complex acoustic settings. The results show that the SS-AAD system is capable of improving speech intelligibility and facilitating tracking of the attended speaker in realistic acoustic environments. Finally, this part presents employing self-supervised learned speech representation in the SS-AAD systems to enhance the neural decoding of attentional selection.

No description available.

<p>This Master’s thesis presents results from two clinical hearing aid studies. Wide dynamic range compression (WDRC), a hearing aid amplification algorithm widely used in the hearing aid industry, is compared against a novel hearing aid called the Neuro-Compensator (NC), which employs a neural-based amplification algorithm based on a computational model of the auditory periphery. The NC strategy involves preprocessing an incoming auditory signal, such that when the signal is presented to a damaged cochlea, auditory nerve output is reconstructed to look similar to the auditory nerve output of a healthy cochlea for the original auditory signal. The NC and WDRC hearing aid technologies are compared across a multitude of auditory domains. Objective measures of speech intelligibility in quiet and in noise, music perception, sound localization, and subjective measures of sound quality are obtained. It was hypothesized that the NC would restore more normal auditory abilities across auditory domains, due to its proposed strategy of restoring more normal auditory nerve output. Results from the clinical hearing aid studies quantified domains in which the NC was superior to WDRC, and vice versa.</p> / Master of Science (MSc)

This report introduces a novel architecture for a General Purpose Digital Signal Processing System and applies the system to implement a digital hearing aid. The theory and implementation of the general purpose digital signal processing system revolve around the architecture of the digital signal processor (DSP) and its use. The system consists of three subsystems: the Analog Interface Board, the DAAD Board, and the DSP Board. The general purpose digital signal processing system described takes into consideration both the basic needs of such a system as well as the many features which make it efficient in a wide range of applications. The system was used as a testbed for implementing various real-time DSP Algorithms. One of these algorithms is concerned with the problem of hearing loss. The final implementation of the digital hearing aid examines both the feasibility of the DHA as well as the usefulness of the general purpose digital signal processing system in a random application. Suggestions for future modification and expansion are discussed.

published_or_final_version / Speech and Hearing Sciences / Master / Master of Science in Audiology

<p>Hearing loss afflicts as many as 10\% of our population.Fortunately, technologies designed to alleviate the effects ofhearing loss are improving rapidly, including cochlear implantsand the increasing computing power of digital hearing aids. Thisthesis focuses on theoretically sound methods for improvinghearing aid technology. The main contributions are documented inthree research articles, which treat two separate topics:modelling of human speech recognition (Papers A and B) andoptimization of diagnostic methods for hearing loss (Paper C).Papers A and B present a hidden Markov model-based framework forsimulating speech recognition in noisy conditions using auditorymodels and signal detection theory. In Paper A, a model of normaland impaired hearing is employed, in which a subject's pure-tonehearing thresholds are used to adapt the model to the individual.In Paper B, the framework is modified to simulate hearing with acochlear implant (CI). Two models of hearing with CI arepresented: a simple, functional model and a biologically inspiredmodel. The models are adapted to the individual CI user bysimulating a spectral discrimination test. The framework canestimate speech recognition ability for a given hearing impairmentor cochlear implant user. This estimate could potentially be usedto optimize hearing aid settings.Paper C presents a novel method for sequentially choosing thesound level and frequency for pure-tone audiometry. A Gaussianmixture model (GMM) is used to represent the probabilitydistribution of hearing thresholds at 8 frequencies. The GMM isfitted to over 100,000 hearing thresholds from a clinicaldatabase. After each response, the GMM is updated using Bayesianinference. The sound level and frequency are chosen so as tomaximize a predefined objective function, such as the entropy ofthe probability distribution. It is found through simulation thatan average of 48 tone presentations are needed to achieve the sameaccuracy as the standard method, which requires an average of 135presentations.</p>

A perda auditiva unilateral (PAUn) é caracterizada pela diminuição da audição em apenas uma orelha. Indivíduos com este tipo de perda auditiva podem apresentar comprometimento nas habilidades auditivas de localização sonora, processamento temporal, ordenação e resolução temporal. O objetivo deste estudo foi verificar as habilidades auditivas de ordenação temporal, resolução temporal e localização sonora, antes e após a adaptação do aparelho de amplificação sonora individual (AASI). Foram avaliados 22 indivíduos, com idades entre 18 e 60 anos, com diagnóstico de PAUn sensorioneural ou mista, de graus leve a severo. O estudo foi dividido em duas etapas: a pré e a pós-adaptação de AASI. Em ambas as etapas, os indivíduos foram submetidos a uma anamnese, aplicação do Questionário de Habilidade Auditiva da Localização da fonte sonora, avaliação simplificada do processamento auditivo (ASPA) e Random Gap Detection Test (RGDT). O presente estudo encontrou diferença estatisticamente significante na avaliação da ASPA, exceto no teste de memória para sons não verbais em sequência (TMSnV), no RGDT e no Questionário de Habilidade Auditiva da Localização Sonora. A conclusão do estudo foi que com o uso efetivo do AASI, indivíduos com PAUn apresentaram melhora nas habilidades auditivas de localização sonora, ordenação e resolução temporal. / The Unilateral hearing loss (UHL) is characterized by decreased hearing in one ear. In individuals with this type of hearing loss the hearing abilities of sound localization, and temporal processing, ordering and temporal resolution, are affected. The objective of this study was to assess the hearing abilities of temporal ordering, temporal resolution and sound localization before and after the fitting of a hearing aid (HA). They evaluated 22 subjects, aged 18 to 60 years, diagnosed with sensorineural or mixed UHL, from mild to severe degrees. The study was divided into two stages: the pre and post-adaptation of HA. In both phases, subjects were submitted to an interview, application of Questionnarie for Desabilities Associated with Impaired Auditory Localization, auditory processing screening protocol (APSP) and Random Gap Detection Test (RGDT). This study found no statistically significant difference in the evaluation of APSP except in the memory test for non-verbal sounds in sequence, in RGDT and Questionnarie for Desabilities Associated with Impaired Auditory Localization. The conclusion was that with the effective use of hearing aids, individuals with UHL showed improvement in the auditory abilities of sound localization, ordering and temporal resolution.

Uma sistemática de teleconsulta para o processo de programação, verificação e adaptação dos aparelhos de amplificação sonora individuais (AASI) foi desenvolvida e sua eficácia avaliada por meio de um estudo prospectivo, randomizado e cego. Participaram do estudo 50 deficientes auditivos (30 homens; 20 mulheres) candidatos ao uso do AASI, com idades variando entre 39 e 88 anos e média ISO dos limiares audiométricos entre 30 e 68,75dB. Os participantes foram alocados em dois grupos (randomização estratificada): controle (n=25), submetidos aos procedimentos face a face (F) e experimental (n=25) submetidos à teleconsulta (T). O procedimento F foi realizado pelo avaliador 1 em todos os participantes, incluindo: programação do AASI, medidas com microfone sonda (equipamento Unity PC Probe, estímulo speech noise, apresentado em 50, 65 e 80dBNPS) e orientação quanto ao uso e cuidados com o AASI. Para a teleconsulta o participante e um facilitador se posicionaram na sala junto aos equipamentos (ambiente de teste). O avaliador 1 posicionou-se em outra sala (ambiente remoto) com um notebook conectado à LAN USP. Webcams e headsets capturaram áudio e vídeo, transmitidos em tempo real pelo software Polycom PVX (384 kbps), também utilizado para compartilhamento de aplicativos. Por meio desse compartilhamento o avaliador 1 realizava todos os procedimentos diretamente no participante, com auxílio do facilitador. O tempo despendido para as consultas F e T foi cronometrado. Após as consultas o avaliador 2, cego quanto aos grupos, realizou a avaliação de percepção da fala (Hearing in Noise Test - HINT Brasil) para obter o limiar de reconhecimento de fala no silêncio (LRF) e a relação sinal/ruído (S/R) em que os participantes reconheciam 50% das sentenças apresentadas. Aproximadamente um mês após as consultas foi realizada (avaliador 2) a análise do tempo diário de uso do AASI (via datalogging do dispositivo) e do benefício e satisfação em vida diária (questionário International Outcome Inventory for Hearing Aids - IOI-HA). Maior tempo para a programação e verificação e menor tempo para orientação foi observado para o grupo experimental, sendo este estatisticamente significativo (teste t de Student). Não houve diferença entre grupos no tempo total do atendimento em minutos (82,2 controle e 81,3 experimental). A equiparação das medidas com microfone sonda aos respectivos targets foi similar para os dois grupos. Não houve diferença significativa (teste t) entre as médias do LRF no silêncio (56,24 controle e 51,78 experimental) e da relação S/R (4,94 controle e 4,17 experimental), o tempo médio de uso do AASI em horas (7,26 controle e 7,44 experimental) e das pontuações total e dos itens do questionário IOI-HA. Houve correlação (Spearman) positiva forte e significativa entre o tempo de uso do AASI e o tempo de uso informado pelo participante, para ambos os grupos. Para alguns participantes durante a teleconsulta houve interrupção da transmissão de áudio e vídeo que foi prontamente solucionada. Concluiu-se que a teleconsulta é um procedimento eficaz para a realização da programação, verificação do AASI e orientação de usuários quando, por algum motivo, serviços face a face não estiverem disponíveis. / A teleconsultation system for hearing aid (HA) programming, verification and fitting has been developed and its efficacy assessed through a prospective, randomized and blind study. Fifty hearing impaired individuals (30 men; 20 women), HA candidates, in the age range 39 - 88 years and ISO mean of audiometric thresholds between 30 and 68.75dBNA, participated in the study. The participants were divided into two groups (stratified randomization): control (n=25), submitted to face to face procedures (F) and experimental (n=25), submitted to teleconsultation (T). Procedure F was performed by evaluator 1 in all participants: HA programming, probe microphone measures (Unity PC Probe equipment, speech noise stimulus presented at 50, 65 and 80dBSPL) and HA use and care orientation. For the teleconsultation, the participant and a facilitator positioned themselves in the room with the equipment (test environment). Evaluator 1 stayed in another room (remote environment) with a notebook connected to the LAN USP. Webcams and headsets captured audio and video, transmitted in real time by software Polycom PVX (384 kbps), utilized as well for application sharing, through which evaluator 1 performed all the procedures directly in the patient, aided by the facilitator. The time spent for consultations F and T was recorded. Following the consultations, evaluator 2, blind to the groups, performed the speech perception assessment (Hearing in Noise Test - HINT Brazil) so as to obtain the speech reception threshold (SRT) in silence and the signal/noise ratio (S/N) in which the participants recognized 50% of the sentences presented. Nearly a month following the consultations the evaluator 2 carried out the analysis of daily time of HA use (via datalogging of the device) as well as benefit and satisfaction (questionnaire International Outcome Inventory for Hearing Aids - IOI-HA). An statistically significant (Students t test) greater time for the programming and verification and a smaller one for HA orientation were observed for the experimental group. No difference was found between the groups for the total consultation time in minutes (control: 82.2; experimental: 81.3). The real ear measures matching to targets were similar for the both groups. No significant difference was observed (Students t test) between the means of SRT in silence (56.24 control and 51.78 experimental) as well as the S/N ratio (4.94 control and 4.17 experimental), the daily amount HA use in hours (7.26 control and 7.44 experimental) and the IOI scores. For both groups there was a positive, strong and significant correlation (Spearman) between the daily amount of HA use (datalogging) and the reported HA use. For some participants, during the teleconsultation, the transmission of audio and video was interrupted, but readily restored. Conclusion: teleconsultation is an effective procedure for hearing aid programming, verification and fitting when face to face services are not available.

Devido a intensa relação entre a fissura labiopalatina e a deficiência auditiva, este trabalho teve como objetivo desenvolver e analisar a eficácia de um ambiente virtual de transmissão de informações sobre fissura labiopalatina e deficiência auditiva para profissionais, com a utilização da Teleducação Interativa. Como metodologia, foi desenvolvido um ambiente de aprendizagem no formato de um blog contendo informações sobre fissura labiopalatina e deficiência auditiva, englobando conteúdo específico direcionado a incidência, etiologia e tipos de fissura; deficiência auditiva; adaptação e seleção de AASI, funcionamento da tuba auditiva, otite média, etiologia da alteração de orelha média nos indivíduos com fissura labiopalatina, influência da otite no desenvolvimento da linguagem; tratamento para otite média: intervenção cirúrgica e acompanhamento sistemático e adaptação de AASI. Este blog foi disponibilizado na internet, sendo realizada a avaliação, por meio de dois questionários, o primeiro é o Emory (Emory University Rollins School of Public Health 1998), traduzido para o português (Bastos 2011), e o segundo questionário corresponde à avaliação específica do conteúdo do blog. A avaliação foi realizada por 75 profissionais do HRAC-USP, Bauru e outros, por demanda espontânea. O blog foi elaborado e disponibilizado na web pelo endereço: www.fissuraeaudicao.wordpress.com. No período analisado, de agosto de 2010 a 25 de junho de 2012, foram obtidas 13.746 visitas, sendo a página mais visitada sobre Fissura labiopalatina tipos de fissura, com 4.064 acessos. Além do Brasil diversos países acessaram o conteúdo do blog. O termo de busca mais utilizado foi tuba auditiva. A maioria dos avaliadores do blog (70,7%) era formada por profissionais Fonoaudiólogos. Destes, 89% consideraram o blog como excelente e 11% como adequado. Quanto ao conteúdo, dos 70 profissionais que responderam a este questionário como um todo, 69% consideraram excelente, 27% adequado e 4% consideraram o conteúdo pobre, sendo estes últimos fonoaudiólogos. Assim, um ambiente virtual, em formato de blog, contendo informações sobre fissura labiopalatina e deficiência auditiva para profissionais da área da saúde, foi desenvolvido e avaliado pela maioria dos juízes como excelente. / Due to the intense relationship between cleft palate and hearing loss, this work had as objective to develop and to analyze the effectiveness of a virtual environment of transmission of information on cleft palate and hearing loss for professionals, with the use of Interactive Teleducation. As methodology, an environment of learning in blog format was developed, containing information about cleft lip and palate and hearing loss, encompassing specific content directed to the incidence, etiology and types of cleft lip and palate; hearing loss; selection and adaptation of hearing aids, functioning of the auditory tube, otitis media, etiology of the alteration of the middle ear in individuals with cleft palate, influence of the otitis in the language development; treatment for otitis media: surgical intervention and systematic follow up and adaptation of hearing aids. This blog was made available on the Internet, and the evaluation was made through two questionnaires, being the first one Emory (Emory University Rollins School of Public Health 1998), translated into Portuguese (Bastos 2011), and the second questionnaire corresponds to the specific evaluation of the blog content. The evaluation was carried through by 75 professionals of the HRAC-USP-Bauru and others, through spontaneous demand. The blog was elaborated and made available on the web on the address: www.fissuraeaudicao.wordpress.com. In the analyzed period from August 2010 to June 25, 2012, there were 13.746 visits, being the most visited page about cleft lip and palate - types of cleft lip and palate, with 4.064 accesses. In addition to Brazil, several countries accessed the contents of the blog. The most used search term was auditory tube. The majority of the blog evaluators (70.7%) consisted of professional Speech and Language Pathologists. Of these, 89% considered the blog excellent and 11% adequate. In regard to the content, out of the 70 professionals who had answered this questionnaire as a whole, 69% had considered it excellent, 27% adequate and 4% considered the content poor, being these last ones Speech and Language Pathologists. Thus, a virtual environment, in blog format, containing information on cleft palate and hearing loss for health professionals was developed and evaluated by the majority of the judges as excellent.