This paper deals with processing and recognition of events in audio signal. The work explores the possibility of using audio signal visualization and subsequent use of convolutional neural networks as a classifier for recognition in real use. Recognized audio events are gunshots placed in a sound background such as street noise, human voice, animal sounds, and other forms of random noise. Before the implementation, a large database with various parameters, especially reverberation and time positioning within the processed section, is created. In this work are used freely available platforms Keras and TensorFlow for work with neural networks.

Sound is one of a human beings most important senses. After vision, it is the sense most used to gather information about the environment. Despite this, comparatively little research has been done into the field of sound recognition. The research that has been done mainly centres around the recognition of speech and music. Our auditory environment is made up of many sounds other than speech and music. This sound information can be taped into for the benefit of specific applications such as security systems. Currently, most researchers are ignoring this sound information. This thesis investigates techniques to recognise environmental non-speech sounds and their direction, with the purpose of using these techniques in an autonomous mobile surveillance robot. It also presents advanced methods to improve the accuracy and efficiency of these techniques. Initially, this report presents an extensive literature survey, looking at the few existing techniques for non-speech environmental sound recognition. This survey also, by necessity, investigates existing techniques used for sound recognition in speech and music. It also examines techniques used for direction detection of sounds. The techniques that have been identified are then comprehensively compared to determine the most appropriate techniques for non-speech sound recognition. A comprehensive comparison is performed using non-speech sounds and several runs are performed to ensure accuracy. These techniques are then ranked based on their effectiveness. The best technique is found to be either Continuous Wavelet Transform feature extraction with Dynamic Time Warping or Mel-Frequency Cepstral Coefficients with Dynamic Time Warping. Both of these techniques achieve a 70% recognition rate. Once the best of the existing classification techniques is identified, the problem of uncountable sounds in the environment can be addressed. Unlike speech recognition, non-speech sound recognition requires recognition from a much wider library of sounds. Due to this near-infinite set of example sounds, the characteristics and complexity of non-speech sound recognition techniques increases. To address this problem, a systematic scheme needs to be developed for non-speech sound classification. Several different approaches are examined. Included is a new design for an environmental sound taxonomy based on an environmental sound alphabet. This taxonomy works over three levels and classifies sounds based on their physical characteristics. Its performance is compared with a technique that generates a structured tree automatically. These structured techniques are compared for different data sets and results are analysed. Comparable results are achieved for these techniques with the same data set as previously used. In addition, the results and greater information from these experiments is used to infer some information about the structure of environmental sounds in general. Finally, conclusions are drawn on both sets of techniques and areas of future research stemming from this thesis are explored.

Dans beaucoup de pays du monde, on observe une importante augmentation du nombre de personnes âgées vivant seules. Depuis quelques années, un nombre significatif de projets de recherche sur l’assistance aux personnes âgées ont vu le jour. La plupart de ces projets utilisent plusieurs modalités (vidéo, son, détection de chute, etc.) pour surveiller l'activité de la personne et lui permettre de communiquer naturellement avec sa maison "intelligente", et, en cas de danger, lui venir en aide au plus vite. Ce travail a été réalisé dans le cadre du projet ANR VERSO de recherche industrielle, Sweet-Home. Les objectifs du projet sont de proposer un système domotique permettant une interaction naturelle (par commande vocale et tactile) avec la maison, et procurant plus de sécurité à l'habitant par la détection des situations de détresse. Dans ce cadre, l'objectif de ce travail est de proposer des solutions pour la reconnaissance des sons de la vie courante dans un contexte réaliste. La reconnaissance du son fonctionnera en amont d'un système de Reconnaissance Automatique de la Parole. Les performances de celui-ci dépendent donc de la fiabilité de la séparation entre la parole et les autres sons. Par ailleurs, une bonne reconnaissance de certains sons, complétée par d'autres sources informations (détection de présence, détection de chute, etc.) permettrait de bien suivre les activités de la personne et de détecter ainsi les situations de danger. Dans un premier temps, nous nous sommes intéressés aux méthodes en provenance de la Reconnaissance et Vérification du Locuteur. Dans cet esprit, nous avons testé des méthodes basées sur GMM et SVM. Nous avons, en particulier, testé le noyau SVM-GSL (SVM GMM Supervector Linear Kernel) utilisé pour la classification de séquences. SVM-GSL est une combinaison de SVM et GMM et consiste à transformer une séquence de vecteurs de longueur arbitraire en un seul vecteur de très grande taille, appelé Super Vecteur, et utilisé en entrée d'un SVM. Les expérimentations ont été menées en utilisant une base de données créée localement (18 classes de sons, plus de 1000 enregistrements), puis le corpus du projet Sweet-Home, en intégrant notre système dans un système plus complet incluant la détection multi-canaux du son et la reconnaissance de la parole. Ces premières expérimentations ont toutes été réalisées en utilisant un seul type de coefficients acoustiques, les MFCC. Par la suite, nous nous sommes penchés sur l'étude d'autres familles de coefficients en vue d'en évaluer l'utilisabilité en reconnaissance des sons de l'environnement. Notre motivation fut de trouver des représentations plus simples et/ou plus efficaces que les MFCC. En utilisant 15 familles différentes de coefficients, nous avons également expérimenté deux approches pour transformer une séquence de vecteurs en un seul vecteur, à utiliser avec un SVM linéaire. Dans le première approche, on calcule un nombre fixe de coefficients statistiques qui remplaceront toute la séquence de vecteurs. La seconde approche (une des contributions de ce travail) utilise une méthode de discrétisation pour trouver, pour chaque caractéristique d'un vecteur acoustique, les meilleurs points de découpage permettant d'associer une classe donnée à un ou plusieurs intervalles de valeurs. La probabilité de la séquence est estimée par rapport à chaque intervalle. Les probabilités obtenues ainsi sont utilisées pour construire un seul vecteur qui remplacera la séquence de vecteurs acoustiques. Les résultats obtenus montrent que certaines familles de coefficients sont effectivement plus adaptées pour reconnaître certaines classes de sons. En effet, pour la plupart des classes, les meilleurs taux de reconnaissance ont été observés avec une ou plusieurs familles de coefficients différentes des MFCC. Certaines familles sont, de surcroît, moins complexes et comptent une seule caractéristique par fenêtre d'analyse contre 16 caractéristiques pour les MFCC / In many countries around the world, the number of elderly people living alone has been increasing. In the last few years, a significant number of research projects on elderly people monitoring have been launched. Most of them make use of several modalities such as video streams, sound, fall detection and so on, in order to monitor the activities of an elderly person, to supply them with a natural way to communicate with their “smart-home”, and to render assistance in case of an emergency. This work is part of the Industrial Research ANR VERSO project, Sweet-Home. The goals of the project are to propose a domotic system that enables a natural interaction (using touch and voice command) between an elderly person and their house and to provide them a higher safety level through the detection of distress situations. Thus, the goal of this work is to come up with solutions for sound recognition of daily life in a realistic context. Sound recognition will run prior to an Automatic Speech Recognition system. Therefore, the speech recognition’s performances rely on the reliability of the speech/non-speech separation. Furthermore, a good recognition of a few kinds of sounds, complemented by other sources of information (presence detection, fall detection, etc.) could allow for a better monitoring of the person's activities that leads to a better detection of dangerous situations. We first had been interested in methods from the Speaker Recognition and Verification field. As part of this, we have experimented methods based on GMM and SVM. We had particularly tested a Sequence Discriminant SVM kernel called SVM-GSL (SVM GMM Super Vector Linear Kernel). SVM-GSL is a combination of GMM and SVM whose basic idea is to map a sequence of vectors of an arbitrary length into one high dimensional vector called a Super Vector and used as an input of an SVM. Experiments had been carried out using a locally created sound database (containing 18 sound classes for over 1000 records), then using the Sweet-Home project's corpus. Our daily sounds recognition system was integrated into a more complete system that also performs a multi-channel sound detection and speech recognition. These first experiments had all been performed using one kind of acoustical coefficients, MFCC coefficients. Thereafter, we focused on the study of other families of acoustical coefficients. The aim of this study was to assess the usability of other acoustical coefficients for environmental sounds recognition. Our motivation was to find a few representations that are simpler and/or more effective than the MFCC coefficients. Using 15 different acoustical coefficients families, we have also experimented two approaches to map a sequence of vectors into one vector, usable with a linear SVM. The first approach consists of computing a set of a fixed number of statistical coefficients and use them instead of the whole sequence. The second one, which is one of the novel contributions of this work, makes use of a discretization method to find, for each feature within an acoustical vector, the best cut points that associates a given class with one or many intervals of values. The likelihood of the sequence is estimated for each interval. The obtained likelihood values are used to build one single vector that replaces the sequence of acoustical vectors. The obtained results show that a few families of coefficients are actually more appropriate to the recognition of some sound classes. For most sound classes, we noticed that the best recognition performances were obtained with one or many families other than MFCC. Moreover, a number of these families are less complex than MFCC. They are actually a one-feature per frame acoustical families, whereas MFCC coefficients contain 16 features per frame

This master's thesis deals with the design and development of mobile aplication for Android platform. The aim of the work is to implement a simple and user-friendly user interface that would support and assist the user in trainning and sport exercises. The thesis also include implementation of sound detection to support during exercises and voice instruction by application. In practice the application should help in making training exercises more comfortable without the user being forced to keep mobile device in hand.

<p dir="ltr">The onset of smart manufacturing signifies a crucial shift in the industrial landscape, underscoring the pressing need for systems capable of adapting to and managing the complex dynamics of modern production environments. In this context, the importance of smart monitoring becomes increasingly apparent, serving as a vital tool for ensuring operational efficiency and reliability. Inspired by the critical role of auditory perception in human decision-making, this study investigated the application of machine sound recognition for practical use in manufacturing environments. Addressing the challenge of utilizing machine sounds in the loud noises of factories, the study employed an Internal Sound Sensor (ISS).</p><p dir="ltr">The study examined how sound propagates through structures and further explored acoustic characteristics of the ISS, aiming to apply these findings in machine monitoring. To leverage the ISS effectively and achieve a higher level of monitoring, a smart sound monitoring framework was proposed to integrate sound monitoring with machine data and human-machine interface. Designed for applicability and cost effectiveness, this system employs real-time edge computing, making it adaptable for use in various industrial settings.</p><p dir="ltr">The proposed framework and ISS deployed across a diverse range of production environments, showcasing a leap forward in the integration of smart technologies in manufacturing. Their application extends beyond continuous manufacturing to include discrete manufacturing systems, demonstrating adaptability. By analyzing sound signals from various production equipment, this study delves into developing machine sound recognition models that predict operational states and productivity, aiming to enhance manufacturing efficiency and oversight on real factory floors. This comprehensive and practical approach underlines the framework's potential to revolutionize operational management and manufacturing productivity. The study progressed to integrating manufacturing context with sound data, advancing towards high-level monitoring for diagnostic predictions and digital twin. This approach confirmed sound recognition's role in manufacturing diagnostics, laying a foundation for future smart monitoring improvements.</p>

Conselho Nacional de Pesquisa e Desenvolvimento Científico e Tecnológico - CNPq / This work presents a study of the spectral dynamics characteristics of audio signals. More specifically, we aim at detecting regularities that can be modeled in typical domestic sounds, in order to classify them. Our starting point is the work of Sehili et al. [2], in which a household sounds classification system based on GMM is proposed. The Sehili system is reproduced in this work as a baseline system. Following the same protocol of experiments, a 73 % recognition rate is achieved. Afterwards, three sets of experiments are performed, arranged so that each new approach incorporates a new technique to highlight a different aspect of the spectral dynamics. The first technique is the insertion of the discrete gradient information of feature vectors, a strategy aimed at a local spectral dynamic analysis, and resultes in a perceptible increase in recognition rate. The next experiment is conducted with a HMM based classifier, in which the spectral dynamic should be encoded in state transition probability matrices. The tests with the HMM do not result in improved recognition rates. The last experiment is based on a features extraction method, proposed by the author, called Patterns of Energy Envelope per Band (PEEB). The PEEB is an extractor that highlight the signal spectral dynamics inside narrow bands. In domestic sounds recognition tests, the classification system based on a combination of PEEB, MFCC and GMM strategies resulted in a significant improvement over all other systems tested. We conclude, based on our results, that the spectral dynamics of the studied dataset plays an important role in the classification task. However, the approaches for spectral dynamic information extraction, studied in this work, are not definitive, for it is clear that they can be further developed. For example, in the case of PEEB, the recognition rate is strongly dependent on the sound class, suggesting more elaborate forms of fusion of PEEB and MFCC features for each class. / Este trabalho é um estudo da característica da dinâmica espectral em sinais sonoros, com vistas a encontrar as regularidades que podem ser modeladas em sons tipicamente domésticos, com o objetivo de classificá-los. O ponto de partida é o trabalho de Sehili et al. [1], no qual é proposto um sistema de classificação de sons domésticos baseado em GMM. O sistema de Sehili é reproduzido neste trabalho como marco zero na análise da dinâmica espectral, seguindo o mesmo roteiro dos experimentos. A partir daí, três conjuntos de experimentos são realizados, organizados de forma que, a cada novo experimento, uma técnica – que destaca um aspecto diferente da dinâmica espectral – seja incorporada. A primeira técnica analisada é a inserção da informação de gradiente discreto dos vetores de características, estratégia que representa uma análise de dinâmica espectral local e que resulta num aumento perceptível na taxa de classificação. O próximo experimento é realizado com um classificador baseado em HMM, no qual a informação de dinâmica espectral deve ser codificada na matriz de probabilidades de transição de estados do modelo. Os testes com o HMM não resultam em melhora na taxa de reconhecimento das classes de sons. O último experimento é baseado num extrator de características proposto pelo autor, chamado de Padrões de Envelopes de Energia por Banda (PEEB). O PEEB é um extrator que destaca os padrões de evolução espectro-temporais do sinais. Nos testes de reconhecimento de sons domésticos, o sistema de classificação baseado numa combinação das estratégias PEEB, MFCC e GMM resultam numa melhora significativa em relação a todos os outros sistemas testados. Conclui-se, com base nos resultados, que a dinâmica espectral dos sinais da base estudada é relevante à tarefa de classificação. No entanto, as maneiras de extração da informação de dinâmica espectral estudadas neste trabalho não são definitivas, pois ainda há muito espaço para desenvolvê-las. Por exemplo, no caso do PEEB, nota-se que a taxa de classificação fortemente é dependente da classe sonora, sugerindo formas mais elaboradas de fusão das características PEEB e MFCC para cada classe.

Conselho Nacional de Pesquisa e Desenvolvimento Científico e Tecnológico - CNPq / This work presents a study of the spectral dynamics characteristics of audio signals. More specifically, we aim at detecting regularities that can be modeled in typical domestic sounds, in order to classify them. Our starting point is the work of Sehili et al. [2], in which a household sounds classification system based on GMM is proposed. The Sehili system is reproduced in this work as a baseline system. Following the same protocol of experiments, a 73 % recognition rate is achieved. Afterwards, three sets of experiments are performed, arranged so that each new approach incorporates a new technique to highlight a different aspect of the spectral dynamics. The first technique is the insertion of the discrete gradient information of feature vectors, a strategy aimed at a local spectral dynamic analysis, and resultes in a perceptible increase in recognition rate. The next experiment is conducted with a HMM based classifier, in which the spectral dynamic should be encoded in state transition probability matrices. The tests with the HMM do not result in improved recognition rates. The last experiment is based on a features extraction method, proposed by the author, called Patterns of Energy Envelope per Band (PEEB). The PEEB is an extractor that highlight the signal spectral dynamics inside narrow bands. In domestic sounds recognition tests, the classification system based on a combination of PEEB, MFCC and GMM strategies resulted in a significant improvement over all other systems tested. We conclude, based on our results, that the spectral dynamics of the studied dataset plays an important role in the classification task. However, the approaches for spectral dynamic information extraction, studied in this work, are not definitive, for it is clear that they can be further developed. For example, in the case of PEEB, the recognition rate is strongly dependent on the sound class, suggesting more elaborate forms of fusion of PEEB and MFCC features for each class. / Este trabalho é um estudo da característica da dinâmica espectral em sinais sonoros, com vistas a encontrar as regularidades que podem ser modeladas em sons tipicamente domésticos, com o objetivo de classificá-los. O ponto de partida é o trabalho de Sehili et al. [1], no qual é proposto um sistema de classificação de sons domésticos baseado em GMM. O sistema de Sehili é reproduzido neste trabalho como marco zero na análise da dinâmica espectral, seguindo o mesmo roteiro dos experimentos. A partir daí, três conjuntos de experimentos são realizados, organizados de forma que, a cada novo experimento, uma técnica – que destaca um aspecto diferente da dinâmica espectral – seja incorporada. A primeira técnica analisada é a inserção da informação de gradiente discreto dos vetores de características, estratégia que representa uma análise de dinâmica espectral local e que resulta num aumento perceptível na taxa de classificação. O próximo experimento é realizado com um classificador baseado em HMM, no qual a informação de dinâmica espectral deve ser codificada na matriz de probabilidades de transição de estados do modelo. Os testes com o HMM não resultam em melhora na taxa de reconhecimento das classes de sons. O último experimento é baseado num extrator de características proposto pelo autor, chamado de Padrões de Envelopes de Energia por Banda (PEEB). O PEEB é um extrator que destaca os padrões de evolução espectro-temporais do sinais. Nos testes de reconhecimento de sons domésticos, o sistema de classificação baseado numa combinação das estratégias PEEB, MFCC e GMM resultam numa melhora significativa em relação a todos os outros sistemas testados. Conclui-se, com base nos resultados, que a dinâmica espectral dos sinais da base estudada é relevante à tarefa de classificação. No entanto, as maneiras de extração da informação de dinâmica espectral estudadas neste trabalho não são definitivas, pois ainda há muito espaço para desenvolvê-las. Por exemplo, no caso do PEEB, nota-se que a taxa de classificação fortemente é dependente da classe sonora, sugerindo formas mais elaboradas de fusão das características PEEB e MFCC para cada classe.

Ljudigenkänning möjliggörs genom spektralanalys, som beräknas av den snabba fouriertransformen (FFT), och har under senare år nått stora genombrott i samband med ökningen av datorprestanda och artificiell intelligens. Tekniken är nu allmänt förekommande, i synnerhet inom bioakustik för identifiering av djurarter, en viktig del av miljöövervakning. Det är fortfarande ett växande vetenskapsområde och särskilt igenkänning av fågelsång som återstår som en svårlöst utmaning. Även de främsta algoritmer i området är långt ifrån felfria. I detta kandidatexamensarbete implementerades och utvärderades enkla algoritmer för att para ihop ljud med en ljuddatabas. En filtreringsmetod utvecklades för att urskilja de karaktäristiska frekvenserna vid fem tidsramar som utgjorde basen för jämförelsen och proceduren för ihopparning. Ljuden som användes var förinspelad fågelsång (koltrast, näktergal, kråka och fiskmås) så väl som egeninspelad mänsklig röst (4 unga svenska män). Våra resultat visar att framgångsgraden normalt är 50–70%, den lägsta var fiskmåsen med 30% för en liten databas och den högsta var koltrasten med 90% för en stor databas. Rösterna var svårare för algoritmen att särskilja, men de hade överlag framgångsgrader mellan 50% och 80%. Dock gav en ökning av databasstorleken generellt inte en ökning av framgångsgraden. Sammanfattningsvis visar detta kandidatexamensarbete konceptbeviset bakom fågelsångigenkänning och illustrerar såväl styrkorna som bristerna av dessa enkla algoritmer som har utvecklats. Algoritmerna gav högre framgångsgrad än slumpen (25%) men det finns ändå utrymme för förbättring eftersom algoritmen vilseleddes av ljud av samma frekvenser. Ytterligare studier behövs för att bedöma den utvecklade algoritmens förmåga att identifiera ännu fler fåglar och röster. / Sound recognition is made possible through spectral analysis, computed by the fast Fourier transform (FFT), and has in recent years made major breakthroughs along with the rise of computational power and artificial intelligence. The technology is now used ubiquitously and in particular in the field of bioacoustics for identification of animal species, an important task for wildlife monitoring. It is still a growing field of science and especially the recognition of bird song which remains a hard-solved challenge. Even state-of-the-art algorithms are far from error-free. In this thesis, simple algorithms to match sounds to a sound database were implemented and assessed. A filtering method was developed to pick out characteristic frequencies at five time frames which were the basis for comparison and the matching procedure. The sounds used were pre-recorded bird songs (blackbird, nightingale, crow and seagull) as well as human voices (4 young Swedish males) that we recorded. Our findings show success rates typically at 50–70%, the lowest being the seagull of 30% for a small database and the highest being the blackbird at 90% for a large database. The voices were more difficult for the algorithms to distinguish, but they still had an overall success rate between 50% and 80%. Furthermore, increasing the database size did not improve success rates in general. In conclusion, this thesis shows the proof of concept and illustrates both the strengths as well as short-comings of the simple algorithms developed. The algorithms gave better success rates than pure chance of 25% but there is room for improvement since the algorithms were easily misled by sounds of the same frequencies. Further research will be needed to assess the devised algorithms' ability to identify even more birds and voices.

No description available.

本研究旨在探究以可解碼英文故事書(decodable storybook)為主的英語教學，對低年級英語低成就學童的影響，特別是在此補救教學方案下，學童早期英語閱讀能力及學習態度之改變及表現，並探討造成其改變的可能原因。本研究分兩階段進行，第一階段先進行小規模的預試研究作為正式研究之準備，目的在測試選用的英語可解碼故事書做為教材是否適切、教學活動是否可行，並根據初探結果規劃正式研究。第二階段的正式研究，根據第一階段的結果改良部分測驗、增加閱讀讀本、以及增加教師訪談等質性資料，以期進一步探討此補救教學方案對於學童早期閱讀能力及學習態度的影響。 在正式研究中，研究對象為台北市某國小英語攜手班(補救教學班)之五名二年級學習低成就學童（學業表現為後15%至20%）。研究者選擇Scholastic Company所出版的某系列分級可解碼英語故事書之第一套部分冊數作為讀本，搭配由整體到細部的架構理念設計課程，發展出適合本校學童的補救教學方案，將字母拼讀技巧訓練融入自然有趣的閱讀情境中。 本研究為行動研究，透過以質性為主的資料蒐集法，包含課室錄影錄音觀察、訪談、多方文件彙集及前後測評量，來進行研究結果之分析詮釋。學童每週均接受一次教學(每次三節課，共120分鐘)，共持續十六週。經過一學期補救教學後，對照學童前後測評量及各種質性資料分析，發現學童在字母認讀、單字辨識、單字拼讀及學習態度上均有正向改變。本研究也發現，使用可解碼故事書的內容來設計閱讀及自然發音教學活動，的確有助於低成就學童早期閱讀能力及內在動機之提升。本研究之結果及教學建議可供未來國小補救教學之參考。 / The purpose of this study is to examine the effect of decodable storybook instruction on the early reading performances and learning attitudes of Taiwanese EFL lower-grade underachievers in an elementary school in Taipei. An attempt is also made to explore the possible reasons for any improvement among the underachievers’ with respect to this remedial program. The present study was conducted in two stages. A small-scale pilot study was implemented as a preparation for the formal study in advance. The purpose of the pilot study was to investigate if the reading text of the selected storybook series is appropriate for underachievers, the feasibility of the decodable storybook instruction and the effects of the instruction on learners’ early reading ability and learning attitudes. Afterwards, the formal study was revised in modifying some test words as well as the scoring procedure on the early reading assessments, adding more reading materials and qualitative data sources such as teacher interviews. In the formal study, the participants are one remedial English class of five second graders (whose academic performances were at the bottom 15% to 20% of the grade). The curriculum was based on the contents of a commercial series of decodable story books—Clifford Phonics Fun Reading Program—pack one. The researcher adopted the framework of whole-to-parts phonics instruction to integrate reading activities and phonics training in an interesting and meaningful reading context. A variety of reading activities integrating four skills were designed and based on the content and topic of the storybooks to rouse learners’ interest. Decodable CVC words embedded in the story series were then explicitly taught to learners to apply letter-sound knowledge in their reading process. This study triangulates action research with qualitative data using classroom observations (video or taping), interviews, assessment records, and document analysis. Data analysis interprets the study results. After the sixteen weeks of remedial instruction, a comparison between the results of pre-tests and post-tests showed some changes in learners’ early reading ability, including letter-sound recognition, word recognition and visual blending, as well as a positive change in learning attitudes. The other findings were as follows. (1) Multiple teaching activities generated from the decodable story book can be beneficial to learners’ early reading ability and learning motivation. (2) The decodable story book is an effective language learning medium for learners to apply their decoding skills as well as to provide them with an interesting reading context to lower their anxiety in learning English. Based on the findings, a number of suggestions and pedagogical implications are provided for EFL elementary school teachers and further studies in the remedial instruction field alike.