In the modern computing landscape, the challenge of tuning software systems is exacerbated by the necessity to accommodate multiple divergent execution environments and stakeholders. Achieving optimal performance requires a different configuration for every combination of hardware setups and business requirements. In addition, the state of the art in system tuning can involve complex statistical models and tools which require deep expertise not commonly possessed by the average software engineer. As an alternative approach to performance tuning, this thesis puts forward the use of sonification-conveying information via non-speech audio-to aid software engineers in tuning complex systems. In particular, this thesis designs, develops, and evaluates a tuning system that interactively (i.e., in response to user actions) sonifies the performance metrics of a computer system. This thesis demonstrates that interactive sonification can effectively guide software engineers through performance tuning of a computer system. To that end, a scientific survey determined which sound characteristics (e.g., loudness, panning, pitch, tempo, etc.) are best suited to express information to the engineer. These characteristics were used to create a proof-of-concept tuning system that was applied to tune the parameters of a real world enterprise application server. Equipped with the tuning system, engineers-not experts in enterprise computing nor performance tuning-were able to tune the server, so that its resulting performance surpasses that exhibited under the standard configuration. The results indicate that sound-based tuning approaches can provide valuable solutions to the challenges of configuring complex computer systems. / Master of Science

Immersive analytics is an emerging field of data exploration and analysis in immersive environments. It is an active research area that explores human-centric approaches to data exploration and analysis based on the spatial arrangement and visualization of data elements in immersive 3D environments. The availability of immersive extended reality systems has increased tremendously recently, but it is still not as widely used as conventional 2D displays. In this dissertation, we described an immersive analysis system for spatiotemporal data and performed several user studies to measure the user performance in the developed system, and laid out design guidelines for an immersive analytics environment. In our first study, we compared the performance of users based on specific visual analytics tasks in an immersive environment and on a conventional 2D display. The approach was realized based on the coordinated multiple-views paradigm. We also designed an embodied interaction for the exploration of spatial time series data. The findings from the first user study showed that the developed system is more efficient in a real immersive environment than using it on a conventional 2D display. One of the important challenges we realized while designing an immersive analytics environment was to find the optimal placement and identification of various visual elements. In our second study, we explored the iterative design of the placement of visual elements and interaction with them based on frames of reference. Our iterative designs explored the impact of the visualization scale for three frames of reference and used the collected user feedback to compare the advantages and limitations of these three frames of reference. In our third study, we described an experiment that quantitatively and qualitatively investigated the use of sonification, i.e., conveying information through nonspeech audio, in an immersive environment that utilized empirical datasets obtained from a multi-dimensional geophysical system. We discovered that using event-based sonification in addition to the visual channel was extremely effective in identifying patterns and relationships in large, complex datasets. Our findings also imply that the inclusion of audio in an immersive analytics system may increase users’ level of confidence when performing analytics tasks like pattern recognition. We outlined the sound design principles for an immersive analytics environment using real-world geospace science datasets and assessed the benefits and drawbacks of using sonification in an immersive analytics setting. / Doctor of Philosophy / When it comes to exploring data, visualization is the norm. We make line charts, scatter plots, bar graphs, or heat maps to look for patterns in data using traditional desktop-based approaches. However, biologically humans are optimized to observe the world in three dimensions. This research is motivated by the idea that representing data in immersive 3D environments can provide a new perspective that may lead to the discovery of previously undetected data patterns. Experiencing the data in three dimensions, engaging multiple senses like sound and sight, and leveraging human embodiment, interaction capabilities, and sense of presence may lead to a unique understanding of the data that is not feasible using traditional visual analytics. In this research, we first compared the data analysis process in a mixed reality system, where real and virtual worlds co-exist, versus doing the same analytical tasks in a desktop-based environment. In our second study, we studied where different charts and data visualizations should be placed based on the scale of the environment, such as table-top versus room-sized. We studied the strengths and limitations of different scales based on the visual and interaction design of the developed system. In our third study, we used a real-world space science dataset to test the liabilities and advantages of using the immersive approach. We also used audio and explored what kinds of audio work for which analytical tasks and laid out design guidelines based on audio. Through this research, we studied how to do data analytics in emerging mixed reality environments and presented results and design guidelines for future developers, designers, and researchers in this field.

This dissertation explains two sonification platforms designed for image-sound association study and art, and the study results. A platform for user study was developed first and an artistic audiovisual platform was derived based on it. First, the five image-sound association studies were conducted to see whether people can successfully associate sounds and fundamental shapes (i.e., a circle, a triangle, a square, lines, curves, and other custom shapes) and the correct answer rate was high. Then, the same sonification platform was transformed by adding colors to the audiovisual platform for artistic/musical expression. A line-by-line sonification method and an object-oriented method were newly developed to sonify the background and the shapes separately. To enhance user experience, the sound of each shape was spatialized in a multi-layer speaker environment or a virtual listening environment. / Doctor of Philosophy / This dissertation introduces two sonification platforms designed for image-sound association study and art, and the study results. A platform for user study was developed first and an artistic audiovisual platform was derived based on it. First, the five image-sound association studies were conducted to see whether people can successfully associate sounds and fundamental shapes (i.e., a circle, a triangle, a square, lines, curves, and other custom shapes) and the correct answer rate was high. Then, the same sonification platform was transformed by adding colors to the audiovisual platform for artistic/musical expression. A line-by-line sonification method and an object-oriented method were newly developed to sonify the background and the shapes separately. To enhance user experience, the sound of each shape was spatialized in a multi-layer speaker environment or a virtual listening environment.

Afin de réduire les consommations de carburant et les émissions de CO2, des technologies comme la désactivation de cylindres ont été développées. Deux stratégies ont été à l’étude chez le Groupe PSA, il s’agit de désactivations appelées fixe ou tournante en fonction du nombre de cylindres désactivés au cours d’un cycle moteur. Des modifications importantes du bruit et des vibrations transmises dans l’habitacle de la voiture en sont les conséquences ce qui modifie nettement la perception du confort par le conducteur de la voiture. Ce travail de recherche s’intéresse aux modifications de confort perçu par le conducteur dues à ces différentes stratégies appliquées au moteur ainsi qu’aux solutions potentielles qui permettraient d’améliorer ce confort. Parmi ces solutions se trouve le principe de sonification en temps réel du bruit moteur. Pour répondre à ces questions, cinq tests perceptifs sont réalisés. Les deux premiers consistent en l’évaluation du confort global dans différentes configurations ainsi qu’à la validation de l’utilisation d’un simulateur vibro-acoustique. Ces expériences ont montré qu’une des stratégies de désactivation était jugée significativement plus inconfortable que les autres. Par la suite, une seule des stratégies sera retenue. La troisième expérience permet de déterminer un seuil d’acceptabilité entre les sollicitations du moteur avec désactivation et celles du moteur habituel. Ceci afin de fixer une cible pour laquelle l’environnement vibro-acoustique serait acceptable. Dans les deux dernières expériences, deux solutions permettant d’atteindre cette cible sont simulées. La première consiste à modifier la plage de régime moteur ou à lieu la désactivation, la deuxième consiste à assouplir les suspensions horizontales afin de limiter les résonances vibratoires à faible régime. / The technology of cylinder of deactivation has been developed in order to reduce gas consumption and CO2 emissions. Two strategies were studied at groupe PSA called fixed and rotating deactivations depending on the number of cylinders deactivated per engine cycle. This implies non neglecting modifications of sound and vibrations transmitted to the car cabin depredating the global comfort of the car. This research work focused on how driver’s comfort was altered by these engine configurations and how it would be possible to improve this comfort. Among the solutions possible, appears the principle of real-time sonification of the engine noise. To answer these questions, five perceptual experiments have been conducted. First, the aim was to evaluate global comfort with different engine configurations and validate the use of a vibro-acoustic simulator. This showed that one deactivation strategy was significantly reducing the comfort evaluation. Then, we focused on the second strategy which was also considered as not comfortable. On the third experiment, we were interested in finding an acceptable threshold between the vibrations and sounds with the deactivation and with the usual engine configuration. This led to a target of signal to reach in order to provide acceptable situations in terms of sound and vibrations. In the last two experiments, we were interested in the simulation of two solutions about the deactivation settings that would reduce the annoyance: the modification of the engine speed range in which the deactivation occurs and the softening of horizontal suspensions part in order to reduce vibrations resonances at low engine speed.

Cette thèse s'est intéressée aux relations entre les mouvements biologiques et la perception sonore en considérant le cas spécifique des mouvements graphiques et des sons de frottement qu'ils génèrent. L'originalité de ces travaux réside dans l'utilisation d'un modèle de synthèse sonore basé sur un principe perceptif issu de l'approche écologique de la perception et contrôlé par des modèles de gestes. Des stimuli sonores dont le timbre n'est modulé que par des variations de vitesse produites par un geste ont ainsi pu être générés permettant de se focaliser sur l'influence perceptive de cet invariant transformationel. Une première étude a ainsi montré que l'on reconnait la cinématique des mouvements biologiques (la loi en puissance 1/3), et que l'on peut discriminer des formes géométriques simples juste à partir des sons de frottement produits. Une seconde étude a montré l'existence de prototypes dynamiques sonores caractérisant les trajectoires elliptiques, mettant ainsi en évidence que les prototypes géométriques peuvent émerger d'un couplage sensorimoteur. Enfin, une dernière étude a montré qu'une cinématique évoquée par un sonore influence significativement la cinématique et la géométrie d'un geste dans une tâche de reproduction graphique du mouvement d'un point lumineux. Ce résultat révèle l'importance de la modalité auditive dans l'intégration multisensorielle des mouvements continus dans une situation jamais explorée. Ces résultats ont permis le contrôle de modèles de synthèse par des descriptions gestuelles et la création d'outils de sonification pour l'apprentissage de gestes et la réhabilitation d'une pathologie motrice, la dysgraphie. / This thesis focused on the relations between biological movements and auditory perception in considering the specific case of graphical movements and the friction sounds they produced. The originality of this work lies in the use of sound synthesis processes that are based on a perceptual paradigm and that can be controlled by gesture models. The present synthesis model made it possible to generate acoustic stimuli which timbre was directly modulated by the velocity variations induced by a graphic gesture in order to exclusively focus on the perceptual influence of this transformational invariant. A first study showed that we can recognize the biological motion kinematics (the 1/3 power law) and discriminate simple geometric shapes simply by listening to the timbre variations of friction sounds that solely evoke velocity variations. A second study revealed the existence of dynamic prototypes characterized by sounds corresponding to the most representative elliptic trajectory, thus revealing that prototypical shapes may emerged from sensorimotor coupling. A final study showed that the kinematics evoked by friction sounds may significantly affect the dynamic and geometric dimension in the visuo-motor coupling. This shed critical light on the relevance of auditory perception in the multisensory integration of continuous motion in a situation never explored. All of these theoretical results enabled the gestural control of sound synthesis models from a gestural description and the creation of sonification tools for gesture learning and rehabilitation of a graphomotor disease, dysgraphia.

Computer usage today is predominantly based on graphical interaction, where the visual presentation of information is essential both for input (hand-eye coordination when using a computer mouse), and output (seeing the information on a computer screen). This can create difficulties for blind computer users, both at an individual level when interacting with a computer, and also when collaborating with other computer users. The work presented in this thesis has investigated interaction for blind computer users in three stages. First investigating access to information by making studies on an interactive audio-only game, drawing conclusions about auditory direct manipulation and auditory interface design. Second studying collaboration between blind and sighted computer users in two different contexts, leading to questioning of commonly expressed design principles regarding access to collaboration. Finally studying accessibility in a working environment, finding out how technology, the assistive device used by the blind person, communication with others and professional knowledge interplayed to create an accessible work environment. Based on these empirical studies, the main conclusion from this work is a proposal of a research perspective, Assistive interfaces as cooperative interfaces. Here, the context where the interface is going to be used is in focus, and cooperative and social dimensions of interaction are acknowledged and highlighted. The design and analysis of assistive devices should be highly sensitive to the socio-interactional environment, and not just focusing on the single individual using an assistive device. / QC 20100921

In this project, I have developed and built a new prototype of the ColoroPhone visual aid. This new prototype has sensors to measure the distance to objects in front of the wearer, which then communicates with a mobile application that generates sounds depending on the distance to the object. The user will hear this sound through bone-conducting headphones This addition to the ColoroPhone is intended to further enhance the assistance the device can offer. The current prototype is a functioning version and will be used in further research done by a group of psychologists, which will study how such devices affect users both on a biological and social level. They will also evaluate how to approach user training. The prototype in its current state have several segments that have considerable room for improvement in further development of the platform. Some of these improvements can be in form of better Bluetooth implementation, better or more sensors, or improvements in the current set of sensors regarding noise handling and data processing. This type of device can have large impact on the daily life for a large and growing amount of people across the world, as independence is a large factor in a persons feel of self-worth and long term psychological health.

Orientadora: Profª. Drª. Márcia Aparecida da Silva Spinacé / Dissertação (mestrado) - Universidade Federal do ABC. Programa de Pós-Graduação em Ciência e Tecnologia/Química, 2015. / Na atualidade, tem aumentado o debate e interesse por processos de produção limpos e economicamente viáveis de celulose nanoestruturada, devido a sua importância em diversas áreas, notoriamente, nos ramos automobilístico e aeronáutico. Um dos processos que vem ganhando destaque é o método de sonificação, de forma alternativa à extração por meio de hidrólise ácida. As fibras oriundas de resíduos agroindustriais podem ser submetidas ao processo de sonificação em diferentes condições de temperatura, quantidades e tempos, as quais também podem ser funcionalizadas, a fim de seu uso como reforço de matrizes poliméricas. Neste trabalho, estudou-se as condições de sonificação em meio aquoso para obtenção de celulose nanoestruturada, com base no resíduo têxtil. Para essa finalidade, foram preparados compósitos com o uso do polietileno de baixa densidade como matriz 3, 6 e 9 % em massa de celulose nanoestruturada com alta cristalinidade, que foi obtida por sonificação ou celulose nanoestruturada com alta cristalinidade obtida por sonificação + poli (óxido de etileno), adicionando-se o óleo mineral puro, a fim de otimizar o processamento durante a moldagem. O poli (óxido de etileno) foi usado na proporção 1:10 em função da celulose nanoestruturada com alta cristalinidade obtida por sonificação + poli (óxido de etileno), para melhorar a interação fibra-matriz. Os resultados mostraram que foi possível a obtenção da celulose nanoestruturada, por meio do método de sonificação, que é um método sem a geração de resíduos. Portanto, a celulose nanoestruturada apresentou picos cristalinos característicos dos cristais polimórficos de uma mistura de celulose I e celulose II, evidenciando um valor de cristalinidade superior a celulose nanoestruturada obtida via química, que, por sua vez, demonstrou a importância do estudo das condições de otimização desse processo mecânico. Nessa direção, a análise por espectroscopia, na região do infravermelho, indicou que não há mudanças na estrutura química da celulose nanoestruturada, após a sonificação. A análise térmica mostrou estabilidade da celulose nanoestruturada sonificada superior à fibra sem tratamento e à fibra obtida via hidrólise química, indicando a possibilidade de uso como matriz em polímeros com temperaturas de fusão superiores às poliolefinas. Já a análise morfológica mostrou a redução nas dimensões da fibra e boa dispersão das fibras após ciclos de moldagem por compressão. Sendo assim, os resultados mecânicos de tração são comparáveis aos verificados na literatura para compósitos similares obtidos por processo de ix extrusão o qual permite uma melhor mistura e dispersão dos compósitos. A celulose nanoestruturada obtida mostrou ser interessante para aplicação em compósitos poliméricos. / At the present time, it has increased the debate and interest in cleaner production processes and economically viable nanostructured cellulose, due to its importance in several areas, notably in the automotive and aeronautical branches. One process that has been gaining importance is the sonication method, the extraction alternatively by acid hydrolysis. The fibers coming from agro-industrial waste can be subjected to sonification process in different conditions of temperature, amounts and times, which can also be functionalized, in order of their use as reinforcing polymeric matrices. In this study, we studied the conditions of sonication in an aqueous medium to obtain nanostructured cellulose-based textile residue. For this purpose, composites were prepared by using low density polyethylene as the matrix 3, 6 and 9 mass% of nanostructured cellulose with high crystallinity, which was obtained by sonication or nanostructured cellulose having high crystallinity obtained by sonification + poly (oxide ethylene), adding pure mineral oil in order to optimize the processing during molding. The poly (ethylene oxide) was used at 1:10 depending on the nanostructured cellulose having high crystallinity obtained by sonification + poly (ethylene oxide) to improve fiber-matrix interactions. The results showed that it was possible to obtain nanostructured cellulose, by means of sonication method, a method that is without generating waste. Therefore, the nanostructured cellulose was characteristic peaks of crystalline polymorphic crystals of a mixture of cellulose I and cellulose II, showing a higher value crystallinity cellulose obtained via the nanostructured chemical which, in turn, has demonstrated the importance of studying optimization conditions this mechanical process. In this sense, the analysis by spectroscopy in the infrared region, indicated that no changes in the structure of the nanostructured chemical pulp, after sonication. Thermal analysis showed stability of nanostructured sonicated above the pulp fiber and untreated fiber obtained via chemical hydrolysis, indicating the possibility of use as matrix polymers with melting temperatures higher than those polyolefins. Since the morphological analysis showed a reduction in fiber dimensions and good dispersion of the fibers after compression molding cycles. Thus, the mechanical traction results are comparable to those observed in the literature for similar composites obtained by the extrusion process which allows better mixing and dispersion of the composite. The nanostructured cellulose obtained proved to be interesting for application in polymeric composites.

This portfolio of compositions investigates the adaptation of state-space models, frequently used in engineering control theory, to the electroacoustic composition context. These models are mathematical descriptions of physical systems that provide several variables representing the system’s behaviours. The composer adapts a set of state-space models of either abstract, mechanical or electrical systems to a music creation environment. She uses them in eight compositions: five mixed media multi-channel pieces and three mixed media pieces. In the portfolio, the composer investigates multiple ways of meaningfully mapping these system’s behaviours into music parameters. This is done either by exploring and creating timbre in synthetic sound, or by transforming existing sounds. The research also involves the process of incorporating state-space models as a real-time software tool using Max and SuperCollider. As real-time models offer several variables of continuous evolutions, the composer mapped them to different dimensions of sound simultaneously. The composer represented the model’s evolutions with either short/interrupted, long or indefinitely evolving sounds. The evolution implies changes in timbre, length and dynamic range. The composer creates gestures, textures and spaces based on the model’s behaviours. The composer explores how the model’s nature influences the musical language and the integration of these with other music sources such as recordings or musical instruments. As the models represent physical processes, the composer observes that the resulting sounds evolve in organic ways. Moreover, the composer not only sonifies the real-time models, but actually excites them to cause changes. The composer develops a compositional methodology which involves interacting with the models while observing/designing changes in sound. In that sense, the composer regards real-time state-space models as her own instruments to create music. The models are regarded as additional forces and as sound transforming agents in mixed media pieces. In fixed media pieces, the composer additionally exploits their linearity to create space through sound de-correlation.

Hur låter miljöförstöring – Självgenererande och slumpmässig sprungen ur statistiska data är ett mastersarbete som fokuserar på hur man kan omvandla data till att kontrollera musik och ljud. Om vi tillåter konstnärliga uttryck, med data som utgångspunkt, kan det få oss att uppleva och förstår original data på ett nytt sätt? Projektets resultat består av sju generativa kompositioner, där parametrar är kontrollerade av olika typer av miljödata, och försöker utforska forskningsområdet sonifikation och generativ musik genom att ställa frågan: Hur låter miljöförstöring? Med generativ musik menas att musiken skapar, utvecklar och förändrar sig själv utifrån de verktyg som bildats inom detta projekt. Texten går också djupare in på metoden av att utveckla dessa verktyg för att möjliggöra liknande kompositioner i framtiden. Med erfarenheter av att använda slumpmässiga parametrar för att manipulera bakgrundsdetaljer, utforskar det här projekt istället möjligheten att utveckla de metoderna och applicera det på hela kompositioner. De sju kompositionerna ligger också till grund för utforskandet av området sonifikation. I tidigare forskning har ämnet främst bemöts ifrån ett vetenskapligt perspektiv. Syftet med det här projektet har istället varit att bemöta det inom ramarna för musikalisk gestaltning och ett konstnärligt perspektiv. Begreppet sonifikation betyder, användandet av icke-talande ljud som uppmärksammar data och statistik, med målet att agera som ett substitut, eller ett komplement, till att visualisera data. Utifrån dessa kompositioner reflekterar sedan texten kring generativ musik i allmänhet, och sonifikation i synnerhet, där bland annat möjligheterna, framtida forskning och autenticiteten inom sonifikation tas upp. / What does environmental pollution sound like? – Self generative and randomized music interprets data is a master thesis focused on transforming data and letting it control music and sounds. If we create artistic outputs out of data, will it allow us to experience and understand the original data in a new way? The core, and the result, of this project where seven compositions, created and controlled by different environmental data which tries to explore the research areas of sonification and generative music by asking the question: What does environmental pollution sound like? Generative music means that the music creates, develops and changes itself based on the established tools that this project provides. This thesis also focuses on the method of developing these tools in order to enable similar productions in the future. With previous experiences in using randomized events, to manipulate details in a production, this project delves deeper into applying the same technique to a whole composition. The seven compositions were formed in order to understand and reflect upon the research areas of sonification. Earlier research tends to approach the subject from a scientific perspective. The purpose of this project was to instead approach it from a more artistic perspective. Sonification means, the use of non-speech audio to perceptualize data which enables the possibilities as an alternative, or complement, to visualize the original data. Drawing from these seven compositions, this thesis also discusses generative music and sonification in general, as well as the opportunities, future research and authenticity of sonification. / <p>Bifogad ljudfil är ett kollage av de sju kompositionerna som arbetet resulterat i, med anledning av att i framtiden kunna publiceras i sin helhet.</p>