A constantly growing corpus of interdisciplinary studies support the idea that music is a complex multimodal medium that is experienced not only by means of sounds but also through body movement. From this perspective, musical instruments can be seen as technological objects coupled with a repertoire of performance gestures. This repertoire is part of an ecological knowledge shared by musicians and listeners alike. It is part of the engine that guides musical experience and has a considerable expressive potential. This thesis explores technical and conceptual issues related to the analysis and creative use of music-related body movements in instrumental music performance. The complexity of this subject required an interdisciplinary approach, which includes the review of multiple theoretical accounts, quantitative and qualitative analysis of data collected in motion capture laboratories, the development and implementation of technologies for the interpretation and interactive use of motion data, and the creation of short musical pieces that actively employ the movement of the performers as an expressive musical feature. The theoretical framework is informed by embodied and enactive accounts of music cognition as well as by systematic studies of music-related movement and expressive music performance. The assumption that the movements of a musician are part of a shared knowledge is empirically explored through an experiment aimed at analysing the motion capture data of a violinist performing a selection of short musical excerpts. A group of subjects with no prior experience playing the violin is then asked to mime a performance following the audio excerpts recorded by the violinist. Motion data is recorded, analysed, and compared with the expert’s data. This is done both quantitatively through data analysis xii as well as qualitatively by relating the motion data to other high-level features and structures of the musical excerpts. Solutions to issues regarding capturing and storing movement data and its use in real-time scenarios are proposed. For the interactive use of motion-sensing technologies in music performance, various wearable sensors have been employed, along with different approaches for mapping control data to sound synthesis and signal processing parameters. In particular, novel approaches for the extraction of meaningful features from raw sensor data and the use of machine learning techniques for mapping movement to live electronics are described. To complete the framework, an essential element of this research project is the com- position and performance of études that explore the creative use of body movement in instrumental music from a Practice-as-Research perspective. This works as a test bed for the proposed concepts and techniques. Mapping concepts and technologies are challenged in a scenario constrained by the use of musical instruments, and different mapping ap- proaches are implemented and compared. In addition, techniques for notating movement in the score, and the impact of interactive motion sensor systems in instrumental music practice from the performer’s perspective are discussed. Finally, the chapter concluding the part of the thesis dedicated to practical implementations describes a novel method for mapping movement data to sound synthesis. This technique is based on the analysis of multimodal motion data collected from multiple subjects and its design draws from the theoretical, analytical, and practical works described throughout the dissertation. Overall, the parts and the diverse approaches that constitute this thesis work in synergy, contributing to the ongoing discourses on the study of musical gestures and the design of interactive music systems from multiple angles.

Smarta kläder är en kroppsnära teknik som består av kläder som har i sig integrerade/invävda sensorer vilka mäter kroppssignaler, arbetsställningar och rörelser och visar information om eventuella överbelastningar. Syftet i denna studie var att utvärdera prototyp 1 av ett smart arbetsklädersystem genom att utreda användarnas upplevelse (user experience) och användbarhet av prototypen samt prototypens påverkan på testpersonernas arbetsbeteende. En kombinerad studiedesign valdes. Den experimentella studien kompletterades med enkät och intervju. Tolv testpersoner deltog i undersökningen, fördelade lika på tre grupper: en kontrollgrupp och två experimentgrupper. Via tekniska mätningar samlades kinematisk data gällande: handledens vinkelhastighet och tummens tryckkraft; överarmens vinkel för höger- och vänster arm; överarmens vinkelhastighet för höger och vänster arm samt bålens vinkel vid fram- och bakåtböjningar. Signifikanta arbetsbeteendeförändringar kunde ej konstateras, men det förekom stora skillnader i de individuella resultaten i varje grupp. Dock kunde inte något specifikt mönster i arbetsbeteendeförändringar från de olika grupperna urskiljas. Hypotesen att arbetsbeteendeförändringar kan tillskrivas påverkan från prototypen stöddes inte av data. Resultatet från intervju- och enkätundersökning kring användarnas upplevelse och användbarhet visade att prototypen skattades som användarvänlig och användbar i sin helhet. Testpersonerna önskade förändringar i prototypens konstruktion gällande prototypens material och storlek, typ av feedback samt placering av sensorer. Vissa brister i design och utförandet av användartesterna och experimentmätningen konstateras och deras inverkan på studiens validitet och reliabilitet diskuteras. Rekommendationer ges gällande framtida testning av nästa prototyp utifrån de upptäckta bristerna. / Smart clothes are a group of technical aids that consist of clothing that has integrated/built-in sensors that measure body postures and movements and display information about possible physical overload. The purpose of this study was to evaluate prototype 1 of a smart workwear system by investigating its user experience and usability, as well as the impact of the prototype on the test subjects' work behavior. A combined study design was chosen. The experimental study was supplemented with questionnaire and interview. Twelve test subjects split in three equal groups participated in the study: one control group and two experimental groups. Through technical measurements, kinematic data was collected: angular velocity of the wrist and thumb pressure; right and left arm's angle; angular velocity for the right and left arm as well as the torso angle during forward and backward bending. Changes in work behavior were found. However, large differences in the individual results within each group were observed, with no obvious pattern of changes in work behavior between groups. The collected data did not support the hypothesis that work behavior changes can be associated with the impact of the prototype.  The result of analysis of the interviews and questionnaires about user experience and usability showed that the prototype was considered user-friendly and useful in general. Test subjects requested changes in the prototype’s construction regarding the prototype’s material and size, type of feedback and location of sensors.  Some shortcomings are observed in the test design, data collection and also in how the tests were conducted. Their impact on the validity and reliability of the study is discussed; accordingly, recommendations addressing the detected shortcomings are given regarding the future testing of the next prototype.

<p>The last two decades have shown a boom in the field of wearable sensing technology. Particularly in the consumer industry, growing trends towards personalized health have pushed new devices to report many vital signs, with a demand for high accuracy and reliability. The most common technique used to gather these vitals is photoplethysmography or PPG. PPG devices are ideal for wearable applications as they are simple, power-efficient, and can be implemented on almost any area of the body. Traditionally PPGs were utilized for capturing just heart rate, however, recent advancements in hardware and digital processing have led to other metrics including respiratory rate (RR) and peripheral oxygen saturation (SpO2), to be reported as well. Our research investigates the potential for wearable devices to be used for outpatient apnea monitoring, and particularly the ability to detect opioid misuse resulting in respiratory depression. Ultimately, the long-term goal of this work is to develop a wearable device that can be used in the rehabilitation process to ensure both accountability and safety of the wearer. This document details contributions towards this goal through the design, development, and evaluation of a device called “Kick Ring”. Primarily, we investigate the ability of Kick Ring to record heartrate (HR), RR, and SpO2. Moreover, we show that the device can calculate RR in real time and can provide an immediate indication of abnormal events such as respiratory depression. Finally, we explore a novel method for reporting apnea events through the use of several PPG characteristics. Kick Ring reliably gathers respiratory metrics and offers a combination of features that does not exist in the current wearables space. These advancements will help to move the field forward, and eventually aid in early detection of life-threatening events.</p>

Traditional fabrication methods used to manufacture biosensors for physiological, therapeutics, or health monitoring purposes are complex and rely on costly materials, which has hindered their adoption as single-use medical devices. The development of a new kind of wearable and implantable electronics relying on inexpensive materials for their manufacturing will pave the way towards the ubiquitous adoption of sticker-like health tracking devices.<div>One of growing and most promising applications for biosensors is the continuous health monitoring using mechanically soft, stretchable sensors. While these healthcare devices showed an excellent compatibility with human tissues, they still need highly trained personnel to perform multi-step, prolonged fabrication for several functioning layers of the device. In this dissertation, I propose low-cost, scalable, simple, and rapid manufacturing techniques to fabricate multifunctional epidermal and implantable sensors to monitor a range of biosignals including heart, muscle, or eye activity to characterizing of biofuids such as sweat. I have also used these devices as an implant to provide heat therapy for muscle regeneration and optical stimulation of neurons using optogenetics. These devices have also combined with those of triboelectric<br>nanogenerators to realize self-powered sensors for monitoring imperceptible mechanical biosignals such as respiratory and pulse rate.</div><div>Food health and safety has also emerged as another important frontier to develop biosensors and improve the human health and quality of life. The recent progresses on detecting microbial activity inside foods or their packages rely on development of highly functional materials. The existing materials for fabrication of food sensors, however,<br>are often costly and toxic for human health or the environment. In this dissertation, I proposed biocompatible food sensors using protein/PCL microfibers to reinforce the protein microfibrous structure in humid conditions and exploit their excellent hygroscopic properties to sense biogenic gas, as an indicator for early detection of food spoilage. Finally, my battery-free food sensors are capable of monitoring food safety with no need of extra measurement devices. Collectively, this dissertation proposes cost-effective solutions to solve human health issues, enabled by developing low-cost, functional materials and exploiting simple fabrication techniques.<br></div>

<p>There are no commercially available wearable shoe in-sole sensors that are capable of measuring and distinguishing between shear and compression forces. Companies have already developed shoe sensors that simply measure pressure and make general inferences on the collected data with elaborate software [2, 3, 4, 5]. Researchers have also attempted making sensors that are capable of measuring shear forces, but they are not well suited for biomedical applications [61, 62, 63, 64]. This work focuses on the development of a novel wearable sensor system that is capable of identifying and measuring shear and compression forces through the use of capacitive sensing. Custom hardware and software tools such as materials test systems and capacitive measurement systems were developed during this work. Numerous sensor prototypes were developed, characterized, and optimized during the scope of this project. Upon analysis of the data, the best capacitive measurement system developed in this work utilized the CAV444 IC chip, whereas the use of the Arduino-derived measurement system required data filtering using median and Butterworth zero phase low pass filters. The highest dielectric constant reported from optimization experiments yielded 9.7034 (+/- 0.0801 STD) through the use of 60.2% by weight calcium copper titanate and ReoFlex-60 silicone. The experiments suggest certain sensors developed in this work feasibly measure and distinguish between shear and compressional forces. Applications for such technology focus on improving quality of life in areas such as managing diabetic ulcer formation, preventing injuries, optimizing performance for athletes and military personnel, and augmenting the scope of motion capture in biomechanical studies.</p>