This thesis develops and presents a design methodology that enables designers to work with the moving body in the design and evaluation of interactive, immersive environments built on motion-sensing technologies. The notion of making strange, that underpins the methodology, calls for designers to re-examine and revitalise their assumptions and conceptions of the moving body through bodily-based movement inquiries. This thesis addresses research questions about ways of understanding human movement, of describing and representing human movement and of accessing the felt experience of the moving body in the emerging field of movement-based interaction design. The research questions were explored through a series of three distinct, yet related, projects, each one focusing on different aspects of designing for moving bodies in interactive, immersive environments. The first project analysed an existing interactive product, Sony Playstation2c EyetoyTM, as a prototype of future movement-based interactive, immersive environments. The second project involved the design and development of a specific interactive, immersive artwork, Bystander. The third project worked with trained dancers and physical performers in a constructed design situation. The contributions of this research are first and foremost the design methodology of Moving and Making Strange: a design approach to movement-based interaction that prioritises the lived experience of movement by both designers and users and values the creative potential of the experiential, moving body. It consists of methods and tools for exploring, experiencing, describing, representing and generating movement that enable designers to shift between the multiple perspectives of the mover, the observer and the machine. It makes particular contributions as follows: • Laban movement analysis and Labanotation as a design tool. • Moving-Sensing schema: Suchman’s analytic framework adapted as a design tool. • Extension of existing human-centred design tools to explicitly represent moving bodies, in the form of movement-oriented personas and movementoriented scenarios. • Patterns of watching: a catalogue of audience behaviour in terms of movements and stillness in relation to engagement with a specific interactive, immersive artwork. • New methods for generating, enacting and experiencing movement, sourced from dance and movement improvisation practices.

This thesis develops and presents a design methodology that enables designers to work with the moving body in the design and evaluation of interactive, immersive environments built on motion-sensing technologies. The notion of making strange, that underpins the methodology, calls for designers to re-examine and revitalise their assumptions and conceptions of the moving body through bodily-based movement inquiries. This thesis addresses research questions about ways of understanding human movement, of describing and representing human movement and of accessing the felt experience of the moving body in the emerging field of movement-based interaction design. The research questions were explored through a series of three distinct, yet related, projects, each one focusing on different aspects of designing for moving bodies in interactive, immersive environments. The first project analysed an existing interactive product, Sony Playstation2c EyetoyTM, as a prototype of future movement-based interactive, immersive environments. The second project involved the design and development of a specific interactive, immersive artwork, Bystander. The third project worked with trained dancers and physical performers in a constructed design situation. The contributions of this research are first and foremost the design methodology of Moving and Making Strange: a design approach to movement-based interaction that prioritises the lived experience of movement by both designers and users and values the creative potential of the experiential, moving body. It consists of methods and tools for exploring, experiencing, describing, representing and generating movement that enable designers to shift between the multiple perspectives of the mover, the observer and the machine. It makes particular contributions as follows: • Laban movement analysis and Labanotation as a design tool. • Moving-Sensing schema: Suchman’s analytic framework adapted as a design tool. • Extension of existing human-centred design tools to explicitly represent moving bodies, in the form of movement-oriented personas and movementoriented scenarios. • Patterns of watching: a catalogue of audience behaviour in terms of movements and stillness in relation to engagement with a specific interactive, immersive artwork. • New methods for generating, enacting and experiencing movement, sourced from dance and movement improvisation practices.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第21917号 / 情博第700号 / 新制||情||120(附属図書館) / 京都大学大学院情報学研究科システム科学専攻 / (主査)教授 加納 学, 教授 杉江 俊治, 教授 大塚 敏之 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

The growth in the adoption of sensing technologies in the construction industry has triggered the need for graduating construction engineering students equipped with the necessary skills for deploying the technologies. For construction engineering students to acquire technical skills for implementing sensing technologies, it is pertinent to engage them in hands-on learning with the technologies. However, limited opportunities for hands-on learning experiences on construction sites and in some cases, high upfront costs of acquiring sensing technologies are encumbrances to equipping construction engineering students with the required technical skills. Inspired by opportunities offered by mixed reality, this study presents an interactive holographic learning environment that can afford learners an experiential opportunity to acquire competencies for implementing sensing systems on construction projects. Firstly, this study explores the required competencies for deploying sensing technologies on construction projects. The current state of sensing technologies in the industry and sensing technology education in construction engineering and management programs were investigated. The learning contents of the holographic learning environment were then driven by the identified competencies. Afterwards, a learnability study was conducted with industry practitioners already adopting sensing technologies to assess the learning environment. Feedback from the learnability study was implemented to further improve the learning environment after which a usability evaluation was conducted. To investigate the pedagogical value of the learning environment in construction education, a summative evaluation was conducted with construction engineering students. This research contributes to the definition of the domain-specific skills required of the future workforce for implementing sensing technologies in the construction industry and how such skills can be developed and enhanced within a mixed reality learning environment. Through concise outline and sequential design of the user interface, this study further revealed that knowledge scaffolding can improve task performance in a holographic learning environment. This study contributes to the body of knowledge by advancing immersive experiential learning discourses previously confined by technology. It opens a new avenue for both researchers and practitioners to further investigate the opportunities offered by mixed reality for future workforce development. / Doctor of Philosophy / The construction industry is getting technically advanced and adopting various sensing technologies for improving construction project performance, reducing cost, and mitigating health and safety hazards. As a result, there is a demand in the industry for graduates that can deploy these sensing technologies on construction projects. However, for construction engineering students to acquire the skills for deploying sensing technologies, it is necessary that they are trained through hands-on interactions with these technologies. It is also imperative to take these students to construction sites for experiential learning of sensing technologies. This is difficult because most institutions often experience barriers and hindrances like weather constraints, difficulty in accessing jobsites, and schedule constraints. Also, while some institutions can afford these sensing technologies, others cannot, making it difficult to train students adequately. Due to the benefits of virtual learning environments (such as mixed reality and virtual reality), this study investigates a mixed reality (holographic) environment that can allow learners an experiential opportunity to acquire competencies for implementing sensing systems on construction projects. To achieve this, this research first investigated the required competencies such as skills, knowledge, and abilities for implementing sensing technologies on construction projects. The current state of sensing technologies in the industry and sensing technology education in construction engineering and management programs were investigated. The results from the first study in this research informed the learning contents of the learning environment. Afterwards, a learnability study was conducted with industry practitioners already adopting sensing technologies to assess the learning environment. Feedback from the learnability study was implemented to further improve the learning environment after which a usability evaluation was conducted. To investigate the pedagogical value of the learning environment in construction education, a summative evaluation was conducted with construction engineering students. The research contributes to the definition of the domain-specific skills required of the future workforce for implementing sensing technologies in the construction industry and how such skills can be developed and enhanced within a mixed reality learning environment. The design features such as the concise outline and sequential design of the user interface, further revealed that knowledge scaffolding can improve task performance in a mixed reality environment. This research further contributes to the body of knowledge by promoting immersive hands-on learning discourses previously confined by technology. It opens a new avenue for both researchers and practitioners to further investigate the opportunities offered by mixed reality for future workforce development.

Au cours des dernières années, l’avènement des technologies de captation du mouvement a radicalement transformé l’univers de la pratique artistique tout en ouvrant des perspectives inédites pour la recherche scientifique. La musique est actuellement l’un des domaines les plus impliqués dans ce renouvellement expressif et épistémologique. Dans ce cadre, les processus d’interaction entre médiation technologique, mouvement et son, semblent se décliner selon deux modalités majeures : d’une part, les technologies d’analyse du mouvement permettent d’étudier expérimentalement la connexion mutuelle entre phénomène acoustique et système sensori-moteur; de l’autre, la compréhension de la nature incarnée de l’expérience musicale oriente la conception et le développement de technologies interactives pour la performance vers un modèle plus holistique. En partant de ces prémisses, cette thèse porte sur la manière dont la transformation des aspects imperceptibles du mouvement en données perceptibles – sous forme de son – permet de prendre conscience des processus physiologiques et figuratifs qui sont à la base du geste. Dans ce contexte, la relation entre mouvement et feedback sonore est analysée selon une perspective écologique visant à mettre en lumière comment la médiation technologique induit un processus d’extension et d’intensification autopoïétique de l’anatomie corporelle. Notamment dans le cas de la pratique performative, l’interaction sonore offre alors au performeur la possibilité de redéfinir sa propre organisation perceptive sur la base d’un un nouveau répertoire des données sensorielles, lui permettant ainsi de repenser la composition expressive du mouvement. / During the last years, motion sensing technologies have radically transformed the universe of the artistic practice. This dramatic change has recently inspired new perspectives in scientific research. Music is actually among the most affected domaines by this expressive and epistemological renewal. The interactive relation between mediation technology, movement and sound, seems to be declined into two main modalities : on one hand, movement analysis’ technologies allow to study mutual connections between acoustic phenomenon and sensorimotor system, on the other hand, embodied understanding of musical experience can help to devise an holistic approach to interactive systems conception and development. Given this background scenario, this thesis focuses on how movement’s qualities transformation into sound allows the performer to become aware of physiological and imaginative processes in gesture composition. In this framework, sound feedback-movement relation is analyzed from an ecological point of view. According to this approach, mediation technology seems to elicit an autopoietic process of extension and intensification of corporeality. Especially in the artistic performance, sound interaction offers to performer a new sensorial geography that allows him/her to renew his/her perceptive organization and thereby rethink expressive composition of movement.