Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2016. / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 63-65). / We present Sensei, the first system designed to understand social interaction and learning in an early-childhood classroom using a distributed sensor network. Our unobtrusive sensors measure proximity between each node in a dynamic range-based mesh network. The sensors can be worn in the shoes, attached to selected landmarks in the classroom, and placed on lessons. This data, accessible to teachers in a web dashboard, enables teachers to derive deeper insights from their classrooms. Further, the anonymized data can be used in large-scale research in early childhood. Sensei is currently deployed in three Montessori schools and we have evaluated the effectiveness of the system with teachers. Our user evaluations have shown that Sensei helps discover insights that would have otherwise been lost. / by Nazmus Saquib. / S.M.

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2017 / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 121-126). / This thesis uses machine learning methods to construct emotional arcs in movies, calculate families of arcs, and demonstrate the ability for certain arcs to predict audience engagement. The system is applied to Hollywood films and high quality shorts found on the web. We begin by harnessing deep convolutional neural networks for audio and visual sentiment analysis. These models are trained on both new and existing large-scale datasets, after which they can be used to compute separate audio and visual emotional arcs for any video. We then crowd source annotations for 30-second video clips extracted from highs and lows in the arcs in order to assess the micro-level precision of the system. Precision is measured in terms of agreement in polarity between the system's predictions and annotators' ratings. The final model combining audio and visual features achieves a precision of 0.894. Next, we look at macro-level characterizations of movies by investigating whether there exist 'universal shapes' of emotional arcs. In particular, we develop a clustering approach to discover distinct classes of emotional arcs. Finally, we show on a sample corpus of short web videos that certain emotional arcs are statistically significant predictors of the number of comments a video receives. These results suggest that the emotional arcs learned by our approach successfully represent macroscopic aspects of a video story that drive audience engagement. Such machine understanding could be used to predict audience reactions to video stories, ultimately improving our ability as storytellers to communicate with each other. / by Eric Chu. / S.M. / S.M. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences

Thesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, May, 2020 / Cataloged from the official PDF of thesis. / Includes bibliographical references (pages 204-217). / Digital technology holds many promises for supporting early literacy development. To stimulate both l earning achievement and children's interest in literacy, i t i s beneficial f or a l earning activity to be playful, support children's agency and self-efficacy, and meaningfully connect to their life. However, nearly all current literacy technology, designed within the instructionist paradigm, lack these qualities. This work attempts to address this issue by exploring the design space of technology that is: ( 1) "child-driven" -- allowing initiative and ideas to come from the learner; ( 2) expressive -- fostering the creation of messages or artistic artifacts; and ( 3) scaffolded -- / assisting the child, in real time, in accomplishing his/her self-selected goals. Several forms of scaffolding were explored: ( 1) direct guidance routines with input from the child, ( 2) facilitating invented spelling, and ( 3) phoneme-based building blocks aimed at eschewing the orthographic complexities of English. The exploration was conducted through two apps, primarily aimed at phonological awareness development -- / minimalistic SpeechBlocks I and scaffolded SpeechBlocks II. They were evaluated in four exploratory studies, both in classrooms and homes. The following was l earned: ( 1) The media sparked intrinsic motivation, supported agency and self-efficacy, and allowed f or non-trivial expression; ( 2) They were used i n markedly different ways: from chaotic, impulsive exploration to sophisticated imaginative play; ( 3) The media encouraged literacy-oriented social play; ( 4) Real-time, built-in scaffolding was essential i n supporting the meaningful participation of early literacy learners. I t allowed children to engage i n high-level creativity, while simplifying the necessary l ow-level routine; ( 5) Different scaffolding types fulfilled different functions, such as responding to children's specific requests and facilitating the search f or ideas; ( 6) The distinction between letter and phoneme blocks was ultimately less important than originally thought. / However, onomatopoeic mnemonics ( designed f or phoneme blocks) were helpful or a certain category of children; ( 7) Initial phonological awareness and executive function appear to be moderators in how productive children's engagement was with the media. This work can provide insights to researchers, educators, and designers on how to combine children's agency with supportive guidance. / by Ivan Sysoev. / Ph. D. / Ph.D. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences

Thesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, September, 2020 / Cataloged from student-submitted PDF of thesis. / Includes bibliographical references (pages 189-197). / The language and formalization of mathematics historically evolved as an interplay between abstractions and their grounding in real world objects and events. The embodied mathematics philosophy posits that our mathematical capabilities are centered around our embodied experiences, and abstract math concepts are layers of metaphors that are based on simple arithmetic capabilities such as categorizing objects, subitizing, and part-whole analysis. Unfortunately, current practice of abstract math often requires us to memorize rules of manipulation, which are cognitively arbitrary, severed from intuitive grounding, and have mechanics and metaphors of their own. While journeying into mathematics, many of us therefore lose the intuitive underpinnings of abstractions. In this dissertation, I develop a design framework and an interactive sketch interface to combine computer algebra algorithms with layers of sketched, visually interpretable compositions. / The framework reimagines some abstract mathematical activities to sketching manipulable structures and performing actions that utilize our natural arithmetic capabilities, enabling the user to form personalized, embodied mathematical representations. The framework solves the challenge of defining a mapping between sketching, iconic objects, symbol algebra, and functions. It converts the symbolic representations of computer algebra to three kinds of sketched primitives and two key interactions. To evaluate the framework and interface, I present examples from the Common Core curriculum, demonstrating how the primitives and the interactions cover a wide range of exercises from Kindergarten to 8th Grade. Qualitative findings from playtesting studies in Bangladesh and USA are then compared to point out the strengths and weaknesses in the design. / An online drawing experiment is also presented that evaluates user preferences for drawing descriptive forms for the proposed mathematical compositions. Next, I describe collaborations with scientists and mathematicians (in astrophysics, neuroscience, and epidemiology) that explore how the proposed methods can reimagine some advanced mathematics, such as numerical integration and differentiation. The ability to do symbol algebra on iconic representations opens up opportunities for doing mathematics with a wide range of physical objects and media. Such affordances are demonstrated through examples. I finish the dissertation by discussing future research opportunities in mathematics, scientific computing, and HCI, and how the landscape of abstract mathematics can adapt embodied mathematics. / by Nazmus Saquib. / Ph. D. / Ph.D. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, September, 2020 / Cataloged from the official PDF version of thesis. / Includes bibliographical references (pages 104-115). / This thesis proposes the vision of Venous Materials which intends to explore the idea that fluid can be considered and manipulated as a tangible digital-physical element for interaction design. Venous Materials suggests a new approach to design interactive materials with means of designing fluidic channels inside deformable materials. These materials are demonstrated as a new type of interactive material that acts as embedded analog fluidic sensors by responding to human input (such as pressure and bending) and by utilizing the flow in an enclosed mechanism for display and color change. This thesis demonstrates the design space and its utilities through several applications, and presents how Venous Materials can be implemented as a tangible interactive tool to augment everyday scenarios. In order to increase the accessibility of Venous Materials for researchers and designers to utilize, a software platform and user interface was built to allow users a simple way to control, predict, and program the fluidic mechanisms. It serves as a design tool that allows users to design the geometry, input mechanical force, and simulate the flow results dynamically. Furthermore, this thesis presents the fabrication process that was developed for Venous Materials, which was documented and demonstrated in a hands-on workshop. Finally, this thesis will discuss the future developments and opportunities of Venous Materials towards Interactive Fluidic Mechanisms. / by Hila Mor. / S.M. / S.M. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, May, 2020 / Cataloged from the official PDF version of thesis. / Includes bibliographical references. / Emergence, when an entity is observed to have properties its parts do not have on their own, is an awe inducing phenomenon seen within nature, complex organization theory, physics, art, and philosophy. Humans can experience emergence or interdependence, a perhaps less potent version of emergence, when they come together, transform, connect, and grow in a way they could not alone through co-creation, playing games and sports, and telling stories. As such moments are scarce, cultures and groups ancient and new have developed technologies (in the sociological sense meaning techniques, processes, and material objects to produce goods, provide services, and connect people) to help groups reach those moments more easily, technologies I define as ancient social technologies (ASTs). Some ASTs, such as narrative coaching, circle practice, and dialogue across differences, have been developed for decades, even generations, to help groups reach emergence together in person. / Now, as computational social technologies like video conferencing and social media have been ground breaking by connecting across distance, many facilitators have shown great creativity and resourcefulness as they use these platforms to implement ASTs online. Yet, computational social technologies do not scaffold or ease the implementation of key AST components that are deemed essential to the practice, making virtual AST use challenging. I present in this thesis Keeper, a tool designed to augment virtual communication by scaffolding the use of ancient social technologies within modern computational social technologies. With a design informed by a deep investigation into four ancient social technologies, Keeper visualizes and mediates online, synchronous, audio and video conversations. Keeper challenges a traditional two dimensional interface through use of "space" and tone. / The tool scaffolds ASTs with features like a talking stick and guidelines, but retains key affordances of the digital medium by incorporating private messaging and conversation data visualization. Keeper fosters socially beneficial group dynamics by making visible conversation measures to promote equitability, and it prompts reflection and learning by offering visual maps of a conversation over time. The reception of this tool through experiments and interviews is discussed, and reflections on future work offered. / by Margaret Hughes. / S.M. / S.M. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2016. / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 75-79). / We present Amphibian, a simulator to experience scuba diving virtually in a terrestrial setting. Amphibian is novel because it simulates a wider variety of sensations experienced underwater compared with to existing diving simulators that mostly focus on visual and aural displays. Users rest their torso on a motion platform to feel buoyancy. Their outstretched arms and legs are placed in a suspended harness to simulate drag as they swim. An Oculus Rift head-mounted display (HMD) and a pair of headphones delineate the visual and auditory ocean scene. Additional senses simulated in Amphibian are breathing-induced motion, temperature changes, and tactile feedback through various sensors. Twelve experienced divers compared Amphibian to real-life scuba diving. We analyzed the system factors that influenced the users' sense of being there while using our simulator. We present future user interface improvements for enhancing immersion in Virtual reality diving simulators. / by Dhruv Jain. / S.M.

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2019 / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 101-107). / I introduce SCALE, a project aiming to further understand Human-Object Interaction through the real-time analysis of force vector signals, which I have defined as "Force-based Interaction" in this thesis. Force conveys fundamental information in Force-based Interaction, including force intensity, its direction, and object weight - information otherwise difficult to be accessed or inferred from other sensing modalities. To explore the design space of force-based interaction, I have developed the SCALE toolkit, which is composed of modularized 3d-axis force sensors and application APIs. In collaboration with big industry companies, this system has been applied to a variety of application domains and settings, including a retail store, a smart home and a farmers market. In this thesis, I have proposed a base system SCALE, and two additional advanced projects titled KI/OSK and DepthTouch, which build upon the SCALE project. / by Takatoshi Yoshida. / S.M. / S.M. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2019 / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 99-102). / Powered ankle prostheses have been proven to improve the walking economy of transtibial amputees although these powered systems are usually much heavier in weight than conventional prostheses. All commercial powered ankle prostheses that are currently available can only perform one-degree-of-freedom motion in a limited range. However, the human ankle can perform both frontal and sagittal plane motions. Studies have shown that the frontal plane motion during ambulation is associated with balancing. As more advanced neural interfaces have become available for amputees, it is possible to fully recover ankle function by combining neural signals and a robotic ankle. Accordingly, there is a need for a powered ankle prosthesis that can have active control on not only plantarflexion and dorsiflexion but also eversion and inversion. The objective of this thesis is to design and evaluate a two-degree-of-freedom powered ankle-foot prosthesis that is untethered and can support an average size human for level-ground walking with full power. At present, a system with such capabilities only exists as tethered. The prosthesis presented in this thesis is a second-iteration design based on its predecessor. The new design features a larger joint range of motion, a more robust transmission, and a more powerful battery module. Benchtop tests and walking trials were conducted to evaluate the system. The results demonstrate system characteristics and dynamics and the ability to support body weight in level-ground walking. / by Tsung-Han Hsieh. / S.M. / S.M. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, February, 2021 / Cataloged from the official PDF version of thesis. / Includes bibliographical references (pages 85-95). / Virtual representations of ourselves can influence the way we think, feel and behave. While this phenomenon has been explored heavily in the realm of Virtual Reality, little is known about the utility of synthetic media and real-time camera filters to reshape the perspective we have of ourselves and our capabilities. At the same time, the prevalence and popularity of these technologies have surged, coupled with greater usage of online communication tools. Motivated by a desire for self-improvement in an age of online digital communication, this thesis aims to investigate how synthetic media and real-time camera filters can be used to influence performance in target tasks, particularly in the realm of communication and creativity. This work encompasses the results of an extensive online survey (174 respondents) regarding the professional use of video-based online communication tools. It unveiled that there is an interest in self-improvement in this context and that the self-view feature of such tools may serve as a potential channel for helpful user input. Building upon this, a user study was conducted (28 participants) in which generative AI was used to synthesize videos of participants excelling in confident communication. It was found that exposure to this form of personalized media may alter feelings of confidence and stress and be implicitly helpful to some by serving as personalized role-models and guides. Following this, a second user study was conducted to explore real-time self-image manipulation via real-time camera filters (21 participants) in video calls. It was observed that applying such filters may trigger various responses in people with regards to mood, embodiment, and creativity. This thesis extends research into the topic of self-image manipulation and opens up a novel perspective that such technologies may help to serve as accessible and scalable approaches to manipulating one's self-view to nurture personal growth. / by Joanne S.L. Leong. / S.M. / S.M. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences