Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2017. / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 135-139). / Tangible Interfaces allow us to utilize our natural propensity for kinesthetic manipulation to control digital computation and touch virtual information. As the technology advances, these interfaces are re-envisioned as programmable materials, able to emulate dynamic physical properties to provide material-based affordances. In this work, I review the motivation and a brief history of Tangible User Interfaces (TUIs) and examine the implications of the Radical Atoms vision for programmable Material User Interfaces (MUIs). I identify two current limitations to Radical Atoms in practice: 1) material rendering capability and 2) affordance prediction for general-purpose MUIs. I propose force-controlled material displays as a framework for future advancement in material property rendering. I also discuss the use of Al for contextual interaction recognition and introduce Spatial Behaviors as an alternative method to allow interfaces to infer appropriate interaction modes from their location in space. This thesis presents the context, motivation, framework, implementation, evaluation, and future roadmaps towards these visions. I present examples of each proposed paradigm, focusing on inFORCE, a force-controlled material display, and ReVeal, a spatial shape display for tangible rendering in Augmented and Virtual Reality. I analyze the technical performance of this system and assess the interface through user studies. Finally, I discuss potential applications of the current system, as well as limitations and premises for future development and improvement in the context of Radical Atoms. / by Daniel John Fitzgerald. / S.M.

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2018. / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 136-139). / THE USE of biological agents, such as bacteria, fungi, plant material, and insects, is proposed as a viable design methodology that can include end-of-life material treatment and re-use as part of the design process. Decay, typically thought of as a destructive process with spatially and temporally unpredictable results can be potentially designed to high levels of spatial and temporal tunability, such that the product of decay can still be of value to the designer. In particular, could microbes, in combination with water, be guided to selectively disintegrate, and potentially also then be harnessed as agents of assembly, enabling design construction instead of destruction? What would the ecology of this system that includes the microbes, biocompatible substrate, and contextual environment, look like? How might it behave and how might this affect how architects design the built environment? The main goal of this thesis is to understand how water-induced decay of biocomposite-based artifacts might be designed such that the deterioration process has use-value for the designed artifact and how this decay might be harnessed as a formative agent instead of only as a deconstructive one. This understanding will be informed by the execution and analysis of a series of case study projects that include: Aguahoja: Artifacts and Hexes Aguahoja: Colonization Aguahoja: Pavilion For these projects, water-based composites of chitosan, cellulose, and pectin, will be additively printed into architectural-scale artifacts and panels which are used as the skin of a small pavilion. The use of the chitin-cellulose-pectin composite introduces a water-dependent biocompatible material system that can act as the base scaffold for biofilms and microbial colonies. A series of material characterization tests were done to understand the nuances of the different bio-composites. Working challenges for this system included warping and uncontrolled deflections due to changing relative humidity and the gradual and constant dehydration of the material. This hydration dependent activity also dictated flexibility rigidity, and colour. Water is used to initiate the disintegration of the panels as well as initiate the growth of microbial cultures on the panels. The projects described here are intended to serve as examples of how a designer might be able to incorporate biological agency into her process. It is not intended as a comprehensive guidebook, but rather as case studies of how one might use nascent biological tools in current design methodology and how that methodology might have to change in order to leverage biology in her process. / by Andrea Ling. / S.M.

Thesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2017. / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 64-73). / While biologists routinely record neural activity with multi-electrodes, spike sorting-- the process of attributing spikes to particular neurons-- remains a challenge that typically requires human curation. Due to technical limitations, there have been very few multi-electrode recordings done in concert with techniques such as patch clamp, which report the "ground truth" voltage state of a single neuron in a population. Such recordings would allow for the direct evaluation of spike sorting, which in turn could lead to further development and refinement of spike sorting methods. We developed a technique to establish a whole-cell or cell-attached patch recording in a cortical neuron of an awake or lightly anesthetized head-fixed mouse, with simultaneous extracellular recording of the same neuron and its neighbors with arrays of close-packed, "ultra-dense," electrodes (64-256, 9 x 9[mu]m electrodes spaced 2[mu]m apart on a shank). Our recordings constitute ground truth for spike sorting evaluation, and allow for the direct evaluation and improvement of an algorithm for automatic spike sorting that benefits from high electrode density relative to neuron packing density. Using this technique we show the patch-triggered extracellular waveforms of neurons at a high level of granularity distributed across cortex, and give a glimpse into the spiking activity of the network surrounding a patched neuron in vivo. We explore the dataset generated with this technique and discover a spike-bursting trajectory exhibiting apparent spike-frequency adaptation. This bursting trajectory was readily apparent in deep but not shallow cortical neurons in patch recordings, but was somewhat obscured in extracellular recordings, where spikes from neighboring neurons may overlap in time to contribute "noise." We show how this trajectory can be easily seen in a high-amplitude extracellular recording, and propose how it may be accentuated in lower amplitude recording through the use of blind source separation. / by Brian D. Allen. / Ph. D.

Thesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2017. / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 235-260). / Current interactive projected augmented reality systems are not designed to support rapid development and deployment of applications beyond the confines of research labs. I developed a series of self-contained interactive projector-sensor systems (collectively LuminAR devices) and a web-based software development framework. The design goal of this research work was to advance the state of the art of projected AR interfaces and to explore how they can manifest in day-to-day objects. This novel, tightly integrated approach allows developers who are not versed in computer graphics, vision algorithms, and augmented reality techniques to implement projected AR applications rapidly. In this work, I review several real-world uses of the system for retail presentation, desktop interaction and collaboration applications, manufacturing, and education. The work is evaluated through extensive use of the hardware and software by developers as well as two user studies that specifically explored applications for manufacturing and education. The evaluation methodology focused both on basic interaction and system usability as well as the implications of using augmented interfaces in the specific application domains of education and manufacturing. I also discuss the results of the first large-scale user studies of projected augmented reality rapid application development. Finally, I provide a set of design principles for projected augmented reality applications, and recommendations concerning how to deploy such applications in the real world. / by Natan Linder. / Ph. D.

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2017. / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 83-85). / Every 98 seconds, a person in the United States is sexually abused. Every 16 hours, a woman in the United States is murdered by her romantic partner or ex-partner. Sexual abuse, assault, and harassment are regarded as some of the most common human rights violations in the world by the United Nations. Our work examines methods to prevent sexual assault, from pre-historic times to latest technologies, to inform contemporary designs. In this thesis, we investigate multiple methods to detect initial signs of assault and develop methods for communication and prevention of assault. We also explore olfactory stimuli as a potential means to prevent sexual assault in real-time. We present three technological interventions which can seamlessly integrate with existing clothing to respond to initial signs of assault like forced disrobing. The proposed solutions aim to combat Child Sexual Abuse (CSA), College Campus assault and abuse of elderly and disabled. The proposed solution operates in two modes, an active mode for instances when the victim is unconscious or cannot fight against the assaulter, for example in case infants, bed-ridden patients, elderly, disabled, intoxicated people and the passive mode where the victim can self-actuate the safety mechanism. Both modes release distress signals to prevent an assault in real-time, also alert the victim's friends and family, and call emergency services for help. Our clothing design is based on input from sexual assault survivors, 338 on-line participants, 67 volunteers and 20 users who helped us understand the real world feasibility of our system. Users evaluated the clothing appeal, functionality, cultural sensitivity and provided feedback on their general sense of security wearing the smart clothing. We demonstrate the practicality of our unobtrusive design with user studies that support our technological development and use of olfactory stimuli by showing the effect of smells on sexual arousal and partner selection. We believe our technosocial approach can help improve user safety and prevent sexual assault. / by Manisha Mohan. / S.M.

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2016. / Cataloged from PDF version of thesis. / Includes bibliographical references (page 56). / There is an increasingly vital awareness that our world is an aggregate of complex systems, emergent behavior, and system dynamics. The perceptual and analytical tools for exploring and studying these systems, however, have generally been relegated to scientists (whether mathematicians, physicists, biologists, economists, or computer scientists). Thus, as more and more people become aware of such systems, most people are still excluded from engaging with complex systems. By inventing a new tool and interface, consisting of playful objects called AutomaTiles, I propose a new approach for fostering a more aware society of systems thinkers. AutomaTiles provide a three-tiered approach to making systems thinking more accessible. Firstly, AutomaTiles are a friendly and approachable set of playful objects; seen simply as toys, they afford the surprising effects of emergent behavior when brought together in aggregate. Secondly, AutomaTiles can be a tool for exploring collective behavior, distributed algorithms, and models of systems (whether forest fires or social phenomena) from a hands-on perspective. Lastly, AutomaTiles are a new kind of platform for games, bringing computational intelligence to table-top games, bringing together the social dynamics of face-to-face interaction with the complexity afforded by conventional video-games. Expanding the work on the future of board games from Playful Systems, we have created a novel digital-physical interface for playing games that allows for modes of gameplay never before possible in a table-top game. This thesis will illustrate the design decisions and affordances of AutomaTiles as a platform for engaging with these three tiers of the exploration and manipulation of complex systems. / by Jonathan Bobrow. / S.M.

Thesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2016. / Cataloged from PDF version of thesis. / Includes bibliographical references. / In order to map the dynamics of neural circuits in mammalian brains, there is a need for tools that can record activity over large volumes of tissue and correctly attribute the recorded signals to the individual neurons that generated them. High-resolution neural activity maps will be critical for the discovery of new principles of neural coding and neural computation, and to test computational models of neural circuits. Extracellular electrophysiology is a neural recording method that has been developed to record from large populations of neurons, but well-known problems with signal attribution pose an existential threat to the viability of further system scaling, as analyses of network function become more sensitive to errors in attribution. A key insight is that blind-source separation algorithms such as Independent Component Analysis may ameliorate problems with signal attribution. These algorithms require recording signals at much finer spatial resolutions than existing probes have accomplished, which places demands on recording system bandwidth. We present several advances to technologies in neural recording systems, and a complete neural recording system designed to investigate the challenges of scaling electrophysiology to whole brain recording. We have developed close-packed microelectrode arrays with the highest density of recording sites yet achieved, for which we built our own data acquisition hardware, developed with a computational architecture specifically designed to scale to over several orders of magnitude. We also present results from validation experiments using colocalized patch clamp recording to obtain ground-truth activity data. This dataset provides immediate insight into the nature of electrophysiological signals and the interpretation of data collected from any electrophysiology recording system. This data is also essential in order to optimize probe development and data analysis algorithms which will one day enable whole-brain activity mapping. / by Jacob G. Bernstein. / Ph. D.

Thesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2018. / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 177-183). / What opportunities does virtual reality offer to improve the way we learn? In this thesis, I investigate the ways that constructivist approaches, in particular exploratory and experiential learning, can be uniquely supported by immersive virtual worlds. Against the background of these learning theories, I introduce a design framework that centers around defining a medium of virtuality that is fundamentally social, and uses capture of movement and interaction as a key means for creating interactive scenarios and narrative. Within the world conjured by this medium, the Equipped Explorer learns, reviews, creates and communicates using tools that I propose and classify according to a taxonomy. A series of prototypes and design explorations are used as proofs of concept for aspects of the design framework. Experimental studies are used to investigate foundational questions concerning the learning benefits of using VR over 2D interactive media, and the viability of social interaction and collaboration in VR. I reflect on the implications of this framework and my experimental results to extrapolate how they might impact the future classroom and the practice of learning and discovery more broadly. Finally, I discuss what kinds of research might be needed to maximize that impact moving forward. / by Scott Wilkins Greenwald. / Ph. D.

Thesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2018. / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 151-160). / When we listen closely, there is a pervading sense that we could hear more if we could only focus a little more intently. Our own perceptual limits are a moving target that we cannot delineate and rarely reach. This dissertation introduces technologies that operate at that mysterious boundary. I envision sensor(y) landscapes, physical sites that meld distributed sensing and sensory perception to afford new perceptual sensibilities. Today's mainstream technologies are well designed for rapid consumption of information and linear, sequential action. A side effect of their effectiveness to task, however, is a loss of undirected, curiosity-driven exploration in the world. I propose alternative technologies that would extend perceptual presence, amplify attention, and leverage intuitions. My focus is on turning rich sensor data into compelling sensory input, and as such, a substantial component of my work involved deploying sensor infrastructure in beautiful places. My projects center on a wetland restoration site, called Tidmarsh, where environmental data are densely and continuously collected and streamed. Using sound and vibration as the medium and nature as the setting, I undertook this work in two steps. The first constructs environments suffused with sensing and built for being present in. My projects in this space comprise sensor-driven virtual worlds, glass elevator sound installations, and vibrating forests that give oral histories. Building on lessons and infrastructure from the first approach, my culminating work uses non-occluding spatial audio to create situated perceptions of data. I developed a bone-conduction headphone device, called HearThere, that renders a live soundscape from distributed microphones and sensors, fully merged with the user's natural hearing. HearThere combines its wearer's inferred listening state with classification output from an Al engine to adjust the mix and spatial parameters of virtual audio sources. The device was developed based on findings from lab studies into spatial hearing and attention, and evaluated in a human subjects study with a panel of experts. Through these projects, I found that deriving meaning in the medium is a matter of possessing or developing perceptual sensibilities, intuitions for how the permeated data can be teased out and contemplated. Carefully composed perceptual confusion-a blurring of place and distributed media-becomes an opportunity for the development of new transpresence sensibilities. How do users make sense of these new dimensions of perception, and how can technologies be designed to facilitate perceptual sense-making? / by Gershon Dublon. / Ph. D.

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2018 / Cataloged from PDF version of thesis. / Includes bibliographical references (pages 95-99). / Science concepts lie at the heart of our everyday experiences, yet people feel disconnected from science because of the abstract way it is taught in schools. We wanted people to learn science concepts in the real world in playful ways, and used Mixed Reality (MR) to allow people to visualize and play with science concepts in the real world. We focused on Newtonian physics as our first science concept in Wonderland because Newtonian physics is commonly experienced by people in their everyday lives, especially in playful contexts, e.g., when they throw a ball. We created simple Newtonian physics tools, which served as building blocks of Newtonian physics systems to allow learners to build their own Newtonian physics models and puzzles for constructionist learning. We include different types of custom visualizations, e.g., graphs, velocity and acceleration vectors, etc, to allow the users to visualize the underlying physics of objects in scientifically accurate, yet intuitive ways. Our rewinding interface also enables users to play, pause, rewind, replay, speed up and slow down physics so that users can learn from repeated physics experimentation. We created two versions of Wonderland: a Hololens version for an immersive head-worn MR experience, and an ARKit version for a more widely accessible MR experience on iOS devices. Our experiments show that users enjoy solving Newtonian physics puzzles in Wonderland, and find the visuals and simulations helpful in understanding Newtonian physics concepts. We aim to further develop and deploy Wonderland to promote science learning and exploration in the real world. / by Mina Khan. / S.M. / S.M. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences