Artistic painting involves mastery of haptic interaction with tools. Each tool brings unique physical affordances which determines an aesthetic expression of the finished work. For instance, a pen offers an ability to make a precise stroke in a realism painting, whereas a thick brush or a sponge works perfectly with dynamic arm movement in the abstract art such as action painting. Yet the selection of a tool is just a beginning. It requires repetitive training to understand the full capability of the tool affordance and to master the painting of preferred aesthetic strokes. Such physical act of an artistic expression cannot be captured by the computational tools today. Due to the increasing market adoption of augmented reality and virtual reality, and the decades of studies in haptics, we see an opportunity for advancing 3D painting experiences in non-conventional approach. In this research, we focus on force haptic interaction for 3D painting art in a room-scale virtual reality. We explore virtual tangibility and tool affordance of its own medium. In addition to investigating the fidelity of a physical interactivity, we seek ways to extend the painting capabilities by computationally customized force feedback and metaphor design. This system consists of a wearable force feedback device that sits on user’s hand, a software for motor control and real-time 3D stroke generation, and their integration to VR platform. We work closely with an artist to refine the 3D painting application and to evaluate the system’s usability.

Technostress is an emerging and significant psychological phenomenon associated with the use of technology. It impacts human behavior and distracts from living a healthy and meaningful life. As humans increasingly encounter computational technology on a daily basis, there is a need to understand and manage the anxieties and tensions that can result from these interactions. Using the lens of critical design, this thesis explores this concept of technology induced stress and promotes reflection, personal growth and awareness through three different design research methods. It further builds on the topic of slow technology which unfolds in the form of a design fiction, design probe and design artifacts, challenging our understanding of technostress while embracing constructive discussions and creative designs to speculate the human-technology relationship. / Master of Fine Arts

La plupart des métiers du travail de l’information requièrent maintenant de passer la majeure partie de nos journée devant un écran. Ceci s’ajoute au temps déjà consacré à ce médium dans nos temps libres pour le divertissement et la communication, que ce soit en utilisant des téléphones intelligents, tablettes ou ordinateurs. Alors que les avancées technologiques dans le domaine des écrans tactiles nous ont permis d’interagir avec ces appareils de manière plus expressive, par exemple en utilisant nos doigts pour interagir directement avec le contenu numérique, ce que nous voyons et manipulons sur écran reste “intouchable” ; nos doigts ne pouvant pénétrer la surface de l’écran pour toucher le contenu numérique qui se trouve derrière. Pour que l’interaction avec le monde numérique soit écologique dans le futur, elle doit mettre à profit l’ensemble des différentes capacités de l’humain au lieu de ne se concentrer que sur certaines d’entre elles (comme le toucher et la vision), laissant les autres sens s’atrophier. Une façon de considérer le problème est d’utiliser le monde réel physique comme support pour le monde numérique, permettant d’imaginer un futur où les objets du quotidien auront de riches et expressives fonctions numériques, tout en étant ancrés dans le monde réel. La réalité augmentée spatiale est une modalité permettant d’aller dans cette direction. Cette thèse s’intéresse principalement à deux aspects en lien avec ces objets tangibles augmentés. Dans un premier temps, nous soulevons la question de comment interagir avec du contenu numérique lorsqu’il est supporté par des objets physiques. Comme point de départ de notre investigation, nous avons étudié différentes modalités qui utilisent des dispositifs d’entrée/sortie typiquement retrouvés dans un environnement de bureau. Cette approche est justifiée par le désir d’utiliser au maximum l’expérience que les utilisateurs ont déjà acquise avec leurs outils numériques tout en se dirigeant vers un espace d’interaction comprenant des éléments physiques. Dans un second temps, nous sommes allés au delà du thème de l’interaction avec le contenu numérique pour se questionner sur le potentiel des objets tangibles augmentés comme support pour un médium plus humain. Nous avons investigué comment ces artéfacts augmentés, combinés à différents capteurs physiologiques, pourraient permettre d’améliorer notre conscience des processus internes de notre corps et de notre esprit, pour éventuellement favoriser l’introspection. Cette partie a pris la forme de deux projets où un avatar tangible a été proposé pour laisser les utilisateurs explorer et personnaliser le retour d’information sur leurs propres états internes en temps réel. / Most of our waking hours are now spent staring at a screen. While the advances in touch screens have enabled a more expressive interaction space with our devices, by using our fingers to interact with digital content, what we see and manipulate on screen is still being kept away from us, locked behind a glassy surface. The range of capabilities of the human senses is much richer than what screens can currently offer. In order to be sustainable in the future, interaction with the digital world should leverage these human capabilities instead of letting them atrophy. One way to provide richer interaction and visualization modalities is to rely on the physical world itself as a host for digital content. Spatial Augmented Reality provides a technical mean towards this idea, by using projectors to shed digitally controlled light onto real-world objects to augment them and their environment with features and content. This paves the way to a future where everyday objects will be embedded with rich and expressive capabilities, while still being anchored in the real world. In this thesis, we are interested in two main aspects related to these tangible augmented objects. In a first time, we are raising the question on how to interact with digital content when it is hosted on physical objects. As a basis for our investigation, we studied interaction modalities that leverage traditional input and output devices found in a typical desktop environment. Our rationale for this approach is to leverage the experience of users with traditional digital tools – tools which researchers and developers spent decades to make simpler and more efficient to use – while at the same time steering towards a physically enriched interaction space. In a second time, we go beyond theinteraction with the digital content of augmented objects and reflect on their potential as a humane medium support. We investigate how these augmented artifacts, combined with physiological computing, can be used to raise our awareness of the processes of our own bodies and minds and, eventually, foster introspection activities. This took the form of two different projects where we used tangible avatars to let users explore and customize real-time physiological feedback of their own inner states.

Tangible User Interaction (TUI) has gathered much momentum over recent years as a way of physically representing and controlling digital information. Tangible User Interfaces privilege the control and action we, as humans, have with our highly developed motor and spatial skills. This physical mode of interaction with computation contrasts significantly with the visual mode of interaction we have with the Graphical User Interface (GUI) of the personal computer. The context for this research inquiry is the design of a TUI, the Sonic Blocks, to encourage exploratory and expressive soundtrack making activities. This thesis gives an account of the design, development and deployment of the Sonic Blocks, with a small group of children in a primary school, to gain an understanding of their unique capabilities to enable a physical and embodied interaction for exploratory and expressive purposes.

The rapidly increasing popularity in wearables in combination with countries such as Sweden moving towards a cashless society creates a market for wearable payment devices. NFC technology is already used in traditional payment cards to provide contactless payment and is now also being used in other forms such as rings or bracelets to provide smart payment devices. However, despite NFC technology being commonly used consumers are not familiar with using the technology in alternative forms or connecting them with their smartphones. Something that Fidesmo, a Swedish company that provides a platform for wearable payment devices, has observed amongst their users. Fidesmo found that users faced so many problems attempting to link their payment card to their wearable on their own they no longer recommend users attempt this. This thesis aims to investigate this problem and present insights that can be utilized to create an effective onboarding experience. The onboarding experience aims to guide users through the following: connecting wearables to smartphones, linking a payment card to the wearable through the smartphone and how to use the wearable to make payment at a payment terminal. Through research into the field, traditional user research workshops and co-design workshops exploring the existing onboarding experience within Fidesmo’s smartphone application the following insights could be derived: Visualize Information, Thorough Instructions, Building Confidence, Celebrating Success, Locate "Connection Points", Designing the Physical and Relate to the Familiar. These insights are strongly indicated by the results and some are supported by previous research. A high fidelity prototype was designed using these insights to demonstrate how to apply them. During user tests the design proposed showed significant improvements compared to Fidesmo’s current onboarding experience. / Det snabbt växande intresset för wearables i kombination med att länder såsom Sverige går mot ett kontantlöst samhälle har skapat en marknad för betallösningar i formen av wearables. NFC teknologi används redan i traditionella betalkort för funktionen kontaktlös betalning och används nu även i andra former såsom ringar eller armband för att tillhandahålla smarta betalningsenheter. Men trots den breda användingen av NFC teknologi så tycks konsumenter inte vara bekanta med att använda tekniken i alternativa former eller att utföra kopplingen med sina smartphones. Detta är något som Fidesmo, ett svenskt företag som erbjuder en plattform för wearable betalningsenheter, upptäckt bland sina användare. Fidesmo fann att sina användare stötte på så pass många problem när de försökte länka sitt betalkort till sin wearable själva att detta inte längre rekommenderas. Detta examensarbete avser att undersöka detta problem och presentera insikter som är ämnade för att skapa en effektiv onboarding upplevelse. Onboarding upplevelsen syftar på att guida användare genom följande moment: koppla wearables till smartphones, länka betalkort till wearables genom smartphones och hur man ska använda denna wearable för att betala vid en kortterminal. Genom forskning kring området, traditionella användarforsknings workshops och co-design workshops som utforskade den befintliga onboarding upplevelsen i Fidesmos smartphone applikation kunde följande insikter anskaffas: Visualicera Information, Nogranna Instruktioner, Bygga Självförtroende, Fira Framgång, Indikera “Anslutningspunkter”, Designa det Fysiska, och Relatera till det Familjära. Dessa Insikter är starkt indikerade av resultatet från denna studie och vissa stöds av tidigare forskning. En high fidelity prototyp var därefter designad baserat på insikterna för att demonstrera hur de kunde appliceras. Under användartester visade den föreslagna designen tecken på en tydlig förbättring i jämförelse med Fidesmos nuvarande onboarding upplevelse.

What if a computer wasn’t necessarily the best place for kids to learn the fundamentals of coding? A new wave of digital coding teaching tools has been ushered into America, with the country progressively gaining more interest in having kids learn code.  The goal of this project has been to propose an alternative teaching method, one focused and tailored to students who learn best through kinesthetic and visual means.  The approach has been to combine tangible interaction principles to help make the introductory stages of learning code as approachable and intuitive as possible.  The final result of this thesis is a modular toy system which gradually introduces kids to the fundamentals of coding independent of a computer, prompting exploration and problem solving.

En dépit de leur omniprésence et de leur rôle essentiel dans nos vies professionnelles et personnelles, les représentations graphiques, qu'elles soient numériques ou sur papier, ne sont pas accessibles aux personnes déficientes visuelles car elles ne fournissent pas d'informations tactiles. Par ailleurs, les inégalités d'accès à ces représentations ne cessent de s'accroître ; grâce au développement de représentations graphiques dynamiques et disponibles en ligne, les personnes voyantes peuvent non seulement accéder à de grandes quantités de données, mais aussi interagir avec ces données par le biais de fonctionnalités avancées (changement d'échelle, sélection des données à afficher, etc.). En revanche, pour les personnes déficientes visuelles, les techniques actuellement utilisées pour rendre accessibles les cartes et les diagrammes nécessitent l'intervention de spécialistes et ne permettent pas la création de représentations interactives. Cependant, les récentes avancées dans le domaine de l'adaptation automatique de contenus laissent entrevoir, dans les prochaines années, une augmentation de la quantité de contenus adaptés. Cette augmentation doit aller de pair avec le développement de dispositifs utilisables et abordables en mesure de supporter l'affichage de représentations interactives et rapidement modifiables, tout en étant accessibles aux personnes déficientes visuelles. Certains prototypes de recherche s'appuient sur une représentation numérique seulement : ils peuvent être instantanément modifiés mais ne fournissent que très peu de retour tactile, ce qui rend leur exploration complexe d'un point de vue cognitif et impose de fortes contraintes sur le contenu. D'autres prototypes s'appuient sur une représentation numérique et physique : bien qu'ils puissent être explorés tactilement, ce qui est un réel avantage, ils nécessitent un support tactile qui empêche toute modification rapide. Quant aux dispositifs similaires à des tablettes Braille, mais avec des milliers de picots, leur coût est prohibitif. L'objectif de cette thèse est de pallier les limitations de ces approches en étudiant comment développer des cartes et diagrammes interactifs physiques, modifiables et abordables. Pour cela, nous nous appuyons sur un type d'interface qui a rarement été étudié pour des utilisateurs déficients visuels : les interfaces tangibles, et plus particulièrement les interfaces tangibles sur table. Dans ces interfaces, des objets physiques représentent des informations numériques et peuvent être manipulés par l'utilisateur pour interagir avec le système, ou par le système lui-même pour refléter un changement du modèle numérique - on parle alors d'interfaces tangibles sur tables animées, ou actuated. Grâce à la conception, au développement et à l'évaluation de trois interfaces tangibles sur table (les Tangible Reels, la Tangible Box et BotMap), nous proposons un ensemble de solutions techniques répondant aux spécificités des interfaces tangibles pour des personnes déficientes visuelles, ainsi que de nouvelles techniques d'interaction non-visuelles, notamment pour la reconstruction d'une carte ou d'un diagramme et l'exploration de cartes de type " Pan & Zoom ". D'un point de vue théorique, nous proposons aussi une nouvelle classification pour les dispositifs interactifs accessibles. / Despite their omnipresence and essential role in our everyday lives, online and printed graphical representations are inaccessible to visually impaired people because they cannot be explored using the sense of touch. The gap between sighted and visually impaired people's access to graphical representations is constantly growing due to the increasing development and availability of online and dynamic representations that not only give sighted people the opportunity to access large amounts of data, but also to interact with them using advanced functionalities such as panning, zooming and filtering. In contrast, the techniques currently used to make maps and diagrams accessible to visually impaired people require the intervention of tactile graphics specialists and result in non-interactive tactile representations. However, based on recent advances in the automatic production of content, we can expect in the coming years a growth in the availability of adapted content, which must go hand-in-hand with the development of affordable and usable devices. In particular, these devices should make full use of visually impaired users' perceptual capacities and support the display of interactive and updatable representations. A number of research prototypes have already been developed. Some rely on digital representation only, and although they have the great advantage of being instantly updatable, they provide very limited tactile feedback, which makes their exploration cognitively demanding and imposes heavy restrictions on content. On the other hand, most prototypes that rely on digital and physical representations allow for a two-handed exploration that is both natural and efficient at retrieving and encoding spatial information, but they are physically limited by the use of a tactile overlay, making them impossible to update. Other alternatives are either extremely expensive (e.g. braille tablets) or offer a slow and limited way to update the representation (e.g. maps that are 3D-printed based on users' inputs). In this thesis, we propose to bridge the gap between these two approaches by investigating how to develop physical interactive maps and diagrams that support two-handed exploration, while at the same time being updatable and affordable. To do so, we build on previous research on Tangible User Interfaces (TUI) and particularly on (actuated) tabletop TUIs, two fields of research that have surprisingly received very little interest concerning visually impaired users. Based on the design, implementation and evaluation of three tabletop TUIs (the Tangible Reels, the Tangible Box and BotMap), we propose innovative non-visual interaction techniques and technical solutions that will hopefully serve as a basis for the design of future TUIs for visually impaired users, and encourage their development and use. We investigate how tangible maps and diagrams can support various tasks, ranging from the (re)construction of diagrams to the exploration of maps by panning and zooming. From a theoretical perspective we contribute to the research on accessible graphical representations by highlighting how research on maps can feed research on diagrams and vice-versa. We also propose a classification and comparison of existing prototypes to deliver a structured overview of current research.

<p>This thesis explores new approaches to making and playing with programming materials, especially the forms provided with screen-based digital media. Designing with these media expressions can be very attractive to children, but they are usually not made available to them in the same degree as are physical materials.</p><p>Inspired by children's play with physical materials, this work includes design explorations of how different resources alter, scaffold and support children in activities of making dynamic, screen-based systems. How tangibles turn the activity of programming into a more physical, social and collaborative activity is emphasised. A specific outcome concerns the importance of considering 'offline' and socially oriented action when designing tangible technologies. The work includes the design of a tangible programming system, Patcher, with which groups of children can program systems displayed on a large screen surface.</p><p>The character of children's programming is conceptualised through the notion of a digital patchwork, emphasising (1) children's programming as media-sensitive design, (2) making programming more concrete by combining and reusing readily available programming constructs, and (3) the use of tangibles for social interaction.</p>

This field study examined how elderly and cognitively disabled people used and experienced a social companion robot. The following pages explores the questions: Which are the physical, social and affective qualities during the interaction? The aim was to through observations see how qualities of interaction could activate different forms of behavior. The results show that motion, sound and the eyes together created communicative and emotional changes for users who felt joy and were willing to share the activity with others. The robot stimulated to some extent users to create their own imaginative experiences but often failed to involve user or group for a long time and was also considered too large and heavy to handle. / Denna fältstudie undersökte hur äldre och kognitivt funktionsnedsatta personer använde och upplevde en social robot. Följande sidor utforskar frågorna: Vilka fysiska, sociala och affektiva kvaliteter finns i interaktionen? Målet var att genom observationer se hur kvaliteterna i interaktionen kunde aktivera olika typer av beteenden. Resultatet visar att rörelse, ljud och ögon tillsammans skapade kommunikativa och känslomässiga förändringar hos användarna som visade glädje och som gärna delade upplevelsen med andra. Roboten stimulerade till viss del användarna att skapa egna fantasifulla upplevelser men lyckades inte ofta involvera användare eller grupp under någon längre tid och ansågs även vara för stor och tung att hantera.

Recent years have witnessed great improvements in ways of game controlling yielding to higher level of interaction. Release of motion controller devices radically changed the conventional ways of interaction that have been used for controlling games so far, also giving developers the opportunity of exploring various new possible ways of interaction. One of these off the shelf tools, Microsoft Kinect for Xbox 360, recognizes motions of the players as game controlling inputs. Although touchless interaction is perceived to be attractive, games that mimic real life activities such as table tennis, sword fighting, baseball and golf may benefit from the player&rsquo / s holding a tangible object to get more involved into game, sensing the actions deeply. In this thesis, a tangible gameplay interaction method that senses whether or not the player holds an object in the hand / if so, detects its dimensions and incorporates the hand-held object into gameplay by projecting motions of the player accordingly, is developed using Microsoft Kinect for Xbox 360. Developed algorithm is implemented on an experimental game and a user study is performed which revealed that an improved gameplay with more natural and accurate motion controlling yielding to new possible actions is achieved with the developed system.