Computers are increasingly being used in situations where a conclusion must be reached based on a wide range of knowledge. As the volume of this stored knowledge increases, so does the possibility of inconsistency within that knowledge. Argumen tation is one solution to this problem, and argumentation based upon propositional logic has been extensively explored in the literature. However, this is insufficient for many applications, such as the use of temporal data. First order logic is a possi ble solution from the literature, but not all of the issues concerning this have been explored. This thesis discusses aspects of argumentation using extensions to propositional logic in order to explore the space of possibilities between propositional logic and full first order logic. In particular, logics with some form of generalisation are considered, incorporating variable, function and predicate symbols. The first form of generalisation considered includes an argumentation system using a calculus which is capable of expressing temporal knowledge. This general isation offers two new and complementary notions of an argument, and these are shown to have some advantages over more traditional arguments in some circum stances. The second form of generalisation concerns predicate and function symbols, and is based around applications of causal mapping, which is a technique using a graphical representation of a logical database in order to show how letters within that database influence each other. Argumentation using this generalisation is con sidered, and a method for representing arguments graphically in causal maps is introduced. The third form of generalisation considers full first order logic, and the methods proposed in previous chapters are used as an insight into problems within first order argumentation. An alternative definition of an argument is proposed as a solution to these problems, and these are considered within argument trees. In conclusion, in this thesis I have shown how first order extensions to proposi tional logic can be used within argumentation systems. I have shown some prob lems when using generalisations, and presented solutions to these problems. These problems and solutions are explored when using first order logic directly in argu mentation.

Herbert Simon (1969) set out an ambitious programme of research for a science of design. This thesis examines the routes taken and the problems encountered with this programme as it is found in the flagship domain of Human-Computer Interaction. It examines the emergence of Human-Computer Interaction as a discipline and its problems (Chapter 1). It considers recent attempts to reformulate many of the foundations of HCI's design theory (Chapter 2) including work of my own (Chapter 3). It describes the development of a new field of Computer-Supported Cooperative Work (Chapter 4) and a new approach to understanding the environment from a social rather than a cognitive point of view by using ethnographic methods imported from sociology and social anthropology and now closely associated with the field of CSCW (Chapter 5). This new orientation is seen by many to offer new hope for informing design and overcoming some of the problems that have so far beset HCI. It represents one of the most significant contributions to design to have emerged from sociology and social anthropology. It is not, however, without its own difficulties in terms of being able to effectively 'speak to' the design process. Ethnographic methods have also been used to examine the very process of design itself and I report on a number of such studies and some of the issues which they raise (Chapter 6). In particular I will report in detail on an ethnographic study of my own which sought simultaneously to investigate the worth of this research orientation, to understand some of the ways in which ethnographic studies can contribute to system design and particularly the design of CSCW technologies and to do this through a study of another area of design work - fashion design (Chapter 7). Studies such as this of design in practice do, however, raise a number of challenging issues for the type of science of design envisaged by Simon and his associates. Empirically examining what design is and how design is done questions some of the abstract formulations of design. Additionally, I suggest that some care is needed in schematizing how a method such as ethnography can contribute to a design process. I consider these results in my concluding chapter (Chapter 8) where I argue that the ethnographic approach turns out to contrast fundamentally with Simon's view of design for his conception of design is also crucially a perspective upon action itself. It is this perspective which ultimately motivates Simon to claim that the proper study of mankind is the science of design, while it is an ethnographic orientation which motivates me to argue that design is practical action and that practical action is designed.

This research applies psychological theory to human-computer interaction and the creation of a virtual assistant (VA) for use in self-‐service checkouts (SSCOs). Existing research investigating the impact of interface agents and avatars in virtual shopping, gaming, and teaching environments presents a range of findings, yet there is comparatively little investigating the impact of a VA in kiosk-based applications. Based on the premise that human-computer interaction resembles human-human interaction, the potential cognitive and social benefits of implementing a visually present VA in SSCOs are examined. Further, this research addresses how the impact of VAs is affected by its gender and human­‐likeness. To this end, this thesis brings together two lines of research: human-likeness (in the human-computer interaction literature) and gender (in the psychology literature). Through four studies, it is demonstrated that a VA's similarity to the user, in terms of gender and human-likeness, has an impact on performance with, attitudes towards, and willingness to use a VA. The first study showed automatic orienting to onscreen locations indicated by the gaze of an animated human-like virtual character. Observers could also respond to this cue based on their own knowledge and goals (unlike peripheral cues or a static virtual character). In the second study, women strongly preferred to gender-match with their VAs, but this did not affect their reaction times in a visual search task; men showed no overall preference for gender but were sometimes slower to respond when a female VA was on screen. The tendency to gender-match in males steadily decreased with age, such that the majority of younger males preferred to gender-match whilst the majority of older males preferred to interact with a female VA. Both men and women preferred more realistic VAs, resulting in faster reaction times. The third study focussed on the development of a questionnaire for measuring perceived VA usability and credibility, and, after reliability and validity testing, offered a new measure for assessing these perceptions. This measure consisted of four sub-constructs: warmth and capacity (sub-constructs of credibility) and usefulness and ease-of-use (sub-constructs of usability). Responses to items evaluated in Study 3, in parallel with Study 4, showed that similar agents tend to be perceived as warmer, higher in capacity, and more useful – but only when participants were assigned a gender-matched, humanoid VA. When participants chose their VA, similarity had no impact on the newly developed scales for measuring perceived usability and credibility. Moreover, the positive ratings of similar VAs that had been assigned to participants did not impact willingness to take its advice; in direct contrast, when participants had chosen their VA, positive ratings of the VA (regardless of similarity) correlated with willingness to take its advice. Finally, consistent with the findings of Chapter 4, female participants who chose their VAs tended to gender-‐match (to the equal exclusion of the male and non-human VAs) and half the male participants tended to choose a male VA. Unexpectedly, this was followed by a preference for the non-human, non-gendered VA rather than for the female VA, attributed to its novelty. In general, participants tended to choose a VA taking human form, rating these options more attractive and likable. Overall, the results suggest a preference for and performance-based benefit of human-like VAs. Preferences for VA gender can be interpreted in terms of differential social networking strategies employed by men and women. The research also shows that the perceptions, preferences, and behaviours induced by a VA that is matched to the user (in terms of gender and human- likeness) are not always in line. Thus, retailers should weigh the importance of each of these outcomes (e.g. perceptions of usability or checkout speed) before choosing VA gender and its level of human-likeness, or should allow users to make their own decisions about the appearance of their VA.

This thesis presents a framework for the automatic strategy synthesis from quantitative specifications, in order to control autonomous systems. We model systems as turn-based two-player zero-sum stochastic games, which are able to express both stochastic and nondeterministic environmental uncertainty. Given a system model and a specification, we define the strategy synthesis problem as that of finding a strategy for the system that is winning against every environment. Since large, complex systems are typically built from multiple components, we consider synthesis of strategies for the individual components separately, which can then be composed to a winning strategy for the full system. Modelling interaction between components is facilitated using assume-guarantee rules, which can express contracts such as "maintain the room temperature of at least 20°C, as long as the windows are closed at least 30% of the time." For the synthesis of strategies for the individual components, we develop synthesis algorithms for Boolean combinations of long-run objectives, specifically, maintaining above a threshold (i) a mean-payoff, almost surely; (ii) an expected mean-payoff; (iii) a ratio of rewards, almost surely; or (iv) a ratio of expected rewards. We implement our algorithms in the PRISM-games 2.0 tool and demonstrate their viability on four case studies.

Human-Computer Interaction (HCI) design has historically involved people from different fields. Designing HCI systems with people of varying background and expertise can bring different perspectives and ideas, but discipline-specific language and design methods can hinder such collaborations. The application of visualisation methods is a way to overcome these challenges, but to date selection tools tend to focus on a facet of HCI design methods and no research has been attempted to assemble a collection of HCI visualisation methods. To fill this gap, this research seeks to establish an inventory of HCI visualisation methods and identify ways of selecting amongst them. Creating the inventory of HCI methods would enable designers to discover and learn about methods that they may not have used before or be familiar with. Categorising the methods provides a structure for new and experienced designers to determine appropriate methods for their design project. The aim of this research is to support designers in the development of Human-Computer Interaction (HCI) systems through better selection and application of visualisation methods. This is achieved through four phases. In the first phase, three case studies are conducted to investigate the challenges and obstacles that influence the choice of a design approach in the development of HCI systems. The findings from the three case studies helped to form the design requirements for a visualisation methods selection and application guide. In the second phase, the Guide is developed. The third phase aims to evaluate the Guide. The Guide is employed in the development of a serious training game to demonstrate its applicability. In the fourth phase, a user study was designed to evaluate the serious training game. Through the evaluation of the serious training game, the Guide is validated. This research has contributed to the knowledge surrounding visualisation tools used in the design of interactive systems. The compilation of HCI visualisation methods establishes an inventory of methods for interaction design. The identification of Selection Approaches brings together the ways in which visualisation methods are organised and grouped. By mapping visualisation methods to Selection Approaches, this study has provided a way for practitioners to select a visualisation method to support their design practice. The development of the Selection Guide provided five filters, which helps designers to identify suitable visualisation methods based on the nature of the design challenge. The development of the Application Guide presented the methodology of each visualisation method in a consistent format. This enables the ease of method comparison and to ensure there is comprehensive information for each method. A user study showing the evaluation of a serious training game is presented. Two learning objectives were identified and mapped to Bloom’s Taxonomy to advocate an approach for like-to-like comparison with future studies.

Action recognition involves automatically labelling videos that contain human motion with action classes. It has applications in diverse areas such as smart surveillance, human computer interaction and content retrieval. The recent advent of depth sensing technology that produces depth image sequences has offered opportunities to solve the challenging action recognition problem. The depth images facilitate robust estimation of a human skeleton’s 3D joint positions and a high level action can be inferred from a sequence of these joint positions. A natural way to model a sequence of joint positions is to use a graphical model that describes probabilistic dependencies between the observed joint positions and some hidden state variables. A problem with these models is that the number of hidden states must be fixed a priori even though for many applications this number is not known in advance. This thesis proposes nonparametric variants of graphical models with the number of hidden states automatically inferred from data. The inference is performed in a full Bayesian setting by using the Dirichlet Process as a prior over the model’s infinite dimensional parameter space. This thesis describes three original constructions of nonparametric graphical models that are applied in the classification of actions in depth videos. Firstly, the action classes are represented by a Hidden Markov Model (HMM) with an unbounded number of hidden states. The formulation enables information sharing and discriminative learning of parameters. Secondly, a hierarchical HMM with an unbounded number of actions and poses is used to represent activities. The construction produces a simplified model for activity classification by using logistic regression to capture the relationship between action states and activity labels. Finally, the action classes are modelled by a Hidden Conditional Random Field (HCRF) with the number of intermediate hidden states learned from data. Tractable inference procedures based on Markov Chain Monte Carlo (MCMC) techniques are derived for all these constructions. Experiments with multiple benchmark datasets confirm the efficacy of the proposed approaches for action recognition.

Eye tracking for pervasive displays in everyday computing is an emerging area in research. There is an increasing number of pervasive displays in our surroundings, such as large displays in public spaces, digital boards in offices and smart televisions at home. Gaze is an attractive input modality for these displays, as people naturally look at objects of interest and use their eyes to seek information. Existing research has applied eye tracking in a variety of fields, but tends to be in constrained environments for lab applications. This thesis investigates how to enable robust gaze sensing in pervasive contexts and how eye tracking can be applied for pervasive displays that we encounter in our daily life. To answer these questions, we identify the technical and design challenges posed by using gaze for pervasive displays. Firstly, in out-of-lab environments, interactions are usually spontaneous where users and systems are unaware of each other beforehand. This poses the technical problem that gaze sensing should not need prior user training and should be robust in unconstrained environments. We develop novel vision-based systems that require only off-the-shelf RGB cameras to address this issue. Secondly, in pervasive contexts, users are usually unaware of gaze interactivity iii of pervasive displays and the technical restrictions of gaze sensing systems. However, there is little knowledge about how to enable people to use gaze interactive systems in daily life. Thus, we design novel interfaces that allow novice users to interact with contents on pervasive displays, and we study the usage of our systems through field deployments. We demonstrate that people can walk up to a gaze interactive system and start to use it immediately without human assistance. Lastly, pervasive displays could also support multiuser co-located collaborations. We explore the use of gaze for collaborative tasks. Our results show that sharing gaze information on shared displays can ease communications and improve collaboration. Although we demonstrate benefits of using gaze for pervasive displays, open challenges remain in enabling gaze interaction in everyday computing and require further investigations. Our research provides a foundation for the rapidly growing field of eye tracking for pervasive displays.

Our bodies are the medium through which we experience the world around us, so human-computer interaction can highly benefit from the richness of body movements and postures as an input modality. In recent years, the widespread availability of inertial measurement units and depth sensors led to the development of a plethora of applications for the body in human-computer interaction. However, the main focus of these works has been on using the upper body for explicit input. This thesis investigates the research space of full-body human-computer interaction through three propositions. The first proposition is that there is more to be inferred by natural users’ movements and postures, such as the quality of activities and psychological states. We develop this proposition in two domains. First, we explore how to support users in performing weight lifting activities. We propose a system that classifies different ways of performing the same activity; an object-oriented model-based framework for formally specifying activities; and a system that automatically extracts an activity model by demonstration. Second, we explore how to automatically capture nonverbal cues for affective computing. We developed a system that annotates motion and gaze data according to the Body Action and Posture coding system. We show that quality analysis can add another layer of information to activity recognition, and that systems that support the communication of quality information should strive to support how we implicitly communicate movement through nonverbal communication. Further, we argue that working at a higher level of abstraction, affect recognition systems can more directly translate findings from other areas into their algorithms, but also contribute new knowledge to these fields. The second proposition is that the lower limbs can provide an effective means of interacting with computers beyond assistive technology To address the problem of the dispersed literature on the topic, we conducted a comprehensive survey on the lower body in HCI, under the lenses of users, systems and interactions. To address the lack of a fundamental understanding of foot-based interactions, we conducted a series of studies that quantitatively characterises several aspects of foot-based interaction, including Fitts’s Law performance models, the effects of movement direction, foot dominance and visual feedback, and the overhead incurred by using the feet together with the hand. To enable all these studies, we developed a foot tracker based on a Kinect mounted under the desk. We show that the lower body can be used as a valuable complementary modality for computing input. Our third proposition is that by treating body movements as multiple modalities, rather than a single one, we can enable novel user experiences. We develop this proposition in the domain of 3D user interfaces, as it requires input with multiple degrees of freedom and offers a rich set of complex tasks. We propose an approach for tracking the whole body up close, by splitting the sensing of different body parts across multiple sensors. Our setup allows tracking gaze, head, mid-air gestures, multi-touch gestures, and foot movements. We investigate specific applications for multimodal combinations in the domain of 3DUI, specifically how gaze and mid-air gestures can be combined to improve selection and manipulation tasks; how the feet can support the canonical 3DUI tasks; and how a multimodal sensing platform can inspire new 3D game mechanics. We show that the combination of multiple modalities can lead to enhanced task performance, that offloading certain tasks to alternative modalities not only frees the hands, but also allows simultaneous control of multiple degrees of freedom, and that by sensing different modalities separately, we achieve a more detailed and precise full body tracking.

In the end, we are all dead. But for some of us, our deaths become entangled online. Our vast data legacies and the appropriation of social media by the bereaved can result in online networks being used to mediate loss, mourning and memory in the event of a death. Recognising this phenomenon between death and technologies has resulted in researchers and designers being asked to become ‘thanatosensitive’, or death-sensitive. In particular, designers have been presented with Thanatosensitive Design [TSD] as an optimistic and non-prescriptive design methodology, devised by Massimi, for researching, designing and developing thanatosensitive technologies within sensitive end-of-life contexts (Massimi, 2012). This thesis is an invitation to reconsider TSD, to rethink what sensitive design practice could look like and the kinds of commitments and claims it is making to bereaved people. This reconsideration takes place through the development of an interdisciplinary conceptual framework that supports ‘thinking with’ and ‘caring for’ other elements in a situation of inquiry (Diprose, 2009; Puig de la Bellacasa, 2012; Puig de la Bellacasa, 2011). Empirically, this exposes a messy human landscape of loss, non-living ‘ghosts’ and non-human networks, the presences and agencies of which unsettle the human-centred ethico-political assumptions that lie within the TSD agenda. This thesis embraces the disturbances that have arisen from empirical and theoretical commitments and uses them as a way to reconsider what thanatosensitivity looks like when it embraces a more inclusive ethico-political landscape that decentres the human. Therefore, this thesis contributes to emerging literatures at the recent intersection between death and technology studies, firstly, by exposing a complex and previously unaccounted for messy ecology of loss across networks online, and, secondly, by reflexivity, exploring how this messy ecology disturbs the centrality of the human in TSD framings. These contributions cumulate in a reconfiguration of TSD that draws out an alternative approach and considerations for practitioners interested in designing sensitively for the end of life. This reconfiguration aims to be socially responsible, inclusive and ecologically sensitive in ways that set it apart from Massimi’s original concept of TSD. This new vision of sensitive design is summarised into a design statement and a polemic design manifesto to aid practitioners who wish to sensitively design for the end of life. The thesis leaves us with a speculative afterword, to consider future work and envision what other forms designing for death might take if we continue to push at the human-centeredness within design ecologies in light of the apocalyptic shadow of the Anthropocene.

Notifications from mobile devices frequently prompt us with information, either to merely inform us or to elicit a reaction. This has led to increasing research interest in considering an individual’s interruptibility prior to issuing notifications, in order for them to be positively received. To achieve this, predictive models need to be built from previous response behaviour where the individual’s interruptibility is known. However, there are several degrees of freedom in achieving this, from different definitions in what it means to be interruptible and a notification to be successful, to various methods for collecting data, and building predictive models. The primary focus of this thesis is to improve upon the typical convention used for labelling interruptibility, an area which has had limited direct attention. This includes the proposal of a flexible framework, called the decision-on-information-gain model, which passively observes response behaviour in order to support various interruptibility definitions. In contrast, previous studies have largely surrounded the investigation of influential contextual factors on predicting interruptibility, using a broad labelling convention that relies on notifications being responded to fully and potentially a survey needing to be completed. The approach is supported through two in-the-wild studies of Android notifications, one with 11,000 notifications across 90 users, and another with 32,000,000 across 3000 users. Analysis of these datasets shows that: a) responses to notifications is a decisionmaking process, whereby individuals can be reachable but not receptive to their content, supporting the premise of the approach; b) the approach is implementable on typical Android devices and capable of adapting to different notification designs and user preferences; and c) the different labels produced by the model are predictable using data sources that do not require invasive permissions or persistent background monitoring; however there are notable performance differences between different machine learning strategies for training and evaluation.