No description available.

Research into the usability of multi-modal systems has tended to be device-led, with a resulting lack of theory about multi-modal interaction and how it might differ from more conventional interaction. This is compounded by a confusion over the precise definition of modality between the various disciplines within the HCI community, how modalities can be effectively classified, and their usability properties. There is a consequent lack of appropriate methodologies and notations to model such interactions and assess the usability implications of these interfaces. The role of expertise and craft skill in using HCI techniques is also poorly understood. This thesis proposes a new definition of modality, and goes on to identify issues of importance to multi-modal usability, culminating in the development of a new methodology to support the identification of such usability issues. It additionally explores the role of expertise and craft skill in using usability modelling techniques to assess usability issues. By analysing the problems inherent in current definitions and approaches, as well as issues relevant to cognitive science, a clear understanding of both the requirements for a suitable definition of modality and the salient usability issues are obtained. A novel definition of modality, based on the three elements of sense, information form and temporal nature is proposed. Further, an associated taxonomy is produced, which categorises modalities within the sensory dimension as visual, acoustic and haptic. This taxonomy classifies modalities within the information form dimension as lexical, symbolic or concrete, and classifies the temporal form dimension modalities as discrete, continuous, or dynamic. This results in a twenty-seven cell taxonomy, with each cell representing one taxon, indicating one particular type of modality. This is a faceted classification system, with the modality named after the intersection of the categories, building the category names into a compound modality name. The issues surrounding modality are examined and refined into the concepts of modality types, properties and clashes. Modalities are identified as belonging to either the system or the user, and being expressive or receptive in type. Various properties are described based on issues of granularity and redundancy. The five different types of clashes are described. Problems relating to the modelling of multi-modal interaction are examined by means of a motivating case study based on a portion of an interface for a robotic arm. The effectiveness of five modelling techniques, STN, CW, CPM-GOMS, PUM and Z, in representing multi-modal issues are assessed. From this, and using the collated definition, taxonomy and theory, a new methodology, Evaluating Multi-modal Usability (EMU), is developed. This is applied to a previous case study of the robotic arm to assess its application and coverage. Both the definition and EMU are used by students in a case study to test the definition and methodology's effectiveness, and to examine the leverage such an approach may give. The results shows that modalities can be successfully identified within an interactive context, and that usability issues can be described. Empirical video data of the robotic arm in use is used to confirm the issues identified by the previous analyses, and to identify new issues. A rational re-analysis of the six approaches (STN, CW, CPM-GOMS, PUM, Z and EMU) is conducted in order to distinguish between issues identified through craft skill, based on general HCI expertise and familiarity with the problem, and issues identified due to the core of the method for each approach. This is to gain a realistic understanding of the validity of claims made by each method, and to identify how else issues might be identified, and the consequent implications. Craft skill is found to have a wider role than anticipated, and the importance of expertise in using such approaches emphasised. From the case study and the re-analyses the implications for EMU are examined, and suggestions made for future refinement. The main contributions of this thesis are the new definition, taxonomy and theory, which significantly contribute to the theoretical understanding of multi-modal usability, helping to resolve existing confusion in this area. The new methodology, EMU, is a useful technique for examining interfaces for multi-modal usability issues, although some refinement is required. The importance of craft skill in the identification of usability issues has been explicitly explored, with implications for future work on usability modelling and the training of practitioners in such techniques.

Executive function (EF) comprises a series of interrelated cognitive and self-regulatory skills which are required in nearly every facet of everyday life, particularly in novel circumstances. EF skills begin developing from birth and continue to grow well into adulthood but are most crucial for children as they are associated with academic and life success as well as mental and physical health. There is now strong evidence that these skills can be trained through targeted intervention in a diverse range of approaches, such as computer games, physical activity, and social play settings. This thesis presents the process of the design and evaluation of an active EF-training game (BrainQuest) for smart-phones, in participation with end-users: a group of 11-12-year-old children from a local Primary School. The design process placed emphasis on creating an engaging user experience, a phenomenon which has eluded many serious games, by building upon motivational game design theory and satisfying end-user requirements. However, in the pursuit of promoting particular executive functions: working memory; inhibitory control; planning and strategizing, the design integrated aspects of a cognitive assessment while also utilizing a range of alternative approaches for training EF, including physical activity and social play. Following an iterative design process which included many single session prototype evaluations, a mixed methods evaluation was undertaken during a 5-week study with twenty-eight 11-12-year-old school children. The study gathered exploratory qualitative and quantitative evidence regarding the game’s potential benefits which was evaluated by triangulating a range of data sources: multi-observer observations notes, interviews with children and teachers, game performance data and logs, and cognitive assessment outcomes. The analysis describes the statistical relationships between game and executive function ability, before exploring user experiences and evidence of cognitive challenge during gameplay through a series of triangulated case studies and general whole-class observations. The analysis presents the game to be engaging and enjoyable throughout the study and, for most children, able to generate a sustainable challenge. Though there were initial difficulties in understanding the complex game rules and technology, the game became increasingly usable and learnable for the target user group and created opportunities for goal setting. It also encouraged feelings of pride and self-confidence as well as facilitating positive social interactions and requiring regulation of emotion, which are considered to be pathways to developing executive functions (Diamond, 2012). There was also promising initial evidence that the game’s variable difficulty level system was able to challenge executive functions: planning and strategizing, working memory, and inhibitory control. Most notably, the game appeared to support improvements in strategizing ability by demanding increasing strategic complexity in response to evolving and increasingly difficult task demands. Supporting BrainQuest’s cognitive challenge, several statistical relationships emerged between executive function ability and game performance measures. However, the game’s ability to significantly improve cognitive outcomes could not yet be concluded. Nevertheless, these findings have implications for both the future design and evaluation practices undertaken by cognitive training researchers. From a design perspective, less credence should be paid to simply gamifying cognitive assessments while greater emphasis should be placed on integration of formal game design and motivational theories. With regards to evaluation, researchers should understand the importance of establishing first whether CTGs can remain engaging over time as well as the feasibility of their challenge to cognitive functions.

Through eye tracking technology, scientists have explored the eyes diverse aspects and capabilities. There are many potential applications that benefit from eye tracking. Each benefit from advances in computer technology as this results in improved quality and decreased costs for eye-tracking systems.This thesis presents a computer vision-based eye tracking system for human computer interaction. The eye tracking system allows the user to indicate a region of interest in a large data space and to magnify that area, without using traditional pointer devices. Presented is an iris tracking algorithm adapted from Camshift; an algorithm originally designed for face or hand tracking. Although the iris is much smaller and highly dynamic. the modified Camshift algorithm efficiently tracks the iris in real-time. Also presented is a method to map the iris centroid, in video coordinates to screen coordinates; and two novel calibration techniques, four point and one-point calibration. Results presented show that the accuracy for the proposed one-point calibration technique exceeds the accuracy obtained from calibrating with four points. The innovation behind the one-point calibration comes from using observed eye scanning behaviour to constrain the calibration process. Lastly, the thesis proposes a non-linear visualisation as an eye-tracking application, along with an implementation.

Children with Autism Spectrum Conditions (ASC) have difficulties in learning social and communication skills. This leads to impairments in social interaction, including lack of understanding others’ intentions, emotions, and mental states, and impairments in communication both verbal and nonverbal. One of the most widely used interventions that addresses social and communication skills is the “social story”. A social story aims to support children with ASC in coping with their own behaviour. Practitioners use social stories to present specific scenarios and to help children understand how they should respond. However, the development of social stories is time consuming, and teachers comment that it is difficult to share them as a resource for others or to customise them to individual children, using their current tools. This thesis explores how a social story authoring tool can be designed, developed and evaluated. The final aim is to better support practitioners in writing, using and assessing social stories for children with ASC compared with their current approaches. A series of studies with practitioners and researchers was carried out to inform the design of a social story authoring tool and to evaluate it. A framework for social stories was built with the purpose of informing the design. Based on this framework, a prototype was iteratively designed and developed. The final prototype (ISISSImproving Social Interaction through Social Stories) was evaluated with practitioners with experience in social story interventions. The evaluation showed that ISISS is perceived by practitioners to be a considerable improvement over their current approaches. The methodology employed in this research combines Action Research, User-Centred Design and Participatory Design. Practitioners and researchers were empowered with different roles at different research stages in order to maximise their contributions to the development process.

Recent trends show digital devices utilized with increasing frequency in most crimes committed. Investigating crime involving these devices is labor-intensive for the practitioner applying digital forensics tools that present possible evidence with results displayed in tabular lists for manual review. This research investigates how enhanced digital forensics tool interface visualization techniques can be shown to improve the investigator's cognitive capacities to discover criminal evidence more efficiently. This paper presents visualization graphs and contrasts their properties with the outputs of The Sleuth Kit (TSK) digital forensic program. Exhibited is the textual-based interface proving the effectiveness of enhanced data presentation. Further demonstrated is the potential of the computer interface to present to the digital forensic practitioner an abstract, graphic view of an entire dataset of computer files. Enhanced interface design of digital forensic tools means more rapidly linking suspicious evidence to a perpetrator. Introduced in this study is a mixed methodology of ethnography and cognitive load measures. Ethnographically defined tasks developed from the interviews of digital forensics subject matter experts (SME) shape the context for cognitive measures. Cognitive load testing of digital forensics first-responders utilizing both a textual-based and visualized-based application established a quantitative mean of the mental workload during operation of the applications under test. A t-test correlating the dependent samples' mean tested for the null hypothesis of less than a significant value between the applications' comparative workloads of the operators. Results of the study indicate a significant value, affirming the hypothesis that a visualized application would reduce the cognitive workload of the first-responder analyst. With the supported hypothesis, this work contributes to the body of knowledge by validating a method of measurement and by providing empirical evidence that the use of the visualized digital forensics interface will provide a more efficient performance by the analyst, saving labor costs and compressing time required for the discovery phase of a digital investigation.

Despite two decades of empirical studies focusing on programmers and the problems with programming, usability of textual programming languages is still hard to achieve. Its younger relation, visual programming languages (VPLs) also share the same problem of poor usability. This research explores and investigates the usability issues relating to VPLs in order to suggest a set of design principles that emphasise usability. The approach adopted focuses on issues arising from the interaction and communication between the human (programmers), the computer (user interface), and the program. Being exploratory in nature, this PhD reviews the literature as a starting point for stimulating and developing research questions and hypotheses that experimental studies were conducted to investigate. However, the literature alone cannot provide a fully comprehensive list of possible usability problems in VPLs so that design principles can be confidently recommended. A commercial VPL was, therefore, holistically evaluated and a comprehensive list of usability problems was obtained from the research. Six empirical studies employing both quantitative and qualitative methodology were undertaken as dictated by the nature of the research. Five of these were controlled experiments and one was qualitative-naturalistic. The experiments studied the effect of a programming paradigm and of representation of program flow on novices' performances. The results indicated superiority of control-flow programs in relation to data-flow programs; a control-flow preference among novices; and in addition that directional representation does not affect performance while traversal direction does - due to cognitive demands imposed upon programmers. Results of the qualitative study included a list of 145 usability problems and these were further categorised into ten problem areas. These findings were integrated with other analytical work based upon the review of the literature in a structured fashion to form a checklist and a set of design principles for VPLs that are empirically grounded and evaluated against existing research in the literature. Furthermore, an extended framework for Cognitive Dimensions of Notations is also discussed and proposed as an evaluation method for diagrammatic VPLs on the basis of the qualitative study. The above consists of the major findings and deliverables of this research. Nevertheless, there are several other findings identified on the basis of the substantial amount of data obtained in the series of experiments carried out, which have made a novel contribution to knowledge in the fields of Human-Computer Interaction, Psychology of Programming, and Visual Programming Languages.

Teleoperation system is an important tool to control a device or model in an isolated area remotely where the operator cannot perform the task locally. The vast majority of teleoperation systems provides the operator with visual and haptic control to accomplish the assignment as naturally as possible. However, on a teleoperation system with considerable distance, the time delay could cause a drop in performance. This thesis aims to minimize delay problem by implementing a prediction approach using Kalman Filter. Kalman Filter algorithm has been widely used to estimate user movement for tracking systems. Kalman filter provides an efficient mechanism to predict future state based on Bayesian estimation to sequentially predict future states and measure an actual system to update system parameters. The primary objective of this work is to extract information generated by our prototyping model and visualizing the data to reflect the performance of the system. We use Phantom Omni devices and 3D arm as a model. Different type of Kalman filter algorithms is used to test the accuracy and performance of predicted state generated by the filter. The result shows that the implementation of Extended Kalman Filter (EKF) and smoothing function could overcome the networking delay on certain degrees. The comparison shows that the EKF has better accuracy and performance compared to Unscented Kalman Filter (UKF) when estimating the future state. Additionally, the implementation of smoothing function could improve the stability of teleoperation system. / Teleoperation systemet är ett viktigt verktyg för att styra en enhet eller modell i ett isolerat område på distans där operatören inte kan utföra uppgiften lokalt. De allra flesta av teleoperation system ger föraren visuell och haptisk kontroll för att utföra uppdraget så naturligt som möjligt. Men på en teleoperation system med stort avstånd, kan tidsfördröjningen medföra en nedgång i prestanda. Denna avhandling syftar till att minimera förseningar problem genom att implementera en förutsägelse tillvägagångssätt med Kalman Filter. Kalman filteralgoritm har i stor utsträckning används för att uppskatta användarens rörlighet för spårning. Kalman filter ger en effektiv mekanism för att förutsäga framtida stat grundad på Bayesian uppskattningen att sekventiellt förutsäga framtida tillstånd och mäta ett verkligt system för att uppdatera systemparametrar. Det primära syftet med detta arbete är att extrahera information som genereras av vår prototypmodell och visualisera data för att återspegla systemets prestanda. Vi använder Phantom Omni enheter och 3D-arm som en modell. Olika typer av Kalman filter algoritmer används för att testa riktigheten och prestandan hos förutsagda tillståndet genereras av filtret. Resultatet visar att genomförandet av Extended Kalman filter (EKF) och utjämningsfunktionen kan övervinna nätverk dröjsmålsvissa grader. Jämförelsen visar att EKF har bättre noggrannhet och prestanda jämfört med Unscented Kalman Filter (UKF) vid bedömningen av framtida tillstånd. Dessutom, genomförandet av utjämningsfunktionen skulle kunna förbättra stabiliteten hos teleoperation systemet.

<p>When humans delegate tasks---whether to human workers or robots---they do so either to trade money for time, or to leverage additional knowledge and capabilities. For complex tasks, however, describing the work to be done requires substantial effort, which reduces the benefit to the requester who delegates tasks. On one hand, human workers---e.g., crowd workers, friends or colleagues on social network, factory workers---have diverse knowledge and level of commitment, making it difficult to achieve joint efforts towards the requester's goal. In contrast, robots and machines have clearly defined capabilities and full commitment, but the requester lacks an efficient way to coordinate them for flexible workflows. </p><p> </p><p> This dissertation presents a series of workflows and systems to enable efficient work transfer to human workers or robots. First, I present BlueSky, a system that can automatically coordinate hundreds of crowd workers to enumerate ideas for a given topic. The latent structure of the idea enumeration task is decomposed into a three-step workflow to guide the crowd workers. Second, I present CoStory, a system that requests alternative designs from friends or colleagues by decomposing the design task into hierarchical chunks. Third, I present AdapTutAR, a system that delegates machine operation tasks to workers through adaptive Augmented Reality tutorials. Finally, I present Vipo, a system that allows requesters to customize tasks for robots and smart machines through spatial-visual programming. This dissertation demonstrates that decomposing latent task structure enables task delegation in an on-demand, scalable, and distributed way.</p>

This thesis book represents an exploration of the relationship between architecture and robotics, tailored to meet the requirements of both architecture students and professionals and any other creative user. The investigation encompasses three distinct robotic arm applications for architecture students, introduces and evaluates an innovative 3D printing application with robotic arms, and presents projects focused on the design of human-robot interaction techniques and their system development. Furthermore, the thesis showcases the development of a more intuitive human-robot interaction system and explores various user interaction methods with robotic arms for rapid prototyping and fabrication. Each experiment describes the process, level of interaction, and key takeaways. The narrative of the thesis unfolds as a journey through different applications of robotic fabrication, emphasizing the creative human as the focal point of these systems. This thesis underscores the significance of user experience research and anticipates future innovations in the evolving landscape of the creative field. The discoveries made in this exploration lay a foundation for the study and design of interfaces and interaction techniques, fostering seamless collaboration between designers and robotic systems. Keywords: Robotic Fabrication - Human-Robot Interaction (HRI) - Human-Computer Interaction (HCI) - User Experience Research - Human-Centered Design - Architecture - Art - Creative Application / Master of Architecture