Mobile phones, tablet computers, laptops, desktops, and large screen displays are increasingly available to individuals for information access, often simultaneously. Dominant content access protocols, such as HTTP/1.1, do not take advantage of this device multiplicity and support information access from single devices only. Changing devices means restarting an information session. Using devices in conjunction with each other poses several challenges, which include the presentation of content on devices with diverse form factors and propagation of the content changes across these devices. In this dissertation, I report on the design and implementation of MIDAS - architecture and a prototype system for multi-device presentations. I propose a framework, called 12C, for characterizing multi-device systems and evaluate MIDAS within this framework. MIDAS is designed as a middleware that can work with multiple client-server architectures, such as the Web and context-aware Trellis, a non-Web hypertext system. It presents information content simultaneously on devices with diverse characteristics without requiring sensor-enhanced environments. The system adapts content elements for optimal presentation on the target device while also striving to retain fidelity with the original form from a human perceptual perspective. MIDAS reconfigures its presentation in response to user actions, availability of devices, and environmental context, such as a user's location or the time of day. I conducted a pilot study that explored human perception of similarity when image attributes such as size and color depth are modified in the process of presenting images on different devices. The results indicated that users tend to prefer scaling of images to color-depth reduction but gray scaling of images is preferable to either modification. Not all images scale equally gracefully; those dominated by natural elements or manmade structures scale exceptionally well. Images that depict recognizable human faces or textual elements should be scaled only to an extent that these features retain their integrity. Attributes of the 12C framework describe aspects of multi-device systems that include infrastructure, presentation, interaction, interface, and security. Based on these criteria, MIDAS is a flexible infrastructure, which lends itself to several content distribution and interaction strategies by separating client- and server-side configuration.

Future applications will run in a highly heterogeneous and dynamic execution environment that forces them to adapt their behavior and offered functionality depending on the user's or the system's current situation. Since application components in such heterogeneous multi-device systems will be distributed over multiple interconnected devices and cooperate to achieve a common goal, a coordinated adaptation is required to ensure a consistent system behavior. In this paper we present a decentralized adaptation middleware to adapt a distributed software system. Our approach supports the reliable execution of multiple adaptation operations that depend on each other and are performed transactionally even in unsteady environments coined by message loss or node failures. We implemented our approach in a search-and-rescue robot scenario to show its feasibility and conduct first performance evaluations.

Mobile devices equipped with features (e.g., camera, network connectivity and media player) are increasingly being used for different tasks such as web browsing, document reading and photography. While the portability of mobile devices makes them desirable for pervasive access to information, their small screen real-estate often imposes restrictions on the amount of information that can be displayed and manipulated on them. On the other hand, large displays have become commonplace in many outdoor as well as indoor environments. While they provide an efficient way of presenting and disseminating information, they provide little support for digital interactivity or physical accessibility. Researchers argue that mobile phones provide an efficient and portable way of interacting with large displays, and the latter can overcome the limitations of the small screens of mobile devices by providing a larger presentation and interaction space. However, distributing user interface (UI) elements across a mobile device and a large display can cause switching of visual attention and that may affect task performance. This thesis specifically explores how the switching of visual attention across a handheld mobile device and a vertical large display can affect a single user's task performance during mobile interaction with large displays. It introduces a taxonomy based on the factors associated with the visual arrangement of Multi Display User Interfaces (MDUIs) that can influence visual attention switching during interaction with MDUIs. It presents an empirical analysis of the effects of different distributions of input and output across mobile and large displays on the user's task performance, subjective workload and preference in the multiple-widget selection task, and in visual search tasks with maps, texts and photos. Experimental results show that the selection of multiple widgets replicated on the mobile device as well as on the large display, versus those shown only on the large display, is faster despite the cost of initial attention switching in the former. On the other hand, a hybrid UI configuration where the visual output is distributed across the mobile and large displays is the worst, or equivalent to the worst, configuration in all the visual search tasks. A mobile device-controlled large display configuration performs best in the map search task and equal to best (i.e., tied with a mobile-only configuration) in text- and photo-search tasks.

We present a method for developing multi-device services which allows for the creation of services that are adapted to a wide range of devices. Users have a wide selection of electronic services at their disposal such as shopping, banking, gaming, and messaging. They interact with these services using the computing devices they prefer or have access to, which can vary between situations. In some cases, the services that they want to use func-tions with the device they have access to, and sometimes it does not. Thus, in order for users to experience their full benefits, electronic services will need to become more flexible. They will need to be multi-device services, i.e. be accessible from different devices. We show that multi-device services are often used in different ways on different devices due to variations in device capabilities, purpose of use, context of use, and usability. This suggests that multi-device services not only need to be accessible from more than one device, they also need to be able to present functionality and user interfaces that suit various devices and situations of use. The key problem addressed in this work is that there are too many device-service combinations for developing a service version for each device. In-stead, there is a need for new methods for developing multi-device services which allows the creation of services that are adapted to various devices and situations. The challenge of designing and implementing multi-device services has been addressed in two ways in the present work: through the study of real-life use of multi-device services and through the creation of a development method for multi-device services. Studying use of multi-device services has gener-ated knowledge about how to design such services which give users the best worth. The work with development methods has resulted in a design model building on the separation of form and content, thus making it possible to create different presentations to the same content. In concrete terms, the work has resulted in design guidelines for multi-device services and a system prototype based on the principles of separation between form and content, and presentation control.

Inom modern informationsteknologi har självloggning fått en allt större roll. Dessa tjänster består vanligtvis av flera delar som behöver samverka för att ge en bra användarupplevelse. Eftersom fysiska enheter kompletteras med appar och webbtjänster behövs det en samspelad gränssnittsdesign för att användarna skall känna ett samband mellan tjänsterna och för att användarna skall använda flera delar. Detta kan göras på flera olika sätt, men genom olika designmål kan utvecklarna bland annat forma de olika delarna av tjänsterna mot samma mål eller spetsa dem för att uppfylla olika delar av tjänsten. Studien undersöker användarnas relation till olika gränssnitt och även olika tjänster. Denna uppsats xxnner att engagemang, personlighet och motivation påverkar valet av tjänst men även hur aktiva användarna är i ett flergränssnittsanvändande och vad de tycker om att behöva hantera flera gränssnitt. Denna uppsats belyser även hur användarna i sin jakt på större självin- sikt samlar och reflekterar över insamlad data över flera gränssnitt och enheter. Genom en kvalitativ intervjustudie med aktiva användare undersöks det närmare hur användarna upplever sitt samlande och hur tjänsterna fungerar och används över flera gränssnitt. Resultaten visar att användare ofta har problem med insamlingen av data och dessutom sällan använder sig av mer än ett eller kanske två gränssnitt. / In modern information technology Personal Informatics(PI) has gained an increasingly important role. Usually PI consists of several parts that need to work together to provide a great user experience. Since physical devices supplemented with apps and web services, to allow users to feel the connection between the services it is very important with a unified interface design. This can be done in several different ways, but through different design objectives you can shape services towards the same goal, or focus them to meet goals in other parts of the service. The study examines users' relationship with various interfaces and also various services. This paper finds that dedication, personality and motivation influences the choice of service but also, how active users are in a multi-use interface and what they think about having to manage multiple interfaces. This paper also highlights how the users in their quest for greater self-awareness collect and reflect on collected data over multiple interfaces and devices. Through qualitative interviews with active users we look closely at how users perceive their collecting and how the services works and are used across multiple interfaces. The results show that users often have problems with the collection of data and that they also rarely use more than one or maybe two interfaces.

With the continued development towards a digitalized and data-driven world, the importance of visual data analysis is increasing as well. Visual data analysis enables people to interactively explore and reason on certain data through the combined use of multiple visualizations. This is relevant for a wide range of application domains, including personal, professional, and public ones. In parallel, a ubiquity of modern devices with very heterogeneous characteristics has spawned. These devices, such as smartphones, tablets, or digital whiteboards, can enable more flexible workflows during our daily work, for example, while on-the-go, in meetings, or at home. One way to enable flexible workflows is the combination of multiple devices in so-called device ecologies. This thesis investigates how such a combined usage of devices can facilitate the visual data analysis of multivariate data sets. For that, new approaches for both visualization and interaction are presented here, allowing to make full use of the dynamic nature of device ecologies. So far, the literature on these aspects is limited and lacks a broader consideration of data analysis in device ecologies. This doctoral thesis presents investigations into three main parts, each addressing one research question: (i) how visualizations can be adapted for heterogeneous devices, (ii) how device pairings can be used to support data exploration workflows, and (iii) how visual data analysis can be supported in fully dynamic device ecologies. For the first part, an extended analytical investigation of the notion of responsive visualization is contributed. This investigation is then complemented by the introduction of a novel matrix-based visualization approach that incorporates such responsive visualizations as local focus regions. For the two other parts, multiple conceptual frameworks are presented that are innovative combinations of visualization and interaction techniques. In the second part, such work is conducted for two selected display pairings, the extension of smartwatches with display-equipped watchstraps and the contrary combination of smartwatch and large display. For these device ensembles, it is investigated how analysis workflows can be facilitated. Then, in the third part, it is explored how interactive mechanisms can be used for flexibly combining and coordinating devices by utilizing spatial arrangements, as well as how the view distribution process can be supported through automated optimization processes. This thesis’s extensive conceptual work is accompanied by the design of prototypical systems, qualitative evaluations, and reviews of existing literature.

[pt] Neste trabalho, investigamos como garantir duas propriedades no desenvolvimento de aplicações multimídia distribuídas interativas: determinismo e consistência. Determinismo é uma propriedade individual dos nós em uma aplicação distribuída e refere-se à característica de um programa sempre produzir a mesma saída a partir de uma mesma entrada. Consistência é uma propriedade de todo o sistema e está relacionada a todos os nós terem sempre a mesma visão da ordem dos eventos. Avaliamos o uso da linguagem síncrona Céu no contexto de programação multimídia para garantir o determinismo. Em relação à consistência, avaliamos se a arquitetura GALS (Globally Asynchronous Local Synchronous) é capaz de prover consistência. Tradicionalmente, aplicações multimídia são desenvolvidas usando linguagens de domínio específico ou linguagens de propósito geral utilizando frameworks especializados. Nenhuma dessas duas abordagens promove o desenvolvimento de aplicações multimídia distribuídas interativas determinísticas e consistentes. Nossa investigação sobre o uso de linguagens síncronas no campo de multimídia levou ao desenvolvimento de Céu-Media, uma biblioteca multimídia determinística para a linguagem síncrona Céu, e Mars, um middleware GALS para aplicações multimídia distribuídas interativas. Os resultados desta tese indicam que, usando as garantias da linguagem síncrona Céu, é possível desenvolver aplicações multimídia determinísticas usando Céu-Media. Além disso, eles também indicam que o modelo de consistência implementado pelo middleware GALS Mars garante que todos os nós sempre concordem com a ordem dos eventos em uma apresentação distribuída. Nós validamos nossa proposta discutindo o desenvolvimento de aplicações multimídia distribuídas propostas pela comunidade de pesquisa usando Céu-Media e Mars, destacando as principais vantagens e também as desvantagens em usar nossa abordagem. / [en] In this, work we investigate how to guarantee two properties in the development of interactive distributed multimedia applications: determinism and consistency. Determinism is a property of individual nodes in a distributed application and states that a program always produces the same output when fed with the same input. Consistency is a property of the whole system and states that all nodes should have the same view of the order of events. We evaluate the use of the synchronous language Céu in the context of multimedia programming for guaranteeing the determinism property. Regarding consistency, we evaluate the GALS (Globally Asynchronous Locally Synchronous) architecture for enforcing consistency. Traditionally, multimedia applications are developed using either a domain specific language or a general purpose language supported by specialized frameworks. Neither of the two approaches promotes the development of deterministic and consistent interactive distributed multimedia applications. Our investigation of the use of synchronous languages in the multimedia field led to the development of Céu-Media, a deterministic multimedia library for the synchronous language Céu, and Mars, a GALS middleware for interactive distributed multimedia applications. The results of this thesis indicate that using the guarantees of the synchronous language Céu it is possible to develop deterministic multimedia applications using Céu-Media. Furthermore, they also indicate that the consistency model enforced by the GALS middleware Mars guarantees that all nodes always agree upon the order of events in a distributed presentation. We validate our proposal by discussing the development of real-world distributed multimedia applications proposed by the research community using both, Céu-Media and Mars, highlighting the main advantages and also the drawbacks of using our approach.

Digital services are increasingly offered across multiple devices for a single user. A challenge for users is that multi-device services are more complex to use. While instructions are supposed to facilitate ease of use of the services, poorly designed instructions become obstacles themselves. This project explores considerations for designing digital instructions of setting up multi-device services. This is done through a user-centred approach by (1) understanding the user expectations and pain points when setting up a multi-device service, (2) designing solutions that align with user expectations and aid in overcoming pain points, and (3) a reflection of the design process and the resulting design solution. To conclude, a set of design recommendations are contributed for designers to arrive at intuitive instructions that facilitate users with setting up multi-device services. / Digitala tjänster erbjuds allt oftare på flera olika enheter för en och samma användare. En utmaning för användare är att det är mer komplicerat att använda tjänster på flera enheter. Även om instruktionerna är avsedda för att underlätta användningen av tjänsterna blir dåligt utformade instruktioner själva ett hinder. I detta projekt undersöks överväganden för utformning av digitala instruktioner för installation av tjänster på flera enheter. Detta görs genom ett användarcentrerat tillvägagångssätt genom 1) att förstå användarnas förväntningar och utmaningar när de installerar en tjänst på flera enheter, 2) att utforma lösningar som möter användarnas förväntningar samtidigt som de hjälper användarna att bemästra utmaningar, och 3) en reflektion av designprocessen och resultatet av designlösning. Avslutningsvis ges en rad rekommendationer för hur designer kan ta fram intuitiva instruktioner som underlättar för användarna att installerar tjänster på flera enheter.

The Internet of Things (IoT) industry has been thriving for the past few years, especially in the household field. The proposal of smart homes has enabled a increasing number of IoT products to enter people’s daily lives. However, the access control in smart homes has gradually grown up to be a prominent problem. Traditional access control mechanisms consider one unique trusted user that controls all access to the devices. However, multiuser and multi-device smart home scenarios pose fundamentally different challenges to the traditional single-user systems. Multiple trusted users are expected to manage the whole smart home system. Furthermore, the access control system of smart homes needs to consider the complexity of user relationships, the convenience of user experience, and the flexibility of access control. So the basic question this project is intended to answer can be concluded as: how to design an access control system based on the user-desired access control mechanism in the context of multiple users and multiple devices? This project first investigates both technical background and user study researches, summarizing current issues. Then, to study user perceptions of current access control systems, this project carries out semi-structured interviews, identifying user needs in access control systems. Finally, a fine-grained prototype satisfying user needs is offered. It consists of three modules, namely the user interaction module, the backend module and the manager module. Permissions are assigned through the user interaction module, translated into access control policies at the backend module and enforced by the manager module. The proposed prototype is then implemented in a simulated smart home environment and evaluated on its feasibility to identified requirements. / Internet of Things (IoT) industrin har blomstrat de senaste åren, särskilt inom hushållsområdet. Förslaget om smarta hem har gjort det möjligt för ett ökande antal IoT produkter att komma in i människors vardag. Tillgångskontrollen i smarta hem har dock gradvis vuxit till att bli ett framstående problem. Traditionella åtkomstkontrollmekanismer överväger en unik betrodd användare som styr all åtkomst till enheterna. Men smarta hemscenarier för flera användare och flera enheter utgör emellertid fundamentalt olika utmaningar för de traditionella enanvändarsystemen. Flera betrodda användare förväntas hantera hela systemet för smarta hem. Dessutom måste system för åtkomstkontroll i smarta hänsyn beakta komplexiteten i användarrelationer, användarupplevelsens bekvämlighet och flexibiliteten i åtkomstkontroll. Så den grundläggande frågan som detta projekt är tänkt att besvara kan slutas som: hur man utformar ett åtkomstkontrollsystem baserat på den användarönskade åtkomstkontrollmekanismen i flera användare och flera enheter? Detta projekt undersöker först både teknisk bakgrund och användarstudier, som sammanfattar aktuella frågor. För att studera användaruppfattningar om nuvarande åtkomstkontrollsystem genomför detta projekt halvstrukturerade intervjuer som identifierar användarens behov i åtkomstkontrollsystem. Slutligen erbjuds en finkornig prototyp som uppfyller användarens behov. Den består av tre moduler, nämligen användarinteraktionsmodulen, backendmodulen och chefsmodulen. Behörigheter tilldelas via användarinteraktionsmodulen, översätts till åtkomstkontrollpolicyer på backend-modulen och tillämpas av chefsmodulen. Den föreslagna prototypen implementeras sedan i en simulerad smart hemmiljö och utvärderas om det är genomförbart för identifierade krav.

In order to be able to perform many tasks, autonomous devices need to understand their environment and know where they are in this environment. Simultaneous Localisation and Mapping (SLAM) is a solution to this problem. When several devices attempt to jointly solve this problem they use Collaborative SLAM (C-SLAM), but this is a very resource-demanding process. In order to enable resource-constrained devices, like small mobile robots or eXtended Reality (XR) devices, to run C-SLAM we look towards a Server-Assisted C-SLAM architecture to lift the computational burden from these devices. In a real-world scenario, sensors might fail, the devices might process sensor data wrongly or a malicious actor might inject wrong data into the system. In order for these solutions to be reliable, they must be able to deal with these \emph{outliers}. This thesis looks into the impact of outliers in Server-Assisted C-SLAM algorithms and presents two novel solutions for a robust algorithm, based on robust estimation of the initial device poses. We show the novel solutions outperform the state of the art both in estimation accuracy, yielding better estimates of the real device trajectories, and computational performance, making it suitable for device-constrained devices. / För att kunna utföra flertalet uppgifter måste autonoma enheter förstå sin miljö och veta var de befinner sig i den här miljön. Simultaneous Localization and Mapping (SLAM) är en lösning på detta problem. När flera enheter försöker lösa detta problem tillsammans använder de Samarbetande SLAM (C-SLAM), men detta är en mycket resurskrävande process. För att möjliggöra att resursbegränsade enheter, så som exempelvis små mobila robotar eller eXtended Reality (XR)-enheter, ska kunna köra C-SLAM föreslås en serverassisterar C-SLAM-arkitektur beräkningsbördan kan lyftas från dessa enheter till servern. I ett verkligt scenario kan sensorer vara felaktiga, enheter behandla sensordata felaktigt eller illvilliga aktörer injicera felaktig data i systemet. Därför undersöker detta arbete effekten av \emph{outliers} i Serverassisterade C-SLAM-algoritmer och presenterar två nya lösningar för en robust algoritm, baserad på robusta uppskattningar av enhetens initiala positioner. Denna lösning visar sig överträffa likartade lösningar i litteraturen både vad gäller uppskattningsnoggrannhet, vilket ger bättre uppskattningar av den verkliga enhetsbanor och beräkningsprestanda, vilket gör den lämplig för enheter med begränsade resurser.