<p>With the advents in hardware techniques and mobile computing powers, Augmented Reality (AR) has been promising in various areas of our everyday life and work. By superimposing virtual assets onto the real world, the boundary between the digital and physical spaces has been significantly blurred, which bridges a large amount of digital augmentation and intelligence with the surroundings of the physical reality. Meanwhile, thanks to the increasing developments of Artificial Intelligence (AI) perception algorithms such as object detection, scene reconstruction, and human tracking, the dynamic behaviors of digital AR content have extensively been associated with the physical contexts of both humans and environments. This context-awareness enabled by the emerging techniques enriches the potential interaction modalities of AR experiences and improves the intuitiveness and effectiveness of the digital augmentation delivered to the consumers. Therefore, researchers are gradually motivated to include more contextual information in the AR domain to create novel AR experiences used for augmenting their activities in the physical world.</p> <p>    </p> <p>On a broader level, our work in this thesis focuses on novel designs and modalities that combine contextual information with AR content behaviors in context-aware AR experiences. In particular, we design the AR experiences by inspecting different types of contexts from the real world, namely 1) human actions, 2) physical entities, and 3) interactions between humans and physical environments. To this end, we explore 1) software and hardware modules, and conceptual models that perceive and interpret the contexts required by the AR experiences, and 2) supportive authoring tools and interfaces that enable users and designers to define the associations of the AR contents and the interaction modalities leveraging the contextual information. In this thesis, we mainly study the following workflows: 1) designing adaptive AR tutoring systems for human-machine-interactions, 2) customizing human-involved context-aware AR applications, 3) authoring shareable semantic-aware AR experiences, and 4) enabling hand-object-interaction datasets collection for scalable context-aware AR application deployment. We further develop the enabling techniques and algorithms including 1) an adaptation model that adaptively vary the AR tutoring elements based on the real-time learner's interactions with the physical machines, 2) a customized video-see-through AR headset for pervasive human-activity detecting, 3) a semantic adaptation model that adjusts the spatial relationships of the AR contents according to the semantic understanding of different physical entities and environments, and 4) an AR-based interface that empowers novice users to collect high-quality datasets used for training user- and cite-specific networks in hand-object-interaction-aware AR applications.</p> <p><br></p> <p>Takeaways from the research series include 1) the usage of the modern AI modules effectively enlarges both the spatial and contextual scalability of AR experiences, and 2) the design of the authoring systems and interfaces lowers the barrier for end-users and domain experts to leverage AI outputs in the creation of AR experiences that are tailored for target users. We conclude that involving AI techniques in both the creation and implementation stages of AR applications is crucial to building an intelligent, adaptive, and scalable ecosystem of context-aware AR applications.</p>

The consumers of today have become used to a constant flow of new content. Whether the content being music, film and series, games or other forms of media. This has created a strain on the developers and creators, to create new and original content for an ever demanding audience. Creating original content can be a costly and time consuming process. Today there are tools to help in this process, many that build on the idea of procedurally generated content. But with these tools alone it can be hard for the creator to leave their mark on the content, since most of it will look all the same from the same tool.  We propose CLAPPY, or Collaborative Learning Algorithm for Predicting Personal Yield. The AI colleague that will learn the patterns of the creator, and help them express themselves in their content, regardless of the tool.  Controlled experiments were conducted where subjects were given a content creation tool in the form of a terrain generator for games. The thesis then compares the results of the content creation tool when the subjects used it on their own and with an AI-collaborator.

As AI develops, grows, and expands, the more benefits we can have from it. AI is used in multiple fields to assist humans, such as object recognition, self-driving cars, or design tools. However, AI could be used for more than assisting humans in their tasks. It could be employed to collaborate with humans as colleagues in shared tasks, which is usually described as Mixed-Initiative (MI) paradigm. This paradigm creates an interactive scenario that leverage on AI and human strengths with an alternating and proactive initiative to approach a task. However, this paradigm introduces several challenges. For instance, there must be an understanding between humans and AI, where autonomy and initiative become negotiation tokens. In addition, control and expressiveness need to be taken into account to reach some goals. Moreover, although this paradigm has a broader application, it is especially interesting for creative tasks such as games, which are mainly created in collaboration. Creating games and their content is a hard and complex task, since games are content-intensive, multi-faceted, and interacted by external users.  Therefore, this thesis explores MI collaboration between human game designers and AI for the co-creation of games, where the AI's role is that of a colleague with the designer. The main hypothesis is that AI can be incorporated in systems as a collaborator, enhancing design tools, fostering human creativity, reducing their workload, and creating adaptive experiences. Furthermore, This collaboration arises several dynamic properties such as control, expressiveness, and initiative, which are all central to this thesis. Quality-Diversity algorithms combined with control mechanisms and interactions for the designer are proposed to investigate this collaboration and properties. Designer and Player modeling is also explored, and several approaches are proposed to create a better workflow, establish adaptive experiences, and enhance the interaction. Through this, it is demonstrated the potential and benefits of these algorithms and models in the MI paradigm.

Processen att skapa innehåll till digitala spel för hand är kostsamt och tidskrävande. Allteftersom spelindustrin expanderar ökar behovet av att minska produktionskostnaderna. En lösning på detta problem som det forskas om idag är procedurell generering av spelinnehåll. Kortfattat innebär detta att en algoritm gör det manuella arbetet istället för en designer. I denna uppsats presenterar vi en sådan metod för att automatisera processen att skapa kartor i digitala spel. Vår metod använder kontrollerbara agenter med blandade initiativ (dvs. designern och algoritmen turas om) för att skapa geometri. Vi använder stokastiska agenter för att skapa variation och deterministiska agenter för att garantera spelbarhet. För att kontrollera dessa agenter använder vi en uppsättning parametrar som kan manipuleras. Däröver har designern tillgång till ett antal verktyg inklusive möjligheten att låsa befintlig geometri, konvertera geometri till andra typer, lägga till geometri manuellt, och även möjligheten att använda agenter på specifika områden på kartan. Vi tittar på spelläget Battle i det digitala spelet Mario Kart 64 och visar hur vår metod kan användas för att skapa sådana kartor. Vi utförde en användarstudie på outputen från metoden och resultatet visar att kvaliteten är i allmänhet gynnsam. / The process of creating content for digital games by hand is a costly and time consumingone. As the game industry expands, the need to reduce costs becomes ever more pressing.One solution to this problem being research today is procedural generation of content.In short, this means that an algorithm does the labor rather than a designer. In thisthesis we present such a method for automating the process of creating maps in digitalgames. Our method uses controllable software agents and mixed initiative (i.e. allowingthe designer and algorithm to take turns) to create geometry. We use stochastic agentsto create variation and deterministic agents to ensure playability. To control these agentswe use a set of input parameters which can be manipulated. Furthermore, the designerhas access to a number of tools including the ability to lock existing geometry, convertgeometry to other types, add geometry manually, as well as the ability to use agents onspecific areas of the map. We look at the game mode Battle in the digital game MarioKart 64 and show how our method can be used to create such maps. We conducted auser study on the output of the method and the results show that the quality is generallyfavorable.

L’interaction homme-robot est un domaine qui en est encore à ses balbutiements.Les développements se sont avant tout concentrés sur l’autonomie et l’intelligence artificielle et doter les robots de capacités avancées pour exécuter des tâches complexes. Dans un proche avenir, les robots développeront probablement la capacité de s’adapter et d’apprendre de leur environnement. Les robots ont confiance, ne s’ennuient pas et peuvent fonctionner dans des environnements hostiles et dynamiques - tous des attributs souhaités à l’exploration spatiale et aux situations d’urgence ou militaires. Ils réduisent également les coûts de mission, augmentent la flexibilité de conception et maximisent la production de données. Cependant, lorsqu’ils sont confrontés à de nouveaux scénarios et à des événements inattendus, les robots sont moins performants par rapport aux êtres humains intuitifs et créatifs (mais aussi faillibles et biaisés). L’avenir exigera que les concepteurs de mission équilibrent intelligemment la souplesse et l’ingéniosité des humains avec des systèmes robotiques robustes et sophistiqués. Ce travail de recherche propose un cadre formel, basé sur la théorie de jeux, pour une équipe de drones qui doit coordonner leurs actions entre eux et fournir à l’opérateur humain des données suffisantes pour prendre des décisions « difficiles » qui maximisent l’efficacité de la mission, selon certaines directives opérationnelles. Notre première contribution a consisté à présenter un cadre décentralisé et une fonction d’utilité pour une mission de patrouille avec une équipe de drones. Ensuite, nous avons considéré l’effet de cadrage, ou « framing effect » en anglais, dans le contexte de notre étude,afin de mieux comprendre et modéliser à terme certains processus décisionnels sous incertitude.Ainsi, nous avons réalisé deux expérimentations avec 20 et 12 participants respectivement. Nos résultats ont révélé que la façon dont le problème a été présenté (effet de cadrage positif ou négatif), l’engagement émotionnel et les couleurs du texte ont affecté statistiquement les choix des opérateurs humains. Les données expérimentales nous ont permis de développer un modèle d’utilité pour l’opérateur humain que nous cherchons à intégrer dans la boucle décisionnelle du système homme-robots. Enfin, nous formalisons et évaluons l’ensemble du cadre proposé où nous "fermons la boucle" à travers une expérimentation en ligne avec 101 participants. Nos résultats suggèrent que notre approche permet d’optimiser le système homme-robots dans un contexte où des décisions doivent être prises dans un environnement incertain. / Human-robot interaction is a field that is still in its infancy. Developments havefocused on autonomy and artificial intelligence, and provide robots with advanced capabilitiesto perform complex tasks. In the near future, robots will likely develop the ability to adapt andlearn from their surroundings. Robots have reliance, do not get bored and can operate in hostileand dynamics environments - all attributes well suited for space exploration, and emergency ormilitary situations. They also reduce mission costs, increase design flexibility, and maximizedata production. However, when coped with new scenarios and unexpected events, robots palein comparison with intuitive and creative human beings. The future will require that missiondesigners balance intelligently the flexibility and ingenuity of humans with robust and sophisticatedrobotic systems. This research work proposes a game-theoretic framework for a drone teamthat must coordinate their actions among them and provide the human operator sufficient datato make “hard” decisions that maximize the mission efficiency, according with some operationalguidelines. Our first contribution was to present a decentralized framework and utility functionfor a drone-team patrolling mission. Then, we considered the framing effect in the context of ourstudy, in order to better understand and model certain human decision-making processes underuncertainty. Hence, two experiments were conducted with 20 and 12 participants respectively.Our findings revealed that the way the problem was presented (positive or negative framing), theemotional commitment and the text colors statistically affected the choices made by the humanoperators. The experimental data allowed us to develop a utility model for the human operatorthat we sought to integrate into the decision-making loop of the human-robot system. Finally,we formalized and evaluated the close-loop of the whole proposed framework with a last onlineexperiment with 101 participants. Our results suggest that our approach allow us to optimize thehuman-robot system in a context where decisions must be made in an uncertain environment.

<p dir="ltr">Data is collected everywhere in our world; however, it often is noisy and incomplete. Different sources of data may have different characteristics, quality levels, or come from dynamic and diverse environments. This poses challenges for both humans who want to gain insights from data and machines which are learning patterns from data. How can we leverage the diversity of data across domains to enhance our understanding and decision-making? In this thesis, we address this question by proposing novel methods and applications that use multiple domains as more holistic sources of information for both human and machine learning tasks. For example, to help human operators understand environmental dynamics, we show the detection and localization of distribution shifts to problematic features, as well as how interpretable distributional mappings can be used to explain the differences between shifted distributions. For robustifying machine learning, we propose a causal-inspired method to find latent factors that are robust to environmental changes and can be used for counterfactual generation or domain-independent training; we propose a domain generalization framework that allows for fast and scalable models that are robust to distribution shift; and we introduce a new dataset based on human matches in StarCraft II that exhibits complex and shifting multi-agent behaviors. We showcase our methods across various domains such as healthcare, natural language processing (NLP), computer vision (CV), etc. to demonstrate that learning from data across domains can lead to more faithful representations of data and its generating environments for both humans and machines.</p>

<p dir="ltr">In the dynamic realm of Artificial Intelligence (AI), this study explores the multifaceted landscape of Trustworthy AI with a dedicated focus on achieving both explainability and acceptance. The research addresses the evolving dynamics of AI, emphasizing the essential role of human involvement in shaping its trajectory.</p><p dir="ltr">A primary contribution of this work is the introduction of a novel "Trustworthy Explainability Acceptance Metric", tailored for the evaluation of AI-based systems by field experts. Grounded in a versatile distance acceptance approach, this metric provides a reliable measure of acceptance value. Practical applications of this metric are illustrated, particularly in a critical domain like medical diagnostics. Another significant contribution is the proposal of a trust-based security framework for 5G social networks. This framework enhances security and reliability by incorporating community insights and leveraging trust mechanisms, presenting a valuable advancement in social network security.</p><p dir="ltr">The study also introduces an artificial conscience-control module model, innovating with the concept of "Artificial Feeling." This model is designed to enhance AI system adaptability based on user preferences, ensuring controllability, safety, reliability, and trustworthiness in AI decision-making. This innovation contributes to fostering increased societal acceptance of AI technologies. Additionally, the research conducts a comprehensive survey of foundational requirements for establishing trustworthiness in AI. Emphasizing fairness, accountability, privacy, acceptance, and verification/validation, this survey lays the groundwork for understanding and addressing ethical considerations in AI applications. The study concludes with exploring quantum alternatives, offering fresh perspectives on algorithmic approaches in trustworthy AI systems. This exploration broadens the horizons of AI research, pushing the boundaries of traditional algorithms.</p><p dir="ltr">In summary, this work significantly contributes to the discourse on Trustworthy AI, ensuring both explainability and acceptance in the intricate interplay between humans and AI systems. Through its diverse contributions, the research offers valuable insights and practical frameworks for the responsible and ethical deployment of AI in various applications.</p>