abstract: The rapid advancements of technology have greatly extended the ubiquitous nature of smartphones acting as a gateway to numerous social media applications. This brings an immense convenience to the users of these applications wishing to stay connected to other individuals through sharing their statuses, posting their opinions, experiences, suggestions, etc on online social networks (OSNs). Exploring and analyzing this data has a great potential to enable deep and fine-grained insights into the behavior, emotions, and language of individuals in a society. This proposed dissertation focuses on utilizing these online social footprints to research two main threads – 1) Analysis: to study the behavior of individuals online (content analysis) and 2) Synthesis: to build models that influence the behavior of individuals offline (incomplete action models for decision-making). A large percentage of posts shared online are in an unrestricted natural language format that is meant for human consumption. One of the demanding problems in this context is to leverage and develop approaches to automatically extract important insights from this incessant massive data pool. Efforts in this direction emphasize mining or extracting the wealth of latent information in the data from multiple OSNs independently. The first thread of this dissertation focuses on analytics to investigate the differentiated content-sharing behavior of individuals. The second thread of this dissertation attempts to build decision-making systems using social media data. The results of the proposed dissertation emphasize the importance of considering multiple data types while interpreting the content shared on OSNs. They highlight the unique ways in which the data and the extracted patterns from text-based platforms or visual-based platforms complement and contrast in terms of their content. The proposed research demonstrated that, in many ways, the results obtained by focusing on either only text or only visual elements of content shared online could lead to biased insights. On the other hand, it also shows the power of a sequential set of patterns that have some sort of precedence relationships and collaboration between humans and automated planners. / Dissertation/Thesis / Doctoral Dissertation Computer Science 2019

Planning is the sub-area of artificial intelligence that studies the process of selecting actions to lead an agent, e.g. a robot or a softbot, to a goal state. In many realistic scenarios, any choice of actions can lead the robot into a dead-end state, that is, a state from which the goal cannot be reached. In such cases, the robot can, pro-actively, resort to human help in order to reach the goal, an approach called symbiotic autonomy. In this work, we propose two different approaches to tackle this problem: (I) contingent planning, where the initial state is partially observable, configuring a belief state, and the outcomes of the robot actions are non-deterministic; and (II) probabilistic planning, where the initial state may be partially or totally observable and the actions have probabilistic outcomes. In both approaches, the human help is considered a scarce resource that should be used only when necessary. In contingent planning, the problem is to find a policy (a function mapping belief states into actions) that: (i) guarantees the agent will always reach the goal (strong policy); (ii) guarantees that the agent will eventually reach the goal (strong cyclic policy), or (iii) does not guarantee achieving the goal (weak policy). In this scenario, we propose a contingent planning system that considers human help to transform weak policies into strong (cyclic) policies. To do so, two types of human help are included: (i) human actions that modify states and/or belief states; and (ii) human observations that modify belief states. In probabilistic planning, the problem is to find a policy (a function mapping between world states and actions) that can be one of these two types: a proper policy, where the agent has probability 1 of reaching the goal; or an improper policy, in the case of unavoidable dead-ends. In general, the goal of the agent is to find a policy that minimizes the expected accumulated cost of the actions while maximizes the probability of reaching the goal. In this scenario, this work proposes probabilistic planners that consider human help to transform improper policies into proper policies however, considering two new (alternative) criteria: either to minimize the probability of using human actions or to minimize the expected number of human actions. Furthermore, we show that optimal policies under these criteria can be efficiently computed either by increasing human action costs or given a penalty when a human help is used. Solutions proposed in both scenarios, contingent planning and probabilistic planning with human help, were evaluated over a collection of planning problems with dead-ends. The results show that: (i) all generated policies (strong (cyclic) or proper) include human help only when necessary; and (ii) we were able to find policies for contingent planning problems with up to 10^15000 belief states and for probabilistic planning problems with more than 3*10^18 physical states. / Planejamento é a subárea de Inteligência Artificial que estuda o processo de selecionar ações que levam um agente, por exemplo um robô, de um estado inicial a um estado meta. Em muitos cenários realistas, qualquer escolha de ações pode levar o robô para um estado que é um beco-sem-saída, isto é, um estado a partir do qual a meta não pode ser alcançada. Nestes casos, o robô pode, pró-ativamente, pedir ajuda humana para alcançar a meta, uma abordagem chamada autonomia simbiótica. Neste trabalho, propomos duas abordagens diferentes para tratar este problema: (I) planejamento contingente, em que o estado inicial é parcialmente observável, configurando um estado de crença, e existe não-determinismo nos resultados das ações; e (II) planejamento probabilístico, em que o estado inicial é totalmente observável e as ações tem efeitos probabilísticos. Em ambas abordagens a ajuda humana é considerada um recurso escasso e deve ser usada somente quando estritamente necessária. No planejamento contingente, o problema é encontrar uma política (mapeamento entre estados de crença e ações) com: (i) garantia de alcançar a meta (política forte); (ii) garantia de eventualmente alcançar a meta (política forte-cíclica), ou (iii) sem garantia de alcançar a meta (política fraca). Neste cenário, uma das contribuições deste trabalho é propor sistemas de planejamento contingente que considerem ajuda humana para transformar políticas fracas em políticas fortes (cíclicas). Para isso, incluímos ajuda humana de dois tipos: (i) ações que modificam estados do mundo e/ou estados de crença; e (ii) observações que modificam estados de crenças. Em planejamento probabilístico, o problema é encontrar uma política (mapeamento entre estados do mundo e ações) que pode ser de dois tipos: política própria, na qual o agente tem probabilidade 1 de alcançar a meta; ou política imprópria, caso exista um beco-sem-saída inevitável. O objetivo do agente é, em geral, encontrar uma política que minimize o custo esperado acumulado das ações enquanto maximize a probabilidade de alcançar a meta. Neste cenário, este trabalho propõe sistemas de planejamento probabilístico que considerem ajuda humana para transformar políticas impróprias em políticas próprias, porém considerando dois novos critérios: minimizar a probabilidade de usar ações do humano e minimizar o número esperado de ações do humano. Mostramos ainda que políticas ótimas sob esses novos critérios podem ser computadas de maneira eficiente considerando que ações humanas possuem um custo alto ou penalizando o agente ao pedir ajuda humana. Soluções propostas em ambos cenários, planejamento contingente e planejamento probabilístico com ajuda humana, foram empiricamente avaliadas sobre um conjunto de problemas de planejamento com becos-sem-saida. Os resultados mostram que: (i) todas as políticas geradas (fortes (cíclicas) ou próprias) incluem ajuda humana somente quando necessária; e (ii) foram encontradas políticas para problemas de planejamento contingente com até 10^15000 estados de crença e para problemas de planejamento probabilístico com até 3*10^18 estados do mundo.

Le concept de systèmes informatisés complexes rassemble tous les systèmes constitués d’un grand nombre de composantes inter-connectées et gérées par ordinateur. La configuration et la gestion de ces systèmes passe par une multitude de tâches critiques à leur bon fonctionnement et leur évolution. La problématique de la mise en place et la maîtrise des procédures de gestion et de configuration de tels systèmes devient un point critique au vu de la complexité croissante et du besoin d’évolution de ces derniers. L’objectif de cette thèse est de répondre à cette problématique, à travers le développement d’un outil de planification en mixed-initiative pour la co-génération automatique d’actions de gestion et de configuration, indépendamment du domaine d’application. Dans cette perspective, nous développons « Grounded Totally Ordered Hierarchical Planner » (GTOHP), un planificateur automatique hiérarchique, en « Hierarchical Task Network » (HTN), qui présente des performances élevées nécessaires à une interaction en mixed-initiative. Pour cela nous proposons un algorithme d’instanciation et de simplification des problèmes de planification HTN qui réduit de manière très significative leur complexité et améliore de ce fait les performances des algorithmes de planification. Nous proposons aussi une extension au langage de définition des domaines de planification automatique PDDL afin de modéliser les connaissances des experts du domaine d’application sous forme de méthodes de décomposition des tâches qui serviront à guider l’algorithme de planification HTN. Ensuite, nous intégrons au planificateur GTOHP des mécanismes de récolte de statistiques et d’in- formations sur les résultats des tests réalisés lors de la recherche de plans et nous l’intégrons dans le système « Mixed-Initiative Planner » (MIP) qui fournit plusieurs fonctionnalités d’interaction en mixed-initiative. Nous démontrons les performances élevées du planificateur GTOHP et les apports de l’algorithme d’instanciation et de simplification en le comparant à un planificateur HTN de l’état de l’art à travers une série d’expérimentations sur des domaines de planification issues de la compé- tition internationale de planification automatique. Enfin, nous proposons des critères d’évaluation pour les systèmes en mixed-initiative qui servent de base à la discussion du système MIP. / The concept of complex IT systems includes all systems consisting of a large number of inter-connected and computer-managed components. The configuration and management of these systems involves a multitude of tasks that are critical to their proper functioning and their evolution. The problem of defining procedures for managing and configuring such systems becomes very critical in view of their increasing complexity and their rapid evolution. The aim of this thesis is to develop a mixed-initiative planning tool for the automatic co-generation of a set of management and configuration actions, regardless of the application domain. In this perspective, we develop GTOHP, a hierarchical automatic planner, with HTN, that present the high performance needed for a mixed-initiative interaction. We propose an algorithm for the instantiation and the simplification of HTN planning problems, which significantly reduces their complexity and improves the performance of the planning algorithms. We also propose an extension to the « Planning Domain Definition Language » (PDDL) in order to modelize the knowledge of domain experts in the form of tasks decomposition methods that will be used to guide the HTN planning algorithm. Then, we integrate some mechanisms to GTOHP for collecting statistics and information about the tests results carried out during the plans search and integrate them into the MIP which provides several features of mixed-initiative interaction. We demonstrate the high performance of the GTOHP planner and the contributions of the instantiation and simplification algorithm, by comparing them to a state-of-the-art HTN planner through a series of experiments on planning domains from the international planning competitions. Finally, we propose a panel of evaluation criteria of mixed-initiative systems that serve as a basis for the discussion about the performances and contributions of the MIP system.

Le sujet de recherche est dans la continuité des travaux réalisés au cours de mon M2R sur la programmation par démonstration appliqué à la cobotique en milieu industriel. Ce sujet est à la croisée de plusieurs domaines (interaction Humain-Robot, planification automatique, apprentissage artificiel). Il s'agit maintenant d'aller au delà de ces premiers résultats obtenus au cours de mon M2R et de trouver un cadre générique pour la programmation de « cobots » (robots collaboratifs) en milieu industriel. L'approche cobotique consiste à ce qu'un opérateur humain, en tant qu'expert métier directement impliqué dans la réalisation des tâches en ligne, apprenne au robot à effectuer de nouvelles tâches et à utiliser le robot comme assistant « agile ». Dans ce contexte la thèse propose un mode d'apprentissage de type « end-user programming », c'est-à-dire simple et ne nécessitant pas d'être expert en robotique pour programmer le robot industriel Baxter. / The increasing presence of robots in industries has not gone unnoticed.Cobots (collaborative robots) are revolutionising industries by allowing robots to work in close collaboration with humans.Large industrial players have incorporated them into their production lines, but smaller companies hesitate due to high initial costs and the lack of programming expertise.In this thesis we introduce a framework that combines two disciplines, Programming by Demonstration and Automated Planning, to allow users without programming knowledge to program a robot.The user constructs the robot's knowledge base by teaching it new actions by demonstration, and associates their semantic meaning to enable the robot to reason about them.The robot adopts a goal-oriented behaviour by using automated planning techniques, where users teach action models expressed in a symbolic planning language.In this thesis we present preliminary work on user experiments using a Baxter Research Robot to evaluate our approach.We conducted qualitative user experiments to evaluate the user's understanding of the symbolic planning language and the usability of the framework's programming process.We showed that users with little to no programming experience can adopt the symbolic planning language, and use the framework.We further present our work on a Programming by Demonstration system used for organisation tasks.The system includes a goal inference model to accelerate the programming process by predicting the user's intended product configuration.

Artificial Intelligence Planning is about acting in order to achieve a desired goal. Under incomplete information, the task of finding the actions needed to achieve the goal can be modelled as a search problem in the belief space. This task is costly, as belief space is exponential in the number of states, which is exponential in the number of variables. Good belief representations and heuristics are thus critical for scaling up in this setting. The translation-based approach to automated planning with incomplete information deals with both issues by casting the problem of search in belief space to a search problem in state space, where each node of the search space represents a belief state. We develop plan synthesis tools that use translated versions of planning problems under uncertainty, with partial or null sensing available. We show formally under which conditions the introduced translations are polynomial, and capture all and only the plans of the original problems. We study empirically the value of these translations. / La Planificación es la disciplina de Inteligencia Artificial que estudia los procesos de razonamiento necesarios para conseguir las acciones que logren un objetivo dado. En presencia de información incompleta, el problema de planificación puede ser modelado como una búsqueda en el espacio de estados de creencia, cada uno de ellos representando un conjunto de estados posibles. Este problema es costoso ya que el numero de estados de creencia puede ser exponencial en el número de estados, lo cual es exponencial en el número de variables del problema. El uso de buenas representaciónes de los estados y de heurísticas informadas resultan cruciales para escalar en este espacio de búsqueda. En esta tesis se presentan traducciones para planificación con información incompleta, que transforman el problema de búsqueda en el espacio de estados de creencia, en búsqueda en espacio de estados, donde cada nodo representa un estado de creencia. Hemos desarrollado herramientas para la generación de planes para el problema traducido, ya sea con percepción parcial o nula. A su vez, demostramos formalmente bajo qué circunstancias las traducciones son polinómicas, completas y correctas. La evaluación empírica remarca el valor de dichas traducciones

L'apprentissage de modèles temporels constitue l'une des grandes problématiques de l'Exploration de Données (Data Mining). Dans cette thèse, nous avons développé un nouveau modèle temporel appelé TITA Rules (Règle associative temporelle basé sur des arbres d'intervalles). Ce modèle permet de décrire des phénomènes ayant un certain degré d'incertitude et/ou d'imprécision. Ce modèle permet entre autres d'exprimer la synchronicité entre évènements, les contraintes temporelles disjonctives et la négation temporelle. De par leur nature, les TITA Rules peuvent êtes utilisées pour effectuer des prédictions avec une grande précision temporel. Nous avons aussi développé un algorithme capable de découvrir et d'extraire de manière efficace des TITA Rules dans de grandes bases de données temporelles. Le cœur de l'algorithme est basé sur des techniques de minimisation d'entropie, de filtrage par Apriori et par des analyses de co-dépendance. Note modèle temporelle et notre algorithme ont été appliqués et évalués sur plusieurs jeux de données issues de phénomènes réels et de phénomènes simulés. La seconde partie de cette thèse à consisté à étudier l'utilisation de notre modèle temporel sur la problématique de la Planification Automatique. Ces travaux ont mené au développement d'un algorithme de planification automatique. L'algorithme prend en entrée un ensemble de TITA Rules décrivant le fonctionnement d'un système quelconque, une description de l'état initial du système, et un but à atteindre. En retour, l'algorithme calcule un plan décrivant la meilleure façon d'atteindre le but donné. Par la nature même des TITA Rules, cet algorithme est capable de gérer l'incertain (probabilités), l'imprécision temporelle, les contraintes temporelles disjonctives, ainsi que les événements exogènes prédictibles mais imprécis. / The learning of temporal patterns is a major challenge of Data mining. We introduce a temporal pattern model called Temporal Interval Tree Association Rules (Tita rules or Titar). This pattern model can be used to express both uncertainty and temporal inaccuracy of temporal events. Among other things, Tita rules can express the usual time point operators, synchronicity, order, and chaining,disjunctive time constraints, as well as temporal negation. Tita rules are designed to allow predictions with optimum temporal precision. Using this representation, we present the Titar learner algorithm that can be used to extract Tita rules from large datasets expressed as Symbolic Time Sequences. This algorithm based on entropy minimization, apriori pruning and statistical dependence analysis. We evaluate our technique on simulated and real world datasets. The problem of temporal planning with Tita rules is studied. We use Tita rules as world description models for a Planning and Scheduling task. We present an efficient temporal planning algorithm able to deal with uncertainty, temporal inaccuracy, discontinuous (or disjunctive) time constraints and predictable but imprecisely time located exogenous events. We evaluate our technique by joining a learning algorithm and our planning algorithm into a simple reactive cognitive architecture that we apply to control a robot in a virtual world.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第18408号 / 情博第523号 / 新制||情||92(附属図書館) / 31266 / 京都大学大学院情報学研究科社会情報学専攻 / (主査)教授 多々納 裕一, 教授 石田 亨, 准教授 畑山 満則 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

[pt] Nesta tese propomos a aplicação de um paradigma de planejamento baseado em uma disciplina de modelagem conceitual para as tarefas de Mineração de Processos. Postulamos que a abordagem apresentada habilita as tarefas de descoberta de processos, checagem de conformidade e melhoria de modelos em domínios educacionais, que tem características de processos não-estruturados – dependências entre tarefas, múltiplas dependências, eventos concorrentes, atividades que falham, atividades repetidas, traços parciais e estruturas de nocaute. Relacionamos os conceitos em ambas as áreas de pesquisa e demonstramos a abordagem aplicada a um exemplo em um domínio acadêmico, implementando os algoritmos como parte de uma Biblioteca de Planos Típicos para Mineração de Processos que constrói sobre a extensa literatura prévia. / [en] In this thesis we propose the application of an automated planning paradigm based on a conceptual modeling discipline for the Process Mining tasks. We posit that the presented approach enables the process discovery, conformance checking and model enhancement tasks for educational domains, comprising characteristics of unstructured processes – with intertask dependencies, multiple dependencies, concurrent events, failing activities, repeated activities, partial traces and knock-out structures. We relate the concepts in both areas of research, and demonstrate the approach applied to an academic domain example, implementing the algorithms as part of a Library for Typical Plans for Process Mining that leverages the extensive prior art in the literature.

Ambientes Pervasivos são ambientes povoados por diversos dispositivos (sensores, atuadores, etc.) e aplicações de software (locais ou distribuídas) incorporadas nestes ambientes físicos de forma transparente para o usuário. Ambientes deste tipo devem ser capazes de interagir e satisfazer as requisições do usuário de forma autônoma e transparente. Nesse sentido, um dos maiores desafios de pesquisa em ambientes pervasivos é a de estabelecer mecanismos automáticos para compor, de forma dinâmica, funcionalidades que satisfaçam as requisições dos usuários. Nesse sentido, nós partimos da hipótese que mecanismos automáticos de interação entre ambientes e usuários podem ser abordados como um problema de composição automática de serviços em ambientes pervasivos. Portanto, nossa proposta é desenvolver um modelo referência, a partir do qual podem ser implementados sistemas que permitam ao ambiente pervasivo interagir com o usuário de forma natural, automática e dinâmica. Desta forma, o sistema de composição de serviços gerencia e automatiza o processo de resolução de requisições feitas pelo usuário (de forma implícita e/ou explicita) através das funcionalidades (serviços) disponíveis no ambiente ou através de novas funcionalidades criadas pelo processo de composição. Para tal fim, este trabalho apresenta um Modelo de Referência que permita projetar, implementar e avaliar sistemas de composição de serviços que gerenciem e automatizem o processo de interação em diversos tipos de ambientes pervasivos. Nossa proposta traz vantagens como: o baixo acoplamento e a interoperabilidade, isto porque é possível selecionar, integrar e reutilizar de forma eficiente e efetiva serviços heterogêneos provenientes de diversos tipos de dispositivos e/ou aplicações. Além disso, o modelo ontológico WSMO (Web Services Modelling Ontology) nos permite descrever semanticamente as capacidades dos serviços como também as informações contextuais presentes no ambiente, o que torna nosso sistema mais perto de um ambiente pervasivo real como o idealizado por Mark Weiser. / Pervasive environments are populated by several devices (sensors, actuators, etc.) and software applications (local or distributed) incorporated these physical environments transparently to the user. Environments of this type should be able to interact and process user requests autonomously and transparently. In this sense, one of the biggest research challenges in pervasive environments is to establish automatic mechanisms to compose dynamically, features that meet the user requirements. In this sense, we set the hypothesis that automatic mechanisms of interaction between users and environments can be addressed as a problem of automatic composition of services in pervasive environments. Therefore, our proposal is to develop a reference model, from which systems can be implemented to enable the pervasive environment interact with the user in a natural, automatic and dynamic. Thus, the system service composition management and automates the process of resolving requests made by the user (implicitly and / or explicitly) through the functionality (services) available in the environment or through new features created by the process of composition. To this end, this paper presents a Reference Model that allows to design, implement and evaluate systems of composition of services that manage and automate the interaction process in different types of pervasive environments. Our proposal brings benefits such as loose coupling and interoperability, because it is possible to select, integrate and reuse in an efficient and effective heterogeneous services from different types of devices and / or applications. Furthermore, the ontological model WSMO (Web Services Modeling Ontology) allows us to semantically describe the capabilities of the services as well as contextual information in the environment, which makes our system closer to a real pervasive environment as conceived by Mark Weiser.

O grande avanço das técnicas de Planejamento em Inteligência Artificial fez com que a Engenharia de Requisitos e a Engenharia do Conhecimento ganhassem extrema importância entre as disciplinas relacionadas a projeto de engenharia (Engineering Design). A especificação, modelagem e análise dos domínios de planejamento automático se tornam etapas fundamentais para melhor entender e classificar os domínios de planejamento, servindo também de guia na busca de soluções. Neste trabalho, é apresentada uma proposta de um ambiente integrado de modelagem e análise de domínios de planejamento, que leva em consideração o ciclo de vida de projeto, representado por uma ferramenta gráfica de modelagem que utiliza diferentes representações: a UML para modelar e analisar as características estáticas dos domínios; XML para armazenar, integrar, e exportar informação para outras linguagens (ex.: PDDL); as Redes de Petri para fazer a análise dinâmica; e a PDDL para testes com planejadores. / The great development in Artificial Intelligence Planning has emphasized the role of Requirements Engineering and Knowledge Engineering among the disciplines that contributes to Engineering Design. The modeling and specification of automated planning domains turn out to be fundamental tasks in order to understand and classify planning domains and guide the application of problem solving techniques. In this work, it is presented the proposed integrated environment for modeling and analyzing automated planning domains, which considered the life cycle of a project, represented by a tool that uses several language representations: UML to model and perform static analyses of planning environments; XML to hold, integrate, share and export information to other language representations (e.g. PDDL); Petri Nets, where dynamic analyses are made; and PDDL for testing models with planners.