[pt] A principal vantagem da TV Digital é a disponibilização da interatividade nos programas televisivos. Entretanto, autores que desejam produzir programas interativos precisam aprender alguma linguagem de programaação, como a NCL, que é a linguagem declarativa do Sistema Brasileiro de TV Digital. Ferramentas de autoria nos fornecem um nível de abstração adequado sobre a linguagem de programação para facilitar esse processo de autoria. Entretanto, atualmente não encontramos ferramentas de autoria que abstraiam completamente a NCL. Este trabalho apresenta uma ferramenta de autoria que permite aos autores visualizar e editar aplicações interativas para a TV Digital sem precisar saber NCL. Além disso, essa ferramenta acelera e facilita a autoria dessas aplicações através de componentes gráficos. / [en] Digital TV enables interaction in television programs. However, authors who want to produce this kind of application have to learn programming languages, such as NCL, the declarative language of the Brazilian standard for digital TV. Authoring tools can provide us with adequate abstractions that facilitates the authoring process. However, nowadays we don’t have authoring tools that do so for the NCL language. In this dissertation, we present a tool that allows authors to visualize and edit content without knowing NCL. This tool accelerates and makes easier the authoring process of Digital TV applications.

<p><a>In animated film production, a great bulk of work goes into animating and scene layout. Currently, the industry is centered around flat screens and keyboard and mouse interfaces, which require a substantial amount of time to lay out scene sets and animate shots, resulting in high production costs and lengthy production schedules. This study was designed to determine whether immersive VR interfaces could reduce the time taken to lay out 3D scene sets and animate those scenes compared to traditional interfaces. More specifically, this study compared the time 38 participants took to set up and animate a 3D scene by using two different interfaces, e.g. a VR immersive interface and a traditional mouse and keyboard interface. Further, the study investigated subjects’ interface preference. Findings show that the virtual reality interface was about twice as fast for laying out the scene as the traditional interface. When surveyed, participants preferred the virtual reality interface by a large margin over the provided traditional interface.</a></p>

<p>Immersive reality technology, such as augmented and virtual reality, has recently become quite prevalent due to innovation in hardware and software, leading to cheaper devices such as Head-mounted displays. There is significant evidence of an improved rate of skill acquisition with immersive reality training. However, the knowledge required to develop content for such immersive media is still relatively high. Subject experts often work together with programmers to create such content. </p> <p><br></p> <p>Our research goal in this thesis can be broadly classified into four distinct but mutually dependent categories. First, we explored the problem of immersive content creation with ProcessAR, an AR-based system to develop 2D/3D content that captures subject matter experts (SMEs) environment-object interactions in situ. The design space for ProcessAR was identified from formative interviews with AR programming experts and SMEs, alongside a comparative design study with SMEs and novice users. To enable smooth workflows, ProcessAR locates and identifies different tools/objects through computer vision within the workspace when the author looks at them. We explored additional features, such as embedding 2D videos with detected objects and user-adaptive triggers. A final user evaluation comparing ProcessAR and a baseline AR authoring environment showed that, according to our qualitative questionnaire, users preferred ProcessAR.</p> <p><br></p> <p>Second, we explored a unified authoring and editing environment, EditAR, that can create content for multiple media, such as AR, VR, and video instructions, based on a single demonstration. EditAR captures the user's interaction within an environment and creates a digital twin, enabling users without programming backgrounds to develop content. We conducted formative interviews with the subject and media experts to design the system. The prototype was developed and reviewed by experts. We also performed a user study comparing traditional video creation with 2D video creation from 3D recordings via a 3D editor, which uses freehand interaction for in-headset editing. Users took five times less time to record instructions and preferred EditAR, giving significantly higher usability scores.</p> <p><br></p> <p>We then explore AnnotateXR, an extended reality (XR) workflow to collect various high fidelity data and auto-annotate it in a single demonstration. AnnotateXR allows users to align virtual models over physical objects, tracked with 6DoF sensors. AnnotateXR utilizes a hand tracking capable XR HMD coupled with 6DoF information and collision detection to enable algorithmic segmentation of different actions in videos through its digital twin. The virtual-physical mapping provides a tight bounding volume to generate semantic segmentation masks for the captured image data. Alongside supporting object and action segmentation, we also support other dimensions of annotation required by modern CV, such as Human-Object, Object-Object, and rich 3D recordings, all with a single demonstration. Our user study shows AnnotateXR produced over 112,000 annotated data points in 67 minutes while maintaining the same performance quality as manual annotations.</p> <p><br></p> <p>Lastly, We conducted two elicitation studies empirically evaluated to determine design guidance for cross-modal devices capable of supporting an immersive interface in VR, allowing for simultaneous interaction with direct hand interaction while allowing for keyboard and mouse input. Recent advances in hand tracking have allowed users to interact with and experience interactions closer and similar to interactions in the physical world. However, these added benefits of natural interaction come at the cost of precision and accuracy offered by legacy input media such as a keyboard/mouse. The results and the guidelines from the two studies were used to develop a prototype called the Immersive Keyboard, which was evaluated against only traditional interface of only the keyboard and mouse. </p> <p><br></p> <p>In this thesis, we have explored a novel extended reality authoring environment that enables users without programming to author asynchronous immersive content in-situ, especially for spatial instructions.</p>

"Intelligent Tutoring Systems (ITSs), while effective in enhancing students’ problem solving skills, are difficult and time-consuming to build. In order to reduce the length and the complexity of ITS construction, authoring tools are used. These tools provide a solid foundation for creating pedagogical exercises for students, and offer graphical user interfaces that eliminate the need for programming expertise. One of the major problems with today’s authoring tools is that they are still quite intricate and time-consuming to utilize, even for users who are familiar with them. Their steep learning curves often intimidate users who are only interested in creating simple tutoring systems. I have designed and implemented an authoring tool, called Mason, which strips away the visual interface design features of today’s top ITSs, and focuses on the creation of sophisticated pedagogical exercises using a hierarchical domain model. The exercise creation process includes the definition of numerous components, such as: a problem statement, the desired answer to the exercise, the strategies for tutoring students on the mistakes they make while trying to formulate the correct answer, and diagnostic rules for launching the appropriate strategies for specific student errors. The ultimate goal of Mason is to be able to significantly reduce the time needed to author text-based ITSs that are able to diagnose student answers and generate pedagogical dialogue accordingly. This goal was verified by using Mason to replicate the architecture of Ms. Lindquist, a sophisticated ITS for algebra that originally took over a year a construct. The replica was finished in less than a week, and was able to emulate Ms. Lindquist’s dialogue generation accurately with minor limitations."

In more than two decades of research and practical experiments in interactive digital narrative (IDN), much insight about the relationship of narrative and digital media has been gained and many successful experiments have been undertaken, as a survey of the field illustrates. However, current approaches also limit the scope of experimentation and constrain theory in interactive narrative forms original to digital media. After reviewing the "interactivisation" of legacy theory (neo-Aristotelian poetics for interactive drama, poststructuralism for hyperfiction, 20th century narratology for interactive fiction and as a general theory for IDN), the thesis introduces a theoretical framework that changes the focus from the product-centered view of legacy media towards system and the process of instantiation. The terms protostory describing the overall space of potential narratives in an IDN system, narrative design for the concrete assemblage of elements and narrative vectors as substructures that enable authorial control are introduced to supersede legacy terms like story and plot. On the practical side, the thesis identifies limitations of existing approaches (e.g. legacy metaphors like the timeline, and authoring tools that support only particular traditions) To overcome these limitations a software toolset built on the principles of robustness, modularity, and extensibility is introduced and some early results are evaluated. Finally, the thesis proposes an inclusive, open-ended iterative process as a structure for future IDN research in which practical implementations and research co-exist in a tightly coupled mutual relationship that allows changes on one side to be integrated on the other.

This dissertation is the developmental research report on the design and development of a multimedia tutorial on Anthropometry for university-level health science students. The practical implication of using Merrill's Model of Instructional Design for developing course work for senior students in the health sciences was investigated. The multimedia tutorial was designed and developed, and the prototype evaluated by a team consisting of subject and education experts. The prototype was tested by third-year dietetic students who had completed the lecture-based and paper-based course. The students were observed while using the program, then completed a structured questionnaire and finally were given the opportunity to express their views in an informal group discussion. It was found that if the activation of relevant existing knowledge does not take place, Merrill's Model will be turned inside-out: the student's own problem replaces the instructional problem. / Thesis (MEd (Computer-integrated Education))--University of Pretoria, 2003. / Social Studies Education / unrestricted

Multimedia production is an elaborate activity composed of multiple information management and transformation tasks that support an underlying creative goal. Examples of these activities are structuring, organization, modification and versioning of media elements, all of which depend on the maintenance of supporting documentation and metadata. In professional productions, which can count on proper human and material resources, such documentation is maintained by the production crew, being key to secure a high quality in the final content. In less resourceful configurations, such as amateur-oriented productions, at least reasonable quality standards are desirable in most cases, however the perceived difficulty in managing and transforming content can inhibit amateurs on producing content with acceptable quality. This problem has been tackled in many fronts, for instance via annotation methods, smart browsing methods and authoring techniques, just to name a few. In this dissertation, the primary objective is to take advantage of user-created annotations in order to aid amateur-oriented multimedia authoring. In order to support this objective, the contributions are built around an authoring approach based on structured multimedia documents. First, a custom language for Web-based multimedia documents is defined, based on SMIL (Synchronized Multimedia Integration Language). This language brings several contributions, such as the formalization of an extended graph-based temporal layout model, live editing of document elements and extended reuse features. Second, a model for document annotation and an algebra for document transformations are defined, both of which allows composition and extraction of multimedia document fragments based on annotations. Third, the previous contributions are integrated into a Web-based authoring tool, which allows manipulating a document while it is active. Such manipulations encompass several interaction techniques for enriching, editing, publishing and extending multimedia documents. The contributions have been instantiated with multimedia sessions obtained from synchronous collaboration tools, in scenarios of video-based lectures, meetings and video-based qualitative research. Such instantiations demonstrate the applicability and utility of the contributions / Produção multimídia é uma atividade complexa composta por múltiplas atividades de gerência e transformação de informação, as quais suportam um objetivo de criar conteúdo. Exemplos dessas atividades são estruturação, organização, modificação e versionamento de elementos de mídia, os quais dependem da manutenção de documentos auxiliares e metadados. Em produções profissionais, as quais podem contar com recursos humanos e materiais adequados, tal documentação é mantida pela equipe de produção, sendo instrumental para garantir a uma alta qualidade no produto final. Em configurações com menos recursos, como produções amadoras, ao menos padrões razoáveis de qualidade são desejados na maioria dos casos, contudo a dificuldade em gerenciar e transformar conteúdo pode inibir amadores a produzir conteúdo com qualidade aceitável. Esse problema tem sido atacado em várias frentes, por exemplo via métodos de anotação, métodos de navegação e técnicas de autoria, apenas para nomear algumas. Nesta tese, o objetivo principal é tirar proveito de anotações criadas pelo usuário com o intuito de apoiar autoria multimídia por amadores. De modo a subsidiar esse objetivo, as contribuições são construídas em torno uma abordagem de autoria baseada em documentos multimídia estruturados. Primeiramente, uma linguagem customizada para documentos multimídia baseados na Web é definida, baseada na linguagem SMIL (Synchronized Multimedia Integration Language). Esta linguagem traz diversas contribuições, como a formalização de um modelo estendido para formatação temporal baseado em grafos, edição ao vivo de elementos de um documento e funcionalidades de reúso. Em segundo, um modelo para anotação de documentos e uma álgebra para transformação de documentos são definidos, ambos permitindo composição e extração de fragmentos de documentos multimídia com base em anotações. Em terceiro, as contribuições anteriores são integradas em uma ferramenta de autoria baseada na Web, a qual permite manipular um documento enquanto o mesmo está ativo. Tais manipulações envolvem diferentes técnicas de interação com o objetivo de enriquecer, editar, publicar e estender documentos multimídia interativos. As contribuições são instanciadas com sessões multimídia obtidas de ferramentas de colaboração síncrona, em cenários de aulas baseadas em vídeos, reuniões e pesquisa qualitativa baseada em vídeos. Tais instanciações demonstram a aplicabilidade e utilidade das contribuições

The purpose of this research was to apply machine learning techniques to automate rule generation in the construction of Intelligent Tutoring Systems. By using a pair of somewhat intelligent iterative-deepening, depth-first searches, we were able to generate production rules from a set of marked examples and domain background knowledge. Such production rules required independent searches for both the“if" and“then" portion of the rule. This automated rule generation allows generalized rules with a small number of sub-operations to be generated in a reasonable amount of time, and provides non-programmer domain experts with a tool for developing Intelligent Tutoring Systems.

Cette thèse aborde la problématique de l’acquisition des connaissances dans le cadre de la conception des Environnements Informatiques pour l’Apprentissage Humain (EIAH). Dans le contexte spécifique de ce travail, nous nous intéressons à des EIAH destinés à enseigner des méthodes de résolution de problèmes. De telles méthodes permettent, dans un domaine donné, de reconnaître la classe d’un problème et d’être capable de savoir quelle technique de résolution appliquer pour le résoudre. Le coût de conception de ces EIAH est cependant très élevé, en particulier du fait de l’élicitation des connaissances, qui nécessite non seulement une expertise dans le domaine concerné mais également en programmation.Afin de réduire le coût de conception de ces EIAH, et permettre à des auteurs (enseignants ou pédagogues plus experts) de pouvoir éliciter sans programmer les connaissances nécessaires à l’EIAH, nous proposons un processus d’acquisition interactive de ces connaissances. Ce processus est mis en œuvre à travers la conception d’un outil auteur : AMBRE-KB. Pour ce faire, nous avons d’abord proposé des méta-modèles qui permettent de décrire la forme des connaissances à acquérir. Ces connaissances ne sont pas celles d’un expert dans un domaine, mais les connaissances telles qu’on voudrait qu’elles fonctionnent chez l’apprenant à l’issue de l’apprentissage. En s’appuyant sur ces méta-modèles, nous avons ensuite proposé un processus d’acquisition de ces connaissances qui permet d’assister l’auteur lors de l’élicitation des connaissances nécessaires, en lui permettant de construire un modèle de connaissances spécifique à un domaine.Nous avons mené deux expérimentations pour évaluer le processus d’acquisition des connaissances et sa mise en œuvre dans l’outil AMBRE-KB. La première porte sur la complétude. Il s’agit de vérifier, pour un domaine donné, si les modèles de connaissances générés par AMBRE-KB permettent à l’EIAH de résoudre des problèmes. L’objectif de la deuxième expérimentation est de mesurer l’utilité et l’utilisabilité de AMBRE-KB. Les résultats des deux expérimentations sont satisfaisants. / The general issue we addressed in this thesis is the challenge of knowledge elicitation in Intelligent Tutoring Systems (ITS). In the context of this work, we are interested in ITS teaching problem solving methods. Teaching methods for solving problems consists in teaching students how to think about the problem before starting its resolution. In a given field, such a method is based on a categorization of problems. Knowing to recognize the class of a problem enables students to choose the resolution technique associated with this class. However, designing such ITS is tedious and costly, and specially require expertise in the application domain and in programming.In order to reduce the design cost of these ITS and to enable an author (for example a teacher) to be able to elicit knowledge needed without programming, we propose an interactive knowledge elicitation process. This process is implemented through the design of an authoring tool: AMBRE-KB. For that, we first propose meta-models for the knowledge to be acquired. This knowledge is not an expert knowledge, but knowledge such as we would want that they work at the end of the learning. Next, we propose a knowledge acquisition process based on these meta-models, which enable the author to be assisted in the elicitation process enabling him/her to build specific knowledge models for a given domain.We conducted two experiments to evaluate the knowledge acquisition process and its implementation in the AMBRE-KB tool. The first relates to completeness. The aim is to verify, for a given domain, whether the knowledge models generated by AMBRE-KB enable the solver to solve problems. The aim of the second experiment is to measure the utility and usability of AMBRE-KB. The results of both experiments are satisfactory.

Multimedia production is an elaborate activity composed of multiple information management and transformation tasks that support an underlying creative goal. Examples of these activities are structuring, organization, modification and versioning of media elements, all of which depend on the maintenance of supporting documentation and metadata. In professional productions, which can count on proper human and material resources, such documentation is maintained by the production crew, being key to secure a high quality in the final content. In less resourceful configurations, such as amateur-oriented productions, at least reasonable quality standards are desirable in most cases, however the perceived difficulty in managing and transforming content can inhibit amateurs on producing content with acceptable quality. This problem has been tackled in many fronts, for instance via annotation methods, smart browsing methods and authoring techniques, just to name a few. In this dissertation, the primary objective is to take advantage of user-created annotations in order to aid amateur-oriented multimedia authoring. In order to support this objective, the contributions are built around an authoring approach based on structured multimedia documents. First, a custom language for Web-based multimedia documents is defined, based on SMIL (Synchronized Multimedia Integration Language). This language brings several contributions, such as the formalization of an extended graph-based temporal layout model, live editing of document elements and extended reuse features. Second, a model for document annotation and an algebra for document transformations are defined, both of which allows composition and extraction of multimedia document fragments based on annotations. Third, the previous contributions are integrated into a Web-based authoring tool, which allows manipulating a document while it is active. Such manipulations encompass several interaction techniques for enriching, editing, publishing and extending multimedia documents. The contributions have been instantiated with multimedia sessions obtained from synchronous collaboration tools, in scenarios of video-based lectures, meetings and video-based qualitative research. Such instantiations demonstrate the applicability and utility of the contributions / Produção multimídia é uma atividade complexa composta por múltiplas atividades de gerência e transformação de informação, as quais suportam um objetivo de criar conteúdo. Exemplos dessas atividades são estruturação, organização, modificação e versionamento de elementos de mídia, os quais dependem da manutenção de documentos auxiliares e metadados. Em produções profissionais, as quais podem contar com recursos humanos e materiais adequados, tal documentação é mantida pela equipe de produção, sendo instrumental para garantir a uma alta qualidade no produto final. Em configurações com menos recursos, como produções amadoras, ao menos padrões razoáveis de qualidade são desejados na maioria dos casos, contudo a dificuldade em gerenciar e transformar conteúdo pode inibir amadores a produzir conteúdo com qualidade aceitável. Esse problema tem sido atacado em várias frentes, por exemplo via métodos de anotação, métodos de navegação e técnicas de autoria, apenas para nomear algumas. Nesta tese, o objetivo principal é tirar proveito de anotações criadas pelo usuário com o intuito de apoiar autoria multimídia por amadores. De modo a subsidiar esse objetivo, as contribuições são construídas em torno uma abordagem de autoria baseada em documentos multimídia estruturados. Primeiramente, uma linguagem customizada para documentos multimídia baseados na Web é definida, baseada na linguagem SMIL (Synchronized Multimedia Integration Language). Esta linguagem traz diversas contribuições, como a formalização de um modelo estendido para formatação temporal baseado em grafos, edição ao vivo de elementos de um documento e funcionalidades de reúso. Em segundo, um modelo para anotação de documentos e uma álgebra para transformação de documentos são definidos, ambos permitindo composição e extração de fragmentos de documentos multimídia com base em anotações. Em terceiro, as contribuições anteriores são integradas em uma ferramenta de autoria baseada na Web, a qual permite manipular um documento enquanto o mesmo está ativo. Tais manipulações envolvem diferentes técnicas de interação com o objetivo de enriquecer, editar, publicar e estender documentos multimídia interativos. As contribuições são instanciadas com sessões multimídia obtidas de ferramentas de colaboração síncrona, em cenários de aulas baseadas em vídeos, reuniões e pesquisa qualitativa baseada em vídeos. Tais instanciações demonstram a aplicabilidade e utilidade das contribuições