Wong Ka Ho. / Thesis submitted in: October 2005. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2006. / Includes bibliographical references (leaves 163-170). / Abstracts in English and Chinese. / Abstract --- p.2 / Acknowledgements --- p.5 / Chapter 1 --- Introduction --- p.10 / Chapter 1.1 --- Multi-modal and Multi-media --- p.10 / Chapter 1.2 --- Overview --- p.11 / Chapter 1.3 --- Thesis Goal --- p.13 / Chapter 1.4 --- Thesis Outline --- p.15 / Chapter 2 --- Background --- p.16 / Chapter 2.1 --- Multi-modal Fission --- p.17 / Chapter 2.2 --- Multi-modal Data collection --- p.21 / Chapter 2.2.1 --- Collection Time --- p.21 / Chapter 2.2.2 --- Annotation and Tools --- p.21 / Chapter 2.2.3 --- Knowledge of Multi-modal Using --- p.21 / Chapter 2.3 --- Text-to-audiovisual Speech System --- p.22 / Chapter 2.3.1 --- Different. Approaches to Generate a Talking Heading --- p.23 / Chapter 2.3.2 --- Sub-tasks in Animating a Talking Head --- p.25 / Chapter 2.4 --- Modality Selection --- p.27 / Chapter 2.4.1 --- Rules-based approach --- p.27 / Chapter 2.4.2 --- Plan-based approach --- p.28 / Chapter 2.4.3 --- Feature-based approach --- p.29 / Chapter 2.4.4 --- Corpus-based approach --- p.30 / Chapter 2.5 --- Summary --- p.30 / Chapter 3 --- Information Domain --- p.31 / Chapter 3.1 --- Multi-media Information --- p.31 / Chapter 3.2 --- "Task Goals, Dialog Acts, Concepts and Information Type" --- p.32 / Chapter 3.2.1 --- Task Goals and Dialog Acts --- p.32 / Chapter 3.2.2 --- Concepts and Information Type --- p.36 / Chapter 3.3 --- User's Task and Scenario --- p.37 / Chapter 3.4 --- Chapter Summary --- p.38 / Chapter 4 --- Multi-modal Response Data Collection --- p.41 / Chapter 4.1 --- Data Collection Setup --- p.42 / Chapter 4.1.1 --- Multi-modal Input Setup --- p.43 / Chapter 4.1.2 --- Multi-modal Output Setup --- p.43 / Chapter 4.2 --- Procedure --- p.45 / Chapter 4.2.1 --- Precaution --- p.45 / Chapter 4.2.2 --- Recording --- p.50 / Chapter 4.2.3 --- Data Size and Type --- p.50 / Chapter 4.3 --- Annotation --- p.52 / Chapter 4.3.1 --- Extensible Multi-Modal Markup Language --- p.52 / Chapter 4.3.2 --- "Mobile, Multi-biometric and Multi-modal Annotation" --- p.53 / Chapter 4.4 --- Problems in the Wizard-of-Oz Setup --- p.56 / Chapter 4.4.1 --- Lack of Knowledge --- p.57 / Chapter 4.4.2 --- Time Deficiency --- p.57 / Chapter 4.4.3 --- Information Availability --- p.58 / Chapter 4.4.4 --- Operation Delay --- p.59 / Chapter 4.4.5 --- Lack of Modalities --- p.59 / Chapter 4.5 --- Data Optimization --- p.61 / Chapter 4.5.1 --- Precaution --- p.61 / Chapter 4.5.2 --- Procedures --- p.61 / Chapter 4.5.3 --- Data Size in Expert Design Responses --- p.63 / Chapter 4.6 --- Analysis and Discussion --- p.65 / Chapter 4.6.1 --- Multi-modal Usage --- p.67 / Chapter 4.6.2 --- Modality Combination --- p.67 / Chapter 4.6.3 --- Deictic term --- p.68 / Chapter 4.6.4 --- Task Goal and Dialog Acts --- p.71 / Chapter 4.6.5 --- Information Type --- p.72 / Chapter 4.7 --- Chapter Summary --- p.74 / Chapter 5 --- Text-to-Audiovisual Speech System --- p.76 / Chapter 5.1 --- Phonemes and Visemes --- p.77 / Chapter 5.2 --- Three-dimensional Facial Animation --- p.82 / Chapter 5.2.1 --- Three-dimensional (3D) Face Model --- p.82 / Chapter 5.2.2 --- The Blending Process for Animation --- p.84 / Chapter 5.2.3 --- Connectivity between Visemes --- p.85 / Chapter 5.3 --- User Perception Experiments --- p.87 / Chapter 5.4 --- Applications and Extension --- p.89 / Chapter 5.4.1 --- Multilingual Extension and Potential Applications --- p.89 / Chapter 5.5 --- Talking Head in Multi-modal Dialogue System --- p.90 / Chapter 5.5.1 --- Prosody --- p.93 / Chapter 5.5.2 --- Body Gesture --- p.94 / Chapter 5.6 --- Chapter Summary --- p.94 / Chapter 6 --- Modality Selection and Implementation --- p.98 / Chapter 6.1 --- Multi-modal Response Examples --- p.98 / Chapter 6.1.1 --- Single Concept-value Example --- p.99 / Chapter 6.1.2 --- Two Concept-values with Different Information Types --- p.102 / Chapter 6.1.3 --- Multiple Concept-values with Same Information Types Example --- p.103 / Chapter 6.2 --- Heuristic Rules for Modality Selection --- p.105 / Chapter 6.2.1 --- General Principles --- p.106 / Chapter 6.2.2 --- Heuristic rules --- p.107 / Chapter 6.2.3 --- Temporal Coordination for Synchronization --- p.109 / Chapter 6.2.4 --- Physical Layout --- p.110 / Chapter 6.2.5 --- Deictic Term --- p.111 / Chapter 6.2.6 --- Example --- p.111 / Chapter 6.3 --- Spoken Content Generation --- p.113 / Chapter 6.4 --- Chapter Summary --- p.115 / Chapter 7 --- Conclusions and Future Work --- p.117 / Chapter 7.1 --- Summary --- p.117 / Chapter 7.2 --- Contributions --- p.118 / Chapter 7.3 --- Future work --- p.119 / Chapter A --- XML Schema for M3 Markup Language --- p.123 / Chapter B --- M3ML Examples --- p.128 / Chapter C --- Domain-Specific Task Goals in the Hong Kong Tourism Do- main --- p.131 / Chapter D --- Dialog Acts for User Request in the Hong Kong Tourism Do- main --- p.133 / Chapter E --- Dialog Acts for System Response in the Hong Kong Tourism Domain --- p.137 / Chapter F --- Information Type and Concepts --- p.141 / Chapter G --- Concepts --- p.143 / Bibliography --- p.149

This thesis consists of the production of an inter-active computer-based artwork, an analysis of its research outcomes, and an exploration of the theoretical issues that influenced the artistic practice. The artwork, Postcard from Tunis, is an Integrationist exploration of writing and its transformation at the human-computer interface. It is set in a personal portrait of Tunis, a city with a rich history of writing. The thesis starts with the theory of writing. The conventional view of real writing as representation of speech is shown to have serious limitations.Postcard from Tunis offers users who are not Arabic-literate the perception that there are actually no fixed boundaries between writing and pictures, as both are based on spatial configurations. User interaction with Postcard from Tunis, particularly rollover activity, creates a variety of dynamic signs that cannot be theorised by a bipartate theory of signs and that transcend a distinction between the verbal and the non-verbal altogether. Postcard from Tunis both extends Integrationist theory into writing and human-computer interaction and also uniquely articulates this integration of activities in a way that is impossible with written words on paper. The research asserts the validity of the Integrationist theory of writing, language and human communication and of uncoupling these from spoken words. A framework is outlined for future Integrationist research into icons and human-computer interaction. / Doctor of Philosophy (PhD)

Abstract: This research is the culmination of a four-year investigation and analysis into the principles of Interaction Design, particularly those that are found to be most suitable when designing and developing interactive navigation systems. The research was undertaken as a Masters degree by project. The project consists of a CD containing an online gallery showcasing works of art and an accompanying exegesis. The exegesis is structured into seven chapters, which consider, analyse and define what the key characteristics of Interaction Design are, where it comes from, and how it improves the quality of interactive multimedia applications. The exegesis includes four case studies that look at how other practitioners in the digital realm have created systems for showcasing narrative or creative content online. I examine alternative artworks and how they have shaped the development of creative media. I investigate what experts in the field define as good Interaction Design and what guidelines and principles they recommend. I show how these guidelines conflict with more creative approaches and how good design and creativity can be merged to be usable and friendly to users. I also look at the history of opponents of guidelines and principles and how their contribution helps make design better. By creating the example gallery I aim to help designers working within the field of ID to understand the principles behind good design in order that they may deliver higher-quality user experiences relevant to the content they are displaying. By creating this gallery I also hope to help artists understand the principles behind good design in order that they may showcase their artworks in ways appropriate to their artwork. By designing and building an example I aim to provide a better understanding of how to construct a feature-rich application in an easy to use and understandable environment.

Generative Multimedia (GM) applications are an increasingly popular way to implement interactive media performances. Our contributions include creating a metric for evaluating Generative Multimedia performances, designing a model for accepting perceiver preferences, and using those preferences to adapt GM performances. The metric used is imprecision, which is the ratio of the actual computation time of a GM element to the computation time of a complete version of that GM element. By taking a perceiver's preferences into account when making adaptation decisions, applications can produce GM performances that meet soft real-time and resource constraints while allocating imprecision to the GM elements the perceiver least cares about. Compared to other approaches, perceiver-directed imprecision best allocates impreciseness while minimizing delay.

In more than two decades of research and practical experiments in interactive digital narrative (IDN), much insight about the relationship of narrative and digital media has been gained and many successful experiments have been undertaken, as a survey of the field illustrates. However, current approaches also limit the scope of experimentation and constrain theory in interactive narrative forms original to digital media. After reviewing the "interactivisation" of legacy theory (neo-Aristotelian poetics for interactive drama, poststructuralism for hyperfiction, 20th century narratology for interactive fiction and as a general theory for IDN), the thesis introduces a theoretical framework that changes the focus from the product-centered view of legacy media towards system and the process of instantiation. The terms protostory describing the overall space of potential narratives in an IDN system, narrative design for the concrete assemblage of elements and narrative vectors as substructures that enable authorial control are introduced to supersede legacy terms like story and plot. On the practical side, the thesis identifies limitations of existing approaches (e.g. legacy metaphors like the timeline, and authoring tools that support only particular traditions) To overcome these limitations a software toolset built on the principles of robustness, modularity, and extensibility is introduced and some early results are evaluated. Finally, the thesis proposes an inclusive, open-ended iterative process as a structure for future IDN research in which practical implementations and research co-exist in a tightly coupled mutual relationship that allows changes on one side to be integrated on the other.

Title from PDF of title page. / Document formatted into pages; contains 97 pages. / Thesis (M.A.)--University of South Florida, 2003. / Includes bibliographical references. / Text (Electronic thesis) in PDF format. / ABSTRACT: Computer-based multimedia offer an alternative means of providing instruction to learners in two primary, yet disparate, ways. Multimedia can be used to convey information to learners, or alternatively, learners can make use of multimedia to impart information. One example of the use of multimedia technologies at the University of South Florida is an interactive computer kiosk installed in the Anthropology Exhibit Gallery. The development of the educational program featured on the kiosk's touchscreen computer is the subject of this paper. The purpose of the kiosk's program was twofold: 1) to introduce the field of anthropology to university students and the general public who visit the Anthropology Exhibit Gallery; and 2) to incorporate training in the creation of multimedia materials into two departmental project-based courses, Museum Methods and Visual Anthropology. / ABSTRACT: Designing effective educational programs that take advantage of multimedia capabilities without losing focus on the user's needs or on the content being presented is a challenging endeavor. In this paper, I present the process of designing an interactive multimedia program, and discuss the critical issues of audience, hardware and software, programming tools and other technical and design considerations. The development of the program, furthermore, must be understood within the broader context of several areas, including anthropology and museums, the role of education in museums, and exhibitions as a form of media and communication. Finally, a summary of the project is presented, including a discussion of the problems and successes encountered and suggested areas for further development. / System requirements: World Wide Web browser and PDF reader. / Mode of access: World Wide Web.

Thesis (DAppLing)--Macquarie University, Faculty of Human Sciences, Dept. of Linguistics, 2009. / Bibliography: p. 208-233.

Thesis (Ph.D)--Computing, Georgia Institute of Technology, 2010. / Committee Chair: Bruckman, Amy; Committee Member: Grinter, Rebecca; Committee Member: Grudin, Jonathan; Committee Member: Guzdial, Mark; Committee Member: Kolodner, Janet. Part of the SMARTech Electronic Thesis and Dissertation Collection.

This evaluation started with preliminary research into the situations and problems in science classrooms and computer laboratories. The preliminary research identified teacher-centred lessons, learner and teacher conceptualisations, large numbers of learners per classroom, assessment, and a lack of interest in biology as some of the major problems in South African classrooms. The current research (because it is continuing) uses two Educational Computer Programmes (ECPs); a Computer-Aided Assessment (CAA) programme which is designed to alleviate problems in assessment, and Zadarh (a constructivist adventure game) designed to solve problems in biology classrooms, to further investigate some of the identified problems and find out the learners' and teachers' views on the utility of these two ECPs. The use of these two ECPs had not previously been investigated appropriately, especially in disadvantaged communities where teachers had little knowledge of the use and of evaluating ECPs. Therefore, a major concern for this study is that previous ECP evaluations excluded teachers and were not comprehensive enough especially for deploying ECPs in disadvantaged communities. A review of the methods that had hitherto been used, indicated that quantitative, mostly, behavioural and cognitive, pre-test post-test methods were prominently used, despite the shift in instructional design to constructional design, which embrace qualitative aspects of learning. Also, instructional design has evolved from behavioural models to include constructivist microworlds, which were unfairly evaluated by excluding qualitative benefits. Thus, this study seeks a more comprehensive evaluation strategy, in which teachers play the role of co-evaluators and which captures the qualitative and quantitative changes that software programs impart upon teachers' classroom practices, with sensitivity to the multiple disciplines in a program, as well as to the value systems of teachers. Comprehensive evaluation processes were facilitated during which 26 teachers in 23 schools in the Eastern Cape, KwaZulu Natal and Mpumalanga Provinces embarked upon the evaluation of the two ECPs. Evaluations were based upon a developmental, constructivist and interpretative approaches, by which teachers took ownership of these evaluations. Comprehensive evaluations revealed benefits from CAA and Zadarh, as well as benefits from direct teacher participations in the evaluations. CAA (Question Mark in this case) instantly provided diagnostic data. However, it was evident that the quality of diagnosis and remediation depended upon the quality of the test items, and the learning as well as the teaching strategies. Factors that could militate against the use and full utilisation of CAA in the schools where the study was done included the cost of software for CAA, teachers' capacity to set diagnostic test items particularly in a multiple-choice format, teachers' ability to interpret data produced by CAA, and teachers' skills in remedying their classroom problems as well as learners' problems. This study found that by playing Zadarh learners were able to construct knowledge through discovery and were attracted to the enjoyable aspects of this educational tool. Learners remembered most of those moments in the game during which they were both stuck and trying to solve problems on their way through Zadarh. Therefore, Zadarh can provide useful learning experiences with fun, and can improve motivation towards learning. Debilitating factors against the use of Zadarh and CAA include school curricula, which do not accommodate innovations, inflexible timetables, and classroom approaches that are teacher-centred. It was clear that the success of using computers in education would depend upon the ability of teachers to evaluate the ECPs, and to integrate ECPs into school curricula. drive these interactions played an important role in the successful integration of ECPs into classroom. One way of achieving such success is to include teachers as evaluators and co-designers of ECPs. Evaluations of ECPs therefore should: i) allow the teachers and learners, through social dialog, to identify how software could solve problems; ii) establish the compatibility of the software with the school curriculum; iii) ascertain the capacity of school computers to execute the software; and iv) provide support to the teachers in the use the software. Evaluations should benefit teachers and learners. The study concluded that a post-modern, developmental, and constructivist evaluation process might be one of the ways of enhancing training teachers in the use of the ECPs, in the concepts that the software deal with, and in evaluation. In that way, a socially contracted evaluation is comprehensive and can serve as a change agent through which teachers reflect and act upon improving their classroom practices. / Thesis (Ph.D.)-University of KwaZulu-Natal, Durban, 2004.

This implementation study set out to establish the impact of a constructivist-informed, / Thesis (M.Ed.) - University of KwaZulu-Natal, 2005.