Miscommunication between controllers and pilots, potentially resulting from a high pilot cognitive load, has been a causal or contributing factor in a large number of aviation accidents. In this context, failure to communicate can be attributed, among other factors, to an inadequate human-system interface design, the related high cognitive load imposed on the pilot, and poor performance reflected by a higher error rate. To date, voice radio remains in service without any means for managing pilot cognitive load by design (as opposed to training or procedures). Such an oversight is what prompted this dissertation. The goals of this study were (a) to investigate the utility of a voice-to-text transcription (V-T-T) of ATC clearances in managing pilot's cognitive load during controller-pilot communications within the context of a modern flight deck environment, and (b) to validate whether a model of variable relationships which is generated in the domain of learning and instruction would "transfer", and to what extend, to an operational domain. First, within the theoretical framework built for this dissertation, all the pertaining factors were analyzed. Second, by using the process of synthesis, and based on guidelines generated from that theoretical framework, a redundant verbal display of ATC clearances (i.e., a V-T-T) was constructed. Third, the synthesized device was empirically examined. Thirty four pilots participated in the study – seventeen pilots with 100-250 total flight hours and seventeen with > 500 total flight hours. All participants had flown within sixty days prior to attending the study. The experiment was conducted one pilot at a time in 2.5-hour blocks. A 2 Verbal Display Redundancy (no-redundancy and redundancy) X 2 Verbal Input Complexity (low and high) X 2 Level of Expertise (novices and experts) mixed-model design was used for the study with 5 IFR clearances in each Redundancy X Complexity condition. The results showed that the amounts of iii reduction of cognitive load and improvement of performance, when verbal display redundancy was provided, were in the range of about 20%. These results indicated that V-T-T is a device which has a tremendous potential to serve as (a) a pilot memory aid, (b) a way to verify a clearance has been captured correctly without having to make a "Say again" call, and (c) to ultimately improve the margin of safety by reducing the propensity for human error for the majority of pilot populations including those with English as a second language. Fourth, the results from the validation of theoretical models "transfer" showed that although cognitive load remained as a significant predictor of performance, both complexity and redundancy also had unique significant effects on performance. Furthermore, these results indicated that the relationship between these variables was not as "clear-cut" in the operational domain investigated here as the models from the domain of learning and instruction suggested. Until further research is conducted, (a) to investigate how changes in the operational task settings via adding additional coding (e.g., permanent record of clearances which can serve as both a memory aid and a way to verify a clearance is captured correctly) affect performance through mechanisms other than cognitive load; and (b) unless the theoretical models are modified to reflect how changes in the input variables impact the outcome in a variety of ways; a degree of prudence should be exercised when the results from the model "transfer" validation are applied to operational environments similar to the one investigated in this dissertation research.

The aim of this thesis is to develop a gamified crowdsourcing platform for transcription and metadata tagging of digitised text documents, and further to examine whether the platform can stimulate engagement in archives and be used as an educational resource. The case, around which the thesis project is conducted, is the governmental Archives of Bosnia and Herzegovina and its historical documents which were damaged in a fire in 2014.  Theories about engagement, archival pedagogy, and gamification guided the development of the platform. Bosnian-Herzegovinian school children, History teachers, and staff of the Archives of Bosnia and Herzegovina tested the platform and shared their thoughts of it in focus group discussions and interviews. The developed platform consists of a game interface, a database, and an interface for results viewing and searching. It is web-based, and coded in HTML, CSS, JavaScript, and PHP. The platform stimulated engagement among the school children who tested it. They expressed interest in reading and learning more about archival documents, in long-term preservation of information, and in the archive’s mission and future. Judging from the discussions and interviews, the platform could well serve as an educational resource. The History teachers showed interest in using the platform in their teaching, and found support for it in the curriculum for the History subject. A gamified crowdsourcing platform has been developed, and the perception among school children, History teachers, and archive staff was overall very positive. Crowdsourcing is not just about sourcing out tasks to the crowd. Co-creation of content could furthermore stimulate learning and engagement, and thereby tie a bound between the public and the archive. Gamification can successfully make the overarching goal of crowdsourcing understandable and concrete for the platform users. / Uppsatsen syftar till att utveckla en spelifierad crowdsourcing-plattform för transkription och metadatataggning av digitaliserade textdokument samt att undersöka huruvida plattformen kan väcka engagemang för arkiv och användas som en utbildningsresurs. Det fall runt vilket uppsatsprojektet kretsar är Bosnien-Hercegovinas centrala arkivmyndighet och dess historiska dokument som skadades i en brand 2014. Teorier om engagemang, arkivpedagogik och gamification vägledde utvecklingen av plattformen. Skolbarn och historielärare i Bosnien-Hercegovina samt personal på arkivmyndigheten provade plattformen och delade med sig av sina tankar om den i fokusgruppsdiskussioner och intervjuer. Den utvecklade plattformen består av ett spelgränssnitt, en databas och ett gränssnitt för resultatvisning och sökning. Den är webbaserad och skriven i HTML, CSS, JavaScript och PHP. Plattformen väckte engagemang hos de barn som provade den. De uttryckte intresse av att läsa och lära sig mer om arkivdokument, av långsiktigt bevarande samt av arkivets uppdrag och framtid. Plattformen skulle, av fokusgruppsdiskussionerna och intervjuerna att döma, kunna användas och fungera väl som utbildningsresurs. Historielärarna visade intresse av att använda plattformen i sin undervisning och fann stöd för det i historieämnets läroplan. En spelifierad crowdsourcing-plattform har utvecklats och mottagandet av den bland skolbarn, historielärare och arkivpersonal var överlag mycket positivt. Crowdsourcing handlar inte enbart om att outsourca arbetsuppgifter till massan. Medskapande kan dessutom stimulera lärande och engagemang och därigenom knyta band mellan allmänheten och arkivet. Spelifiering kan med framgång göra crowdsourcingens övergripande mål förståeligt och konkret för dem som använder plattformen.

This thesis presents scientific contributions to the field of multimodal interac- tive structured prediction (MISP). The aim of MISP is to reduce the human effort required to supervise an automatic output, in an efficient and ergonomic way. Hence, this thesis focuses on the two aspects of MISP systems. The first aspect, which refers to the interactive part of MISP, is the study of strate- gies for efficient human¿computer collaboration to produce error-free outputs. Multimodality, the second aspect, deals with other more ergonomic modalities of communication with the computer rather than keyboard and mouse. To begin with, in sequential interaction the user is assumed to supervise the output from left-to-right so that errors are corrected in sequential order. We study the problem under the decision theory framework and define an optimum decoding algorithm. The optimum algorithm is compared to the usually ap- plied, standard approach. Experimental results on several tasks suggests that the optimum algorithm is slightly better than the standard algorithm. In contrast to sequential interaction, in active interaction it is the system that decides what should be given to the user for supervision. On the one hand, user supervision can be reduced if the user is required to supervise only the outputs that the system expects to be erroneous. In this respect, we define a strategy that retrieves first the outputs with highest expected error first. Moreover, we prove that this strategy is optimum under certain conditions, which is validated by experimental results. On the other hand, if the goal is to reduce the number of corrections, active interaction works by selecting elements, one by one, e.g., words of a given output to be supervised by the user. For this case, several strategies are compared. Unlike the previous case, the strategy that performs better is to choose the element with highest confidence, which coincides with the findings of the optimum algorithm for sequential interaction. However, this also suggests that minimizing effort and supervision are contradictory goals. With respect to the multimodality aspect, this thesis delves into techniques to make multimodal systems more robust. To achieve that, multimodal systems are improved by providing contextual information of the application at hand. First, we study how to integrate e-pen interaction in a machine translation task. We contribute to the state-of-the-art by leveraging the information from the source sentence. Several strategies are compared basically grouped into two approaches: inspired by word-based translation models and n-grams generated from a phrase-based system. The experiments show that the former outper- forms the latter for this task. Furthermore, the results present remarkable improvements against not using contextual information. Second, similar ex- periments are conducted on a speech-enabled interface for interactive machine translation. The improvements over the baseline are also noticeable. How- ever, in this case, phrase-based models perform much better than word-based models. We attribute that to the fact that acoustic models are poorer estima- tions than morphologic models and, thus, they benefit more from the language model. Finally, similar techniques are proposed for dictation of handwritten documents. The results show that speech and handwritten recognition can be combined in an effective way. Finally, an evaluation with real users is carried out to compare an interactive machine translation prototype with a post-editing prototype. The results of the study reveal that users are very sensitive to the usability aspects of the user interface. Therefore, usability is a crucial aspect to consider in an human evaluation that can hinder the real benefits of the technology being evaluated. Hopefully, once usability problems are fixed, the evaluation indicates that users are more favorable to work with the interactive machine translation system than to the post-editing system. / Alabau Gonzalvo, V. (2014). Multimodal interactive structured prediction [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/35135 / TESIS / Premios Extraordinarios de tesis doctorales