Current multimodal interfaces for complex systems, such as those designed using the Ecological Interface Design (EID) methodology, have largely focused on effective design of interfaces that treat each sensory modality as either an independent channel of information or as a way to provide redundant information. However, there are many times when operationally related information is presented in different sensory modalities. There is very little research that has examined how this information in different modalities can be linked at a perceptual level. When related information is presented through multiple sensory modalities, interface designers will require perceptual methods for linking relevant information together across modalities. This thesis examines one possible crossmodal perceptual relationship, temporal synchrony, and evaluates whether the relationship is useful in the design of multimodal interfaces for complex systems. Two possible metrics for the evaluation of crossmodal perceptual relationships were proposed: resistance to changes in workload, and stream monitoring awareness. Two experiments were used to evaluate these metrics. The results of the first experiment showed that temporal rate synchrony was not resistant to changes in workload, manipulated through a secondary visual task. The results of the second experiment showed that participants who used crossmodal temporal rate synchrony to link information in a multimodal interface did not achieve better performance in the monitoring of the two streams of information being presented over equivalent unimodal interfaces. Taken together, these findings suggest that temporal rate synchrony may not be an effective method for linking information across modalities. Crossmodal perceptual relationships may be very different from intra-modal perceptual relationships. However, methods for linking information across sensory modalities are still an important goal for interface designers, and a key feature of future multimodal interface design for complex systems.

The purpose of this thesis is to explore the theoretical and practical issues associated with the design, execution and analysis of an empirical evaluation of novel process control displays in a collaborative control room environment. An important feature of the thesis is how practical constraints associated with limited access to industry controllers were handled and how convergent lines of evidence were used to achieve the evaluation. As a novel research domain, hydropower systems (HPS) present many design challenges, because controllers must coordinate multiple domains across different time frames. If controllers are to maintain effective situation awareness and if they are to exercise effective control, the displays must integrate controllers’ problem solving across water, generation, market, and transmission concerns and across time frames. However, these needs are ignored in the current displays in a hydropower control room. The two new ecologically-inspired displays, called “Functional Displays”, evaluated in this thesis, are intended to overcome the above shortcomings. The evaluation was done with industry hydropower controllers and coordinators on a medium fidelity simulator which represented the working environment of a large hydropower control room. The core issue of this thesis is how the evaluation was conceptualized, operationalized and analysed given practical constraints arising from limited access to industry controllers, limited capacity of the simulator, and the complexity of the hydropower domain. Of key concern was the design of test scenarios, selection of measures of performance, and data analysis in the face of these constraints. The rationale of the scenario design was to combine representative contingencies within and outside the hydropower scheme, emergent storage problems and different market contexts so that controllers or coordinators would be required to act in different domains. Only by placing controllers in such challenging and time pressured situations could we maximize our chances of seeing the expected benefits of the new display, given the limited length of the experiment. Multiple measures were proposed to capture the quality of in-the-loop coupling between human controllers and the system under various disturbances. The measures of performance include: (1) control quality, which investigates how the new displays tame “temporal complexity” by helping the control team construct a more effective pattern of activities to handle contingencies; and (2) control strategy, which exposes how the new display support the coordination between storage, generation and market to meet the strategic or tangible objectives. The measures of cognition and affect include: (1) situation awareness (SA), which represents controllers’ or coordinators’ degree of cognitive coupling with different time frames and different levels of the hydropower scheme and (2) controllers’ or coordinators’ trust in the supporting displays, and their self-confidence in their own ability to control the hydropower scheme. Through the effort of triangulating the different measures, balancing the elements in designing scenarios, and extracting and integrating raw data from the study, convergent lines of evidence were achieved for evaluation. It was found that the new Functional Displays led the participant teams to a better pattern of work which was reflected in their situation awareness, their discretionary control activity, and the outcome of their control with respect to system productivity and stability. However, limitations of these findings because of the constraints of the experiment are discussed. This research contributes to many theoretical and practical issues in Human-System Interface (HSI). For example, it outlines how some of the principles of ecological interface design (EID) can be used and it highlights the value of using time as a basis for distinguishing interfaces. Moreover, this research provides fruitful insights into the selection and development of measures to assess human control in real world complex work environments. Because a key issue of this thesis is how practical constraints were handled; pragmatic approaches for measuring SA and control were developed. Because industry often performs evaluations under similar intensive constraints, the approaches and solutions developed in this thesis for evaluating novel interfaces could be easily adapted to various industrial settings.

L’évolution des technologies et la mutation du rôle attribué aux opérateurs de l’industrie suscitent un essor de la littérature concernant l’assistance aux procédés de fabrication. Cette thèse de psychologie s’inscrit dans cette littérature. Son objectif est de concevoir un assistant de contrôle aux procédés de fabrication, le compagnon virtuel, qui permette de maintenir à la fois les performances et les compétences des opérateurs. Le système étudié, une usine aéronautique, nécessite le recours à la simulation de processus physiques dynamiques. Deux champs de la littérature sont mobilisés : les cadres théoriques autour de la conception d’interfaces pour les environnements dynamiques et ceux sur le développement des compétences. Une méthodologie en trois parties est utilisée : (1) analyse de l’activité des opérateurs dans l’atelier réel, (2) validation d’un micromonde et (3) test d’un compagnon virtuel dans le micromonde. Les résultats montrent que les opérateurs de l’atelier réel mobilisent un schème trompeur lors de leur activité. Ce schème trompeur est également présent chez des participants tout-venants et il a un effet sur les performances des participants entrainés dans le micromonde. Enfin, le compagnon virtuel, conçu selon les principes de l’EID, ne permet pas de faire disparaitre le schème trompeur. Cependant, il permet aux participants qui n’utilisent pas le schème trompeur d’obtenir de bonnes performances. Les résultats obtenus étayent l’idée qu’il est pertinent de vérifier si des opérateurs ne mobilisent pas des schèmes trompeurs qui pourraient être un frein à l’utilisation des interfaces proposées pour les assister. / The evolution of technology and the shift in the role devoted to operators in industry, from simple task execution to complex system control, provoke a development of the literature concerning assistance to fabrication process. This thesis of psychology joins this literature. It aims at designing an assistant to fabrication process, called the virtual companion, which would both support performances and maintain or improve competences. In particular, the virtual companion would be addressed to operators in an aeronautical factory. The case study requires in line simulation of dynamic physical processes. Two fields of the literature were used: frameworks concerning interface design for dynamic situations and those concerning competences. A three-parts methodology was used: (1) activity analysis of the operators in the real factory, (2) conception and validation of a microworld, and (3) design and test of a virtual companion in the microworld. Results show that the operators of the factory use a misleading scheme during their activity. This misleading scheme is also used by random participants. Moreover, this misleading scheme has an effect on the performances of random participants trained in the microworld. Finally, the virtual companion, whose interface is design based on EID principles, does not eliminate the misleading scheme. However, the participants who did not use the misleading scheme performed better after using the virtual companion. This results leads to the conclusion that studying the operators’ use of misleading schemes seems accurate as such schemes may impair the use of interfaces designed to support their activity

No description available.

No description available.

No description available.

No description available.

No description available.

No description available.

No description available.