This thesis explores the use of multimodality to communicate retrieved results of Internet search engines. The investigation aimed to investigate suitable multimodal metaphors which would increase the level of usability of Internet search engine interfaces and enhance users` experience in the search activity. The study consisted of three experiments based on questionnaires and Internet search activities with a set of text-based and multimodal interfaces. These interfaces were implemented in two browsing platforms, named AVBRO and AVBRO II. In the first experiment, the efficiency of specific multimodal metaphors to communicate additional information of retrieved results was investigated. The experiment also sought to obtain users` views of these metaphors with a questionnaire. An experimental multimodal interface of the AVBRO platform, which communicated additional information with a combination of three 2D graphs and musical stimuli, was used as a basis for the experiment, together with the Google search engine. The results obtained led to the planning of a second experiment. The aim of this experiment was to obtain and compare the level of usability of four different experimental multimodal interfaces and one traditional text-based interface, all implemented in the AVBRO II platform. Effectiveness, efficiency and users` satisfaction were used as criteria to evaluate the usability of these interfaces. In the third and final experiment the usability analysis of a traditional text-based interface and the two most suitable experimental multimodal interfaces of the AVBRO II platform was further investigated. Learnability, errors rate, efficiency, memorability and users` satisfaction were used as criteria to evaluate the usability of these interfaces. The analysis of the results obtained from these experiments provided the basis for a set of design guidelines for the development of usable interfaces based on a multimodal approach.

This thesis investigated the use of novel combinations of multimodal metaphors in the presentation of learning information to evaluate the effect of these combinations on the usability of e-learning interfaces and on the users' learning performance. The empirical research described in this thesis comprised three experimental phases. In the first phase, an initial experiment was carried out with 30 users to explore and compare the usability and learning performance of facially animated expressive avatars with earcons and speech, and text with graphics metaphors. The second experimental phase involved an experiment conducted with 48 users to investigate their perception of avatar's facial expressions and body gestures when presented in both the absence and presence of interactive e-learning context. In addition, the experiment aimed at evaluating the role that an avatar could play as virtual lecturer in e-learning interfaces by comparing the usability and learning performance of three different modes of interaction: speaking facially expressive virtual lecturer, speaking facially expressive full-body animated virtual lecturer, and two speaking facially expressive virtual lecturers. In the third phase, a total of 24 users experimentally examined a novel approach for the use of earcons and auditory icons in e-learning interfaces to support an animated facially expressive avatar with body gestures during the presentation of the learning material. The obtained results demonstrated the usefulness of the tested metaphors to enhance e-learning usability and to enable users to attain better learning performance. These results provided a set of empirically derived innovative guidelines for the design and use of these metaphors to generate more usable e-learning interfaces. For example, when designing avatars as animated virtual lecturers in e-learning interfaces, specific facial expression and body gestures should be incorporated due to its positive influence in enhancing learners' attitude towards the learning process.

This thesis investigated the use of novel combinations of multimodal metaphors in the presentation of learning information to evaluate the effect of these combinations on the usability of e-learning interfaces and on the users¿ learning performance. The empirical research described in this thesis comprised three experimental phases. In the first phase, an initial experiment was carried out with 30 users to explore and compare the usability and learning performance of facially animated expressive avatars with earcons and speech, and text with graphics metaphors. The second experimental phase involved an experiment conducted with 48 users to investigate their perception of avatar¿s facial expressions and body gestures when presented in both the absence and presence of interactive e-learning context. In addition, the experiment aimed at evaluating the role that an avatar could play as virtual lecturer in e-learning interfaces by comparing the usability and learning performance of three different modes of interaction: speaking facially expressive virtual lecturer, speaking facially expressive full-body animated virtual lecturer, and two speaking facially expressive virtual lecturers. In the third phase, a total of 24 users experimentally examined a novel approach for the use of earcons and auditory icons in e-learning interfaces to support an animated facially expressive avatar with body gestures during the presentation of the learning material. The obtained results demonstrated the usefulness of the tested metaphors to enhance e-learning usability and to enable users to attain better learning performance. These results provided a set of empirically derived innovative guidelines for the design and use of these metaphors to generate more usable e-learning interfaces. For example, when designing avatars as animated virtual lecturers in e-learning interfaces, specific facial expression and body gestures should be incorporated due to its positive influence in enhancing learners¿ attitude towards the learning process.

This thesis explores the use of multimodality to communicate retrieved results of Internet search engines. The investigation aimed to investigate suitable multimodal metaphors which would increase the level of usability of Internet search engine interfaces and enhance users` experience in the search activity. The study consisted of three experiments based on questionnaires and Internet search activities with a set of text-based and multimodal interfaces. These interfaces were implemented in two browsing platforms, named AVBRO and AVBRO II. In the first experiment, the efficiency of specific multimodal metaphors to communicate additional information of retrieved results was investigated. The experiment also sought to obtain users` views of these metaphors with a questionnaire. An experimental multimodal interface of the AVBRO platform, which communicated additional information with a combination of three 2D graphs and musical stimuli, was used as a basis for the experiment, together with the Google search engine. The results obtained led to the planning of a second experiment. The aim of this experiment was to obtain and compare the level of usability of four different experimental multimodal interfaces and one traditional text-based interface, all implemented in the AVBRO II platform. Effectiveness, efficiency and users` satisfaction were used as criteria to evaluate the usability of these interfaces. In the third and final experiment the usability analysis of a traditional text-based interface and the two most suitable experimental multimodal interfaces of the AVBRO II platform was further investigated. Learnability, errors rate, efficiency, memorability and users` satisfaction were used as criteria to evaluate the usability of these interfaces. The analysis of the results obtained from these experiments provided the basis for a set of design guidelines for the development of usable interfaces based on a multimodal approach.