This simulator-based study examined the use of conventional auditory warnings (tonal, non-verbal sounds) and auditory icons> (representational non-verbal sounds), alone and in combination with a dash-mounted visual display, to present information about impending collision situations to commercial motor vehicle operators. Brake response times were measured for impending front-to-rear collision scenarios under six display configurations, two vehicle speeds, and two levels of headway. Accident occurrence was measured for impending side collision scenarios under two vehicle speeds, two levels of visual workload, two auditory displays, absence/presence of mirrors, and absence/presence of dash-mounted iconic display. Subjective preference data was also obtained from participants. For both front-to-rear and side collision scenarios, auditory icons elicited significantly improved driver performance over conventional auditory warnings. Driver performance improved when collision warning information was presented through multiple modalities. Brake response times were significantly faster for impending front-to-rear collision scenarios using the longer headway criterion. The presence of mirrors significantly reduced the number of accidents for impending side collision scenarios. Subjective preference data indicated that participants preferred multi-modal displays over single-modality displays. A technique for systematically identifying, selecting, and evaluating candidate auditory icons was also developed. The potential exists to expand upon these developments, toward the goals of identifying appropriate auditory icons, improving operator performance, and developing information display techniques to effectively managing workload across multiple modalities. / Master of Science

This thesis investigates the effect of incorporating various information visualisation techniques and non-speech sounds (i.e. auditory icons and earcons) in email browsing. This empirical work consisted of three experimental phases. The first experimental phase aimed at finding out the most usable visualisation techniques for presenting email information. This experiment involved the development of two experimental email visualisation approaches which were called LinearVis and MatrixVis. These approaches visualised email messages based on a dateline together with various types of email information such as the time and the senders. The findings of this experiment were used as a basis for the development of a further email visualisation approach which was called LinearVis II. This novel approach presented email data based on multi-coordinated views. The usability of messages retrieval in this approach was investigated and compared to a typical email client in the second experimental phase. Users were required to retrieve email messages in the two experiments with the provided relevant information such as the subject, status and priority. The third experimental phase aimed at exploring the usability of retrieving email messages by using other type of email data, particularly email threads. This experiment investigated the synergic use of graphical representations with non-speech sounds (Multimodal Metaphors), graphical representations and textual display to present email threads and to communicate contextual information about email threads. The findings of this empirical study demonstrated that there is a high potential for using information visualisation techniques and non-speech sounds (i.e. auditory icons and earcons) to improve the usability of email message retrieval. Furthermore, the thesis concludes with a set of empirically derived guidelines for the use of information visualisation techniques and non-speech sound to improve email browsing.

The purpose of the survey is to answer the following question: How is the learning process in digital games affected by an informative sound design? The survey analyzes the adaptation of sound design in digital games for optimized learning regarding the digital environment's responses to the actor's interaction. The work investigates actors' decisions in a digital environment and the changes in decision after a change in sound design. The study resulted in a reinforced perception when auditory icons occurred in a non-representative environment, these interactions were recorded and recalled to a greater extent in relation to remaining auditory methods. Analysis of Kolb's learning patterns in a digital environment enhanced by informative sound design was documented in the following steps: observation of interaction, reflection on interaction, implementation of hypothesis on a similar interaction, reflection on differences and similarities of varying results of interaction. / Undersökningens syfte är att besvara följande frågeställning: Hur påverkas inlärningsprocessen i digitala spel med hjälp av en informativ ljuddesign?Undersökningen analyserar anpassning av ljuddesign i digitala spel för en optimerad inlärning gällande den digitala miljöns reaktioner till aktörers interaktion. Arbetet undersöker aktörers beslut i en digital miljö och förändringarna i beslut efter en förändring i ljuddesign.Undersökningen resulterade i en förstärkt uppfattningsförmåga då Auditory Icons uppstod i en icke-representativ miljö, dessa interaktioner registrerades och återkallas i en större omfattning i förhållande till resterande auditiva metoder.Analys av Kolb’s inlärningsmönster i en digital miljö stärkt av informativ ljuddesign dokumenterades i följande steg: observation av interaktion, reflektion över interaktion, implementation av hypotes på en liknande interaktion, reflektion över skillnader och likheter av varierande resultat av interaktion.

Informative sound design serves as an indicator to the player that something just happened, it can be urgently needing the players immediate attention, or serve as a feedback for a job well done. This paper studies how players interpret different informative sounds, how confident they are about their interpretation, and why they interpret the sound that way.

This thesis investigates the effect of incorporating various information visualisation techniques and non-speech sounds (i.e. auditory icons and earcons) in email browsing. This empirical work consisted of three experimental phases. The first experimental phase aimed at finding out the most usable visualisation techniques for presenting email information. This experiment involved the development of two experimental email visualisation approaches which were called LinearVis and MatrixVis. These approaches visualised email messages based on a dateline together with various types of email information such as the time and the senders. The findings of this experiment were used as a basis for the development of a further email visualisation approach which was called LinearVis II. This novel approach presented email data based on multi-coordinated views. The usability of messages retrieval in this approach was investigated and compared to a typical email client in the second experimental phase. Users were required to retrieve email messages in the two experiments with the provided relevant information such as the subject, status and priority. The third experimental phase aimed at exploring the usability of retrieving email messages by using other type of email data, particularly email threads. This experiment investigated the synergic use of graphical representations with non-speech sounds (Multimodal Metaphors), graphical representations and textual display to present email threads and to communicate contextual information about email threads. The findings of this empirical study demonstrated that there is a high potential for using information visualisation techniques and non-speech sounds (i.e. auditory icons and earcons) to improve the usability of email message retrieval. Furthermore, the thesis concludes with a set of empirically derived guidelines for the use of information visualisation techniques and non-speech sound to improve email browsing. / Taibah University in Medina and the Ministry of Higher Education in Saudi Arabia.

Internet är för många en självklarhet för att kunna dela och ta del av information. För andra kan användandet av internet vara frustrerande och exkluderande. En sådan grupp är personer med olika former av synnedsättningar. För dessa personer finns vissa verktyg tillgängliga så som screen readers. Dessa underlättar för personer som är helt eller delvis blinda men kan upplevas som långsamma att använda för personer som till viss del klarar av att utan stöd av en screen reader. För att underlätta för dessa personer kan ljud användas i olika delar av användargränssnittet. En studie genomfördes för att undersöka hur en modifierad navigeringsmeny med ljud påverkar tiden det tar för användare att lösa navigeringsrelaterade uppgifter i en webbapplikation. Webbapplikationen använde sig av en simulering av synnedsättningen glaukom (grön starr). Resultaten visade inte på några statistiskt signifikanta skillnader varken gällande tiden det tog att lösa uppgifterna eller hur svår navigeringsmenyn upplevdes att använda. Utifrån detta dras slutsatsen att ljud inte påverkar hur lätt eller svårt det är för användare att lösa navigeringsrelaterade uppgifter.

In the driving task, vision, hearing, and the haptic senses are all used by the driver to gather required information. Future Intelligent Transportation Systems components are likely to further increase the volume of information available to or required by the driver, particularly in the case of commercial vehicle operators. The use of alternate modalities to present in-vehicle information is a possible solution to the potential overload of the visual channel. Auditory icons have been shown to improve operator performance and decrease learning and response times, not only in industrial applications, but also as emergency braking warnings. The use of auditory icons in commercial truck cabs has the potential to increase the number of auditory displays that can be distinguished and understood by commercial vehicle operators, and this experiment sought to determine the utility of auditory icons in that situation. Nine auditory icons were evaluated by commercial vehicle operators as they drove an experimental vehicle over public roads. A comparison of the data collected in the truck-cab environment to data collected in a laboratory study on the same auditory icons revealed some differences in the perceived meaning, perceived urgency, and association with the auditory icons' intended meanings between the two conditions. The presence of these differences indicates that driver evaluations of auditory icons can be affected by the environment, and testing should therefore be conducted in a situation that approximates the end-user environment as closely as possible. A comparison of the drivers' brake response times across the three warning conditions (no warning, auditory icon, and soft braking) was also conducted on a closed, secure handling course. Dependent measures included overall brake reaction time and its components, steering response time, time to initial driver action, and categorical measures of driver responses (steering, swerving, braking, and stopping). The results indicated numerically shorter mean response times (on the order of 0.5 seconds for Total Brake Response Time) for the two conditions with warnings, but the differences were not statistically significant. The most likely reason for this lack of significance is the extreme between-subject variability in response times in the no warning condition. An analysis of the response time variance across the three conditions did indicate significantly less variability in operator responses in the two warning conditions. Two of the five dependent measures (Brake Pedal Contact Time and Total Brake Response Time) exhibited significantly reduced variance in the auditory icon warning condition compared to the no warning condition. The soft braking warning condition exhibited significantly reduced variance for four of the dependent measures (Accelerator Reaction Time, Brake Pedal Contact Time, Total Brake Response Time, and First Reaction Time). These results indicate that a soft braking stimulus like that used in this study could potentially prove to be a more effective emergency braking warning than simple auditory warnings alone. / Master of Science

There has been an increasing demand for commercial organisations to foster real-time interaction with customers, because harnessing customer competencies has been shown to be a major contributor towards various benefits, such as growth, innovation and competition. This may drive organisations to embrace the development of multimodal interaction and complement Electronic Customer Knowledge Management Systems (E-CKMS) with metaphors of audio-visual nature. Although the implementation of E-CKMS encounters several challenges, such as lack of trust and information overload, few empirical studies were devoted to assess the role of audio-visual metaphors, and investigate whether these technologies can be put into practice. Therefore, this thesis describes a comparative evaluation study carried out to examine the implication of incorporating multimodal metaphors into E-CKMS interfaces on not only usability of E-CKMS, but also the user's trust, knowledge and acceptance. An experimental E-CKMS platform was implemented with three different modes of interaction: Visual-only E-CKMS (VCKMS) with text and graphics, Multimodal E-CKMS (MCKMS) with speech, earcons and auditory icons and Avatar-enhanced multimodal E-CKMS (ACKMS). The three platforms were evaluated by three independent groups of twenty participants each (total=60) who carried out eight common tasks of increasing complexity and design based on three different styles. Another dependent group of forty-eight participants (n=48) was instructed to interact with the systems under similar usability conditions by performing six common tasks of two styles, and fill a questionnaire devised to measure the aspects of user acceptance. The results therein revealed that ACKMS was more usable and acceptable than both MCKMS and VCKMS, whereas MCKMS was more usable than VCKMS, but less acceptable. Inferential Statistics indicated that these results were statistically significant.

There has been an increasing demand for commercial organisations to foster real-time interaction with customers, because harnessing customer competencies has been shown to be a major contributor towards various benefits, such as growth, innovation and competition. This may drive organisations to embrace the development of multimodal interaction and complement Electronic Customer Knowledge Management Systems (E-CKMS) with metaphors of audio-visual nature. Although the implementation of E-CKMS encounters several challenges, such as lack of trust and information overload, few empirical studies were devoted to assess the role of audio-visual metaphors, and investigate whether these technologies can be put into practice. Therefore, this thesis describes a comparative evaluation study carried out to examine the implication of incorporating multimodal metaphors into E-CKMS interfaces on not only usability of E-CKMS, but also the user¿s trust, knowledge and acceptance. An experimental E-CKMS platform was implemented with three different modes of interaction: Visual-only E-CKMS (VCKMS) with text and graphics, Multimodal E-CKMS (MCKMS) with speech, earcons and auditory icons and Avatar-enhanced multimodal E-CKMS (ACKMS). The three platforms were evaluated by three independent groups of twenty participants each (total=60) who carried out eight common tasks of increasing complexity and design based on three different styles. Another dependent group of forty-eight participants (n=48) was instructed to interact with the systems under similar usability conditions by performing six common tasks of two styles, and fill a questionnaire devised to measure the aspects of user acceptance. The results therein revealed that ACKMS was more usable and acceptable than both MCKMS and VCKMS, whereas MCKMS was more usable than VCKMS, but less acceptable. Inferential Statistics indicated that these results were statistically significant.

No / This paper introduces a comparative multi-group study carried out to investigate the use of multimodal interaction metaphors (visual, oral, and aural) for improving learnability (or usability from first time use) of interface-design environments. An initial survey was used for taking views about the effectiveness and satisfaction of employing speech and speech-recognition for solving some of the common usability problems. Then, the investigation was done empirically by testing the usability parameters: efficiency, effectiveness, and satisfaction of three design-toolkits (TVOID, OFVOID, and MMID) built especially for the study. TVOID and OFVOID interacted with the user visually only using typical and time-saving interaction metaphors. The third environment MMID added another modality through vocal and aural interaction. The results showed that the use of vocal commands and the mouse concurrently for completing tasks from first time use was more efficient and more effective than the use of visual-only interaction metaphors.