Most events that we encounter in everyday life provide our different senses with correlated information, and audiovisual speech perception is a familiar instance of multisensory integration. Several approaches will be used to further examine the role of cognitive factors on audiovisual speech perception. The main focuses of this thesis will be to examine the influences of cognitive load and selective attention on audiovisual speech perception, as well as the integration of auditory and visual information in talking distractor faces. The influence of cognitive factors on the temporal integration of auditory and visual speech, and gaze behaviour during audiovisual speech will also be addressed. The overall results of the experiments presented here suggest that the integration of auditory and visual speech information is quite robust to various attempts to modulate the integration. Adding a cognitive load task shows minimal disruption of the integration of auditory and visual speech information. Changing attentional instructions to get subjects to selectively attend to either the auditory or visual speech information also has a rather modest influence on the observed integration of auditory and visual speech information. Generally, the integration of temporally offset auditory and visual information seems rather insensitive to cognitive load or selective attentional manipulations. The processing of visual information from distractor faces seems to be limited. The language of the visually articulating distractors doesn't appear to provide information that is helpful for matching together the auditory and visual speech streams. Audiovisual speech distractors are not really any more distracting than auditory distractor speech paired with a still image, suggesting a limited processing or integration of the visual and auditory distractor information. The gaze behaviour during audiovisual speech perception appears to be relatively unaffected by an increase in cognitive load, but is somewhat influenced by attentional instructions to selectively attend to the auditory and visual information. Additionally, both the congruency of the consonant, and the temporal offset of the auditory and visual stimuli have small but rather robust influences on gaze. / Thesis (Ph.D, Psychology) -- Queen's University, 2011-09-30 23:31:07.754

In social interactions it is important for perceivers to be able to differentiate between facial expressions of emotion associated with a congruent emotional experience (genuine expressions) and those that are not (posed expressions). This research investigated the sensitivity of participants with a range of depressive symptom severity and varying levels of rumination to the differences between genuine and posed facial expressions The suggested mechanisms underlying impairments in emotion recognition were also investigated; the effect of cognitive load (as a distraction from deliberate processing of stimuli) and attention, and the relationships between mechanisms and sensitivity across a range of depressive symptoms and level of rumination. Participants completed an emotion categorisation task in which they were asked if targets were showing either happiness or sadness, and then if targets were feeling those emotions. Participants also completed the same task under cognitive load. In addition, a recognition task was used to measure attention. Results showed that when making judgements about whether targets were feeling sad lower sensitivity was related to higher levels of depressive symptoms, but contrary to predictions, only when under cognitive load. Depressive symptoms and rumination were not related to higher levels of bias towards sad expressions. Recognition did not show a relationship with sensitivity, rumination or depression scores. Cognitive load did not show the expected effects or improving sensitivity but instead showed lower sensitivity scores in some conditions compared to conditions without load. Implications of results are discussed, as well as directions for future research.

The overall aim of this study was to examine how the characteristics of two salient stimuli -music and message- of an audio advertisement influence the psychological state of consumers and how such a state subsequently determines their cognitive and affective responses to the advertisement. In achieving this aim, this study was guided by a combination of two cognitive resource utilisation theories, Limited Capacity Model of Motivated Mediated Message Processing (Lang, 2000) and Resource-Matching Hypothesis (Anand & Sternthal, 1989). In particular building upon inconsistency and load theories, this study proposed that certain stimulus characteristics prompted certain states of a consumer’s cognition. These two stimulus characteristics were the congruence of musical stimulus and the complexity of the message stimulus. The model then predicted the potential effect of these characteristics on certain psychological states (Psychological Discomfort and Cognitive Load) leading to affective (Attitude towards Advertisement) and cognitive (encoding, storage, and retention) responses. To empirically examine this model, an online experiment (using a 2 x 2 between-subject x 2 with-in subject mix design) was conducted, in which a mixed sample of 284 subjects was exposed to a set of audio advertisements especially designed for this study. Unfamiliar music in conjunction with a fictitious brand was used and the exposure level was maintained at low. ANCOVA, MANCOVA, two-stage hierarchical regression analysis, and Repeated-measures MANCOVA were administered to test the hypotheses presented in the conceptual model. Among major findings were that the multiple informational structures in a complex message positively influenced cognitive load, while congruent music was capable of attenuating the level of cognitive load. Incongruent music, on the other hand, was capable of generating a dissonance state experienced as psychological discomfort that in turn increased the level of cognitive load as a result of listener’s trying to resolve such a state. Both dissonance and cognitive load negatively influenced attitude towards advertisements, and the affect primacy of attitude formation appeared to be more applicable. Though high cognitive load clearly undermines encoding, storage, and retrieval processes, no evidence was found to support the Resource-matching Hypothesis. Furthermore, the findings suggested that the cognitive load offset by the congruent music would increase advertisement effectiveness by enabling its message to carry more information and by generating more favourable attitudes.

The term ontology is defined as the explicit specification of a conceptualization. While much of the prior research has focused on technical aspects of ontology management, little attention has been paid to the investigation of issues that limit the widespread use of ontologies and the evaluation of the effectiveness of ontologies in improving task performance. This dissertation addresses this void through the development of approaches to ontology creation, refinement, and evaluation. This study follows a multi-paper model focusing on ontology creation, refinement, and its evaluation. The first study develops and evaluates a method for ontology creation using knowledge available on the Web. The second study develops a methodology for ontology refinement through pruning and empirically evaluates the effectiveness of this method. The third study investigates the impact of an ontology in use case modeling, which is a complex, knowledge intensive organizational task in the context of IS development. The three studies follow the design science research approach, and each builds and evaluates IT artifacts. These studies contribute to knowledge by developing solutions to three important issues in the effective development and use of ontologies.

For the last four decades, psycholinguistic research has dealt with the question to what extent elements of simple sentences like “The monk read the book” are planned ahead both on the abstract-lexical and phonological processing level. While a number of studies have shown that all up to the final element can be activated on these two levels, empirical evidence on the flexibility of the respective planning scopes is inconsistent, and a systematic delineation of the influence of different forms of cognitive load has not yet been provided. This thesis presents a series of 9 picture-word interference experiments in which participants produced subject-verb-object sentences while ignoring auditory distractor words. Advance planning was assessed at an abstract-lexical (lemma) level and at a phonological (word form) level under varying working memory load conditions (no load, or visuospatial load, or verbal load). In the absence of a concurrent working memory load and with a concurrent visuospatial working memory load, subject and object nouns were found to be activated at the abstract-lexical and the phonological level prior to speech onset. By contrast, with a concurrent verbal working load, the scope of advance planning at the phonological level was reduced, while the scope of advance planning at the abstract-lexical level remained unaffected. Moreover, sentence planning had a more disruptive effect on verbal working memory performance than on visuospatial working memory performance. Overall, these results suggest that advance planning at the phonological level is more adaptive to external factors than advance planning at the abstract-lexical level. Also, they indicate an overlap of resources allocated to phonological processing in speech production and verbal working memory.

Master of Applied Science / The aim of this investigation was to investigate the effects of computer-based visual feedback in the teaching of singing. Pitch accuracy, a readily-measured parameter of the singing voice, was used in this study to gauge changes in singing for groups with and without visual feedback. The study investigated whether the style of feedback affects the amount of learning achieved, and whether the provision of concurrent visual feedback hampers the simultaneous performance of the singing task. The investigation used a baseline–intervention–post-test between-groups design. Participants of all skill levels were randomly assigned to a control group or one of two experimental groups – with all participants given one hour of singing training. At intervention, the two experimental groups were offered one of two different displays of real-time visual feedback on their vocal pitch accuracy, while control participants had a non-interactive display. All sessions were recorded, and the vocal exercise patterns performed at baseline, intervention and post-test phases were acoustically analysed for pitch accuracy. Questionnaires assessed both general health and the amount of singing and music training of all participants; people in the two experimental groups were also given a further questionnaire about the visual feedback. The results indicate that visual feedback improves pitch accuracy in singing. Cognitive load related to the decoding of visual information was a factor at intervention. At post-test, the two groups who had used real-time visual feedback demonstrated marked improvement on their initial pitch accuracy. There was no significant difference between the results of participants from the two experimental groups, although the participants with some background in singing training showed greater improvement using a simpler visual feedback design. The findings suggest that a hybrid approach integrating standard singing teaching practices with real-time visual feedback of aspects of the singing voice may improve learning.

abstract: The purpose of this experimental study was to investigate the effects of textual and visual annotations on Spanish listening comprehension and vocabulary acquisition in the context of an online multimedia listening activity. 95 students who were enrolled in different sections of first year Spanish classes at a community college and a large southwestern university were randomly assigned to one of four versions of an online multimedia listening activity that contained textual and visual annotations of several key words. Students then took a comprehension and vocabulary posttest and a survey to measure cognitive load and general attitudes towards the program. Results indicated that textual annotations had a significant positive effect on listening comprehension and that visual annotations had a significant positive effect on how successful students felt. No statistically significant differences were found for other variables. Participants also reported positive attitudes towards vocabulary annotations and expressed a desire to see more annotations during multimedia listening activities of this type. These findings provide further evidence of the impact that multimedia may have on language acquisition. These findings have implications for multimedia design and for future research. Language listening activities should include a variety of vocabulary annotations that may help students to understand what they hear and to help them learn new vocabulary. Further research is needed outside of the laboratory, in the online and increasingly-mobile language learning environment in order to align the research with the environment in which many students currently study. The incorporation of motivation into multimedia learning theory and cognitive load should be explored, as well as new measures of cognitive load. / Dissertation/Thesis / Ph.D. Educational Technology 2010

abstract: The purpose of this study was to investigate the impacts of three types of instructional presentation methods on learning, efficiency, cognitive load, and learner attitude. A total of 67 employees of a large southwestern university working in the field of research administration were randomly assigned to one of three conditions. Each condition presented instructional materials using a different method, namely dynamic integrated, dynamic non-integrated, or non-dynamic non-integrated. Participants completed a short survey, pre-test, cognitive load questions, learner attitude questions, and a post-test during their experience. The results reveal that users of the dynamic integrated condition treatment showed significant improvement in both learning and efficiency. The dynamic non-integrated participants had a faster mean time to complete an assigned task, however, they also had significantly lower average test scores. There were no other significant findings in terms of cognitive load or learner attitude. Limitations, implications and future studies are discussed. / Dissertation/Thesis / Doctoral Dissertation Educational Technology 2016

The use of smartphone applications has increased dramatically in the last decade. With the technology being ubiquitous around us it is important to reduce the required cognitive demand for interacting with the technology. A common use of smartphones is calendar applications. This paper investigates if cognitive load can be lowered by using a calendar application specifically designed with cognitive load in mind compared to the widely used Google Calendar. The hypothesis is that the application will generate a lower cognitive load in comparison. An experiment was conducted where participants were instructed to perform a primary task by following given instructions of what to do in the calendar application while also performing a secondary task. There were no significant differences in performance except for two tasks where the new app performed worse. Participants also reported a higher level of frustration for the new app which could mean that their cognitive load was slightly higher. / Användandet av smartphone applikationer har ökat dramatiskt under det senaste decenniet, eftersom att tekniken hela tiden är närvarande runt om oss så är det viktigt att minska de kognitiva kraven som behövs vid användandet. Ett vanligt användande av smartphones är kalenderapplikationer, den här studien undersöker om den kognitiva belastningen kan minskas vid användandet av en kalender app genom att använda applikation som designats med kognitiv belastning i åtanke för att sen jämföras mot Google Kalender. Hypotesen är att applikationen kommer att generera en lägre kognitiv belastning i jämförelse. Ett experiment utfördes där deltagarna instruerades att utföra en huvuduppgift som innebar att utföra givna instruktioner med uppdrag i kalendarapplikationen, samtidigt som en sekundär uppgift utfördes där deltagarna fick trycka på en knapp när en signal hördes. Det var ingen signifikant skillnad i prestation förutom i två uppgifter där appen presterade sämre, deltagarna rapporterade också en högre nivå av frustration vilket skulle kunna innebära att den kognitiva belastningen var något högre.

Safely operating a vehicle requires the full attention of the driver. Should the driver lose focus as a result of performing other tasks simultaneously, there could be disastrous outcomes. To gain insight into a driver’s mental state, the cognitive load experienced by the driver can be investigated. Measuring cognitive load can be done in numerous ways, one popular approach is the use of Electroencephalography (EEG). A lot of the data that can be extracted from EEG-signals, are redundant or irrelevant when trying to classify cognitive load. This thesis focuses on identifying EEG-features relevant to the classification of cognitive load experienced by drivers, through the use of feature selection algorithms. An experimental approach was utilized where three feature selection algorithms (ReliefF, BSS/WSS and BIRS) were applied to the available datasets. The feature subsets produced by the algorithms achieved higher classification accuracies compared to the use of all features. The best performing subset was generated by the ReliefF algorithm which achieved an accuracy of 66%. However, several other unique subsets achieved comparable results, therefore no single feature subset could be identified as most relevant for classification of cognitive load experienced by drivers. To conclude, the proposed approach could not identify features which could be used to confidently predict a driver’s mental state. / Vehicle Driver Monitoring (VDM)