Neurobehavioural problems (i.e. cognitive impairment/behaviour problems) are common in childhood epilepsy. There are very limited data in children with early-onset epilepsy (CWEOE; onset ≤4 years). This study: (1) estimated the incidence of early-onset epilepsy, (2) described the spectrum and prevalence of neurobehavioural problems in CWEOE, and their risk factors, and (3) explored eye-gaze behaviour as a marker of neurobehavioural problems. This two year, prospective, population-based, case-controlled study identified newly diagnosed CWEOE in South East Scotland using active multi-source capture-recapture surveillance. CWEOE and controls completed detailed age-appropriate neuropsychological assessment - including Bayley III/WPPSI III, NEPSY II and social-emotional behaviour questionnaires. Children completed five eye-tracking tasks which assessed memory, attention, and social cognition. 59 CWEOE were identified (36M:23F); ascertainment-adjusted incidence 62/100,000 ≤4yrs/yr (95%CI 40-88). Asian and White-European children were at increased risk of epilepsy. 46 CWEOE (95%CI 62-84, 27M:19F) and 37 sex-age matched controls (18M:19F) underwent neuropsychological assessment. CWEOE had poorer general cognitive ability (p < .001, η²=.24), and increased parent reports of abnormal behaviour – significantly so in adaptive behaviour, ASD behaviours, hyperactivity/inattention, and atypical social behaviour. Overall 63% of CWEOE met criteria for neurobehavioural problems across multiple domains, vs 27% of controls (p < .001). Risk factors varied by domain. Prematurity and symptomatic/cryptogenic aetiology were common risk factors but other seizure-related variables were not. CWEOE with social problems exhibited abnormal eye-gaze behaviour toward social stimuli. Subtle atypicalities in sustained attention were noted in CWEOE, and an unexpected absence of antisaccade production was seen in all children. This is the first population-based study to describe the neurobehavioural profile, and explore eye-gaze behaviour, in CWEOE. Neurobehavioural problems are present, detectable, and highly prevalent in CWEOE, with implications for medical, psychosocial and educational resource provision, and provides an argument for early intervention. Eye-tracking may be a viable marker of neurobehavioural problems, and this study provides impetus for future eye-tracking investigations in CWEOE. Lastly, certain ethnic groups may be at increased risk of early-onset epilepsy in Scotland, providing opportunity for targeted intervention.

abstract: Geoscience educators commonly teach geology by projecting a photograph in front of the class. Geologic photographs often contain animals, people, and inanimate objects that help convey the scale of features in the photograph. Although scale items seem innocuous to instructors and other experts, the presence of such items is distracting and has a profound effect on student learning behavior. To evaluate how students visually interact with distracting scale items in photographs and to determine if cueing or signaling is an effective means to direct students to pertinent information, students were eye tracked while looking at geologically-rich photographs. Eye-tracking data revealed that learners primarily looked at the center of an image, focused on faces of both humans and animals if they were present, and repeatedly returned to looking at the scale item (distractor) for the duration an image was displayed. The presence of a distractor caused learners to look at less of an image than when a distractor was not present. Learners who received signaling tended to look at the distractor less, look at the geology more, and surveyed more of the photograph than learners who did not receive signaling. The San Antonio area in the southern part of the Baja California Peninsula is host to hydrothermal gold deposits. A field study, including drill-core analysis and detailed geologic mapping, was conducted to determine the types of mineralization present, the types of structures present, and the relationship between the two. This investigation revealed that two phases of mineralization have occurred in the area; the first is hydrothermal deposition of gold associated with sulfide deposits and the second is oxidation of sulfides to hematite, goethite, and jarosite. Mineralization varies as a function of depth, whereas sulfides occurring at depth, while minerals indicative of oxidation are limited to shallow depths. A structural analysis revealed that the oldest structures in the study area include low-grade to medium-grade metamorphic foliation and ductile mylonitic shear zones overprinted by brittle-ductile mylonitic fabrics, which were later overprinted by brittle deformation. Both primary and secondary mineralization in the area is restricted to the later brittle features. Alteration-bearing structures have an average NNW strike consistent with northeast-southwest-directed extension, whereas unaltered structures have an average NNE strike consistent with more recent northwest-southeast-directed extension. / Dissertation/Thesis / Ph.D. Geological Sciences 2011

abstract: An array of north-striking, left-stepping, active normal faults is situated along the southwestern margin of the Gulf of California. This normal fault system is the marginal fault system of the oblique-divergent plate boundary within the Gulf of California. To better understand the role of upper-crustal processes during development of an obliquely rifted plate margin, gravity surveys were conducted across the normal-fault-bounded basins within the gulf-margin array and, along with optically stimulated luminescence dating of offset surfaces, fault-slip rates were estimated and fault patterns across basins were assessed, providing insight into sedimentary basin evolution. Additionally, detailed geologic and geomorphic maps were constructed along two faults within the system, leading to a more complete understanding of the role of individual normal faults within a larger array. These faults slip at a low rate (0.1-1 mm/yr) and have relatively shallow hanging wall basins (~500-3000 m). Overall, the gulf-margin faults accommodate protracted, distributed deformation at a low rate and provide a minor contribution to overall rifting. Integrating figures with text can lead to greater science learning than when either medium is presented alone. Textbooks, composed of text and graphics, are a primary source of content in most geology classes. It is essential to understand how students approach learning from text and figures in textbook-style learning materials and how the arrangement of the text and figures influences their learning approach. Introductory geology students were eye tracked while learning from textbook-style materials composed of text and graphics. Eye fixation data showed that students spent less time examining the figure than the text, but the students who more frequently examined the figure tended to improve more from the pretest to the posttest. In general, students tended to examine the figure at natural breaks in the reading. Textbook-style materials should, therefore, be formatted to include a number of natural breaks so that learners can pause to inspect the figure without the risk of losing their place in the reading and to provide a chance to process the material in small chunks. Multimedia instructional materials should be designed to support the cognitive processes of the learner. / Dissertation/Thesis / Ph.D. Geological Sciences 2011

In a world with an increasing flow of written information online and offline, the demand for automatic translation, simplification and summarization technology is growing. The tool StilLett uses several lexical and semantic rules to automatically simplify text, shortening the time from original text to simplified text compared to manual simplification. Four of the simplification types were evaluated in this study, using three different eye tracking measures; total dwell time, number of regressions, and average fixation duration. No statistically significant differences were found for any of the simplification types when compared to the corresponding original sentences, indicating that for this population the difficulty of the sentences was roughly the same.

Vision research has uncovered computational principles that generalize across species and brain area. However, these biological mechanisms are not frequently implemented in computer vision algorithms. In this thesis, models suitable for application in computer vision were developed to address the benefits of two biologically-inspired computational principles: multi-scale sampling and active, space-variant, vision. The first model investigated the role of multi-scale sampling in motion integration. It is known that receptive fields of different spatial and temporal scales exist in the visual cortex; however, models addressing how this basic principle is exploited by species are sparse and do not adequately explain the data. The developed model showed that the solution to a classical problem in motion integration, the aperture problem, can be reframed as an emergent property of multi-scale sampling facilitated by fast, parallel, bi-directional connections at different spatial resolutions. Humans and most other mammals actively move their eyes to sample a scene (active vision); moreover, the resolution of detail in this sampling process is not uniform across spatial locations (space-variant). It is known that these eye-movements are not simply guided by image saliency, but are also influenced by factors such as spatial attention, scene layout, and task-relevance. However, it is seldom questioned how previous eye movements shape how one learns and recognizes an object in a continuously-learning system. To explore this question, a model (CogEye) was developed that integrates active, space-variant sampling with eye-movement selection (the where visual stream), and object recognition (the what visual stream). The model hypothesizes that a signal from the recognition system helps the where stream select fixation locations that best disambiguate object identity between competing alternatives. The third study used eye-tracking coupled with an object disambiguation psychophysics experiment to validate the second model, CogEye. While humans outperformed the model in recognition accuracy, when the model used information from the recognition pathway to help select future fixations, it was more similar to human eye movement patterns than when the model relied on image saliency alone. Taken together these results show that computational principles in the mammalian visual system can be used to improve computer vision models.

Humans can often report a subjective sense of confidence in a decision before knowing its outcome. Such confidence judgements are positively correlated to accuracy in perceptual and memory tasks, but the strength of this relationship (known as metacognitive accuracy) differs across people and contexts. Computationally, confidence judgements are believed to relate to the strength of evidence favouring each option, but it has been suggested that confidence also captures information from other sources, such as response time. This thesis explores the pragmatics of confidence: what factors influence confidence judgements, how accurate confidence judgements are, and how they might influence future behaviour. Our knowledge of the antecedents of confidence is extended by this work in two ways, by introducing novel predictors of confidence and by increasing our understanding of well-known ones. I find that bilinguals have worse metacognitive accuracy than monolinguals. This bilingual deficiency in metacognitive accuracy cannot be explained by response time and stimulus strength, suggesting that there is at least one important predictor of confidence that remains unaccounted for. I introduce such a predictor in a new eye tracking correlate of confidence: Gaze-shift-frequency, the number of saccades between options, negatively predicts subsequent confidence in perceptual and value-based decisions. In the value domain, the total value of the options is shown to positively relate to confidence despite being negatively related to accuracy, the first such dissociation to be recorded, as far as I am aware. The dissertation extends our understanding of response time as a predictor of confidence by showing that it influences confidence more for judgements that are made after a choice, relative to those made simultaneously with the choice. This differential influence of response time explains the higher metacognitive accuracy of sequential confidence reports. I explore the consequences of confidence judgements in the context of value-based choice. Lower levels of confidence are associated with changes of mind when the same options recur in subsequent trials. To test whether these changes of mind are rational, I approximate choice accuracy in the value domain. I propose a novel method based on the transitivity of the full choice set, so that choices that violate the most transitive ordering of the items can be treated as errors. I find that participants who were more metacognitively accurate showed a decrease in transitivity violations over time. These results extend prior work linking confidence judgements to error correction in the perceptual domain.

Preterm birth (defined as birth before 37 weeks) is a leading cause of neurocognitive impairment in childhood, including difficulties in social cognition and executive function. Microstructural divergence from typical brain development in the preterm brain can be quantified using diffusion magnetic resonance imaging (dMRI) tractography during the neonatal period. The relationship between dMRI tractography metrics and later cognitive difficulties remains inconclusive. A general measure of white matter microstructure (gWM) offers a neural basis for cognitive processes in adults, however it remains unclear when gWM is first detectable in the developmental trajectory. Eye-tracking is a technique which assesses eye-gaze behaviour in response to visual stimuli, which permits inference about underlying cognitive processes, such as social cognition and executive function in infancy. The primary aims of this thesis were to test the hypotheses: dMRI tractography reveals significant differences in tract-average fractional anisotropy (FA) and mean diffusivity (MD) between preterm and term infants, and variance in tract-average FA and MD is shared across major tracts. Secondly, infants born preterm have altered social cognition and executive function compared to term born peers, assessed by eye-tracking and finally, neonatal MRI gWM is associated with cognitive function in infancy. Preterm (birth weight ≤ 1500g) and term infants (born ≥ 37 weeks’ post-menstrual age [PMA]) were recruited and underwent a MRI scan at term equivalent age (between 38 - 42 weeks’ PMA) and an eye-tracking assessment six to nine months later. Preterm infants were assessed at two years using the Bayley Scales of Infant and Toddler Development, Third Edition (BSID-III). dMRI tractography metrics were generated using probabilistic neighbourhood tractography (PNT) in eight pre-defined tracts-of-interest. Principal component analyses (PCA) were used to determine the correlations between the eight tracts-of-interest for four tract-averaged water diffusion parameters. dMRI metrics were compared to the eye-tracking performance and two year outcome data. Quantitative microstructural changes were identifiable within the preterm brain when compared to infants born at term. PCA revealed a single variable that accounts for nearly 50% of shared variance between tracts-of-interest, and all tracts showed positive loadings. Eye-tracking revealed group-wise differences in infant social cognition, attributable to preterm birth, but executive functions inferred from eye-tracking did not differ between groups. dMRI tractography metrics within the neonatal period did not relate to later outcome measures. This thesis shows that variance in dMRI parameters is substantially shared across white matter tracts of the developing brain and suggests that anatomical foundations of later intelligence are present by term equivalent age. Social cognition is altered by preterm birth, however social cognitive ability in infancy is independent of gWM.

A central task in computer vision is detecting object classes such as cars and horses in complex scenes. Training an object class detector typically requires a large set of images labeled with tight bounding boxes around every object instance. Obtaining such data requires human annotation, which is very expensive and time consuming. Alternatively, researchers have tried to train models in a weakly supervised setting (i.e., given only image-level labels), which is much cheaper but leads to weaker detectors. In this thesis, we propose new and efficient human annotation schemes for training object class detectors that bypass the need for drawing bounding boxes and reduce the annotation cost while still obtaining high quality object detectors. First, we propose to train object class detectors from eye tracking data. Instead of drawing tight bounding boxes, the annotators only need to look at the image and find the target object. We track the eye movements of annotators while they perform this visual search task and we propose a technique for deriving object bounding boxes from these eye fixations. To validate our idea, we augment an existing object detection dataset with eye tracking data. Second, we propose a scheme for training object class detectors, which only requires annotators to verify bounding-boxes produced automatically by the learning algorithm. Our scheme introduces human verification as a new step into a standard weakly supervised framework which typically iterates between re-training object detectors and re-localizing objects in the training images. We use the verification signal to improve both re-training and re-localization. Third, we propose another scheme where annotators are asked to click on the center of an imaginary bounding box, which tightly encloses the object. We then incorporate these clicks into a weakly supervised object localization technique, to jointly localize object bounding boxes over all training images. Both our center-clicking and human verification schemes deliver detectors performing almost as well as those trained in a fully supervised setting. Finally, we propose extreme clicking. We ask the annotator to click on four physical points on the object: the top, bottom, left- and right-most points. This task is more natural than the traditional way of drawing boxes and these points are easy to find. Our experiments show that annotating objects with extreme clicking is 5 X faster than the traditional way of drawing boxes and it leads to boxes of the same quality as the original ground-truth drawn the traditional way. Moreover, we use the resulting extreme points to obtain more accurate segmentations than those derived from bounding boxes.

In the light of recent changes on the audiovisual scene in Poland, audiences can choose among different AVT modalities. Although voice-over still prevails on Polish TV, subtitles have become more and more popular as an alternative form of film translation on television. Due to rapid technological advances, commercial requirements and differences in Polish viewers’ preferences, it is thus crucial to understand how audiences at different levels of English proficiency (low, medium, high) retrieve meaning, especially complex ironic meaning relayed through different methods of film translation, such as subtitles and voice-over and the extent to which verbal and non-verbal semiotic channels contribute to irony comprehension. Wilson and Sperber’s (1981, 1992; 1995) echoic theory of irony has been selected as the theoretical framework, given its ability to account for multimodal irony in audiovisual texts as well as the significant importance of non-verbal semiotic resources in the generation and interpretation of irony. The study employs triangulation, incorporating descriptive, experimental and interactionist components. The descriptive component involves multimodal transcription (Baldry and Thibault, 2006) of selected fragments in which irony plays a pivotal narrative role. This procedure aims to determine what non-verbal modes contribute to the multimodal construal of irony and how it is relayed in the subtitled and voiced-over translations. In the experimental component, viewers’ eye movements are recorded using eye-tracking technology while watching subtitled and voiced-over fragments of Sherlock Holmes (2009) and Sherlock Holmes: A Game of Shadows (2011). In the interactionist components, a questionnaire is used in order to elucidate how and/or whether they retrieve ironic meaning as intended by the filmmakers in the selected excerpts. The most obvious finding to emerge from the descriptive data analysis is that multimodal irony is not relayed by the film dialogue alone but, rather, in unison with non-verbal semiotic resources. The instances of multimodal irony in the two Sherlock Holmes films were found to perform narrative and comedic functions by combining the visual, kinesic and acoustic modes of film language. The analysis and comparison of SL dialogues and TL translations revealed two broad categories of irony relay, namely: preservation and modification. The majority of the instances of multimodal irony were modified in the subtitled version, while preservation is only sporadically opted for. In its voiced-over counterpart, the intended meaning is preserved and modified in equal proportions. The experimental component showed major differences in gaze patterns among the participants with different language skills in the subtitled clips. For instance, on average, LLPs spent more time reading the subtitles than HLPs or MLPs. Similar visual behavior, on the other hand, was observed among all viewers in the voiced-over clips in which the on-screen character’s face attracted the greatest amount of visual attention. The interactionist strand showed that the viewers retrieved the intended meaning to various extents depending on their English language proficiency. This data undergirds an assessment of the effectiveness of subtitles and voice-over in the translation and reception of multimodal irony on screen.

People with Autism Spectrum Disorder experience difficulties with reading comprehension and information processing, which affect their school performance, employability and social inclusion. The main goal of this work is to investigate new ways to evaluate and improve text and web accessibility for adults with autism. The first stage of this research involved using eye-tracking technology and comprehension testing to collect data from a group of participants with autism and a control group of participants without autism. This series of studies resulted in the development of the ASD corpus, which is the first multimodal corpus of text and gaze data obtained from participants with and without autism. We modelled text complexity and sentence complexity using sets of features matched to the reading difficulties people with autism experience. For document-level classification we trained a readability classifier on a generic corpus with known readability levels (easy, medium and difficult) and then used the ASD corpus to evaluate with unseen user-assessed data. For sentence-level classification, we used for the first time gaze data and comprehension testing to define a gold standard of easy and difficult sentences, which we then used as training and evaluation sets for sentence-level classification. The results showed that both classifiers outperformed other measures of complexity and were more accurate predictors of the comprehension of people with autism. We conducted a series of experiments evaluating easy-to-read documents for people with cognitive disabilities. Easy-to-read documents are written in an accessible way, following specific writing guidelines and containing both text and images. We focused mainly on the image component of these documents, a topic which has been significantly under-studied compared to the text component; we were also motivated by the fact that people with autism are very strong visual thinkers and that therefore image insertion could be a way to use their strengths in visual thinking to compensate for their difficulties in reading. We investigated the effects images in text have on attention, comprehension, memorisation and user preferences in people with autism (all of these phenomena were investigated both objectively and subjectively). The results of these experiments were synthesised in a set of guidelines for improving text accessibility for people with autism. Finally, we evaluated the accessibility of web pages with different levels of visual complexity. We provide evidence of existing barriers to finding relevant information on web pages that people with autism face and we explore their subjective experiences with searching the web through survey questions.