Objective: Saliency-based theories assume that stimulus features (luminance, edges, texture) shape the hierarchy of perception. Predictions about human viewing behavior point out that highly salient objects or regions of a picture will be perceived first (e.g. Itti et al. 2001). Not much attention is paid to top-down factors such as emotion, motivation and other cognitive functions. The present study analyses the influence of the emotional status on visual perception in free-viewing tasks. Following the hypotheses of the study even in artificial contexts such as viewing neutral pictures in an experiment, top-down factors, such as emotion might influence length of fixation and the number of fixations and saccades. Approaches of visual perception and perception in general should focus more on the individual cognitive aspects of this process. Method: In an eyetracking-study 91 participants viewed neutral and emotional relevant pictures without any task. Five emotional pictures (crime, murder, babies), which were taken from the IAPS (Lang, Bradley & Cuthbert, 2005) were followed by one neutral picture (forest, grass) taken from studies of Peters et al (2005) and Einhäuser et al. (2003). Altogether the subjects viewed 130 pictures. Results: There were significant differences in viewing behavior concerning the neutral pictures subject to the emotional condition. When presented in context with emotional pictures, neutral pictures were fixated longer and less frequently in comparison to the neutral condition. There is evidence to suggest that the emotional context influenced visual attention concerning the neutral pictures in that way that attention for the neutral material was reduced in the context of highly arousing emotional pictures. This leads to the conclusion, that even in experimental setups with simple neutral stimuli the process of visual perception is influenced top-down.

Perceptual categorization involves integrating bottom-up sensory information with top-down knowledge which is based on prior experience. Bottom-up information comes from the external world and visual saliency is a type of bottom-up information that is calculated on the differences between the visual characteristics of adjacent spatial locations. There is currently a related debate in municipal law enforcement communities about which are more ‘visible’: white police cars or black and white police cars. Municipalities do not want police cars to be hit by motorists and they also want police cars to be seen in order to promote a public presence. The present study used three behavioral experiments to investigate the effects of visual saliency on object detection and categorization. Importantly, the results indicated that so-called ‘object detection’ is not a valid construct. Rather than identifying objectness or objecthood prior to categorization, object categorization is an obligatory process, and object detection is a postcategorization decision with higher salience objects being categorized easier than lower salience objects. An additional experiment was conducted to examine the features that constitute a police car. Based on salience alone, black and white police cars were better categorized than white police cars and light bars were slightly more important police car defining components than markings.

In a video there are certain regions in the image that viewers focus on more than others, which are called the salient regions or Regions­Of-Interest (ROI). This thesis aims to improve the perceived quality of videos by improving the quality of these ROis while degrading the quality of the other non-ROI regions of a frame to keep the same bitrate as would have been the case otherwise. This improvement is achieved by using saliency maps generated using an eye tracker or a deep neural network and providing this information to a modified video encoder. In this thesis the open source x264 encoder was chosen to make use of this information. The effects of ROI encoding are studied for high quality 720p videos by encoding them at low bitrates. The results indicate that ROI encoding can improve subjective video quality when carefully applied. / I en video £inns <let vissa delar av bilden som tittarna fokuserar mer pa an andra, och dessa kallas Region of Interest". Malet med den har upp­satsen ar att hoja den av tittaren upplevda videokvaliteten genom att minska kompressionsgraden ( och darmed hoja kvaliteten) i de iogon­fallande delarna av bilden, samtid som man hojer kompressionsgra­den i ovriga delar sa att bitraten blir den samma som innan andring­en. Den har forbattringen gors genom att anvanda Saliency Mapsss­om visar de iogonfallande delarna for varje bildruta. Dessa Saliency Maps"har antingen detekterats med hjalp av en Eye Tracker eller sa har de raknats fram av ett Neuralt Natverk. Informationen anvands sedan i en modifierad version av den oppna codecen x264 enligt en egen­designad algoritm. Effekten av forandringen har studerats genom att koda hogkvalitativa kallfiler vid lag bitrate. Resultaten indikerar att denna metod kan forbattra den upplevda kvaliteten av en video om den appliceras med ratt styrka.

Yes / The exponential growth in the volume of digital image databases is making it increasingly difficult to retrieve relevant information from them. Efficient retrieval systems require distinctive features extracted from visually rich contents, represented semantically in a human perception-oriented manner. This paper presents an efficient framework to model image contents as an undirected attributed relational graph, exploiting color, texture, layout, and saliency information. The proposed method encodes salient features into this rich representative model without requiring any segmentation or clustering procedures, reducing the computational complexity. In addition, an efficient graph-matching procedure implemented on specialized hardware makes it more suitable for real-time retrieval applications. The proposed framework has been tested on three publicly available datasets, and the results prove its superiority in terms of both effectiveness and efficiency in comparison with other state-of-the-art schemes. / Supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2013R1A1A2012904).

No / In this paper, we investigate how controlled changes to image properties and orientation affect eye movements for repeated viewings of images of natural scenes. We make changes to images by manipulating low-level image content (such as luminance or chromaticity) and/or inverting the image. We measure the effects of these manipulations on human scanpaths (the spatial and chronological path of fixations), additionally comparing these effects to those predicted by a widely used saliency model (L. Itti & C. Koch, 2000). Firstly we find that repeated viewing of a natural image does not significantly modify the previously known repeatability (S. A. Brandt & L. W. Stark, 1997; D. Noton & L. Stark, 1971) of scanpaths. Secondly we find that manipulating image features does not necessarily change the repeatability of scanpaths, but the removal of luminance information has a measurable effect. We also find that image inversion appears to affect scene perception and recognition and may alter fixation selection (although we only find an effect on scanpaths with the additional removal of luminance information). Additionally we confirm that visual saliency as defined by L. Itti and C. Koch's (2000) model is a poor predictor of real observer scanpaths and does not predict the small effects of our image manipulations on scanpaths.

We propose a method of adaptive image denoising using a support vector machine (SVM) classifier to select between multiple well-performing contemporary denoising algorithms for each pixel of a noisy image. We begin by proposing a simple method for realistically generating noisy images, and also describe a number of novel and pre-existing features based on seam energy, local colour, and saliency which are used as classifier inputs. Our SVM strategic image denoising (SVMSID) results demonstrate better image quality than either candidate denoising algorithm for images of moderate noise level, as measured using the perceptually-based quaternion structural similarity image metric (QSSIM). We also demonstrate a modified training point selection method to improve robustness across many noise levels, and propose various extensions to SVMSID for further exploration. / Thesis / Master of Applied Science (MASc)

With artificial intelligence (AI) becoming ubiquitous in a broad range of application domains, the opacity of deep learning models remains an obstacle to adaptation within safety-critical systems. Explainable AI (XAI) aims to build trust in AI systems by revealing important inner mechanisms of what has been treated as a black box by human users. This thesis specifically aims to improve the transparency and trustworthiness of deep learning algorithms by combining attribution methods with image segmentation methods. This thesis has the potential to improve the trust and acceptance of AI systems, leading to more responsible and ethical AI applications. An exploratory algorithm called ESAX is introduced and shows how performance greater than other top attribution methods on PIC testing can be achieved in some cases. These results lay a foundation for future work in segmentation attribution.

No description available.

No description available.

Neural network models such as Transformer-based BERT, mBERT and RoBERTa are achieving impressive performance (Devlin et al., 2019; Lewis et al., 2020; Liu et al., 2019; Raffel et al., 2020; Y. Sun et al., 2019), but we still know little about their inner working due to the complex technique like multi-head self-attention they implement. Attention is commonly taken as a crucial way to explain the model outputs, but there are studies argue that attention may not provide faithful and reliable explanations in recent years (Jain and Wallace, 2019; Pruthi et al., 2020; Serrano and Smith, 2019; Wiegreffe and Pinter, 2019). Bastings and Filippova (2020) then propose that saliency may give better model interpretations since it is designed to find which token contributes to the prediction, i.e. the exact goal of explanation.  In this thesis, we investigate the extent to which syntactic structure is reflected in BERT, mBERT and RoBERTa trained on English and Chinese by using a gradient-based saliency method introduced by Simonyan et al. (2014). We examine the dependencies that our models and baselines predict.  We find that our models can predict some dependencies, especially those that have shorter mean distance and more fixed position of heads and dependents, even though all our models can handle global dependencies in theory. Besides, BERT usually has higher overall accuracy on connecting dependents to their corresponding heads, followed by mBERT and RoBERTa. Yet all the three model in fact have similar results on individual relations. Moreover, models trained on English have better performances than models trained on Chinese, possibly because of the flexibility of Chinese language.