Under normal viewing conditions, humans find it easy to distinguish between objects made out of different materials such as plastic, metal, or paper. Untextured materials such as these have different surface reflectance properties, including lightness and gloss. With single isolated images and unknown illumination conditions, the task of estimating surface reflectance is highly underconstrained, because many combinations of reflection and illumination are consistent with a given image. In order to work out how humans estimate surface reflectance properties, we asked subjects to match the appearance of isolated spheres taken out of their original contexts. We found that subjects were able to perform the task accurately and reliably without contextual information to specify the illumination. The spheres were rendered under a variety of artificial illuminations, such as a single point light source, and a number of photographically-captured real-world illuminations from both indoor and outdoor scenes. Subjects performed more accurately for stimuli viewed under real-world patterns of illumination than under artificial illuminations, suggesting that subjects use stored assumptions about the regularities of real-world illuminations to solve the ill-posed problem.

This paper describes a machine vision system that classifies reflectance properties of surfaces such as metal, plastic, or paper, under unknown real-world illumination. We demonstrate performance of our algorithm for surfaces of arbitrary geometry. Reflectance estimation under arbitrary omnidirectional illumination proves highly underconstrained. Our reflectance estimation algorithm succeeds by learning relationships between surface reflectance and certain statistics computed from an observed image, which depend on statistical regularities in the spatial structure of real-world illumination. Although the algorithm assumes known geometry, its statistical nature makes it robust to inaccurate geometry estimates.

Humans distinguish materials such as metal, plastic, and paper effortlessly at a glance. Traditional computer vision systems cannot solve this problem at all. Recognizing surface reflectance properties from a single photograph is difficult because the observed image depends heavily on the amount of light incident from every direction. A mirrored sphere, for example, produces a different image in every environment. To make matters worse, two surfaces with different reflectance properties could produce identical images. The mirrored sphere simply reflects its surroundings, so in the right artificial setting, it could mimic the appearance of a matte ping-pong ball. Yet, humans possess an intuitive sense of what materials typically "look like" in the real world. This thesis develops computational algorithms with a similar ability to recognize reflectance properties from photographs under unknown, real-world illumination conditions. Real-world illumination is complex, with light typically incident on a surface from every direction. We find, however, that real-world illumination patterns are not arbitrary. They exhibit highly predictable spatial structure, which we describe largely in the wavelet domain. Although they differ in several respects from the typical photographs, illumination patterns share much of the regularity described in the natural image statistics literature. These properties of real-world illumination lead to predictable image statistics for a surface with given reflectance properties. We construct a system that classifies a surface according to its reflectance from a single photograph under unknown illuminination. Our algorithm learns relationships between surface reflectance and certain statistics computed from the observed image. Like the human visual system, we solve the otherwise underconstrained inverse problem of reflectance estimation by taking advantage of the statistical regularity of illumination. For surfaces with homogeneous reflectance properties and known geometry, our system rivals human performance.

There is general consensus that context can be a rich source of information about an object's identity, location and scale. In fact, the structure of many real-world scenes is governed by strong configurational rules akin to those that apply to a single object. Here we introduce a simple probabilistic framework for modeling the relationship between context and object properties based on the correlation between the statistics of low-level features across the entire scene and the objects that it contains. The resulting scheme serves as an effective procedure for object priming, context driven focus of attention and automatic scale-selection on real-world scenes.

Salient contour detection is an important lowlevel visual process in the human visual system, and has significance towards understanding higher visual and cognitive processes. Salience detection can be investigated by examining the visual cortical response to visual input. Visual response activity in the early stages of visual processing can be approximated by a sequence of convolutions of the input scene with the difference-of-Gaussian (DoG) and the oriented Gabor filters. The filtered responses are unusually high for prominent edge locations in the image, and are uniformly similar across different natural image inputs. Furthermore, such a response follows a power law distribution. The aim of this thesis is to examine how these response properties could be utilized to the problem of salience detection. First, I identify a method to find the best threshold on the response activity (orientation energy) toward the detection of salient contours: compare the response distribution to a Gaussian distribution of equal variance. Second, I justify this comparison by providing an explanation under the framework of Suspicious Coincidence proposed by Barlow [1]. A connection is provided between perceived salience of contours and the neuronal goal of detecting suspiciousness, where salient contours are seen as affording suspicious coincidences by the visual system. Finally, the neural plausibility of such a salience detection mechanism is investigated, and the representational effciency is shown which could potentially explain why the human visual system can effortlessly detect salience.

Salient contour detection is an important lowlevel visual process in the human visual system, and has significance towards understanding higher visual and cognitive processes. Salience detection can be investigated by examining the visual cortical response to visual input. Visual response activity in the early stages of visual processing can be approximated by a sequence of convolutions of the input scene with the difference-of-Gaussian (DoG) and the oriented Gabor filters. The filtered responses are unusually high for prominent edge locations in the image, and are uniformly similar across different natural image inputs. Furthermore, such a response follows a power law distribution. The aim of this thesis is to examine how these response properties could be utilized to the problem of salience detection. First, I identify a method to find the best threshold on the response activity (orientation energy) toward the detection of salient contours: compare the response distribution to a Gaussian distribution of equal variance. Second, I justify this comparison by providing an explanation under the framework of Suspicious Coincidence proposed by Barlow [1]. A connection is provided between perceived salience of contours and the neuronal goal of detecting suspiciousness, where salient contours are seen as affording suspicious coincidences by the visual system. Finally, the neural plausibility of such a salience detection mechanism is investigated, and the representational effciency is shown which could potentially explain why the human visual system can effortlessly detect salience.

Any image can be described in terms of its statistics (i.e. quantitative parameters calculated from the image, for example RMS-contrast, the skewness of image brightness distribution, and slope constant of an average amplitude spectrum). It was previously shown that insect and vertebrate visual systems are optimised to the statistics common among natural scenes. However, the exact mechanisms of this process are still unclear and need further investigation. This thesis presents the results of examining links between some image statistics and visual responses in humans and hoverflies. It was found that while image statistics do not play the main role when hoverflies (Eristalis tenax and Episyrphus balteatus) chose what flowers to feed on, there is a link between hoverfly (Episyrphus balteatus) active behaviours and image statistics. There is a significant difference in the slope constant of the average amplitude spectrum, RMS contrast and skewness of brightness distribution between photos of areas where hoverflies were hovering or flying. These photos were also used to create a prediction model of hoverfly behaviour. After model validation, it was concluded that photos of both the ground and the surround should be used for best prediction of behaviour. The best predictor was skewness of image brightness distribution. By using a trackball setup, the optomotor response in walking hoverflies (Eristalis tenax) was found to be influenced by the slope constant of an average amplitude spectrum.  Intracellular recording showed that the higher-order neuron cSIFE (The centrifugal stationary inhibited flicker excited) in the hoverfly (Eristalis tenax) lobula plate was inhibited by a range of natural scenes and that this inhibition was strongest in a response to visual stimuli with the slope constant of an average amplitude spectrum of 1, which is the typical value for natural environments.  Based on the results of psychophysics study in human subjects it was found that sleep deprivation affects human perception of naturalistic slope constants differently for different image categories (“food” and “real world scenes”). These results help provide a better understanding of the link between visual processes and the spatial statistics of natural scenes.

Two separate lines of research in object recognition are studies of materials perception and studies of real-world object size perception. Recent object size investigations of texture indicate mid-level features may cue representations of object size in the absence of object identity. However, these findings are somewhat controversial, and beyond that what mid-level features cue object size is not clear. Mid-level features have always been the focus of materials perception studies of gloss and specular highlights, but to date no research has been conducted that attempts to link findings on the perception of materials to high-level object features like real-world object size. Three separate experiments were conducted to study the relationship between perceived surface glossiness and specular highlights, and perceived real-world object size. Previous research on the relationship between perceived object size and real-world object size were replicated. A significant two-way interaction between ratings of perceived glossiness, object size, and texture was found. Follow-up analyses indicated that perceptions of gloss were present across categorical differences in real-world object size in both the object image and texture image task groups. For the normal object images, small objects were perceived as being glossier than big objects. For the texture images, big objects were perceived as being glossier than small objects. Between the conditions, small normal and small texture object images were not significantly different in perceived glossiness. Between the conditions, glossiness ratings for big texture object images were significantly greater than those for the normal big object images. / Doctor of Philosophy / The goal of this project was to understand if category level perceptions of surface gloss (i.e. dull/matte surface reflectance versus shiny/glossy surface reflectance) could predict category level differences in the "actual" size of the objects in the real-world (i.e. small objects versus big objects). Previous research on the relationship between perceived object size and real-world object size were replicated. Moreover, in an experiment in which human subjects were tasked with rating the glossiness of images depicting small and large manmade of objects, category level distinctions in the average perceived glossiness of objects also extended to category level distinctions in perceived real-world object size; on average, small objects were perceived as being glossier than big objects. Similar effects were also found for synthetic textures created from the ordinary real-world object images; on average, big objects were rated as being glossier than small objects. Although categorical distinctions in perceived glossiness extended to real-world object size across image conditions, because there were no significant differences in the average perceived glossiness of small objects across the normal image and texture image conditions, the change in perceived glossiness for the big object images suggests that the texture algorithm used may not have preserved the surface reflectance characteristics of the big objects. Furthermore, statistical investigations of the pixel brightness for the stimulus images provided some evidence that the category level differences in perceived glossiness across object size and image condition may have been driven by differences in factors related to naturally occurring optical artifacts that are introduced when photographing small and big objects. Overall, results of this study are important because they indicate that the real-world spatial properties of objects may be jointly encoded with perceptions of object glossiness.

Unnormalised parametric models are an important class of probabilistic models which are difficult to estimate. The models are important since they occur in many different areas of application, e.g. in modelling of natural images, natural language and associative memory. However, standard maximum likelihood estimation is not applicable to unnormalised models, so alternative methods are required. Noise-contrastive estimation (NCE) has been proposed as an effective estimation method for unnormalised models. The basic idea is to transform the unsupervised estimation problem into a supervised classification problem. The parameters of the unnormalised model are learned by training the model to differentiate the given data samples from generated noise samples. However, the choice of the noise distribution has been left open to the user, and as the performance of the estimation may be sensitive to this choice, it is desirable for it to be automated. In this thesis, the ambiguity in the choice of the noise distribution is addressed by presenting the previously unpublished conditional noise-contrastive estimation (CNCE) method. Like NCE, CNCE estimates unnormalised models by classifying data and noise samples. However, the choice of noise distribution is partly automated via the use of a conditional noise distribution that is dependent on the data. In addition to introducing the core theory for CNCE, the method is empirically validated on data and models where the ground truth is known. Furthermore, CNCE is applied to natural image data to show its applicability in a realistic application. / Icke-normaliserade parametriska modeller utgör en viktig klass av svåruppskattade statistiska modeller. Dessa modeller är viktiga eftersom de uppträder inom många olika tillämpningsområden, t.ex. vid modellering av bilder, tal och skrift och associativt minne. Dessa modeller är svåruppskattade eftersom den vanliga maximum likelihood-metoden inte är tillämpbar på icke-normaliserade modeller. Noise-contrastive estimation (NCE) har föreslagits som en effektiv metod för uppskattning av icke-normaliserade modeller. Grundidén är att transformera det icke-handledda uppskattningsproblemet till ett handlett klassificeringsproblem. Den icke-normaliserade modellens parametrar blir inlärda genom att träna modellen på att skilja det givna dataprovet från ett genererat brusprov. Dock har valet av brusdistribution lämnats öppet för användaren. Eftersom uppskattningens prestanda är känslig gentemot det här valet är det önskvärt att få det automatiserat. I det här examensarbetet behandlas valet av brusdistribution genom att presentera den tidigare opublicerade metoden conditional noise-contrastive estimation (CNCE). Liksom NCE uppskattar CNCE icke-normaliserade modeller via klassificering av data- och brusprov. I det här fallet är emellertid brusdistributionen delvis automatiserad genom att använda en betingad brusdistribution som är beroende på dataprovet. Förutom att introducera kärnteorin för CNCE valideras även metoden med hjälp av data och modeller vars genererande parametrar är kända. Vidare appliceras CNCE på bilddata för att demonstrera dess tillämpbarhet.

This thesis explores the visual perception of material change over time, a novel topic that has received little attention so far. We aimed to understand the material surface features and mental representations associated with material change over time by the human visual system, and possibly wider cognitive systems. To this end, we performed a series of experiments with varying methodologies. These included a psychophysics experiment, online experiments, and data collection during a science festival. The latter showed that the general public mentioned “Faded (colour)” most often to describe material change over time and that specific material surface change features clustered around specific materials. In another experiment, material type, but not colour or the geometrical distribution, had a significant effect on perceived material change. Other experiments partially contradicted this finding. It was found that perceived material type showed a significant, non-linear association with perceived material change, replicating earlier findings on the effect of material type. In contrast, material surface lightness, a constituent of colour, was associated with perceived material change. The same held for components of the geometrical distribution. They showed a minor contribution to the perception of material change, but a major one to perceived material type. Together, our findings suggest that the human visual system seems to use constituents of material surface colour as a cue to material change over time. The geometrical distribution seems to play a minor role. Although these contributions may vary with material type, as our findings showed that material type affected the perception of material change over time. / DyViTo (Dynamics in Vision and Touch) has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 765121