With the launch of networked handheld devices which can capture, store, compress, send and display a variety of audiovisual stimuli; high definition television (HDTV); streaming Internet protocol TV (IPTV) and websites such as Youtube, Facebook and Flickr etc., an enormous amount of visual data of visual data is making its way to consumers. Because of this, considerable time and resources are being expanded to ensure that the end user is presented with with a satisfactory quality of experience (QoE). While traditional QoE methods have focused on optimizing delivery networks with respect to throughput, buffer-lengths and capacity, perceptually optimized delivery of multimedia services is also fast gaining importance. This is especially timely given the explosive growth in (especially wireless) video traffic and expected shortfalls in bandwidth. These perceptual approaches attempt to deliver an optimized QoE to the end-user by utilizing objective measures of visual quality. In this thesis, we shall cover a variety of such algorithms that predict overall QoE of an image or a video, depending on the amount of information available for the algorithm design. Typically, quality assessment (QA) algorithms are classiffied on the basis of the amount of information that is available to the algorithm. This thesis will primarily focus on blind QA algorithms, where blind or no-reference (NR) QA refers to automatic quality assessment of an image/video using an algorithm which only utilizes the distorted image/video whose quality is being assessed. NR QA approaches are further classiffied on the basis of whether the algorithm had access to subjective/human opinion prior to deployment. Algorithms which use machine learning techniques along with human judgements of quality during the 'training' phase may be labelled 'opinion aware' algorithms. The first part of the thesis deals with such approaches. While such opinion aware-NR algorithms demonstrate good correlation with human perception on controlled databases, it is impossible to anticipate all of the different distortions that may occur in a practical system and hence train on them. In such cases, it is of interest to design QA algorithms that are not limited in their performance by training data. Approaches which operate without the knowledge of human judgements during the training phase are labelled as 'opinion unaware' (OU) algorithms. We propose such an approach in the second part of the thesis. Further, we propose new VQA algorithms in the last part of the dissertation to address the completely blind VQA problem. The proposed approach quantify disturbances introduced due to distortions and thereby predict the quality of distorted content even without any external knowledge about the pristine natural sources and hence zero shot models. / text

In a natural environment, humans are not able to process all information available to the visual system simultaneously. As a consequence, we attend to different subparts of the input one after the other. Under natural viewing conditions, these shifts of attention are associated with changes in fixation. This so-called overt attention therefore provides an objective measure of attention shifts. In my thesis, I investigate the influence of color on overt visual attention. I present human subjects with different categories of color-calibrated images and record their eye-movements. In the first series of experiments, images of 7 different categories (Face, Flower, Forest, Fractal, Landscape, Man-Made object, and Rainforest) are presented either in natural color or grayscale. With regard to the influence of color on overt attention, I find two extreme categories: while in Rainforest images all color features examined are salient, none is salient in Fractal. In all other categories, color features are selectively salient. This shows that the influence of color on overt attention depends on the type of image. In the second series of experiments, I use only Rainforest images. These are presented to color-normal or deuteranope subjects with several modifications in the color domain. I find a causal influence of color-contrast on overt attention, processed in a way that it is not specific to either the red-green or blue-yellow color channel. In the case of color blindness, a slower high-level compensatory mechanism affects the selection of fixation points. These experiments show that there is no single color feature, which influences overt attention in all possible environments. I provide evidence that different levels of the visual hierarchy are involved in the computation of saliency.

Recent cognitive research indicates that the human brain possesses special abilities for processing information relating to outdoor scenes. Simulated outdoor scenes are presented as an effective method of displaying higher dimensional data in an efficient and comprehensible manner. We demonstrate novel methods of using outdoor objects and scenes to display multidimensional content in a way that that is intuitive for humans to understand, and to exploit various cues commonly found in scenes from the natural world to communicate the values of multiple variables.

Motion in the outside world forms one of the primary uses of visual information for many animals. The ability to interpret motion quickly and accurately permits interaction with and response to events in the outside world. While much is known about some aspects of motion perception, there is less agreement about how feature selectivity leading to motion perception is actually formed in the convergent and divergent pathways of the visual system. It is even less clear how these classical understandings of motion processing, often driven by artificial stimuli with little resemblance to the outside world, correspond to responses of neurons when using more natural stimuli. In this thesis, we probe these gaps, first by demonstrating that synchronization within the visual thalamus leads to efficient representations of motion (through tuning properties) in primary visual cortex, exploiting precise timing across populations in a unique manner compared to traditional models. We then create a novel “minimally-natural” stimulus with the appearance of an infinite hallway wallpapered with sinusoidal gratings, to probe how such minimally natural features modulate our predictions of neural responses based upon feature tuning properties. Through encoding and decoding models we find that measuring a restricted tuning parameter space limits our ability to capture all response properties but preserves relevant information for decoding. We finish with an exploration of ethologically relevant natural features, perspective and complex motion, and show that even moderate amounts of each feature within or near the classical V1 receptive field changes the neural response from what classical feature tuning would predict and improves stimulus classification tremendously. Together all of these results indicate that capturing information about motion in the outside world through visual stimuli requires a more advanced model of feature selectivity that incorporates parameters based on more complex spatial relationships.

Any image can be described in terms of its statistics (i.e. quantitative parameters calculated from the image, for example RMS-contrast, the skewness of image brightness distribution, and slope constant of an average amplitude spectrum). It was previously shown that insect and vertebrate visual systems are optimised to the statistics common among natural scenes. However, the exact mechanisms of this process are still unclear and need further investigation. This thesis presents the results of examining links between some image statistics and visual responses in humans and hoverflies. It was found that while image statistics do not play the main role when hoverflies (Eristalis tenax and Episyrphus balteatus) chose what flowers to feed on, there is a link between hoverfly (Episyrphus balteatus) active behaviours and image statistics. There is a significant difference in the slope constant of the average amplitude spectrum, RMS contrast and skewness of brightness distribution between photos of areas where hoverflies were hovering or flying. These photos were also used to create a prediction model of hoverfly behaviour. After model validation, it was concluded that photos of both the ground and the surround should be used for best prediction of behaviour. The best predictor was skewness of image brightness distribution. By using a trackball setup, the optomotor response in walking hoverflies (Eristalis tenax) was found to be influenced by the slope constant of an average amplitude spectrum.  Intracellular recording showed that the higher-order neuron cSIFE (The centrifugal stationary inhibited flicker excited) in the hoverfly (Eristalis tenax) lobula plate was inhibited by a range of natural scenes and that this inhibition was strongest in a response to visual stimuli with the slope constant of an average amplitude spectrum of 1, which is the typical value for natural environments.  Based on the results of psychophysics study in human subjects it was found that sleep deprivation affects human perception of naturalistic slope constants differently for different image categories (“food” and “real world scenes”). These results help provide a better understanding of the link between visual processes and the spatial statistics of natural scenes.

No description available.

L'analyse visuelle de scènes débute par l'extraction en parallèle de l'information à différentes fréquences spatiales, en suivant un mode de traitement par défaut « coarse-to-fine ». L'analyse rapide de l'information grossière (« coarse ») en basses fréquences spatiales fournirait un aperçu global de la scène, qui serait ensuite affiné par l'analyse des détails de la scène (« fine ») en hautes fréquences spatiales. L'objectif de cette thèse a été de préciser les bases cérébrales du traitement des fréquences spatiales lors de la catégorisation de scènes. A travers deux études comportementales, nous avons tout d'abord montré qu'une analyse « coarse-to-fine » est plus avantageuse pour la catégorisation rapide de scènes, et ce, indépendamment de la valeur de contraste de luminance associée aux différentes fréquences spatiales (Expériences 1 et 2). Des études en IRMf nous ont par la suite permis de mettre en évidence l'implication d'un large réseau cérébral lors de l'analyse « coarse-to-fine » des scènes, incluant les aires visuelles primaires et occipito-temporales, mais également le cortex frontal inférieur (Expérience 3). Une analyse de la connectivité a révélé que lors de cette analyse, le cortex frontal inférieur exercerait une influence « top-down » sur le cortex visuel primaire et les gyri fusiforme et parahippocampique au sein du cortex occipito-temporal. Ces résultats soulignent le rôle du cortex visuel primaire comme région intégrative, codant à la fois les afférences rétino-thalamiques et les influences « top-down » de régions supérieures. Nous avons également observé que le gyrus frontal inférieur et le gyrus fusiforme participaient activement à l'intégration de l'information sémantique contenue dans les basses et hautes fréquences spatiales d'une scène (Expérience 4). Enfin, nous avons spécifiquement étudié le traitement des fréquences spatiales au sein de régions occipito-temporales sélectives aux scènes : la « parahippocampal place area » (PPA), le cortex retrosplenial et l'« occipital place area ». Nous avons montré que ces trois régions participent de façon distincte au traitement des fréquences spatiales dans les scènes (Expérience 5) et qu'une stratégie d'analyse « coarse-to-fine » serait privilégiée par la PPA (Expérience 6). Les résultats de ces travaux nous permettent de conforter et de préciser les modèles actuels de la catégorisation visuelle de scènes basés sur un traitement fréquentiel de l'information visuelle. / Visual analysis begins with the parallel extraction of different attributes at different spatial frequencies following a predominantly coarse-to-fine default processing sequence. Rapid processing of low spatial frequency information would permit a coarse parsing of the visual input, prior to the detailed analysis of fine information in high spatial frequencies. Our aim was to further address the neural bases of spatial frequency processing during scene categorization. We first demonstrated in two behavioral studies that a coarse-to-fine processing is indeed an advantageous strategy for rapid scene categorization, and is independent of the luminance contrast values associated with the different spatial frequencies (Experiments 1 and 2). In two fMRI studies, we showed first the involvement of a large cerebral network during coarse-to-fine processing of scenes, including early visual and occipito-temporal areas, but also the inferior frontal cortex. Effective connectivity analysis revealed that the inferior frontal gyrus exerts top-down influence on the early visual cortex as well as on the parahippocampal and fusiform gyri in the occipito-temporal cortex (Experiment 3). These results highlight the role of the primary visual cortex in integrating top-down influences from frontal areas to retino-thalamic incoming signals. We also evidenced that the inferior frontal and fusiform gyri actively participate to the integration of the semantic information contained in low and high spatial frequency (Experiment 4). Finally, we specifically investigated the spatial frequency processing of scenes within scene-selective areas of the occipito-temporal cortex: the parahippocampal place area (PPA), the retrosplenial cortex, and the occipital place area. We demonstrated that these regions participate differently in the spatial frequency processing of scenes (Experiment 5) and that a coarse-to-fine processing is favored within the PPA (Experiment 6). Overall, results allow us to refine current model of visual scene categorization based on a spatial frequency analysis.

De nombreux arguments, issus notamment de la neurophysiologie visuelle, suggèrent que la perception visuelle d'une scène débute par une extraction des différentes fréquences spatiales en suivant une stratégie d'analyse « coarse-to-fine ». L'analyse rapide de l'information grossière en basses fréquences spatiales (BFS) fournirait un aperçu global de la de la scène qui serait, ensuite, affinée par l'analyse plus tardive de l'information fine en hautes fréquences spatiales (HFS). L'objectif de cette thèse est de spécifier les mécanismes neuro-fonctionnels et cognitifs du traitement des fréquences spatiales et des scènes naturelles et leur évolution au cours du vieillissement normal et pathologique. Dans une première étude en IRMf (Expérience 1), nous avons montré la coexistence, au niveau du cortex occipital, d'un traitement rétinotopique et hémisphérique des fréquences spatiales. Par ailleurs, nous avons montré que des régions sélectives aux scènes, au sein du gyrus parahippocampique et du cortex retrosplenial, étaient également impliquées dans le traitement des fréquences spatiales. Dans les études IRMf suivantes (Expériences 2, 3 et 4), nous nous sommes particulièrement intéressés au traitement des fréquences spatiales et à la stratégie d'analyse « coarseto- fine » dans ces régions sélectives. Dans la seconde partie de ces travaux, nous avons montré que la stratégie de catégorisation « coarse-to-fine » observée chez le jeune adulte sain, devenait flexible avec l'âge (Expérience 5). Afin de préciser les interactions rétino-corticales, nous avons étudié les performances de catégorisation de patients atteints de dégénérescence maculaire liée à l'âge, pathologie caractérisée par des lésions de la rétine centrale supposée à l'origine de la voie de traitement des HFS. Nous avons démontré un déficit comportemental (Expériences 6 et 7) du traitement des HFS, associé à une hypoactivité du cortex occipital chez ces patients (Expérience 8). Ces travaux permettent de préciser les mécanismes impliqués dans la perception de scènes. / As suggested by evidence from visual neurophysiology, scene perception could begin by the extraction of different spatial frequencies following a “coarse-to-fine” analysis. The rapid analysis of coarse information in low spatial frequencies (LSF) would provide a global overview of the scene which would then be refined by later analysis of fine information in high spatial frequencies (HSF). The aim of this thesis is to specify the neuro-functional and cognitive mechanisms of spatial frequency and natural scene processing as well as their evolution during normal and pathological aging. In a first fMRI study (Experiment 1), we showed the coexistence of retinotopic and hemispheric processing for spatial frequencies in occipital cortex. In addition, we showed that scene selective regions in the parahippocampal gyrus and retrosplenial cortex were also involved in the processing of spatial frequencies. Therefore, in the following fMRI studies (Experiments 2, 3 and 4), we were particularly interested in spatial frequency processing and "coarse-to-fine" analysis in these selective regions. In the second part of this work, we showed that the “coarse-to-fine” strategy observed in healthy young adults becomes flexible with increasing age (Experiment 5). To clarify the retina-cortex interactions, we studied the categorization performance of patients with age-related macular degeneration. This pathology is characterized by lesions of the central retina, which is thought to be the origin of the visual pathway conveying HSF. We have demonstrated a behavioral deficit (Experiments 6 and 7) of HSF processing linked to hypoactivity of occipital cortex in these patients (Experiment 8). These works clarify the mechanisms involved in scene perception.

La répercussion d'un trouble visuel sur le traitement de scènes naturelles a été abordée selon trois approches : 1) une approche comportementale dans le but d'étudier la perception visuelle en champ central et en champs latéralisés, chez les participants sains et chez les patients hémianopsiques à la suite d'une lésion occipitale unilatérale ; 2) une approche fonctionnelle à l'aide d'une étude en IRMf afin d'évaluer tout d'abord l'incidence de la demande cognitive sur une tâche visuelle, chez les participants sains et chez les patients hémianopsiques, en champ central, puis d'observer l'impact d'une lésion occipitale droite et gauche sur la réorganisation corticale ; 3) enfin, une approche neuropsychologique dans le but d'étudier l'effet d'un entraînement visuel dans le champ aveugle des patients HLH, en utilisant les capacités visuelles inconscientes. Nous avons observé un impact différent sur le traitement visuel en champ central selon la latéralisation de la lésion occipitale (droite ou gauche). L'analyse visuelle est plus perturbée à la suite d'une lésion occipitale droite. Les données acquises en IRMf ont permis également de constater une différence des patterns d'activation selon la latéralisation de la lésion occipitale. Enfin, nous avons observé que l'entrainement dans le champ visuel aveugle des patients HLH, en utilisant les capacités de perception implicite : blindsight, permet d'obtenir une amélioration du champ visuel de manière quantifiable. / Visual processing of natural scenes images was investigated in healthy individuals and patients with homonymous hemianopia (after a unilateral occipital damage) using three methods : 1) behavioral studies where stimuli were presented in the central and lateralized visual fields; 2) fMRI paradigm to assess the cerebral network underlying natural scenes processing in the central visual field and to assess consequences of right or left occipital lesion on brain reorganization ; 3) a neuropsychological study where the efficiency of visual training in the blind visual field of hemianopics patients, using blindsight, was investigated. Results showed that occipital lesion side selectively modulates visual performances and pattern of cortical activation. In patients with right occipital damage, performances were severely altered although both hemlispheres were recruited. Yet, in patients with left brain damage, performances were less degraded and the right hemisphere was recruited whatever the visual task. Finally, we observed that explicit (conscious) visual detection can be restored (at least partly) in the blind visual field by using implicit (unconscious) visual capacities.

O emprego de sistemas computacionais na Agricultura de Precisão (AP) fomenta a automação de processos e tarefas aplicadas nesta área, precisamente voltadas à inspeção e análise de culturas agrícolas, e locomoção guiada/autônoma de robôs móveis. Neste contexto, no presente trabalho foi proposta a aplicação de técnicas de visão computacional nas tarefas citadas, desenvolvidas em abordagens distintas, a serem aplicadas em uma plataforma de robô móvel agrícola, em desenvolvimento no NEPAS/EESC/USP. Para o problema de locomoção do robô (primeira abordagem), foi desenvolvida uma arquitetura de aquisição, processamento e análise de imagens com o objetivo de segmentar, classificar e reconhecer padrões de navegação das linhas de plantio, como referências de guiagem do robô móvel, entre plantações de laranja, milho e cana. Na segunda abordagem, tais técnicas de processamento de imagens são aplicadas também na inspeção e localização das culturas laranja (primário) e milho (secundário), para análise de suas características naturais, localização e quantificação. Para as duas abordagens, a estratégia adotada nas etapas de processamento de imagens abrange: filtragem no domínio espacial das imagens adquiridas; pré-processamento nos espaços de cores RGB e HSV; segmentação não supervisionada JSEG customizada à quantização de cores em regiões não homogêneas nestes espaços de cores; normalização e extração de características dos histogramas das imagens pré-processadas para os conjuntos de treinamento e teste através da análise das componentes principais; reconhecimento de padrões e classificação cognitiva e estatística. A metodologia desenvolvida contemplou bases de dados para cada abordagem entre 700 e 900 imagens de cenas naturais sob condições distintas de aquisição, apresentando resultados significativos quanto ao algoritmo de segmentação nas duas abordagens, mas em menor grau em relação à localização de gramíneas, sendo que os milhos requerem outras técnicas de segmentação, que não aplicadas apenas em quantização de regiões não homogêneas. A classificação estatística, Bayes e Bayes Ingênuo, mostrou-se superior à cognitiva RNA e Fuzzy nas duas abordagens, e posterior construção dos mapas de classe no espaço de cores HSV. Neste mesmo espaço de cores, a quantificação e localização de frutos apresentaram melhores resultados que em RGB. Com isso, as cenas naturais nas duas abordagens foram devidamente processadas, de acordo com os materiais e métodos empregados na segmentação, classificação e reconhecimento de padrões, fornecendo características intrínsecas e distintas das técnicas de visão computacional propostas a cada abordagem. / The use of computer systems in Precision Agriculture (PA) promotes the processes automation and its applied tasks, specifically the inspection and analysis of agricultural crops, and guided/autonomous locomotion of mobile robots. In this context, it was proposed in the present work the application of computer vision techniques on such mentioned tasks, developed in different approaches, to be applied in an agricultural mobile robot platform, under development at NEPAS/EESC/USP. For agricultural mobile robot locomotion, an architecture for the acquisition, image processing and analysis was built, in order to segment, classify and recognize patterns of planting rows, as references way points for guiding the mobile robot. In the second approach, such image processing techniques were applied also in the inspection and location of the orange crop (primary) and maize crop (secondary) aiming its natural features, location and quantification. For both mentioned approaches, the adopted image processing steps include: filtering in the spatial domain for acquired images; pre-processing in RGB and HSV color spaces; JSEG unsupervised segmentation algorithm, applied to color quantization in non-homogeneous regions; normalization and histograms feature extraction of preprocessed images for training and test sets, fulfilled by the principal components analysis (PCA); pattern recognition and cognitive and statistical classification. The developed methodology includes sets of 700 and 900 images databases for each approach of natural scenes under different conditions of acquisition, providing great results on the segmentation algorithm, but not as appropriate as in the location of maize grass, considering other segmentation techniques, applied not only in the quantization of non-homogeneous regions. Statistical classification, Bayes and Naive Bayes, outperforms the cognitives Fuzzy and ANN on two approaches and subsequent class maps construction in HSV color space. Quantification and localization of fruits had more accurate results in HSV than RGB. Thus, natural scenes in two approaches were properly processed, according to the materials and methods employed in segmentation, classification and pattern recognition, providing intrinsic and different features of the proposed computer vision techniques to each approach.