Orientador: Prof. Dr. André Guilherme Ribeiro Balan / Dissertação (mestrado) - Universidade Federal do ABC, Programa de Pós-Graduação em Ciência da Computação, 2017. / Uma cena real possui uma grande variação de contraste que, quando vista pelo olho humano, resulta em detalhes que sensores de câmeras digitais comuns não conseguem capturar. Isso ocorre devido às limitações dos dispositivos para obter e exibir diferentes valores de cor. Imagens HDR (High Dynamic Range), por sua vez, são representações que conseguem reproduzir essa amplitude de valores. Para gerar e exibir imagens HDR, diante das limitações dos dispositivos, é necessário trabalhar em um domínio de menor alcance, com imagens LDR (Low Dynamic Range). Os algoritmos que mapeiam os valores entre os domínios são chamadas de operadores de tone-mapping. Apenas a aplicação de tone-mapping não gera resultados de alta qualidade, sendo necessárias técnicas de redução de ruídos e decomposição de imagem para tal. Essas técnicas implicam um alto custo computacional e demandam muito tempo quando executados na CPU. Por outro lado, o processamento na GPU oferece um paralelismo natural, por viabilizar operações a serem aplicadas a todos os pixels, simultaneamente. Uma das maneiras de programar essas operações na GPU é através do uso de shaders gráficos, alterando a forma que os pixels da imagem são reproduzidos. Com o constante crescimento da utilização de dispositivos móveis, um tema recorrente é o desempenho e a viabilidade de aplicações de alta performance em tais dispositivos, que atualmente, na maioria dos casos, possuem em sua arquitetura uma GPU programável. Nesse trabalho, desenvolvemos shaders gráficos OpenGL para processar operações de tonemapping, bem como a decomposição multiescala de imagens utilizando filtros não lineares importantes e modernos, a fim de preservar a maioria dos detalhes das imagens. Isso gera resultados mais nítidos quando comparados com técnicas que aplicam os operadores de tone-mapping diretamente nas imagens. Por outro lado, o processamento em GPU representa uma enorme melhoria de velocidade em relação ao processamento da CPU. A aplicação que desenvolvemos é multiplataforma para que ele possa ser executado em desktops e dispositivos móveis. Utilizamos a aplicação para avaliar o desempenho de diferentes operadores de tone-mapping e diferentes filtros de imagem não lineares para executar a decomposição de imagens em vários níveis. / A typical scene may have a highly nonuniform illumination that common digital camera sensors are currently not able to deal with, as well as typical screen monitors. A High Dynamic Range image (HDR) is an image model capable to store much larger illumination range than regular models, what is more similar to our human system view. To generate and display HDR images, given the limitations of the devices (cameras and screen monitors), it is necessary to work in a domain with smaller range, called LDR images (Low Dynamic Range). The algorithms that map HDR images to LDR images are called tone-mapping operators. These algorithms, when operating on very high resolution HDR images, demand very high computational effort that CPU are also not currently capable to deal with. On the other hand, GPU offers a natural parallelism by enabling operations to be applied on thousands of pixels simultaneously. One way to program these operations on the GPU is through the use of graphics shaders, directly changing a graphical pipeline that reproduce pixels of the image, such as OpenGL pipeline. Nowadays, mobile devices are also highly available devices that can have powerful GPUs. Hence, an important research subject is to access the viability of using such devices on HDR image processing and tone-mapping. In this work, we develop OpenGL graphic shaders to process tone-mapping operation as well as image multiscale decomposition using important and modern nonlinear image filters, in order to preserve the most of the images details. This generates sharper results when compared to techniques that directly apply tone-mapping operators on the images. On the other hand, GPU processing represents a huge speed improvement over CPU processing. The application we develop is multiplatform so it can run on desktops and mobile devices. We used it to evaluate the performance of different tone-mapping operators and different nonlinear image filters to perform image multiscale decomposition.

Les travaux de cette thèse sont menés dans le cadre d’un projet porté par le centre de recherche LGI2P, (IMT Mines Ales). L’objectif final de ce projet vise à permettre le retour à l’emploi et à améliorer le quotidien des personnes malvoyantes atteintes de rétinite pigmentaire et de glaucome. Le dispositif final est conçu pour aider les personnes dans leurs activités en mobilité : détection d’obstacles, recherche d’indices visuels, en adressant les problèmes liés à l’éblouissement et à l’héméralopie dont souffrent ces utilisateurs potentiels.La contribution de cette thèse au projet se situe sur plusieurs plans. Tout d’abord, il était demandé de définir les paramètres caractérisant la vision résiduelle de chaque utilisateur. En effet, chaque patient a ses propres conditions de confort lumineux, qui dépendent en particulier de l’état d’avancement de sa pathologie : à chaque étape de l’évolution de leur maladie, les patients ont des limites spécifiques de luminosité minimale en dessous de laquelle ils ne perçoivent plus les détails dans une scène, et aussi de la luminosité maximale au-dessus de laquelle ils ressentent gêne et douleur. La définition de ces limitations en luminosité va permettre de paramétrer le dispositif et l’adapter à chaque utilisateur. Mais il n’existe pas de méthode pour mesurer ces niveaux de luminance limite. Nous avons donc participé à la conception et au développement de tests dédiés, et à la réalisation d’essais sur des sujets déficients visuels, dans le cadre d’un essai clinique piloté par le CHU de Nîmes et l’ARAMAV (institut spécialisé dans la rééducation fonctionnelle basse vision), pour l’aspect médical. Nous avons également proposé un nouveau test pour mesurer la sensibilité au contraste chromatique, toujours dans le but de mieux adapter les images affichées à la vision des utilisateurs.Nous avons ensuite travaillé sur la mise au point d’un prototype du dispositif (caméras et visiocasque de réalité virtuelle). Pour cela, nous avons dû choisir les équipements de capture et d’affichage d’images. Un travail de calibration colorimétrique sur ces équipements nous permet de relier grandeurs numériques (code RGB) et grandeurs physiques (luminance et chrominance). Cette étape est nécessaire pour réaliser les tests précités dans des conditions physiquement connues. Elle nous permet également de définir les caractéristiques physiques que devront posséder les équipements qui seront choisis pour réaliser le produit final, s’ils sont différents de ceux utilisés pendant nos travaux.Enfin, nous avons abordé la question des traitements à appliquer au signal capturé par la caméra. Nous avons proposé des traitements en temps réel sur la luminosité dans le but d’augmenter la luminosité dans les zones sombres de l’image et de baisser la luminosité dans les zones qui éblouissent le patient. Nous avons montré les limitations de l’imagerie classique et la nécessité de travailler sur des images HDR (high dynamic range) Nous avons comparé plusieurs méthodes pour permettre l’affichage de ces images HDR sur les écrans de plus faible dynamique, en recherchant les caractéristiques de l’image que ces méthodes doivent préserver au mieux, et en prenant en compte les performances visuelles des utilisateurs potentiels. Nous avons aussi proposé des traitements sur la couleur en augmentant le contraste et la saturation pour rendre les images mieux perceptibles par les patients qui souffrent de troubles de vision des couleurs. / This thesis is part of a project conducted by the LGI2P research center (IMT Mines Ales). The project's final aim is to help people with vision disorders suffering from retinitis pigmentosa and glaucoma get back to work and improve their daily lives. The final device is designed to help people in their mobility activities: detecting obstacles, searching for visual signals, by addressing problems related to dazzling and haemeralopia affecting these potential users.The research of this thesis has several contributions to the project. First of all, parameters characterizing the residual vision of each user had to be defined. Indeed, each patient has his own light comfort conditions, which depend in particular on his pathology's progress: at each stage of the evolution of their disease, patients have specific minimum luminosity limits below which they no longer perceive the details in a scene, and also on the maximum luminosity above which they feel discomfort and pain. The definition of these limitations in luminosity will make it possible to parameterize the device and adapt it to each user. But there is no method to measure these limiting luminance levels. We have therefore participated in the design and development of specialized tests, and in the conduct of trials on visually impaired subjects, as part of a clinical trial led by the Nîmes University Hospital and the ARAMAV (institute specializing in low vision functional rehabilitation), for medical research. We have also proposed a new test to measure sensitivity to chromatic contrast, always with the aim of better adjusting the images displayed to users' vision.Then, we developed a prototype of the device (cameras and virtual reality video headset). In order to achieve these results, we had to choose the image capture and display equipment. A colorimetric calibration work on these equipments allowed us to link digital quantities (RGB code) and physical quantities (luminance and chrominance). This stage is required to perform the above tests under physically known conditions. It also allowed us to define the physical characteristics of the equipment that would be selected to produce the final product, whether they are different from those required during our work.Finally, we discussed the processing to be applied to the signal captured by the camera. We have proposed real-time brightness treatments to increase brightness in dark areas of the image and decrease brightness in areas that dazzle the patient. We have presented the limitations of conventional imaging and the necessity to work on HDR (high dynamic range) images. We have compared several methods to allow the display of these HDR images on screens with lower dynamic range, looking for the image characteristics that these methods should better preserve, and taking into consideration the visual performance of potential users. We have also suggested color treatments by increasing contrast and saturation to make images more perceptible to patients with color vision disorders.

In this thesis, we discuss three image processing topics: High Dynamic Range (HDR) image creation in scenes with motion, Tone Mapping (TM), and Demosaicking. The first part of this thesis focuses on the creation of HDR images using gradient fusion techniques, and proposes a method that deals with motion and avoids bleeding and ghost artifacts. In the second part, we tackle the TM problem, whose goal is to produce a low dynamic range picture from an HDR image that reproduces the sensation of an observer in the scene. We review the perceptual principles that we find important for TM purposes and present a new method that compares well to the state of the art. Finally, we propose a new method to reconstruct the three color channels of a picture taken with a Bayer filter. This problem is called Demosaicking and will be presented in the third part of this thesis. / En esta tesis tratamos tres temas de procesamiento de imagen: creación de imágenes de alto rango dinámico o HDR, Tone Mapping (TM) y Demosaicking. En la primera parte proponemos un método para la creación de imágenes HDR con movimiento que permite generar resultados sin artefactos de tipo bleeding y ghosting. En la segunda parte de la tesis tratamos el problema de TM cuyo objetivo es comprimir el rango dinámico de una imagen HDR para ser mostrada en una pantalla o impresa, simulando lo mejor posible la percepción de un sujeto en la escena. Presentaremos los principios sicofísicos que consideramos relevantes para TM y propondremos un método nuevo que mejora los resultados del estado del arte. Finalmente, en la tercera parte presentamos un método de Demosaicking o reconstrucción de los tres canales de color de una imagen tomada con un filtro de Bayer.

[pt] A principal meta da computação gráfica é a síntese de imagens. Essas imagens podem ser tanto geradas por computador quanto ser resultado de manipulação digital de fotografias. Diferentes métodos para captura de imagens e fotografia digital mudaram a importância da imagem digital. Em relação a imagens geradas por computador, a busca por imagens mais realistas é importante para a indústria de filmes, de desenvolvimento de jogos entre outras. Uma das maiores revoluções na computação gráfica atual está relacionada a imagens High Dynamic Range. Essas imagens representam o próximo nível em termos de representação de imagens, uma vez que seus valores são verdadeiramente proporcionais às condições de iluminação em uma cena e são capazes de codificar a dynamic range encontrado no mundo real, fato impossível para as imagens tradicionais que apresentam 24 bits por pixel. Quando imagens high dynamic range são utilizadas para codificar as condições de iluminação em uma cena, elas são chamadas mapas de radiância ou mapas de iluminação. O foco principal dessa dissertação é mostrar técnicas de renderização em tempo real utilizando mapas de iluminação. Técnicas conhecidas como Image Based Lighting. Esse trabalho apresenta os conceitos de imagens high dynamic range, seus fundamentos físicos na teoria da luz, uma série de trabalhos importantes em manipulação dessas imagens e uma discussão sobre o pipeline de aplicações em tempo real que utilizam high dynamic range. Finalmente são apresentadas as técnicas para utilização de mapas de iluminação em alta precisão em tempo real. / [en] In 1997, the seminal work by Paul Debevec and Jitendra Malik on the generation of HDR (High Dynamic Range) images, from ordinary LDR (Low Dynamic Range) cameras, facilitated the generation of light probes enormously. In consequence, this caused a boom of works on the rendering of objects with images of light from the real world, which is known as Image- Based Lighting. The present dissertation aims to study this new area, trying to situate itself in the question of real-time compositing of synthetic objects in real images. This dissertation proposes a real-time rendering pipeline for 3D games, in the simple case of static scenes, adapting the non-real-time technique presented by Paul Debevec in 1998. There is no written work about this adaptation in the literature, although exists some reference to developments done by graphics card manufacturers in this direction. Also this dissertation presents an experiment with diffuse objects. Moreover, the author gives ideas towards the solution of shadow problems for diffuse objects.

Cette thèse présente des contributions sur différents aspects du ''display retargeting'' dans le cadre de l'imagerie HDR (pour High Dynamic Range imaging en anglais). Bien que les contributions soient diverses, elles sont motivées par notre conviction que la préservation de l'intention artistique et la prise en compte de caractéristiques en termes de perception du système visuel humain sont essentielles pour un ''display retargeting'' esthétiquement et visuellement confortable. / This thesis presents various contributions in display retargeting under the vast field of High Dynamic Range (HDR) imaging. The motivation towards this work is the conviction that by preserving artistic intent and considering insights from human visual system leads to aesthetic, comfortable and efficient display retargeting.

Les technologies de capture et d'affichage d'images ont fait un grand pas durant la dernière décennie en termes de capacités de reproduction de la couleur, de gamme de dynamique et de détails des scènes réelles. Les caméras et les écrans à large gamme de dynamique (HDR: high dynamic range) sont d'ores et déjà disponibles, offrant ainsi plus de flexibilité pour les créateurs de contenus afin de produire des scènes plus réalistes.Dans le même temps, à cause des limitations des appareils conventionnels, il existe un gap important en terme de reproduction de contenu et d'apparence colorée entre les deux technologies. Cela a accentué le besoin pour des algorithmes prenant en compte ces considérations et assurant aux créateurs de contenus une reproduction cross-média fidèle.Dans cette thèse, nous focalisons sur l'adaptation et la reproduction des contenus à gamme de dynamique standard sur des dispositifs HDR. Tout d'abord, les modèles d'apparence colorée ainsi que les opérateurs de mappage tonal inverse ont été étudiés subjectivement quant à leur fidélité couleur lors de l'expansion de contenus antérieurs. Par la suite, les attributs perceptuels de clarté, chroma et saturation ont été analysés pour des stimuli émissifs ayant des niveaux de luminance de plus hautes gammes et ce en menant une étude psychophysique basée sur la méthode de mise à l'échelle de partitions. Les résultats expérimentaux ont permis de mener à la définition d'un nouvel espace couleur HDR efficace et précis, optimisé pour les applications d'adaptation de la gamme de dynamique. Enfin, dans l'optique de récupérer les détails perdus lors de captures d'images standard et d'améliorer la qualité perçue du contenu antérieur avant d'être visualisé sur un dispositif HDR, deux approches de correction des zones surexposées ou ayant subi un écrêtage couleur ont été proposées. Les modèles et algorithmes proposés ont été évalués en comparaison avec une vérité terrain HDR, menant à la conclusion que les résultats obtenus sont plus proches des scènes réelles que les autres approches de la littérature. / Camera and display technologies have greatly advanced in their capacities of reproducing color, dynamic range and details of real scenes in recent years. New and powerful high dynamic range (HDR) camera and display technologies are currently available in the market and, recently, these new HDR technologies offer higher flexibility to content creators, allowing them to produce a more precise representation of real world scenes.At the same time, limitations of conventional camera and display technologies mean that there is a significant gap in terms of content and color appearance reproduction between new and existing technologies. These mismatches lead to an increased demand for appearance studies and algorithms which take such under consideration and help content creators to perform accurate cross-media reproductions.In this thesis we mainly considered the adaptation and reproduction of standard dynamic range content towards HDR displays. First, existing color appearance models and reverse tone mapping operators were subjectively studied for their color fidelity during dynamic range expansion of legacy contents. Then perceptual lightness, chroma and saturation attributes were analyzed for emissive stimuli with higher range of luminance levels using adapted psycho-visual experimental setups based on the partition scaling method. The experimental results lead to a new, more efficient and accurate HDR color space, specifically optimized for dynamic range adaptation applications. Finally, to recover lost details and enhance the visual quality of legacy content before visualizing on an HDR display, two methods for color-clipping and over-exposure correction were introduced. The models and algorithms presented, were evaluated relative to HDR ground truth content, showing that our results are closer to the real scene than can be achieved with previous methods.

Recent years have seen an increasing focus on electric energy consumption as well as ecology in relation to human health. It is crucial to maintain the so called lighting comfort which is influenced by a wide range of lighting parameters, including illuminance, brightness, balance of illuminance, and glare. Thanks to all these parameters we can design lighting systems in such a way as to prevent energy waste and far too rapid deterioration of eyesight. This study therefore focuses on glare evaluation of indoor and outdoor lighting systems. In order to comprehend the issue thoroughly, a detailed description of human eyes and their functions is provided, followed by definitions of glare and types of glare. An overview of different calculations for indoor and outdoor glare is provided, considering the evaluation of daylight glare in terms of complex lighting systems. The study mentions several options involving unconventional methods for measuring glare (brightness). In the analytical part of the study, outdoor glare on Kolejní Street and indoor glare at the Prof. Brauner Hall at VUT measurements were obtained using brightness analyzer (an unconventional method). The thesis is concluded with a comparison of the conventionally and unconventionally obtained measurements, considering their respective advantages.

The wavelet transform has been used for several decades and it is still an object of research - especially its recent modifications which are using the so-called second generation wavelets. It has several advantages over other integral transformations. The most important of them are the ability to localize both in time and frequency and an ability to decorrelate some real non-stationary signals such as images. For this reasons, the wavelet transform became an often used tool in many image processing tasks, for example in image compression, edge detection or contrast enhancement. In this thesis, the wavelet transform is explained, including the theoretical foundation and implementation for use with two-dimensional discrete signals. Some of the applications of the wavelet transform are presented and described. The wavelet transform showed to be suitable tool for edge detection, noise reduction, contrast enhancement and HDR compression.

Cette thèse se situe à l'interface de deux des sujets de recherche du LEPSi8S, la perception et la réalité virtuelle, appliqués aux transports routiers. L'objectif de la thèse est d'améliorer l'état de l'art concernant le rendu des images de synthèse pour les simulateurs de conduite. L'axe privilégié est le réalisme perceptif des images. L'approche retenue propose un mode de rendu High Dynamic Range, qui permet de générer une image en luminance. La technique proposée permet de réutiliser des environnements virtuels classiques, avec un minimum d'informations supplémentaires concernant les sources lumineuses. Les textures et matériaux existants sont utilisés pour un rendu aussi proche physiquement de la réalité que possible. Ensuite, l'image est traitée avec un opérateur de reproduction de tons, qui compresse la dynamique pour tenir compte des limites liées au dispositif d'affichage, tout en respectant autant que possible un réalisme perceptif du rendu. L'opérateur a été choisi de façon à ce qu'il soit adapté à la simulation de conduite, notamment pour les cas extrêmes (nuit, éblouissement, soleil rasant). Une simulation de l'éblouissement a également été implémentée. L'ensemble du rendu est temps réel, et a été intégré dans la boucle visuelle les simulateurs de conduite du LEPSiS. Enfin, des comparaisons réel-virtuel ont permis de montrer la qualité du rendu HDR obtenu. Des expérimentations avec sujets, sur des photographies (avec une référence réelle) et sur des vidéos, ont de plus montré les meilleures performances d'un opérateur doté d'un modèle visuel humain pour la simulation de conduite, notamment par sa capacité à s'adapter temporellement aux variations de luminance.

We describe the design and implementation of a high dynamic range (HDR) imaging system capable of capturing RGB color images with a dynamic range of 10,000,000 : 1 at 25 frames per second. We use a highly programmable camera unit with high throughput A/D conversion, data processing and data output. HDR acquisition is performed by multiple exposures in a continuous rolling shutter progression over the sensor. All the different exposures for one particular row of pixels are acquired head to tail within the frame time, which means that the time disparity between exposures is minimal, the entire frame time can be used for light integration and the longest expo- sure is almost the entire frame time. The system is highly configurable, and trade-offs are possible between dynamic range, precision, number of exposures, image resolution and frame rate.