A síntese de imagens com alta faixa dinâmica é uma prática cada vez mais comum em computação gráfica. O desafio consiste em relacionar o grande conjunto de intensidades da imagem sintetizada com um sub-conjunto muito inferior suportado por um dispositivo de exibição, evitando a perda de detalhes contrastivos. Os operadores locais de reprodução de tons (local tone-mapping operators) são capazes de realizar tal compressão, adaptando o nível de luminância de cada pixel com respeito à sua vizinhança. Embora produzam resultados significativamente superiores aos operadores globais, o custo computacional é consideravelmente maior, o que vem impedindo sua utilização em aplicações em tempo real. Este trabalho apresenta uma técnica para aproximar o operador fotográfico local de reprodução de tons. Todas as etapas da técnica são implementadas em GPU, adequando-se ao cenário de aplicações em tempo real, sendo significativamente mais rápida que implementações existentes e produzindo resultados semelhantes. A abordagem é baseada no uso de tabelas de áreas acumuladas (summed-area tables) para acelerar a convolução das vizinhanças, usando filtros da média (box-filter), proporcionando uma solução elegante para aplicações que utilizam imagens em alta faixa dinâmica e que necessitam de performance sem comprometer a qualidade da imagem sintetizada. Uma investigação sobre algoritmos para a geração de somatórios pré-fixados (prefix sum) e uma possível melhoria para um deles também são apresentada. / High dynamic range (HDR) rendering is becoming an increasingly popular technique in computer graphics. Its challenge consists on mapping the resulting images’ large range of intensities to the much narrower ones of the display devices in a way that preserves contrastive details. Local tone-mapping operators effectively perform the required compression by adapting the luminance level of each pixel with respect to its neighborhood. While they generate significantly better results when compared to global operators, their computational costs are considerably higher, thus preventing their use in real-time applications. This work presents a real-time technique for approximating the photographic local tone reproduction that runs entirely on the GPU and is significantly faster than existing implementations that produce similar results. Our approach is based on the use of summed-area tables for accelerating the convolution of the local neighborhoods with a box filter and provides an attractive solution for HDR rendering applications that require high performance without compromising image quality. A survey of prefix sum algorithms and possible improvements are also presented.

A síntese de imagens com alta faixa dinâmica é uma prática cada vez mais comum em computação gráfica. O desafio consiste em relacionar o grande conjunto de intensidades da imagem sintetizada com um sub-conjunto muito inferior suportado por um dispositivo de exibição, evitando a perda de detalhes contrastivos. Os operadores locais de reprodução de tons (local tone-mapping operators) são capazes de realizar tal compressão, adaptando o nível de luminância de cada pixel com respeito à sua vizinhança. Embora produzam resultados significativamente superiores aos operadores globais, o custo computacional é consideravelmente maior, o que vem impedindo sua utilização em aplicações em tempo real. Este trabalho apresenta uma técnica para aproximar o operador fotográfico local de reprodução de tons. Todas as etapas da técnica são implementadas em GPU, adequando-se ao cenário de aplicações em tempo real, sendo significativamente mais rápida que implementações existentes e produzindo resultados semelhantes. A abordagem é baseada no uso de tabelas de áreas acumuladas (summed-area tables) para acelerar a convolução das vizinhanças, usando filtros da média (box-filter), proporcionando uma solução elegante para aplicações que utilizam imagens em alta faixa dinâmica e que necessitam de performance sem comprometer a qualidade da imagem sintetizada. Uma investigação sobre algoritmos para a geração de somatórios pré-fixados (prefix sum) e uma possível melhoria para um deles também são apresentada. / High dynamic range (HDR) rendering is becoming an increasingly popular technique in computer graphics. Its challenge consists on mapping the resulting images’ large range of intensities to the much narrower ones of the display devices in a way that preserves contrastive details. Local tone-mapping operators effectively perform the required compression by adapting the luminance level of each pixel with respect to its neighborhood. While they generate significantly better results when compared to global operators, their computational costs are considerably higher, thus preventing their use in real-time applications. This work presents a real-time technique for approximating the photographic local tone reproduction that runs entirely on the GPU and is significantly faster than existing implementations that produce similar results. Our approach is based on the use of summed-area tables for accelerating the convolution of the local neighborhoods with a box filter and provides an attractive solution for HDR rendering applications that require high performance without compromising image quality. A survey of prefix sum algorithms and possible improvements are also presented.

A síntese de imagens com alta faixa dinâmica é uma prática cada vez mais comum em computação gráfica. O desafio consiste em relacionar o grande conjunto de intensidades da imagem sintetizada com um sub-conjunto muito inferior suportado por um dispositivo de exibição, evitando a perda de detalhes contrastivos. Os operadores locais de reprodução de tons (local tone-mapping operators) são capazes de realizar tal compressão, adaptando o nível de luminância de cada pixel com respeito à sua vizinhança. Embora produzam resultados significativamente superiores aos operadores globais, o custo computacional é consideravelmente maior, o que vem impedindo sua utilização em aplicações em tempo real. Este trabalho apresenta uma técnica para aproximar o operador fotográfico local de reprodução de tons. Todas as etapas da técnica são implementadas em GPU, adequando-se ao cenário de aplicações em tempo real, sendo significativamente mais rápida que implementações existentes e produzindo resultados semelhantes. A abordagem é baseada no uso de tabelas de áreas acumuladas (summed-area tables) para acelerar a convolução das vizinhanças, usando filtros da média (box-filter), proporcionando uma solução elegante para aplicações que utilizam imagens em alta faixa dinâmica e que necessitam de performance sem comprometer a qualidade da imagem sintetizada. Uma investigação sobre algoritmos para a geração de somatórios pré-fixados (prefix sum) e uma possível melhoria para um deles também são apresentada. / High dynamic range (HDR) rendering is becoming an increasingly popular technique in computer graphics. Its challenge consists on mapping the resulting images’ large range of intensities to the much narrower ones of the display devices in a way that preserves contrastive details. Local tone-mapping operators effectively perform the required compression by adapting the luminance level of each pixel with respect to its neighborhood. While they generate significantly better results when compared to global operators, their computational costs are considerably higher, thus preventing their use in real-time applications. This work presents a real-time technique for approximating the photographic local tone reproduction that runs entirely on the GPU and is significantly faster than existing implementations that produce similar results. Our approach is based on the use of summed-area tables for accelerating the convolution of the local neighborhoods with a box filter and provides an attractive solution for HDR rendering applications that require high performance without compromising image quality. A survey of prefix sum algorithms and possible improvements are also presented.

Real-world radiance values can range over eight orders of magnitude from starlight to direct sunlight but few digital cameras capture more than three orders in a single Low Dynamic Range (LDR) image. We approach this problem using established High Dynamic Range (HDR) techniques in which multiple images are captured with different exposure times so that all portions of the scene are correctly exposed at least once. These images are then combined to create an HDR image capturing the full range of the scene. HDR capture introduces new challenges; movement in the scene creates faded copies of moving objects, referred to as ghosts. Many techniques have been introduced to handle ghosting, but typically they either address specific types of ghosting, or are computationally very expensive. We address ghosting by first detecting moving objects, then reducing their contribution to the final composite on a frame-by-frame basis. The detection of motion is addressed by performing change detection on exposure-normalized images. Additional special cases are developed based on a priori knowledge of the changing exposures; for example, if exposure is increasing every shot, then any decrease in intensity in the LDR images is a strong indicator of motion. Recent Superpixel over-segmentation techniques are used to refine the detection. We also propose a novel solution for areas that see motion throughout the capture, such as foliage blowing in the wind. Such areas are detected as always moving, and are replaced with information from a single input image, and the replacement of corrupted regions can be tailored to the scenario. We present our approach in the context of a panoramic tele-presence system. Tele-presence systems allow a user to experience a remote environment, aiming to create a realistic sense of "being there" and such a system should therefore provide a high quality visual rendition of the environment. Furthermore, panoramas, by virtue of capturing a greater proportion of a real-world scene, are often exposed to a greater dynamic range than standard photographs. Both facets of this system therefore stand to benefit from HDR imaging techniques. We demonstrate the success of our approach on multiple challenging ghosting scenarios, and compare our results with state-of-the-art methods previously proposed. We also demonstrate computational savings over these methods.

L'un des buts principaux de l'imagerie numérique est d'une part la capture et d'autre part la reproduction de scènes réelles ou synthétiques sur des dispositifs d'affichage aux capacités restreintes. Les techniques d'imagerie traditionnelles sont limitées par la gamme de luminance qu'elles peuvent capturer et afficher. L'imagerie à grande gamme de luminance (High Dynamic Range – HDR) vise à dépasser cette limitation en capturant, représentant et affichant les quantités physique de la lumière présente dans une scène. Cependant, les technologies d'affichage existantes ne vont pas disparaitre instantanément, la compatibilité entre ces nouveaux contenus HDR et les contenus classiques est donc requise. Cette compatibilité est assurée par une opération de réduction des gammes de luminance (tone mapping) qui adapte les contenus HDR aux capacités restreintes des écrans. Bien que de nombreux opérateurs de tone mapping existent, ceux-ci se focalisent principalement sur les images fixes. Les verrous scientifiques associés au tone mapping de vidéo HDR sont plus complexes du fait de la dimension temporelle. Les travaux recherche menés dans la thèse se sont focalisés sur la préservation de la cohérence temporelle du vidéo tone mapping. Deux principaux axes de recherche ont été traités : la qualité subjective de contenus tone mappés et l'efficacité de la compression des vidéos HDR. En effet, tone mapper individuellement chaque image d'une séquence vidéo HDR engendre des artefacts temporels. Ces artefacts affectent la qualité visuelle de la vidéo tone mappée et il est donc nécessaire de les minimiser. Au travers de tests effectués sur des vidéos HDR avec différents opérateurs de tone mapping, nous avons proposé une classification des artefacts temporels en six catégories. Après avoir testé les opérateurs de tone mapping vidéo existants sur les différents types d'artefacts temporels, nous avons observé que seulement trois des six types d'artefacts étaient résolus. Nous avons donc créé une technique de post-traitement qui permet de réduire les 3 types d'artefacts non-considérés. Le deuxième aspect considéré dans la thèse concerne les relations entre compression et tone mapping. Jusque là, les travaux effectués sur le tone mapping et la vidéo compression se focalisaient sur l'optimisation du tone mapping de manière à atteindre des taux de compression élevés. Ces techniques modifient fortement le rendu, c'est à dire l'aspect de la vidéo, modifiant ainsi l'intention artistique initiale en amont dans la chaine de distribution (avant la compression). Dans ce contexte, nous avons proposé une technique qui permet de réduire l'entropie d'une vidéo tone mappée sans en modifier son rendu. Notre méthode adapte la quantification afin d'accroitre les corrélations entre images successives d'une vidéo. / One of the main goals of digital imagery is to improve the capture and the reproduction of real or synthetic scenes on display devices with restricted capabilities. Standard imagery techniques are limited with respect to the dynamic range that they can capture and reproduce. High Dynamic Range (HDR) imagery aims at overcoming these limitations by capturing, representing and displaying the physical value of light measured in a scene. However, current commercial displays will not vanish instantly hence backward compatibility between HDR content and those displays is required. This compatibility is ensured through an operation called tone mapping that retargets the dynamic range of HDR content to the restricted dynamic range of a display device. Although many tone mapping operators exist, they focus mostly on still images. The challenges of tone mapping HDR videos are more complex than those of still images since the temporal dimensions is added. In this work, the focus was on the preservation of temporal coherency when performing video tone mapping. Two main research avenues are investigated: the subjective quality of tone mapped video content and their compression efficiency. Indeed, tone mapping independently each frame of a video sequence leads to temporal artifacts. Those artifacts impair the visual quality of the tone mapped video sequence and need to be reduced. Through experimentations with HDR videos and Tone Mapping Operators (TMOs), we categorized temporal artifacts into six categories. We tested video tone mapping operators (techniques that take into account more than a single frame) on the different types of temporal artifact and we observed that they could handle only three out of the six types. Consequently, we designed a post-processing technique that adapts to any tone mapping operator and reduces the three types of artifact not dealt with. A subjective evaluation reported that our technique always preserves or increases the subjective quality of tone mapped content for the sequences and TMOs tested. The second topic investigated was the compression of tone mapped video content. So far, work on tone mapping and video compression focused on optimizing a tone map curve to achieve high compression ratio. These techniques changed the rendering of the video to reduce its entropy hence removing any artistic intent or constraint on the final results. That is why, we proposed a technique that reduces the entropy of a tone mapped video without altering its rendering.

This work focuses on analysing and improving feature detection and matching. After creating an initial framework of study, four main areas of work are researched. These areas make up the main chapters within this thesis and focus on using the Scale Invariant Feature Transform (SIFT).The preliminary analysis of the SIFT investigates how this algorithm functions. Included is an analysis of the SIFT feature descriptor space and an investigation into the noise properties of the SIFT. It introduces a novel use of the a contrario methodology and shows the success of this method as a way of discriminating between images which are likely to contain corresponding regions from images which do not. Parameter analysis of the SIFT uses both parameter sweeps and genetic algorithms as an intelligent means of setting the SIFT parameters for different image types utilising a GPGPU implementation of SIFT. The results have demonstrated which parameters are more important when optimising the algorithm and the areas within the parameter space to focus on when tuning the values. A multi-exposure, High Dynamic Range (HDR), fusion features process has been developed where the SIFT image features are matched within high contrast scenes. Bracketed exposure images are analysed and features are extracted and combined from different images to create a set of features which describe a larger dynamic range. They are shown to reduce the effects of noise and artefacts that are introduced when extracting features from HDR images directly and have a superior image matching performance. The final area is the development of a novel, 3D-based, SIFT weighting technique which utilises the 3D data from a pair of stereo images to cluster and class matched SIFT features. Weightings are applied to the matches based on the 3D properties of the features and how they cluster in order to attempt to discriminate between correct and incorrect matches using the a contrario methodology. The results show that the technique provides a method for discriminating between correct and incorrect matches and that the a contrario methodology has potential for future investigation as a method for correct feature match prediction.

Orientadores: Paulo Sergio Scarazzato, Hélio Pedrini / Tese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Civil, Arquitetura e Urbanismo / Made available in DSpace on 2018-08-26T13:49:42Z (GMT). No. of bitstreams: 1 Souza_DennisFloresde_D.pdf: 57790978 bytes, checksum: b6823404106d40d260ee7c43d24ecaa9 (MD5) Previous issue date: 2014 / Resumo: O uso de imagens de grande alcance dinâmico (HDR) nos estudos de iluminação vem se tornando um expediente frequente pela capacidade de armazenamento de dados referentes à distribuição de luminâncias em uma cena. Diversos estudos comprovaram, por exemplo, as possibilidades de registro da luz natural por imagens digitais, uma vez que as características das imagens HDR puderam melhorar os resultados. Dentre as diferentes aplicações, o registro da abóbada celeste é um dos que mais pode se beneficiar dessa ferramenta, pois tal procedimento é mais simples do que aqueles realizados a partir de medições feitas por luminancímetros ou escâneres de céu. Além disso, atualmente a identificação dos tipos de céu ainda é feita em sua maioria utilizando métodos subjetivos. Isto se deve ao fato de não existir uma metodologia para comparar imagens HDR com modelos matemáticos, apenas métodos unidimensionais que focalizam um ou outro aspecto. Esta pesquisa teve por objetivo desenvolver um método multidimensional de identificação, classificação e extração de dados de iluminação natural a partir de imagens HDR da abóbada celeste. As imagens das câmeras foram calibradas segundo métodos disponíveis para estabelecer a confiabilidade da análise e interpretação dos dados, e foram obtidas em localidade com o mínimo de obstrução à visão da abóbada celeste. O método multidimensional de análise foi desenvolvido juntamente a uma rotina em MATLAB, que serviu ao propósito de verificar sua viabilidade e a precisão. Os dados extraídos foram testados na plataforma Flash, usando a linguagem ActionScript 3, para brevemente demonstrar as possibilidades de uso. Este método utiliza um sistema de classificação baseado na relevância das características identificadas na imagem, como a cobertura de nuvens e a distribuição de luminâncias, para escolher o tipo de céu da norma ISO 15469:2004 (e) / CIE S 011/E:2003 mais apropriado. Os resultados apontam para a viabilidade desse método em escolher o tipo de céu mais relevante de acordo com os dados extraídos da imagem HDR. A proposição deste método multidimensional de análise pode contribuir para a criação de um sistema de classificação e de um banco de dados digital úteis para futuros programas de simulação, providenciando dados de entrada obtidos a partir de medições de uma realidade física, facilmente registrada com precisão e confiabilidade a partir de imagens fotográficas / Abstract: Lately, the high dynamic range images (HDR) have experienced a significant growth in their usage in lighting studies, due to their capacity to store data of luminance distribution in a scene. Various studies have attested, for instance, the possibilities of using digital images in the register of daylighting, since the features of HDR images could enhance the results. Among different applications, the record of the light on the sky vault is one that can benefit most from HDR techniques, because this procedure is simpler than those performed by luminance meters or sky scanners measurements. Besides, the identification and classification of sky types are still done mostly by subjective methods. This can be explained by the unavailability of a methodology able to compare HDR images with mathematical models, although there are unidimensional methods that focus on one or another aspect of digital images. This research aimed at the development of a multidimensional method of identification, classification and extraction of daylight data from HDR images of the sky vault. The images registered by the camera were calibrated using available methods to establish the reliability of the analysis and interpretation of data. They were then obtained on a site with minimal obstruction to the vision of the sky vault. The multidimensional analysis method was developed in conjunction with a routine in MATLAB, which served the purpose of verifying its feasibility and accuracy. The extracted data were tested in Flash platform using ActionScript 3 language to briefly demonstrate the usage possibilities. This method relies on a classification system based on the relevance of the features identified in the image, such as cloud covering and luminance distribution, to choose the most appropriate sky type according to ISO 15469:2004 (e) / CIE S 011/E:2003 Standard. The results demonstrate the feasibility of this method in choosing the most relevant sky type according to the data extracted from the HDR image. The proposition of this multidimensional analysis method may contribute to the creation of a classification system and a digital database useful for future simulation software, providing input data from measurements of a physical reality, easily recorded with accuracy and confidence by photographic images / Doutorado / Arquitetura, Tecnologia e Cidade / Doutor em Arquitetura, Tecnologia e Cidade

This work will be a web application for image and video quality assessment, especially those with a high dynamic range (HDR). The metrics are used for the evaluation, the results of which will be used for scientific purposes. Broader public citizens will be acquainted with this issue. This app will have an assistant available for a simple understanding of the whole application. The application also tries to be autonomous and dynamic. For example, new metrics can be added from the administration environment and editing on the server side is not required. The administrator will be able to upload a metric, a local library, a startup script, and define the parameters. The parameters provide metrics necessary data from the processed image or video. Only installation of system applications and libraries will be required on the server side.

Cílem práce je vytvoření podpory pro editovaní videa v HDR formátu. Pro editaci videa je zvolen program Adobe Premiere Pro a na dosažení požadovaného výsledku je vytvořen plugin do zmiňovaného softwaru, který poskytuje požadovanou funkcionalitu. V práci jsou vysvětleny principy vytváření, zobrazení a ukládáni obsahu z rozšíreným dynamickým rozsahem. Zároveň jsou vysvětleny principy vytváření pluginů pro Adobe Premiere Pro za pomoci SDK verze CS6. V praktické části této práce jsou vysvětleny detaily implementace, problémy, které byly řešené, a popis samotného pluginu. Návrh pluginů je vytvořen tak, aby byla možná další práce na tomto softwaru, přidaní další funkcionality a pro umožnění využití tohoto díla na rozvoji obsahu s rozšíreným dynamickým rozsahem.

Diese Arbeit befasst sich mit dem Entwurf eines asynchron arbeitenden, zeitbasierten CMOS–Bildsensors mit erhöhtem Dynamikbereich und Unterdrückung zeitlich redundanter Daten. Aufgrund immer kleinerer Strukturgrößen in modernen Prozessen zur Fertigung von Halbleitern und einer gleichzeitig physikalisch bedingt immer geringeren Skalierbarkeit konventioneller Bildsensoren wird es zunehmend möglich und praktikabel, Signalverarbeitungsansätze auf Pixelebene zu implementieren. Unter Berücksichtigung dieser Entwicklungen befasst sich die folgende Arbeit mit dem Entwurf eines neuartigen CMOS–Bildsensors mit nahezu vollständiger Unterdrückung zeitlich redundanter Daten auf Pixelebene. Jedes photosensitive Element in der Matrix arbeitet dabei vollkommen autonom. Es detektiert selbständig Änderungen in der Bestrahlung und gibt den Absolutwert nur beim Auftreten einer solchen Änderung mittels asynchroner Signalisierung nach außen. Darüber hinaus zeichnet sich der entwickelte Bildaufnehmer durch einen, gegenüber herkömmlichen Bildsensoren, deutlich erhöhten Dynamikbereich und eine niedrige Energieaufnahme aus, wodurch das Prinzip besonders für die Verwendung in Systemen für den mobilen Einsatz oder zur Durchführung von Überwachungsaufgaben geeignet ist. Die Realisierbarkeit des Konzepts wurde durch die erfolgreiche Implementierung eines entsprechenden Bildaufnehmers in einem Standard–CMOS–Prozess nachgewiesen. Durch die Größe des Designs von 304 x 240 Bildelementen, die den Umfang üblicher Prototypen-Realisierungen deutlich übersteigt, konnte speziell die Anwendbarkeit im Bereich größerer Sensorfelder gezeigt werden. Der Schaltkreis wurde erfolgreich getestet, wobei sowohl das Gesamtsystem als auch einzelne Schaltungsteile messtechnisch analysiert worden sind. Die nachgewiesene Bildqualität deckt sich dabei in guter Näherung mit den theoretischen Vorbetrachtungen.