The introduction of input devices with natural user interfaces in gaming hardware has changed the way we interact with games. Hardware with motion-sensing and gesture recognizing capabilities remove the constraint of interacting with games through typical traditional devices like mouse-keyboard and gamepads. This changes the way we approach games and how the game communicates back to us as the player opening new levels of interactivity. This thesis covers how eye tracker technology can be used to affect rendering effects in games.

Shader programming is important for realistic material and global illumination real-time rendering, especially in 3D industrial fields nowadays, more and more customers of Visual Components Oy, a Finnish 3D software company have been found to be no longer only content with the correct simulation result, but also the result of realistic real-time rendering. This thesis project will provide a deep research on real world material classification, property definition and global illumination techniques in industrial fields. On the other hand, the Shader program for different materials and global illumination techniques are also created according to the classification and definition in this thesis work. Moreover, an external rendering tool Redway3D is evaluated as the reference and regarded as the considerable solution in the future development work.

In this thesis, we discuss three image processing topics: High Dynamic Range (HDR) image creation in scenes with motion, Tone Mapping (TM), and Demosaicking. The first part of this thesis focuses on the creation of HDR images using gradient fusion techniques, and proposes a method that deals with motion and avoids bleeding and ghost artifacts. In the second part, we tackle the TM problem, whose goal is to produce a low dynamic range picture from an HDR image that reproduces the sensation of an observer in the scene. We review the perceptual principles that we find important for TM purposes and present a new method that compares well to the state of the art. Finally, we propose a new method to reconstruct the three color channels of a picture taken with a Bayer filter. This problem is called Demosaicking and will be presented in the third part of this thesis. / En esta tesis tratamos tres temas de procesamiento de imagen: creación de imágenes de alto rango dinámico o HDR, Tone Mapping (TM) y Demosaicking. En la primera parte proponemos un método para la creación de imágenes HDR con movimiento que permite generar resultados sin artefactos de tipo bleeding y ghosting. En la segunda parte de la tesis tratamos el problema de TM cuyo objetivo es comprimir el rango dinámico de una imagen HDR para ser mostrada en una pantalla o impresa, simulando lo mejor posible la percepción de un sujeto en la escena. Presentaremos los principios sicofísicos que consideramos relevantes para TM y propondremos un método nuevo que mejora los resultados del estado del arte. Finalmente, en la tercera parte presentamos un método de Demosaicking o reconstrucción de los tres canales de color de una imagen tomada con un filtro de Bayer.

A síntese de imagens com alta faixa dinâmica é uma prática cada vez mais comum em computação gráfica. O desafio consiste em relacionar o grande conjunto de intensidades da imagem sintetizada com um sub-conjunto muito inferior suportado por um dispositivo de exibição, evitando a perda de detalhes contrastivos. Os operadores locais de reprodução de tons (local tone-mapping operators) são capazes de realizar tal compressão, adaptando o nível de luminância de cada pixel com respeito à sua vizinhança. Embora produzam resultados significativamente superiores aos operadores globais, o custo computacional é consideravelmente maior, o que vem impedindo sua utilização em aplicações em tempo real. Este trabalho apresenta uma técnica para aproximar o operador fotográfico local de reprodução de tons. Todas as etapas da técnica são implementadas em GPU, adequando-se ao cenário de aplicações em tempo real, sendo significativamente mais rápida que implementações existentes e produzindo resultados semelhantes. A abordagem é baseada no uso de tabelas de áreas acumuladas (summed-area tables) para acelerar a convolução das vizinhanças, usando filtros da média (box-filter), proporcionando uma solução elegante para aplicações que utilizam imagens em alta faixa dinâmica e que necessitam de performance sem comprometer a qualidade da imagem sintetizada. Uma investigação sobre algoritmos para a geração de somatórios pré-fixados (prefix sum) e uma possível melhoria para um deles também são apresentada. / High dynamic range (HDR) rendering is becoming an increasingly popular technique in computer graphics. Its challenge consists on mapping the resulting images’ large range of intensities to the much narrower ones of the display devices in a way that preserves contrastive details. Local tone-mapping operators effectively perform the required compression by adapting the luminance level of each pixel with respect to its neighborhood. While they generate significantly better results when compared to global operators, their computational costs are considerably higher, thus preventing their use in real-time applications. This work presents a real-time technique for approximating the photographic local tone reproduction that runs entirely on the GPU and is significantly faster than existing implementations that produce similar results. Our approach is based on the use of summed-area tables for accelerating the convolution of the local neighborhoods with a box filter and provides an attractive solution for HDR rendering applications that require high performance without compromising image quality. A survey of prefix sum algorithms and possible improvements are also presented.

A síntese de imagens com alta faixa dinâmica é uma prática cada vez mais comum em computação gráfica. O desafio consiste em relacionar o grande conjunto de intensidades da imagem sintetizada com um sub-conjunto muito inferior suportado por um dispositivo de exibição, evitando a perda de detalhes contrastivos. Os operadores locais de reprodução de tons (local tone-mapping operators) são capazes de realizar tal compressão, adaptando o nível de luminância de cada pixel com respeito à sua vizinhança. Embora produzam resultados significativamente superiores aos operadores globais, o custo computacional é consideravelmente maior, o que vem impedindo sua utilização em aplicações em tempo real. Este trabalho apresenta uma técnica para aproximar o operador fotográfico local de reprodução de tons. Todas as etapas da técnica são implementadas em GPU, adequando-se ao cenário de aplicações em tempo real, sendo significativamente mais rápida que implementações existentes e produzindo resultados semelhantes. A abordagem é baseada no uso de tabelas de áreas acumuladas (summed-area tables) para acelerar a convolução das vizinhanças, usando filtros da média (box-filter), proporcionando uma solução elegante para aplicações que utilizam imagens em alta faixa dinâmica e que necessitam de performance sem comprometer a qualidade da imagem sintetizada. Uma investigação sobre algoritmos para a geração de somatórios pré-fixados (prefix sum) e uma possível melhoria para um deles também são apresentada. / High dynamic range (HDR) rendering is becoming an increasingly popular technique in computer graphics. Its challenge consists on mapping the resulting images’ large range of intensities to the much narrower ones of the display devices in a way that preserves contrastive details. Local tone-mapping operators effectively perform the required compression by adapting the luminance level of each pixel with respect to its neighborhood. While they generate significantly better results when compared to global operators, their computational costs are considerably higher, thus preventing their use in real-time applications. This work presents a real-time technique for approximating the photographic local tone reproduction that runs entirely on the GPU and is significantly faster than existing implementations that produce similar results. Our approach is based on the use of summed-area tables for accelerating the convolution of the local neighborhoods with a box filter and provides an attractive solution for HDR rendering applications that require high performance without compromising image quality. A survey of prefix sum algorithms and possible improvements are also presented.

A síntese de imagens com alta faixa dinâmica é uma prática cada vez mais comum em computação gráfica. O desafio consiste em relacionar o grande conjunto de intensidades da imagem sintetizada com um sub-conjunto muito inferior suportado por um dispositivo de exibição, evitando a perda de detalhes contrastivos. Os operadores locais de reprodução de tons (local tone-mapping operators) são capazes de realizar tal compressão, adaptando o nível de luminância de cada pixel com respeito à sua vizinhança. Embora produzam resultados significativamente superiores aos operadores globais, o custo computacional é consideravelmente maior, o que vem impedindo sua utilização em aplicações em tempo real. Este trabalho apresenta uma técnica para aproximar o operador fotográfico local de reprodução de tons. Todas as etapas da técnica são implementadas em GPU, adequando-se ao cenário de aplicações em tempo real, sendo significativamente mais rápida que implementações existentes e produzindo resultados semelhantes. A abordagem é baseada no uso de tabelas de áreas acumuladas (summed-area tables) para acelerar a convolução das vizinhanças, usando filtros da média (box-filter), proporcionando uma solução elegante para aplicações que utilizam imagens em alta faixa dinâmica e que necessitam de performance sem comprometer a qualidade da imagem sintetizada. Uma investigação sobre algoritmos para a geração de somatórios pré-fixados (prefix sum) e uma possível melhoria para um deles também são apresentada. / High dynamic range (HDR) rendering is becoming an increasingly popular technique in computer graphics. Its challenge consists on mapping the resulting images’ large range of intensities to the much narrower ones of the display devices in a way that preserves contrastive details. Local tone-mapping operators effectively perform the required compression by adapting the luminance level of each pixel with respect to its neighborhood. While they generate significantly better results when compared to global operators, their computational costs are considerably higher, thus preventing their use in real-time applications. This work presents a real-time technique for approximating the photographic local tone reproduction that runs entirely on the GPU and is significantly faster than existing implementations that produce similar results. Our approach is based on the use of summed-area tables for accelerating the convolution of the local neighborhoods with a box filter and provides an attractive solution for HDR rendering applications that require high performance without compromising image quality. A survey of prefix sum algorithms and possible improvements are also presented.

L'un des buts principaux de l'imagerie numérique est d'une part la capture et d'autre part la reproduction de scènes réelles ou synthétiques sur des dispositifs d'affichage aux capacités restreintes. Les techniques d'imagerie traditionnelles sont limitées par la gamme de luminance qu'elles peuvent capturer et afficher. L'imagerie à grande gamme de luminance (High Dynamic Range – HDR) vise à dépasser cette limitation en capturant, représentant et affichant les quantités physique de la lumière présente dans une scène. Cependant, les technologies d'affichage existantes ne vont pas disparaitre instantanément, la compatibilité entre ces nouveaux contenus HDR et les contenus classiques est donc requise. Cette compatibilité est assurée par une opération de réduction des gammes de luminance (tone mapping) qui adapte les contenus HDR aux capacités restreintes des écrans. Bien que de nombreux opérateurs de tone mapping existent, ceux-ci se focalisent principalement sur les images fixes. Les verrous scientifiques associés au tone mapping de vidéo HDR sont plus complexes du fait de la dimension temporelle. Les travaux recherche menés dans la thèse se sont focalisés sur la préservation de la cohérence temporelle du vidéo tone mapping. Deux principaux axes de recherche ont été traités : la qualité subjective de contenus tone mappés et l'efficacité de la compression des vidéos HDR. En effet, tone mapper individuellement chaque image d'une séquence vidéo HDR engendre des artefacts temporels. Ces artefacts affectent la qualité visuelle de la vidéo tone mappée et il est donc nécessaire de les minimiser. Au travers de tests effectués sur des vidéos HDR avec différents opérateurs de tone mapping, nous avons proposé une classification des artefacts temporels en six catégories. Après avoir testé les opérateurs de tone mapping vidéo existants sur les différents types d'artefacts temporels, nous avons observé que seulement trois des six types d'artefacts étaient résolus. Nous avons donc créé une technique de post-traitement qui permet de réduire les 3 types d'artefacts non-considérés. Le deuxième aspect considéré dans la thèse concerne les relations entre compression et tone mapping. Jusque là, les travaux effectués sur le tone mapping et la vidéo compression se focalisaient sur l'optimisation du tone mapping de manière à atteindre des taux de compression élevés. Ces techniques modifient fortement le rendu, c'est à dire l'aspect de la vidéo, modifiant ainsi l'intention artistique initiale en amont dans la chaine de distribution (avant la compression). Dans ce contexte, nous avons proposé une technique qui permet de réduire l'entropie d'une vidéo tone mappée sans en modifier son rendu. Notre méthode adapte la quantification afin d'accroitre les corrélations entre images successives d'une vidéo. / One of the main goals of digital imagery is to improve the capture and the reproduction of real or synthetic scenes on display devices with restricted capabilities. Standard imagery techniques are limited with respect to the dynamic range that they can capture and reproduce. High Dynamic Range (HDR) imagery aims at overcoming these limitations by capturing, representing and displaying the physical value of light measured in a scene. However, current commercial displays will not vanish instantly hence backward compatibility between HDR content and those displays is required. This compatibility is ensured through an operation called tone mapping that retargets the dynamic range of HDR content to the restricted dynamic range of a display device. Although many tone mapping operators exist, they focus mostly on still images. The challenges of tone mapping HDR videos are more complex than those of still images since the temporal dimensions is added. In this work, the focus was on the preservation of temporal coherency when performing video tone mapping. Two main research avenues are investigated: the subjective quality of tone mapped video content and their compression efficiency. Indeed, tone mapping independently each frame of a video sequence leads to temporal artifacts. Those artifacts impair the visual quality of the tone mapped video sequence and need to be reduced. Through experimentations with HDR videos and Tone Mapping Operators (TMOs), we categorized temporal artifacts into six categories. We tested video tone mapping operators (techniques that take into account more than a single frame) on the different types of temporal artifact and we observed that they could handle only three out of the six types. Consequently, we designed a post-processing technique that adapts to any tone mapping operator and reduces the three types of artifact not dealt with. A subjective evaluation reported that our technique always preserves or increases the subjective quality of tone mapped content for the sequences and TMOs tested. The second topic investigated was the compression of tone mapped video content. So far, work on tone mapping and video compression focused on optimizing a tone map curve to achieve high compression ratio. These techniques changed the rendering of the video to reduce its entropy hence removing any artistic intent or constraint on the final results. That is why, we proposed a technique that reduces the entropy of a tone mapped video without altering its rendering.

The diploma thesis provides the theoretical background needed to understand the complex topic of high dynamic range, and deals with the implementation of an innovative method for temporal video tone-mapping. At first, high dynamic range image acquisition, storage and processing are described. Afterwards, a general introduction to tone-mapping, description of undesirable image artifacts and an overview of existing state-of-the-art algorithms are presented. The part which follows focuses on the implementation of the chosen method and proposes several additional enhancements of the initially selected algorithm. Based on the information provided, an application with an intuitive graphical interface, which can be used for temporal HDR video tone-mapping, has been created. Finally, the the achieved results are demonstrated and visually compared with existing methods for temporal video tone-mapping.

This dissertation focuses on the many issues that arise from the visual rendering problem. Of primary consideration is light transport simulation, which is known to be computationally expensive. Monte Carlo methods represent a simple and general class of algorithms often used for light transport computation. Unfortunately, the images resulting from Monte Carlo approaches generally suffer from visually unacceptable noise artifacts. The result of any light transport simulation is, by its very nature, an image of high dynamic range (HDR). This leads to the issues of the display of such images on conventional low dynamic range devices and the development of data compression algorithms to store and recover the corresponding large amounts of detail found in HDR images. This dissertation presents our contributions relevant to these issues. Our contributions to high dynamic range image processing include tone mapping and data compression algorithms. This research proposes and shows the efficacy of a novel level set based tone mapping method that preserves visual details in the display of high dynamic range images on low dynamic range display devices. The level set method is used to extract the high frequency information from HDR images. The details are then added to the range compressed low frequency information to reconstruct a visually accurate low dynamic range version of the image. Additional challenges associated with high dynamic range images include the requirements to reduce excessively large amounts of storage and transmission time. To alleviate these problems, this research presents two methods for efficient high dynamic range image data compression. One is based on the classical JPEG compression. It first converts the raw image into RGBE representation, and then sends the color base and common exponent to classical discrete cosine transform based compression and lossless compression, respectively. The other is based on the wavelet transformation. It first transforms the raw image data into the logarithmic domain, then quantizes the logarithmic data into the integer domain, and finally applies the wavelet based JPEG2000 encoder for entropy compression and bit stream truncation to meet the desired bit rate requirement. We believe that these and similar such contributions will make a wide application of high dynamic range images possible. The contributions to light transport simulation include Monte Carlo noise reduction, dynamic object rendering and complex scene rendering. Monte Carlo noise is an inescapable artifact in synthetic images rendered using stochastic algorithm. This dissertation proposes two noise reduction algorithms to obtain high quality synthetic images. The first one models the distribution of noise in the wavelet domain using a Laplacian function, and then suppresses the noise using a Bayesian method. The other extends the bilateral filtering method to reduce all types of Monte Carlo noise in a unified way. All our methods reduce Monte Carlo noise effectively. Rendering of dynamic objects adds more dimension to the expensive light transport simulation issue. This dissertation presents a pre-computation based method. It pre-computes the surface radiance for each basis lighting and animation key frame, and then renders the objects by synthesizing the pre-computed data in real-time. Realistic rendering of complex scenes is computationally expensive. This research proposes a novel 3D space subdivision method, which leads to a new rendering framework. The light is first distributed to each local region to form local light fields, which are then used to illuminate the local scenes. The method allows us to render complex scenes at interactive frame rates. Rendering has important applications in mixed reality. Consistent lighting and shadows between real scenes and virtual scenes are important features of visual integration. The dissertation proposes to render the virtual objects by irradiance rendering using live captured environmental lighting. This research also introduces a virtual shadow generation method that computes shadows cast by virtual objects to the real background. We finally conclude the dissertation by discussing a number of future directions for rendering research, and presenting our proposed approaches.

This thesis focuses on the high dynamic range (HDR) imaging problematic. It describes process of capturing and storage of HDR images in brief. Main part of thesis deals with techniques of how to display HDR images on common visual display devices, which are not able to represent them directly. This process is called tone-mapping. It is firstly described in general and than the text focuses on several techniques of tone-mapping, mainly on tone-mapping using bilateral filtering.