Real-world radiance values can range over eight orders of magnitude from starlight to direct sunlight but few digital cameras capture more than three orders in a single Low Dynamic Range (LDR) image. We approach this problem using established High Dynamic Range (HDR) techniques in which multiple images are captured with different exposure times so that all portions of the scene are correctly exposed at least once. These images are then combined to create an HDR image capturing the full range of the scene. HDR capture introduces new challenges; movement in the scene creates faded copies of moving objects, referred to as ghosts. Many techniques have been introduced to handle ghosting, but typically they either address specific types of ghosting, or are computationally very expensive. We address ghosting by first detecting moving objects, then reducing their contribution to the final composite on a frame-by-frame basis. The detection of motion is addressed by performing change detection on exposure-normalized images. Additional special cases are developed based on a priori knowledge of the changing exposures; for example, if exposure is increasing every shot, then any decrease in intensity in the LDR images is a strong indicator of motion. Recent Superpixel over-segmentation techniques are used to refine the detection. We also propose a novel solution for areas that see motion throughout the capture, such as foliage blowing in the wind. Such areas are detected as always moving, and are replaced with information from a single input image, and the replacement of corrupted regions can be tailored to the scenario. We present our approach in the context of a panoramic tele-presence system. Tele-presence systems allow a user to experience a remote environment, aiming to create a realistic sense of "being there" and such a system should therefore provide a high quality visual rendition of the environment. Furthermore, panoramas, by virtue of capturing a greater proportion of a real-world scene, are often exposed to a greater dynamic range than standard photographs. Both facets of this system therefore stand to benefit from HDR imaging techniques. We demonstrate the success of our approach on multiple challenging ghosting scenarios, and compare our results with state-of-the-art methods previously proposed. We also demonstrate computational savings over these methods.

Shader programming is important for realistic material and global illumination real-time rendering, especially in 3D industrial fields nowadays, more and more customers of Visual Components Oy, a Finnish 3D software company have been found to be no longer only content with the correct simulation result, but also the result of realistic real-time rendering. This thesis project will provide a deep research on real world material classification, property definition and global illumination techniques in industrial fields. On the other hand, the Shader program for different materials and global illumination techniques are also created according to the classification and definition in this thesis work. Moreover, an external rendering tool Redway3D is evaluated as the reference and regarded as the considerable solution in the future development work.

This thesis explores synthesis by example as a paradigm for rendering real-world phenomena. In particular, phenomena that can be visually described as texture are considered. We exploit, for synthesis, the self-repeating nature of the visual elements constituting these texture exemplars. Techniques for unconstrained as well as constrained/controllable synthesis of both image and video textures are presented. For unconstrained synthesis, we present two robust techniques that can perform spatio-temporal extension, editing, and merging of image as well as video textures. In one of these techniques, large patches of input texture are automatically aligned and seamless stitched with each other to generate realistic looking images and videos. The second technique is based on iterative optimization of a global energy function that measures the quality of the synthesized texture with respect to the given input exemplar. We also present a technique for controllable texture synthesis. In particular, it allows for generation of motion-controlled texture animations that follow a specified flow field. Animations synthesized in this fashion maintain the structural properties like local shape, size, and orientation of the input texture even as they move according to the specified flow. We cast this problem into an optimization framework that tries to simultaneously satisfy the two (potentially competing) objectives of similarity to the input texture and consistency with the flow field. This optimization is a simple extension of the approach used for unconstrained texture synthesis. A general framework for example-based synthesis and rendering is also presented. This framework provides a design space for constructing example-based rendering algorithms. The goal of such algorithms would be to use texture exemplars to render animations for which certain behavioral characteristics need to be controlled. Our motion-controlled texture synthesis technique is an instantiation of this framework where the characteristic being controlled is motion represented as a flow field.

Generating natural phenomena in a virtual world has a number of practical applications. Thanks to the rich and complicated details in the real world, the goal of realistically and efficiently reproducing natural phenomena is well known as an open problem for graphics researchers. In this dissertation, three different issues in modeling liquid animations have been addressed. First, a virtual surface method is proposed to account for surface tension effects and their interactions with solid surfaces in physically based fluid simulation. This allows us to generate various surface tension behaviors in small scale liquid. The second issue that is addressed is how to make small scale fluid simulation more efficient. The proposed solution is a general shallow wave equation model, extended from the original shallow wave equations. By simplifying 3D incompressible fluid dynamics into 2D, small scale liquid can be stably and efficiently simulated over arbitrarily curved surfaces using implicit numerical schemes. The third contribution is a novel hybrid framework that combines image based reconstruction techniques with physically based fluid simulation. While image based methods cannot correctly generate fluid animations alone frame by frame, physics is used as a refinement tool to enforce physical soundness by propagating shape information back and forth in space and time. In this way, water animations can be realistically and faithfully generated from images without error accumulation or stability issues.

Real-world radiance values can range over eight orders of magnitude from starlight to direct sunlight but few digital cameras capture more than three orders in a single Low Dynamic Range (LDR) image. We approach this problem using established High Dynamic Range (HDR) techniques in which multiple images are captured with different exposure times so that all portions of the scene are correctly exposed at least once. These images are then combined to create an HDR image capturing the full range of the scene. HDR capture introduces new challenges; movement in the scene creates faded copies of moving objects, referred to as ghosts. Many techniques have been introduced to handle ghosting, but typically they either address specific types of ghosting, or are computationally very expensive. We address ghosting by first detecting moving objects, then reducing their contribution to the final composite on a frame-by-frame basis. The detection of motion is addressed by performing change detection on exposure-normalized images. Additional special cases are developed based on a priori knowledge of the changing exposures; for example, if exposure is increasing every shot, then any decrease in intensity in the LDR images is a strong indicator of motion. Recent Superpixel over-segmentation techniques are used to refine the detection. We also propose a novel solution for areas that see motion throughout the capture, such as foliage blowing in the wind. Such areas are detected as always moving, and are replaced with information from a single input image, and the replacement of corrupted regions can be tailored to the scenario. We present our approach in the context of a panoramic tele-presence system. Tele-presence systems allow a user to experience a remote environment, aiming to create a realistic sense of "being there" and such a system should therefore provide a high quality visual rendition of the environment. Furthermore, panoramas, by virtue of capturing a greater proportion of a real-world scene, are often exposed to a greater dynamic range than standard photographs. Both facets of this system therefore stand to benefit from HDR imaging techniques. We demonstrate the success of our approach on multiple challenging ghosting scenarios, and compare our results with state-of-the-art methods previously proposed. We also demonstrate computational savings over these methods.

Image-based 3D reconstruction refers to the capture and virtual reconstruction of real scenes, through the use of ordinary camera sensors. A common approach is the use of the algorithms Structure from Motion, Multi-view Stereo and Poisson Surface Reconstruction, that fares well for many types of scenes. However, a problem that this pipeline suffers from is that it often falters when it comes to texture-less surfaces and areas, such as those found in man-made environments. Building facades, roads and walls often lack detail and easily trackable feature points, making this approach less than ideal for such scenes. To remedy this weakness, this thesis investigates an expanded approach, incorporating line segment detection and line cloud generation into the already existing point cloud-based pipeline. Texture-less objects such as building facades, windows and roofs are well-suited for line segment detection, and line clouds are fitting for encoding 3D positional data in scenes consisting mostly of objects featuring many straight lines. A number of approaches have been explored in order to determine the usefulness of line clouds in this context, each of them addressing different aspects of the reconstruction procedure.

Des optiques à longue focale ont été souvent utilisées dans le cinéma 2D et la télévision, soit dans le but de se rapprocher de la scène, soit dans le but de produire un effet esthétique grâce à la déformation de la perspective. Toutefois, dans le cinéma ou la télévision 3D, l'utilisation de longues focales crée le plus souvent un "effet carton” ou de la divergence oculaire.Pour résoudre ce problème, les méthodes de l'état de l'art utilisent des techniques de transformation de la disparité, qui sont une généralisation de l'interpolation de points de vue.Elles génèrent de nouvelles paires stéréoscopiques à partir des deux séquences d'images originales. Nous proposons d'utiliser plus de deux caméras pour résoudre les problèmes non résolus par les méthodes de transformation de la disparité.Dans la première partie de la thèse, nous passons en revue les causes de la fatigue visuelle et de l'inconfort visuel lors de la visualisation d'un film stéréoscopique. Nous modélisons alors la perception de la profondeur de la vision stéréoscopique d'une scène filmée en 3D avec deux caméras, et projetée dans une salle de cinéma ou sur un téléviseur 3D. Nous caractérisons mathématiquement cette distorsion 3D, et formulons les contraintes mathématiques associées aux causes de la fatigue visuelle et de l'inconfort. Nous illustrons ces distorsions 3D avec un nouveau logiciel interactif, la “salle de projection virtuelle".Afin de générer les images stéréoscopiques souhaitées, nous proposons d'utiliser le rendu basé image. Ces techniques comportent généralement deux étapes. Tout d'abord, les images d'entrée sont transformées vers la vue cible, puis les images transformées sont mélangées. Les transformations sont généralement calculés à l'aide d'une géométrie intermédiaire (implicite ou explicite). Le mélange d'images a été largement étudié dans la littérature et quelques heuristiques permettent d'obtenir de très bonnes performances.Cependant, la combinaison des heuristiques proposées n'est pas simple et nécessite du réglage manuel de nombreux paramètres.Dans cette thèse, nous proposons une nouvelle approche bayésienne au problème de synthèse de nouveaux points de vue, basé sur un modèle génératif.Le modèle génératif proposé tient compte de l'incertitude sur la transformation d'image. Le formalisme bayésien nous permet de déduire l'énergie du modèle génératif et de calculer les images désirées correspondant au maximum a posteriori. La méthode dépasse en termes de qualité les techniques de l'état de l'art du rendu basé image sur des jeux de données complexes. D'autre part, les équations de l'énergie fournissent une formalisation des heuristiques largement utilisés dans les techniques de rendu basé image.Le modèle génératif proposé aborde également le problème de la super-résolution, permettant de rendre des images à une résolution plus élevée que les images de départ.Dans la dernière partie de cette thèse, nous appliquons la nouvelle technique de rendu au cas du zoom stéréoscopique et nous montrons ses performances. / Optics with long focal length have been extensively used for shooting 2D cinema and television, either to virtually get closer to the scene or to produce an aesthetical effect through the deformation of the perspective. However, in 3D cinema or television, the use of long focal length either creates a “cardboard effect” or causes visual divergence. To overcome this problem, state-of-the-art methods use disparity mapping techniques, which is a generalization of view interpolation, and generate new stereoscopic pairs from the two image sequences. We propose to use more than two cameras to solve for the remaining issues in disparity mapping methods.In the first part of the thesis, we review the causes of visual fatigue and visual discomfort when viewing a stereoscopic film. We then model the depth perception from stereopsis of a 3D scene shot with two cameras, and projected in a movie theater or on a 3DTV. We mathematically characterize this 3D distortion, and derive the mathematical constraints associated with the causes of visual fatigue and discomfort. We illustrate these 3D distortions with a new interactive software, “The Virtual Projection Room”.In order to generate the desired stereoscopic images, we propose to use image-based rendering. Those techniques usually proceed in two stages. First, the input images are warped into the target view, and then the warped images are blended together. The warps are usually computed with the help of a geometric proxy (either implicit or explicit). Image blending has been extensively addressed in the literature and a few heuristics have proven to achieve very good performance. Yet the combination of the heuristics is not straightforward, and requires manual adjustment of many parameters.In this thesis, we propose a new Bayesian approach to the problem of novel view synthesis, based on a generative model taking into account the uncertainty of the image warps in the image formation model. The Bayesian formalism allows us to deduce the energy of the generative model and to compute the desired images as the Maximum a Posteriori estimate. The method outperforms state-of-the-art image-based rendering techniques on challenging datasets. Moreover, the energy equations provide a formalization of the heuristics widely used in image-based rendering techniques. Besides, the proposed generative model also addresses the problem of super-resolution, allowing to render images at a higher resolution than the initial ones.In the last part of this thesis, we apply the new rendering technique to the case of the stereoscopic zoom and show its performance.

Les champs de lumière ont attisé la curiosité durant ces dernières décennies. Capturés par une caméra plénoptique ou un ensemble de caméras, ils échantillonnent la fonction plénoptique qui informe sur la radiance de n'importe quel rayon lumineux traversant la scène observée. Les champs lumineux offrent de nombreuses applications en vision par ordinateur comme en infographie, de la reconstruction 3D à la segmentation, en passant par la synthèse de vue, l'inpainting ou encore le matting par exemple.Dans ce travail nous nous attelons au problème de reconstruction du champ de lumière dans le but de synthétiser une image, comme si elle avait été prise par une caméra plus proche du sujet de la scène que l'appareil de capture plénoptique. Notre approche consiste à formuler la reconstruction du champ lumineux comme un problème de rendu basé image (IBR). La plupart des algorithmes de rendu basé image s'appuient dans un premier temps sur une reconstruction 3D approximative de la scène, appelée proxy géométrique, afin d'établir des correspondances entre les points image des vues sources et ceux de la vue cible. Une nouvelle vue est générée par l'utilisation conjointe des images sources et du proxy géométrique, bien souvent par la projection des images sources sur le point de vue cible et leur fusion en intensité.Un simple mélange des couleurs des images sources ne garantit pas la cohérence de l'image synthétisée. Nous proposons donc une méthode de rendu direct multi-échelles basée sur les pyramides de laplaciens afin de fusionner les images sources à toutes les fréquences, prévenant ainsi l'apparition d'artefacts de rendu.Mais l'imperfection du proxy géométrique est aussi la cause d'artefacts de rendu, qui se traduisent par du bruit en haute fréquence dans l'image synthétisée. Nous introduisons une nouvelle méthode de rendu variationnelle avec des contraintes sur les gradients de l'image cible dans le but de mieux conditionner le système d'équation linéaire à résoudre et supprimer les artefacts de rendu dus au proxy.Certaines scènes posent de grandes difficultés de reconstruction du fait du caractère non-lambertien éventuel de certaines surfaces~; d'autre part même un bon proxy ne suffit pas, lorsque des réflexions, transparences et spécularités remettent en cause les règles de la parallaxe. Nous proposons méthode originale basée sur l'approximation locale de l'espace plénoptique à partir d'un échantillonnage épars afin de synthétiser n'importe quel point de vue sans avoir recours à la reconstruction explicite d'un proxy géométrique. Nous évaluons notre méthode à la fois qualitativement et quantitativement sur des scènes non-triviales contenant des matériaux non-lambertiens.Enfin nous ouvrons une discussion sur le problème du placement optimal de caméras contraintes pour le rendu basé image, et sur l'utilisation de nos algorithmes pour la vision d'objets dissimulés derrière des camouflages.Les différents algorithmes proposés sont illustrés par des résultats sur des jeux de données plénoptiques structurés (de type grilles de caméras) ou non-structurés. / Light fields have gathered much interest during the past few years. Captured from a plenoptic camera or a camera array, they sample the plenoptic function that provides rich information about the radiance of any ray passing through the observed scene. They offer a pletora of computer vision and graphics applications: 3D reconstruction, segmentation, novel view synthesis, inpainting or matting for instance.Reconstructing the light field consists in recovering the missing rays given the captured samples. In this work we cope with the problem of reconstructing the light field in order to synthesize an image, as if it was taken by a camera closer to the scene than the input plenoptic device or set of cameras. Our approach is to formulate the light field reconstruction challenge as an image-based rendering (IBR) problem. Most of IBR algorithms first estimate the geometry of the scene, known as a geometric proxy, to make correspondences between the input views and the target view. A new image is generated by the joint use of both the input images and the geometric proxy, often projecting the input images on the target point of view and blending them in intensity.A naive color blending of the input images do not guaranty the coherence of the synthesized image. Therefore we propose a direct multi-scale approach based on Laplacian rendering to blend the source images at all the frequencies, thus preventing rendering artifacts.However, the imperfection of the geometric proxy is also a main cause of rendering artifacts, that are displayed as a high-frequency noise in the synthesized image. We introduce a novel variational rendering method with gradient constraints on the target image for a better-conditioned linear system to solve, removing the high-frequency noise due to the geometric proxy.Some scene reconstructions are very challenging because of the presence of non-Lambertian materials; moreover, even a perfect geometric proxy is not sufficient when reflections, transparencies and specularities question the rules of parallax. We propose an original method based on the local approximation of the sparse light field in the plenoptic space to generate a new viewpoint without the need for any explicit geometric proxy reconstruction. We evaluate our method both quantitatively and qualitatively on non-trivial scenes that contain non-Lambertian surfaces.Lastly we discuss the question of the optimal placement of constrained cameras for IBR, and the use of our algorithms to recover objects that are hidden behind a camouflage.The proposed algorithms are illustrated by results on both structured (camera arrays) and unstructured plenoptic datasets.

[pt] Modelagem e renderização baseadas em imagem tem sido uma área de pesquisa muito ativa nas últimas décadas, tendo recebido grande atenção como uma alternativa às técnicas tradicionais de síntese de imagens baseadas primariamente em geometria. Nesta área, algoritmos de visão computacional são usados para processar e interpretar fotos ou vídeos do mundo real a fim de construir um modelo representativo de uma cena, ao passo que técnicas de computação gráfica são usadas para tomar proveito desta representação e criar cenas foto-realistas. O propósito deste trabalho é investigar técnicas de renderização capazes de gerar vistas virtuais de alta qualidade de uma cena, em tempo real. Para garantir a performance interativa do algoritmo, além de aplicar otimizações a métodos de renderização existentes, fazemos uso intenso da GPU para o processamento de geometria e das imagens para gerar as imagens finais. Apesar do foco deste trabalho ser a renderização, sem reconstruir o mapa de profundidade a partir das fotos, ele implicitamente contorna possíveis problemas na estimativa da profundidade para que as cenas virtuais geradas apresentem um nível aceitável de realismo. Testes com dados públicos são apresentados para validar o método proposto e para ilustrar deficiências dos métodos de renderização baseados em imagem em geral. / [en] Image-based modeling and rendering has been a very active research topic as a powerful alternative to traditional geometry-based techniques for image synthesis. In this area, computer vision algorithms are used to process and interpret real-world photos or videos in order to build a model of a scene, while computer graphics techniques use this model to create photorealistic images based on the captured photographs or videos. The purpose of this work is to investigate rendering techniques capable of delivering visually accurate virtual views of a scene in real-time. Even though this work is mainly focused on the rendering task, without the reconstruction of the depth map, it implicitly overcomes common errors in depth estimation, yielding virtual views with an acceptable level of realism. Tests with publicly available datasets are also presented to validate our framework and to illustrate some limitations in the IBR general approach.

SILVA JÚNIOR, Erasmo Artur da. Método dinâmico para troca de representação em sistemas híbridos de renderização de multidões. 2013. 52 f. : Dissertação (mestrado) - Universidade Federal do Ceará, Centro de Ciências, Departamento de Computação, Fortaleza-CE, 2013. / Submitted by guaracy araujo (guaraa3355@gmail.com) on 2016-06-21T19:52:02Z No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) / Approved for entry into archive by guaracy araujo (guaraa3355@gmail.com) on 2016-06-21T19:53:00Z (GMT) No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) / Made available in DSpace on 2016-06-21T19:53:00Z (GMT). No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) Previous issue date: 2013 / Environments populated with crowds are employed in various applications, such as games, simulators and editors. Many of these environments require not only a realistic and detailed rendering, but it must run smoothly in real-time. This task easily exhausts the system’s resources, even considering the current state-of-the-art hardware. Therefore, crowd rendering in real-time remains a challenge in computer graphics. Approaches exploiting levels of detail, visibility culling and image-based rendering are presented in order to facilitate this task. The first two increase the efficiency of rendering, but sometimes are not enough to keep an interactive frame rate. Some researches on this subject focus on image-based rendering techniques, specifically with the use of impostors. In this work it is proposed a method that balances the computational demand of rendering job by varying the threshold’s distance of the representation switch between full geometry (mesh) and image-based(impostors) models in accordance with the available resources. / Ambientes providos de multidões são empregados em diversas aplicações, como jogos, simuladores e editores. Muitas destas aplicações não requerem somente a renderização de agentes animados de forma realística e detalhada, mas que seja executada suavemente em tempo real, tarefa que facilmente esgota os recursos do sistema (mesmo considerando hardware no estado da arte). Por conta disso,a renderização de multidões em tempo real permanece como um desafio dentro da computação gráfica. Abordagens explorando nível de detalhe, descarte por visibilidade e renderização baseada em imagens foram propostas no intuito de viabilizar esta tarefa. As duas primeiras aumentam a eficiência da renderização, mas as vezes não são suficientes para manter taxas de quadros por segundo interativas. Grande parte dos estudos acerca do tema se concentra em técnicas de renderização baseadas em imagem, especificamente com o emprego de impostores. Neste trabalho é proposto um método que faz o balanço da demanda computacional da renderização através da variação da distância do limiar onde ocorre a troca de representação entre os modelos de geometria completa (malhas) e os baseados em imagem (impostores) de acordo com os recursos disponíveis.