[pt] Modelagem e renderização baseadas em imagem tem sido uma área de pesquisa muito ativa nas últimas décadas, tendo recebido grande atenção como uma alternativa às técnicas tradicionais de síntese de imagens baseadas primariamente em geometria. Nesta área, algoritmos de visão computacional são usados para processar e interpretar fotos ou vídeos do mundo real a fim de construir um modelo representativo de uma cena, ao passo que técnicas de computação gráfica são usadas para tomar proveito desta representação e criar cenas foto-realistas. O propósito deste trabalho é investigar técnicas de renderização capazes de gerar vistas virtuais de alta qualidade de uma cena, em tempo real. Para garantir a performance interativa do algoritmo, além de aplicar otimizações a métodos de renderização existentes, fazemos uso intenso da GPU para o processamento de geometria e das imagens para gerar as imagens finais. Apesar do foco deste trabalho ser a renderização, sem reconstruir o mapa de profundidade a partir das fotos, ele implicitamente contorna possíveis problemas na estimativa da profundidade para que as cenas virtuais geradas apresentem um nível aceitável de realismo. Testes com dados públicos são apresentados para validar o método proposto e para ilustrar deficiências dos métodos de renderização baseados em imagem em geral. / [en] Image-based modeling and rendering has been a very active research topic as a powerful alternative to traditional geometry-based techniques for image synthesis. In this area, computer vision algorithms are used to process and interpret real-world photos or videos in order to build a model of a scene, while computer graphics techniques use this model to create photorealistic images based on the captured photographs or videos. The purpose of this work is to investigate rendering techniques capable of delivering visually accurate virtual views of a scene in real-time. Even though this work is mainly focused on the rendering task, without the reconstruction of the depth map, it implicitly overcomes common errors in depth estimation, yielding virtual views with an acceptable level of realism. Tests with publicly available datasets are also presented to validate our framework and to illustrate some limitations in the IBR general approach.

SILVA JÚNIOR, Erasmo Artur da. Método dinâmico para troca de representação em sistemas híbridos de renderização de multidões. 2013. 52 f. : Dissertação (mestrado) - Universidade Federal do Ceará, Centro de Ciências, Departamento de Computação, Fortaleza-CE, 2013. / Submitted by guaracy araujo (guaraa3355@gmail.com) on 2016-06-21T19:52:02Z No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) / Approved for entry into archive by guaracy araujo (guaraa3355@gmail.com) on 2016-06-21T19:53:00Z (GMT) No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) / Made available in DSpace on 2016-06-21T19:53:00Z (GMT). No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) Previous issue date: 2013 / Environments populated with crowds are employed in various applications, such as games, simulators and editors. Many of these environments require not only a realistic and detailed rendering, but it must run smoothly in real-time. This task easily exhausts the system’s resources, even considering the current state-of-the-art hardware. Therefore, crowd rendering in real-time remains a challenge in computer graphics. Approaches exploiting levels of detail, visibility culling and image-based rendering are presented in order to facilitate this task. The first two increase the efficiency of rendering, but sometimes are not enough to keep an interactive frame rate. Some researches on this subject focus on image-based rendering techniques, specifically with the use of impostors. In this work it is proposed a method that balances the computational demand of rendering job by varying the threshold’s distance of the representation switch between full geometry (mesh) and image-based(impostors) models in accordance with the available resources. / Ambientes providos de multidões são empregados em diversas aplicações, como jogos, simuladores e editores. Muitas destas aplicações não requerem somente a renderização de agentes animados de forma realística e detalhada, mas que seja executada suavemente em tempo real, tarefa que facilmente esgota os recursos do sistema (mesmo considerando hardware no estado da arte). Por conta disso,a renderização de multidões em tempo real permanece como um desafio dentro da computação gráfica. Abordagens explorando nível de detalhe, descarte por visibilidade e renderização baseada em imagens foram propostas no intuito de viabilizar esta tarefa. As duas primeiras aumentam a eficiência da renderização, mas as vezes não são suficientes para manter taxas de quadros por segundo interativas. Grande parte dos estudos acerca do tema se concentra em técnicas de renderização baseadas em imagem, especificamente com o emprego de impostores. Neste trabalho é proposto um método que faz o balanço da demanda computacional da renderização através da variação da distância do limiar onde ocorre a troca de representação entre os modelos de geometria completa (malhas) e os baseados em imagem (impostores) de acordo com os recursos disponíveis.

SILVA JÚNIOR, Erasmo Artur da. Método dinâmico para troca de representação em sistemas híbridos de renderização de multidões. 2013. 61 f. Dissertação (Mestrado em ciência da computação)- Universidade Federal do Ceará, Fortaleza-CE, 2013. / Submitted by Elineudson Ribeiro (elineudsonr@gmail.com) on 2016-07-11T13:53:21Z No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) / Approved for entry into archive by Rocilda Sales (rocilda@ufc.br) on 2016-07-15T13:37:49Z (GMT) No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) / Made available in DSpace on 2016-07-15T13:37:49Z (GMT). No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) Previous issue date: 2013 / Environments populated with crowds are employed in various applications, such as games, simulators and editors. Many of these environments require not only a realistic and detailed rendering, but it must run smoothly in real-time. This task easily exhausts the system’s resources, even considering the current state-of-the-art hardware. Therefore, crowd rendering in real-time remains a challenge in computer graphics. Approaches exploiting levels of detail, visibility culling and image-based rendering are presented in order to facilitate this task. The first two increase the efficiency of rendering, but sometimes are not enough to keep an interactive frame rate. Some researches on this subject focus on image-based rendering techniques, specifically with the use of impostors. In this work it is proposed a method that balances the computational demand of rendering job by varying the threshold’s distance of the representation switch between full geometry (mesh) and image-based(impostors) models in accordance with the available resources. / Ambientes providos de multidões são empregados em diversas aplicações, como jogos, simuladores e editores. Muitas destas aplicações não requerem somente a renderização de agentes animados de forma realística e detalhada, mas que seja executada suavemente em tempo real, tarefa que facilmente esgota os recursos do sistema (mesmo considerando hardware no estado da arte). Por conta disso,a renderização de multidões em tempo real permanece como um desafio dentro da computação gráfica. Abordagens explorando nível de detalhe, descarte por visibilidade e renderização baseada em imagens foram propostas no intuito de viabilizar esta tarefa. As duas primeiras aumentam a eficiência da renderização, mas as vezes não são suficientes para manter taxas de quadros por segundo interativas. Grande parte dos estudos acerca do tema se concentra em técnicas de renderização baseadas em imagem, especificamente com o emprego de impostores. Neste trabalho é proposto um método que faz o balanço da demanda computacional da renderização através da variação da distância do limiar onde ocorre a troca de representação entre os modelos de geometria completa (malhas) e os baseados em imagem (impostores) de acordo com os recursos disponíveis.

Le travail présenté dans ce mémoire a pour cadre le dispositif de réalité mixte ray-on, conçu par la société on-situ. Ce dispositif, dédié à la mise en valeur du patrimoine architectural et en particulier d'édifices historiques, est installé sur le lieu de l'édifice et propose à l'utilisateur une vision uchronique de celui-ci. Le parti pris étant celui du photo-réalisme, deux pistes ont été suivies : l'amélioration du mélange réel virtuel par la reproduction de l'éclairage réel sur les objets virtuels, et la mise en place d'une méthode de segmentation d'image résiliente aux changements lumineux.Pour la reproduction de l'éclairage, une méthode de rendu basé-image est utilisée et associée à une capture haute dynamique de l'environnement lumineux. Une attention particulière est portée pour que ces deux phases soient justes photométriquement et colorimétriquement. Pour évaluer la qualité de la chaîne de reproduction de l'éclairage, une scène test constituée d'une mire de couleur calibrée est mise en place, et capturée sous de multiples éclairages par un couple de caméra, l'une capturant une image de la mire, l'autre une image de l'environnement lumineux. L'image réelle est alors comparée au rendu virtuel de la même scène, éclairée par cette seconde image.La segmentation résiliente aux changements lumineux a été développée à partir d'une classe d'algorithmes de segmentation globale de l'image, considérant celle-ci comme un graphe où trouver la coupe minimale séparant l'arrière plan et l'avant plan. L'intervention manuelle nécessaire à ces algorithmes a été remplacée par une pré-segmentation de moindre qualité à partir d'une carte de profondeur, cette pré-segmentation étant alors utilisée comme une graîne pour la segmentation finale. / The work described in this report has as a target the mixed reality device ray-on, developed by the on-situ company. This device, dedicated to cultural heritage and specifically architectural heritage, is meant to be installed on-site and shows the user an uchronic view of its surroundings. As the chosen stance is to display photo-realistic images, two trails were followed: the improvement of the real-virtual merging by reproducing accurately the real lighting on the virtual objects, and the development of a real-time segmentation method which is resilient to lighting changes.Regarding lighting reproduction, an image-based rendering method is used in addition to a high dynamic range capture of the lighting environment. The emphasis is put on the photometric and colorimetric correctness of these two steps. To measure the quality of the lighting reproduction chain, a test scene is set up with a calibrated color checker, captured by a camera while another camera is grabbing the lighting environment. The image of the real scene is then compared to the simulation of the same scene, enlightened by the light probe.Segmentation resilient to lighting changes is developed from a set of global image segmentation methods, which consider an image as a graph where a cut of minimal energy has to be found between the foreground and the background. These methods being semi-automatic, the manual part is replaced by a low resolution pre-segmentation based on the depthmap of the scene which is used as a seed for the final segmentation.

Nous présentons une approche de rendu à base d'images (IBR) qui permet, à partir de photos, de naviguer librement et générer des points de vue quelconques dans des scènes urbaines. Les approches précédentes dépendent des modèles 3D et donnent lieu à des rendus de qualité réduite avec beaucoup d'artefacts. Dans cette thèse, nous proposons une approximation basée sur l'image pour compenser le manque de précision de la géométrie 3D. Nous utilisons un warp d'image guidé par des cartes de profondeur quasi-denses qui donnent lieu à beaucoup moins d'artefacts. En se basant sur cette approche, nous avons développé une méthode entièrement automatique permettant de traiter les scènes complexes. Nous sur-segmentons les images d’entrées en superpixels qui limitent les occlusions sur les bords des objets. Nous introduisons la synthèse de profondeur pour créer une approximation de cette profondeur mal reconstruite dans certaines régions et calculons les warps sur les superpixels pour synthétiser le résultat final. Nous comparons nos résultats à de nombreuses approches récentes. Nous avons analysé les artefacts de l'IBR d'un point de vue perceptif en comparant les artefacts générés par le mélange de plusieurs images avec ceux des transitions temporelles brusques et avons élaboré une méthodologie pour la sélection d'un compromis idéal entre les deux. Nous avons également analysé les distorsions perspectives et avons développé un modèle quantitatif qui permet de prédire les distorsions en fonction des paramètres de capture et de visualisation. Comme application, nous avons mis en œuvre un système de réalité virtuelle qui utilise l'IBR à la place de l'infographie traditionnelle. / We present image-based rendering (IBR) approaches that allow free viewpoint walkthroughs of urban scenes using just a few photographs as input. Previous approaches depend upon 3D models which give artifacts as the quality of 3D model degrades. In this thesis, we propose image-based approximations to compensate for the lack of accurate 3D geometry. In the first project, we use discontinuous shape-preserving image warp guided by quasi-dense depth maps which gives far fewer rendering artifacts than previous approaches. We build upon this approach in the second project by developing a completely automated solution that is capable of handling more complex scenes. We oversegment input images into superpixels that capture all occlusion boundaries. We introduce depth synthesis to create approximate depth in very poorly reconstructed regions and compute shape-preserving warps on superpixels to synthesize the final result. We also compare our results to many recent approaches. We analyze IBR artifacts from a perceptual point of view. In the first study, we compare artifacts caused by blending multiple images with abrupt temporal transitions and develop guidelines for selecting the ideal tradeoff. We use vision science in another study to investigate perspective distortions and develop a quantitative model that predicts distortions as a function of capture and viewing parameters. We use guidelines from these experiments to motivate the design of our own IBR systems. We demonstrate the very first virtual reality system that uses IBR instead of traditional computer graphics. This drastically reduces the cost of modeling 3D scenes while producing highly realistic walkthroughs.

Le rendu photo-réaliste traditionnel exige un effort manuel et des calculs intensifs pour créer des scènes et rendre des images réalistes. C'est principalement pour cette raison que la création de contenus pour l’imagerie numérique de haute qualité a été limitée aux experts et le rendu hautement réaliste nécessite encore des temps de calcul significatifs. Le rendu basé image (IBR) est une alternative qui a le potentiel de rendre les applications de création et de rendu de contenus de haute qualité accessibles aux utilisateurs occasionnels, puisqu'ils peuvent générer des images photo-réalistes de haute qualité sans subir les limitations mentionnées ci-dessus. Nous avons identifié trois limitations importantes des méthodes actuelles de rendu basé image : premièrement, chaque algorithme possède des forces et faiblesses différentes, en fonction de la qualité de la reconstruction 3D et du contenu de la scène, et un seul algorithme ne permet souvent pas d’obtenir la meilleure qualité de rendu partout dans l’image. Deuxièmement, ces algorithmes présentent de forts artefacts lors du rendu d’objets manquants ou partiellement reconstruits. Troisièmement, la plupart des méthodes souffrent encore d'artefacts visuels significatifs dans les régions de l’image où la reconstruction est de faible qualité. Dans l'ensemble, cette thèse propose plusieurs améliorations significatives du rendu basé image aussi bien en termes de vitesse de rendu que de qualité d’image. Ces nouvelles solutions sont basées sur le rendu sélectif, la substitution de modèle basé sur l'apprentissage, et la prédiction et la correction des erreurs de profondeur. / Traditional photo-realistic rendering requires intensive manual and computational effort to create scenes and render realistic images. Thus, creation of content for high quality digital imagery has been limited to experts and highly realistic rendering still requires significant computational time. Image-Based Rendering (IBR) is an alternative which has the potential of making high-quality content creation and rendering applications accessible to casual users, since they can generate high quality photo-realistic imagery without the limitations mentioned above. We identified three important shortcomings of current IBR methods: First, each algorithm has different strengths and weaknesses, depending on 3D reconstruction quality and scene content and often no single algorithm offers the best image quality everywhere in the image. Second, such algorithms present strong artifacts when rendering partially reconstructed objects or missing objects. Third, most methods still result in significant visual artifacts in image regions where reconstruction is poor. Overall, this thesis addresses significant shortcomings of IBR for both speed and image quality, offering novel and effective solutions based on selective rendering, learning-based model substitution and depth error prediction and correction.

Real-world radiance values can range over eight orders of magnitude from starlight to direct sunlight but few digital cameras capture more than three orders in a single Low Dynamic Range (LDR) image. We approach this problem using established High Dynamic Range (HDR) techniques in which multiple images are captured with different exposure times so that all portions of the scene are correctly exposed at least once. These images are then combined to create an HDR image capturing the full range of the scene. HDR capture introduces new challenges; movement in the scene creates faded copies of moving objects, referred to as ghosts. Many techniques have been introduced to handle ghosting, but typically they either address specific types of ghosting, or are computationally very expensive. We address ghosting by first detecting moving objects, then reducing their contribution to the final composite on a frame-by-frame basis. The detection of motion is addressed by performing change detection on exposure-normalized images. Additional special cases are developed based on a priori knowledge of the changing exposures; for example, if exposure is increasing every shot, then any decrease in intensity in the LDR images is a strong indicator of motion. Recent Superpixel over-segmentation techniques are used to refine the detection. We also propose a novel solution for areas that see motion throughout the capture, such as foliage blowing in the wind. Such areas are detected as always moving, and are replaced with information from a single input image, and the replacement of corrupted regions can be tailored to the scenario. We present our approach in the context of a panoramic tele-presence system. Tele-presence systems allow a user to experience a remote environment, aiming to create a realistic sense of "being there" and such a system should therefore provide a high quality visual rendition of the environment. Furthermore, panoramas, by virtue of capturing a greater proportion of a real-world scene, are often exposed to a greater dynamic range than standard photographs. Both facets of this system therefore stand to benefit from HDR imaging techniques. We demonstrate the success of our approach on multiple challenging ghosting scenarios, and compare our results with state-of-the-art methods previously proposed. We also demonstrate computational savings over these methods.

No / This volume contains the papers presented at Computer Graphics International 2002, in July, at the University of Bradford, UK. These papers represent original research in computer graphics from around the world.

No description available.

<p dir="ltr">3D effect synthesis is crucial in image editing to enhance realism or visual appeal. Unlike classical graphics rendering, which relies on complete 3D geometries, 3D effect synthesis in im- age editing operates solely with 2D images as inputs. This shift presents significant challenges, primarily addressed by data-driven methods that learn to synthesize 3D effects in an end-to-end manner. However, these methods face limitations in the diversity of 3D effects they can produce and lack user control. For instance, existing shadow generation networks are restricted to produc- ing hard shadows without offering any user input for customization.</p><p dir="ltr">In this dissertation, we tackle the research question: <i>how can we synthesize controllable and realistic 3D effects in image editing when only 2D information is available? </i>Our investigation leads to four contributions. First, we introduce a neural network designed to create realistic soft shadows from an image cutout and a user-specified environmental light map. This approach is the first attempt in utilizing neural network for realistic soft shadow rendering in real-time. Second, we develop a novel 2.5D representation Pixel Height, tailored for the nuances of image editing. This representation not only forms the foundation of a new soft shadow rendering pipeline that provides intuitive user control, but also generalizes the soft shadow receivers to be general shadow receivers. Third, we present the mathematical relationship between the Pixel Height representation and 3D space. This connection facilitates the reconstruction of normals or depth from 2D scenes, broadening the scope for synthesizing comprehensive 3D lighting effects such as reflections and refractions. A 3D-aware buffer channels are also proposed to improve the synthesized soft shadow quality. Lastly, we introduce Dr.Bokeh, a differentiable bokeh renderer that extends traditional bokeh effect algorithms with better occlusion modeling to correct flaws existed in existing methods. With the more precise lens modeling, we show that Dr.Bokeh not only achieves the state-of-the-art bokeh rendering quality, but also pushes the boundary of depth-from-defocus problem.</p><p dir="ltr">Our work in controllable 3D effect synthesis represents a pioneering effort in image editing, laying the groundwork for future lighting effect synthesis in various image editing applications. Moreover, the improvements to filtering-based bokeh rendering could significantly enhance com- mercial products, such as the portrait mode feature on smartphones.</p>