No description available.

<p dir="ltr">3D effect synthesis is crucial in image editing to enhance realism or visual appeal. Unlike classical graphics rendering, which relies on complete 3D geometries, 3D effect synthesis in im- age editing operates solely with 2D images as inputs. This shift presents significant challenges, primarily addressed by data-driven methods that learn to synthesize 3D effects in an end-to-end manner. However, these methods face limitations in the diversity of 3D effects they can produce and lack user control. For instance, existing shadow generation networks are restricted to produc- ing hard shadows without offering any user input for customization.</p><p dir="ltr">In this dissertation, we tackle the research question: <i>how can we synthesize controllable and realistic 3D effects in image editing when only 2D information is available? </i>Our investigation leads to four contributions. First, we introduce a neural network designed to create realistic soft shadows from an image cutout and a user-specified environmental light map. This approach is the first attempt in utilizing neural network for realistic soft shadow rendering in real-time. Second, we develop a novel 2.5D representation Pixel Height, tailored for the nuances of image editing. This representation not only forms the foundation of a new soft shadow rendering pipeline that provides intuitive user control, but also generalizes the soft shadow receivers to be general shadow receivers. Third, we present the mathematical relationship between the Pixel Height representation and 3D space. This connection facilitates the reconstruction of normals or depth from 2D scenes, broadening the scope for synthesizing comprehensive 3D lighting effects such as reflections and refractions. A 3D-aware buffer channels are also proposed to improve the synthesized soft shadow quality. Lastly, we introduce Dr.Bokeh, a differentiable bokeh renderer that extends traditional bokeh effect algorithms with better occlusion modeling to correct flaws existed in existing methods. With the more precise lens modeling, we show that Dr.Bokeh not only achieves the state-of-the-art bokeh rendering quality, but also pushes the boundary of depth-from-defocus problem.</p><p dir="ltr">Our work in controllable 3D effect synthesis represents a pioneering effort in image editing, laying the groundwork for future lighting effect synthesis in various image editing applications. Moreover, the improvements to filtering-based bokeh rendering could significantly enhance com- mercial products, such as the portrait mode feature on smartphones.</p>

No / This volume contains the papers presented at Computer Graphics International 2002, in July, at the University of Bradford, UK. These papers represent original research in computer graphics from around the world.

Fotorealistisches Rendering ist eines der Hauptziele der Computer Grafik. Mittels physikalischer Simulation ist eine fotorealistische Darstellung immer noch rechenaufwändig. Diese Arbeit stellt neue Methoden für Bild-basiertes Rendering komplexer Objekte am Beispiel von Kleidung vor. Die vorgestellten Methoden nutzen Kamerabilder und deren fotorealistische Eigenschaften für komplexe Animationen und Texturmodifikationen. Basierend auf der Annahme, dass für eng anliegende Kleidung Faltenwurf hauptsächlich von der Pose des Trägers beeinflusst wird, schlägt diese Dissertation ein neues Bild-basiertes Verfahren vor, das neue Bilder von Kleidungsstücken abhängig von der Körperpose einer Person aus einer Datenbank von Bildern synthetisiert. Posen-abhängige Eigenschaften (Textur und Schattierung) werden über Abbildungsvorschriften zwischen den Bildern extrahiert und im Posenraum interpoliert. Um die Erscheinung eines Objekts zu verändern, wird ein Verfahren vorgestellt, das den Austausch von Texturen ohne Kenntnis der zugrundeliegenden Szeneneigenschaften ermöglicht. Texturdeformation und Schattierung werden über Bildregistrierung zu einem geeigneten Referenzbild extrahiert. Im Gegensatz zu klassischen Bild-basierten Verfahren, in denen die Synthese auf Blickpunktänderung beschränkt und eine Veränderung des Objekts nicht möglich ist, erlauben die vorgestellten Verfahren komplexe Animationen und Texturmodifikation. Beide Verfahren basieren auf örtlichen und photometrischen Abbildungen zwischen Bildern. Diese Abbildungen werden basierend auf einem angepassten Brightness Constancy Constraint mit Gitternetz-basierten Modellen optimiert. Die vorgestellten Verfahren verlagern einen großen Teil des Rechenaufwands von der Darstellungsphase in die vorangegangene Trainingsphase und erlauben eine realistische Visualisierung von Kleidung inklusive charakteristischer Details, ohne die zugrundeliegenden Szeneneigenschaften aufwändig zu simulieren. / One principal intention of computer graphics is the achievement of photorealism. With physically-based methods, achieving photorealism is still computationally demanding. This dissertation proposes new approaches for image-based visualization of complex objects, concentrating on clothes. The developed methods use real images as appearance examples to guide complex animation or texture modification processes, combining the photorealism of images with the ability to animate or modify an object. Under the assumption that wrinkling depends on the pose of a human body (for tight-fitting clothes), a new image-based rendering approach is proposed, which synthesizes images of clothing from a database of images based on pose information. Pose-dependent appearance and shading information is extracted by image warps and interpolated in pose-space using scattered data interpolation. To allow for appearance changes in image-based methods, a retexturing approach is proposed, which enables texture exchange without a-priori knowledge of the underlying scene properties. Texture deformation and shading are extracted from the input image by a warp to an appropriate reference image. In contrast to classical image-based visualization methods, where animation is restricted to viewpoint change and appearance modification is not possible, the proposed methods allow for complex pose animations and appearance changes. Both approaches build on image warps, not only in the spatial but also in the photometric domain. A new framework for joint spatial and photometric warp optimization is introduced, which estimates mesh-based warp models under a modified brightness constancy assumption. The presented approaches shift computational complexity from the rendering to an a-priori training phase and allow a photo-realistic visualization and modification of clothes, including fine and characteristic details without computationally demanding simulation of the underlying scene and object properties.

<div>In conventional computer graphics and visualization, images are synthesized following the planar pinhole camera (PPC) model. The PPC approximates physical imaging devices such as cameras and the human eye, which sample the scene with linear rays that originate from a single viewpoint, i.e. the pinhole. In addition, the PPC takes a snapshot of the scene, sampling it at a single instant in time, or timepoint, for each image. Images synthesized with these single viewpoint and single timepoint constraints are familiar to the user, as they emulate images captured with cameras or perceived by the human visual system. However, visualization using the PPC model suffers from the limitation of occlusion, when a region of interest (ROI) is not visible due to obstruction by other data. The conventional solution to the occlusion problem is to rely on the user to change the view interactively to gain line of sight to the scene ROIs. This approach of sequential navigation has the shortcomings of (1) inefficiency, as navigation is wasted when circumventing an occluder does not reveal an ROI, (2) inefficacy, as a moving or a transient ROI can hide or disappear before the user reaches it, or as scene understanding requires visualizing multiple distant ROIs in parallel, and (3) user confusion, as back-and-forth navigation for systematic scene exploration can hinder spatio-temporal awareness.</div><div><br></div><div>In this thesis we propose a novel paradigm for handling occlusions in visualization based on generalizing an image to incorporate samples from multiple viewpoints and multiple timepoints. The image generalization is implemented at camera model level, by removing the same timepoint restriction, and by removing the linear ray restriction, allowing for curved rays that are routed around occluders to reach distant ROIs. The paradigm offers the opportunity to greatly increase the information bandwidth of images, which we have explored in the context of both desktop and head-mounted display visualization, as needed in virtual and augmented reality applications. The challenges of multi-viewpoint multi-timepoint visualization are (1) routing the non-linear rays to find all ROIs or to reach all known ROIs, (2) making the generalized image easy to parse by enforcing spatial and temporal continuity and non-redundancy, (3) rendering the generalized images quickly as required by interactive applications, and (4) developing algorithms and user interfaces for the intuitive navigation of the compound cameras with tens of degrees of freedom. We have addressed these challenges (1) by developing a multiperspective visualization framework based on a hierarchical camera model with PPC and non-PPC leafs, (2) by routing multiple inflection point rays with direction coherence, which enforces visualization continuity, and without intersection, which enforces non-redundancy, (3) by designing our hierarchical camera model to provide closed-form projection, which enables porting generalized image rendering to the traditional and highly-efficient projection followed by rasterization pipeline implemented by graphics hardware, and (4) by devising naturalistic user interfaces based on tracked head-mounted displays that allow deploying and retracting the additional perspectives intuitively and without simulator sickness.</div>

For producing realistic images, reflection is an important visual effect. Reflections of the environment are important not only for highly reflective objects, such as mirrors, but also for more common objects such as brushed metals and glossy plastics. Generating these reflections accurately at real-time rates for interactive applications, however, is a difficult problem. Previous works in this area have made assumptions that sacrifice accuracy in order to preserve interactivity. I will present an algorithm that tries to handle reflection accurately in the general case for real-time rendering. The algorithm uses a database of prerendered environment maps to render both the original object itself and an additional bidirectional reflection distribution function (BRDF). The algorithm performs image-based rendering in reflection space in order to achieve accurate results. It also uses graphics processing unit (GPU) features to accelerate rendering.

For producing realistic images, reflection is an important visual effect. Reflections of the environment are important not only for highly reflective objects, such as mirrors, but also for more common objects such as brushed metals and glossy plastics. Generating these reflections accurately at real-time rates for interactive applications, however, is a difficult problem. Previous works in this area have made assumptions that sacrifice accuracy in order to preserve interactivity. I will present an algorithm that tries to handle reflection accurately in the general case for real-time rendering. The algorithm uses a database of prerendered environment maps to render both the original object itself and an additional bidirectional reflection distribution function (BRDF). The algorithm performs image-based rendering in reflection space in order to achieve accurate results. It also uses graphics processing unit (GPU) features to accelerate rendering.