A polygon mesh, or a 3D mesh, consisting of a collection of vertices, edges, and polygons in three-dimensional space, is the standard way of representing 3D objects. In practice, polygon meshes acquired from the 3D scanning process fail to meet the quality requirements for most practical applications. Mesh defects like holes, duplicate elements, non-manifold elements are introduced during the scanning process, which lowers the quality of the output meshes. In this thesis, we describe a complete mesh-repairing process that fixes all defects within a polygon mesh. This process is divided into two parts: the mesh-cleaning part and the hole-filling part. In the mesh-cleaning part, we describe the ways of repairing different types of mesh defects. In the hole-filling part, we discuss two main hole-filling approaches: the surface-based method and the volumetric. In addition, we present a hybrid algorithm by combining the surface-based approach and the volumetric approach. We compare the meshes created by different hole-filing algorithms and show that the new algorithm is a good alternative to the existing ones. / Thesis (Master, Computing) -- Queen's University, 2013-09-11 23:45:08.591

Esta dissertação investiga soluções para o problema genérico de geração de vistas sintéticas a partir de um conjunto de imagens utilizando a abordagem Depth-Image-Based Rendering. Essa abordagem utiliza um formato compacto para a representação de imagens 3D, composto basicamente por duas imagens, uma colorida para a vista de referência e outra em tons de cinza com a correspondência de disparidade para cada pixel. Soluções para esse problema beneficiam aplicações como Free Viewpoint Television. O maior desafio é o preenchimento de regiões sem informação de projeção considerando o novo ponto de vista, genericamente denominados holes, além de outros artefatos como cracks e ghosts que ocorrem por oclusões e erros no mapa de disparidade. Nesta dissertação apresentamos técnicas para remoção e tratamento de cada uma das classes de potenciais artefatos. O conjunto de métodos propostos apresenta melhores resultados quando comparado com o atual estado da arte em geração de vistas sintéticas com o modelo DIBR para o conjunto de dados Middlebury, considerando-se as métricas SSIM e PSNR. / This dissertation investigates solutions to the general problem of generating synthetic views from a set of images using the Depth-Image-Based Rendering approach. This approach uses a compact format for the 3D image representation, composed basically of two images, one color image for the reference view and other grayscale image with the disparity information available for each pixel. Solutions to this problem benefit applications such as Free Viewpoint Television. The biggest challenge is filling in regions without projection information considering the new viewpoint, usually called holes, and other artifacts such as cracks and ghosts that occur due to occlusions and errors in the disparity map. In this dissertation we present techniques for removal and treatment of each of these classes of potential artifacts. The set of proposed methods shows improved results when compared to the current state of the art generation of synthetic views using the DIBR model applied to the Middlebury dataset, considering the SSIM and PSNR metrics.

Depth-Based Image Rendering (DIBR) is a popular method to generate 3D virtual image at different view positions using an image and a depth map. In general, DIBR consists of two major operations: image warping and hole filling. Image warping calculates the disparity from the depth map given some information of viewers and display screen. Hole filling is to calculate the color of pixel locations that do not correspond to any pixels in the original image after image warping. Although there are many different hole filling methods that determine the colors of the blank pixels, some undesirable artifacts are still observed in the synthesized virtual image. In this thesis, we present an approach that examines the geometry information near the region of blank pixels in order to reduce the artifacts near the edges of objects. Experimental results show that the proposed design can generate more natural shape around the edges of objects at the cost of more hardware and computation time.

Esta dissertação investiga soluções para o problema genérico de geração de vistas sintéticas a partir de um conjunto de imagens utilizando a abordagem Depth-Image-Based Rendering. Essa abordagem utiliza um formato compacto para a representação de imagens 3D, composto basicamente por duas imagens, uma colorida para a vista de referência e outra em tons de cinza com a correspondência de disparidade para cada pixel. Soluções para esse problema beneficiam aplicações como Free Viewpoint Television. O maior desafio é o preenchimento de regiões sem informação de projeção considerando o novo ponto de vista, genericamente denominados holes, além de outros artefatos como cracks e ghosts que ocorrem por oclusões e erros no mapa de disparidade. Nesta dissertação apresentamos técnicas para remoção e tratamento de cada uma das classes de potenciais artefatos. O conjunto de métodos propostos apresenta melhores resultados quando comparado com o atual estado da arte em geração de vistas sintéticas com o modelo DIBR para o conjunto de dados Middlebury, considerando-se as métricas SSIM e PSNR. / This dissertation investigates solutions to the general problem of generating synthetic views from a set of images using the Depth-Image-Based Rendering approach. This approach uses a compact format for the 3D image representation, composed basically of two images, one color image for the reference view and other grayscale image with the disparity information available for each pixel. Solutions to this problem benefit applications such as Free Viewpoint Television. The biggest challenge is filling in regions without projection information considering the new viewpoint, usually called holes, and other artifacts such as cracks and ghosts that occur due to occlusions and errors in the disparity map. In this dissertation we present techniques for removal and treatment of each of these classes of potential artifacts. The set of proposed methods shows improved results when compared to the current state of the art generation of synthetic views using the DIBR model applied to the Middlebury dataset, considering the SSIM and PSNR metrics.

Esta dissertação investiga soluções para o problema genérico de geração de vistas sintéticas a partir de um conjunto de imagens utilizando a abordagem Depth-Image-Based Rendering. Essa abordagem utiliza um formato compacto para a representação de imagens 3D, composto basicamente por duas imagens, uma colorida para a vista de referência e outra em tons de cinza com a correspondência de disparidade para cada pixel. Soluções para esse problema beneficiam aplicações como Free Viewpoint Television. O maior desafio é o preenchimento de regiões sem informação de projeção considerando o novo ponto de vista, genericamente denominados holes, além de outros artefatos como cracks e ghosts que ocorrem por oclusões e erros no mapa de disparidade. Nesta dissertação apresentamos técnicas para remoção e tratamento de cada uma das classes de potenciais artefatos. O conjunto de métodos propostos apresenta melhores resultados quando comparado com o atual estado da arte em geração de vistas sintéticas com o modelo DIBR para o conjunto de dados Middlebury, considerando-se as métricas SSIM e PSNR. / This dissertation investigates solutions to the general problem of generating synthetic views from a set of images using the Depth-Image-Based Rendering approach. This approach uses a compact format for the 3D image representation, composed basically of two images, one color image for the reference view and other grayscale image with the disparity information available for each pixel. Solutions to this problem benefit applications such as Free Viewpoint Television. The biggest challenge is filling in regions without projection information considering the new viewpoint, usually called holes, and other artifacts such as cracks and ghosts that occur due to occlusions and errors in the disparity map. In this dissertation we present techniques for removal and treatment of each of these classes of potential artifacts. The set of proposed methods shows improved results when compared to the current state of the art generation of synthetic views using the DIBR model applied to the Middlebury dataset, considering the SSIM and PSNR metrics.

This project, completed in collaboration with Piktiv AB, aimed to develop an automated surface hole-filling feature for static meshes in Unreal Engine 4.27, with the goal of making repaired surfaces visually indistinguishable from their surrounding areas. The solution was primarily designed to address holes that arose from, but were not limited to, the use of Reduction Settings within Unreal Engine on static meshes. The functionality encompassed four key stages: boundary detection, where all holes on the mesh were identified; triangulation, which involved patching the hole using vertices from the boundary; refinement, entailing the addition of vertices and triangles to the patched area to mimic the density of the surrounding surface; and fairing, which smoothed the patched surface. Additionally, the project introduced a straightforward method for determining the texture coordinates of newly added vertices and a technique for ensuring that triangle normals correctly faced outward from the mesh. The Static Mesh Hole Filler, as implemented, demonstrates efficiency in filling an arbitrary amount of small, planar holes, which commonly result from polygon reduction using Reduction Settings in Unreal Engine. However, this function falls short in preserving unique texture details and maintaining the curvature of surfaces when dealing with larger holes. This limitation necessitates users to seek alternative methods for effectively repairing the mesh.

Image completion is a process of removing an area from a photograph and replacing it with suitable data. Earlier methods either search for this relevant data within the image itself, or extends the search to some form of additional data, usually some form of database. Methods that search for suitable data within the image itself has problems when no suitable data can be found in the image. Methods that extend their search has in earlier work either used some form of database with labeled images or a massive database with photos from the Internet. For the labels in a database to be useful they typically needs to be entered manually, which is a very time consuming process. Methods that uses databases with millions of images from the Internet has issues with copyrighted images, storage of the photographs and computation time. This work shows that a small database of the user’s own private, or professional, photos can be used to improve the quality of image completions. A photographer today typically take many similar photographs on similar scenes during a photo session. Therefore a smaller number of images are needed to find images that are visually and structurally similar, than when random images downloaded from the internet are used. Thus, this approach gains most of the advantages of using additional data for the image completions, while at the same time minimizing the disadvantages. It gains a better ability to find suitable data without having to process millions of irrelevant photos.

Three Dimensional Video (3DV) has become increasingly popular with the success of 3D cinema. Moreover, emerging display technology offers an immersive experience to the viewer without the necessity of any visual aids such as 3D glasses. 3DV applications, Three Dimensional Television (3DTV) and Free Viewpoint Television (FTV) are auspicious technologies for living room environments by providing immersive experience and look around facilities. In order to provide such an experience, these technologies require a number of camera views captured from different viewpoints. However, the capture and transmission of the required number of views is not a feasible solution, and thus view rendering is employed as an efficient solution to produce the necessary number of views. Depth-image-based rendering (DIBR) is a commonly used rendering method. Although DIBR is a simple approach that can produce the desired number of views, inherent artifacts are major issues in the view rendering. Despite much effort to tackle the rendering artifacts over the years, rendered views still contain visible artifacts. This dissertation addresses three problems in order to improve 3DV quality: 1) How to improve the rendered view quality using a direct approach without dealing each artifact specifically. 2) How to handle disocclusions (a.k.a. holes) in the rendered views in a visually plausible manner using inpainting. 3) How to reduce spatial inconsistencies in the rendered view. The first problem is tackled by an edge-aided rendering method that uses a direct approach with one-dimensional interpolation, which is applicable when the virtual camera distance is small. The second problem is addressed by using a depth-based inpainting method in the virtual view, which reconstructs the missing texture with background data at the disocclusions. The third problem is undertaken by a rendering method that firstly inpaint occlusions as a layered depth image (LDI) in the original view, and then renders a spatially consistent virtual view. Objective assessments of proposed methods show improvements over the state-of-the-art rendering methods. Visual inspection shows slight improvements for intermediate views rendered from multiview videos-plus-depth, and the proposed methods outperforms other view rendering methods in the case of rendering from single view video-plus-depth. Results confirm that the proposed methods are capable of reducing rendering artifacts and producing spatially consistent virtual views. In conclusion, the view rendering methods proposed in this dissertation can support the production of high quality virtual views based on a limited number of input views. When used to create a multi-scopic presentation, the outcome of this dissertation can benefit 3DV technologies to improve the immersive experience.

Three dimensional television (3DTV) is believed to be the future of television broadcasting that will replace current 2D HDTV technology. In the future, 3DTV will bring a more life-like and visually immersive home entertainment experience, in which users will have the freedom to navigate through the scene to choose a different viewpoint. A desired view can be synthesized at the receiver side using depth image-based rendering (DIBR). While this approach has many advantages, one of the key challenges in DIBR is generating high quality synthesized views. This work presents novel methods to measure and enhance the quality of 3D videos generated through DIBR. For quality measurements we describe a novel method to characterize and measure distortions by multiple cameras used to capture stereoscopic images. In addition, we present an objective quality measure for DIBR-based 3D videos by evaluating the elements of visual discomfort in stereoscopic 3D videos. We also introduce a new concept called the ideal depth estimate, and define the tools to estimate that depth. Full-reference and no-reference profiles for calculating the proposed measures are also presented. Moreover, we introduce two innovative approaches to improve the quality of the synthesized views generated by DIBR. The first approach is based on hierarchical blending of the background and foreground information around the disocclusion areas which produces a natural looking, synthesized view with seamless hole-filling. This approach yields virtual images that are free of any geometric distortions, unlike other algorithms that preprocess the depth map. In contrast to the other hole-filling approaches, our approach is not sensitive to depth maps with high percentage of bad pixels from stereo matching. The second approach further enhances the results through a depth-adaptive preprocessing of the colored images. Finally, we propose an enhancement over depth estimation algorithm using the depth monocular cues from luminance and chrominance. The estimated depth will be evaluated using our quality measure, and the hole-filling algorithm will be used to generate synthesized views. This application will demonstrate how our quality measures and enhancement algorithms could help in the development of high quality stereoscopic depth-based synthesized videos.

This thesis describes theory and algorithms for use with Radial Basis Functions (RBFs), emphasising techniques motivated by three particular application areas. In Part I, we apply RBFs to the problem of interpolating to integral data. While the potential of using RBFs for this purpose has been established in an abstract theoretical context, their use has been lacking an easy to check sufficient condition for finding appropriate parent basic functions, and explicit methods for deriving integral basic functions from them. We present both these components here, as well as explicit formulations for line segments in two dimensions and balls in three and five dimensions. We also apply these results to real-world track data. In Part II, we apply Hermite and pointwise RBFs to the problem of surface reconstruction. RBFs are used for this purpose by representing the surface implicitly as the zero level set of a function in 3D space. We develop a multilevel piecewise technique based on scattered spherical subdomains, which requires the creation of algorithms for constructing sphere coverings with desirable properties and for blending smoothly between levels. The surface reconstruction method we develop scales very well to large datasets and is very amenable to parallelisation, while retaining global-approximation-like features such as hole filling. Our serial implementation can build an implicit surface representation which interpolates at over 42 million points in around 45 minutes. In Part III, we apply RBFs to the problem of animation control in the area of motion synthesis---controlling an animated character whose motion is entirely the result of simulated physics. While the simulation is quite well understood, controlling the character by means of forces produced by virtual actuators or muscles remains a very difficult challenge. Here, we investigate the possibility of speeding up the optimisation process underlying most animation control methods by approximating the physics simulator with RBFs.