Calibrating multiple projectors to create a distortion free environment is required in many fields e.g. simulators and the calibration may be done in a series different ways. This report will cover an automatic single camera projector calibration algorithm.The algorithm handles multiple projectors and can handle projectors covering bigger field of view than a camera by supporting image stitching. A proof of concept blending algorithm is also presented. The algorithm includes a new developed interpolation method building on spline surfaces and an orientation calculation algorithm that calculates the orientation difference between two camera views. Using the algorithm to calibrate, gives pixel accuracy of less than 1 camera pixel after interpolation and the relation between two views are calculated accurately. The images created using the algorithm is distortion free and close to seamless. The algorithm is limited to a controlled projector environment and calibrates the projectors for a single viewpoint. Furthermore, the camera needs to be calibrated positioned in the sweet spot although it can be arbitrary rotated.

A parameter-free solution is presented for data projector calibration using a single camera and Gray coded structured light patterns. The proposed method assumes that both camera and projector exhibit significant non-linear distortion, and that projection surfaces can be either planar or freeform. The camera is calibrated first through traditional methods, and the calibrated images are then used to detect Gray coded patterns displayed on a surface by the data projector. Projector to camera correspondences are created by decoding the patterns in the camera images to form a 2D correspondence map. Calibrated systems produce geometrically correct, ex- tremely short throw projections, while maintaining or exceeding the projection size of a standard configuration. Qualitative experiments are performed on two baseline images, while quantitative data is recovered from the projected image of a chessboard pattern. A typical throw ratio of 0.5 can be achieved with a pixel distance error below 1. / Thesis (Master, Electrical & Computer Engineering) -- Queen's University, 2010-06-29 09:33:50.311

This thesis focuses on the application of the tangible user interface and spatial augmented reality principals for playing board games. For this purpose, a system called interactive table was developed. It is both hardware and software solution. The hardware part solves the installation of a depth camera and a projector located above the table. The software solution deals with calibration of the depth camera and projector system using known procedures for camera calibration on the one hand, on the other hand it solves tasks from the field of computer vision. This perceptional part is the core of the whole thesis and for the purposes of playing board games it provides information about the location of the physical game object, physical game board and the calculations of mask for masking distracting objects. An educational application was developed for the purposes of demonstration of the system options. The interactive table offers new possibilities for playing board games in a real-world environment by the combination of the augmented reality elements with real-world objects and the related new user experience.

A realistic visualization that enhances a virtually authentic experience is the ambition of any visualization system in a simulator environment. But calibrating and maintaining a high quality visualization system entail large costs, resources and time. Hence, a soft- ware has been developed to facilitate the measuring of three useful metrics that lay the foundation for automatic calibration of visual projector systems. The software and associ- ated method conducts relevant measurements which are compatible with arbitrary surface shapes and can be utilized as an assurance of quality measurement tool. The measure- ments include absolute surface shape, absolute geometry and projector blending zone con- tributions. The implementation decrease manual complexity for users, adds environmen- tal flexibility, increase density of measurement points and is more cost and time efficient then existing methods. / <p>Examensarbetet är utfört vid Institutionen för teknik och naturvetenskap (ITN) vid Tekniska fakulteten, Linköpings universitet</p>

Submitted by Renata Lopes (renatasil82@gmail.com) on 2017-03-02T14:44:36Z No. of bitstreams: 1 rogercorreiapinheirosilva.pdf: 22838442 bytes, checksum: 0bd115f462fc7572058a542e9ed91fcc (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2017-03-06T19:52:42Z (GMT) No. of bitstreams: 1 rogercorreiapinheirosilva.pdf: 22838442 bytes, checksum: 0bd115f462fc7572058a542e9ed91fcc (MD5) / Made available in DSpace on 2017-03-06T19:52:42Z (GMT). No. of bitstreams: 1 rogercorreiapinheirosilva.pdf: 22838442 bytes, checksum: 0bd115f462fc7572058a542e9ed91fcc (MD5) Previous issue date: 2011-08-26 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Um sistema câmera-projetor é capaz de capturar informação geométrica tridimensional de objetos e ambientes do mundo real. A captura de geometria em tal sistema baseia-se na projeção de luz estruturada sobre um objeto através do projetor, e na captura da cena modulada através da câmera. Com o sistema previamente calibrado, a deformação da luz projetada causada pelo objeto fornece a informação necessária para reconstruir a geometria do mesmo por meio de triangulação. Este trabalho descreve o desenvolvimento de um digitalizador câmera-projetor de alta definição (com resoluções de até 1920x1080 e 1280x720); são detalhadas as etapas e processos que conduzem à reconstrução de geometria, como calibração câmera-projetor, calibração de cores, processamento da imagem capturada e triangulação. O digitalizador desenvolvido utiliza a codificação de luz estruturada (b; s)-BCSL, que emprega a projeção de uma sequência de faixas verticais coloridas sobre a cena. Este esquema de codificação flexível oferece um número variado de faixas para projeção: quanto maior o número de faixas, mais detalhada a geometria capturada. Um dos objetivos deste trabalho é estimar o número limite de faixas (b,s)-BCSL possível dentro das resoluções atuais de vídeo de alta definição. Este número limite é aquele que provê reconstrução densa da geometria alvo, e ao mesmo tempo possui baixo nível de erro. Para avaliar a geometria reconstruída pelo digitalizador para os diversos números de faixas, é proposto um protocolo para avaliação de erro. O protocolo desenvolvido utiliza planos como objetos para mensurar a qualidade de reconstrução geométrica. A partir da nuvem de pontos gerada pelo digitalizador, a equação do plano para a mesma é estimada por meio de mínimos quadrados. Para um número fixo de faixas, são feitas cinco digitalizações independentes do plano: cada digitalização leva a uma equação; também é computado o plano médio, estimado a partir da união das cinco nuvens de pontos. Uma métrica de distância no espaço projetivo é usada para avaliar a precisão e a acurácia de cada número de faixas projetados. Além da avaliação quantitativa, a geometria de vários objetos é apresentada para uma avaliação qualitativa. Os resultados demonstram que a quantidade de faixas limite para vídeos de alta resolução permite uma grande densidade de pontos mesmo em superfícies com alta variação de cores. / A camera-projector system is capable of capturing three-dimensional geometric information of objects and real-world environments. The capture of geometry in such system is based on the projection of structured light over an object by the projector, and the capture of the modulated scene through the camera. With a calibrated system, the deformation of the projected light caused by the object provides the information needed to reconstruct its geometry through triangulation. The present work describes the development of a high definition camera-projector system (with resolutions up to 1920x1080 and 1280x720). The steps and processes that lead to the reconstruction of geometry, such as camera-projector calibration, color calibration, image processing and triangulation, are detailed. The developed scanner uses the (b; s)-BCSL structured light coding, which employs the projection of a sequence of colored vertical stripes on the scene. This coding scheme offers a flexible number of stripes for projection: the higher the number of stripes, more detailed is the captured geometry. One of the objectives of this work is to estimate the limit number of (b; s)-BCSL stripes possible within the current resolutions of high definition video. This limit number is the one that provides dense geometry reconstruction, and at the same has low error. To evaluate the geometry reconstructed by the scanner for a different number of stripes, we propose a protocol for error measurement. The developed protocol uses planes as objects to measure the quality of geometric reconstruction. From the point cloud generated by the scanner, the equation for the same plane is estimated by least squares. For a fixed number of stripes, five independent scans are made for the plane: each scan leads to one equation; the median plane, estimated from the union of the five clouds of points, is also computed. A distance metric in the projective space is used to evaluate the precision and the accuracy of each number of projected stripes. In addition to the quantitative evaluation, the geometry of many objects are presented for qualitative evaluation. The results show that the limit number of stripes for high resolution video allows high density of points even on surfaces with high color variation.

Measuring 3D surface geometry with precision and accuracy is an important part of many engineering and scientific tasks. 3D Scanning techniques measure surface geometry by estimating the locations of sampled surface points. In recent years, Structured-Light 3D scanners have gained significant popularity owing to their ability to produce highly accurate scans in real-time at a low cost. In this thesis we describe an approach for Structured-Light 3D scanning using a digital camera and a digital projector. We utilise the projective geometric relationships between the projector and the camera to carry out both an implicit calibration of the system and to solve for 3D structure. Our approach to geometric calibration is flexible, reliable and amenable to robust estimation. In addition, we model and account for the radiometric non-linearities in the projector such as gamma distortion. Finally, we apply a post-processing step to efficiently smooth out high-frequency surface noise while retaining the structural details. Consequently, the proposed work reduces the computational load and set-up time of a Structured-Light 3D scanner; thereby speeding up the whole scanning process while retaining the ability to generate highly accurate results. We demonstrate the accuracy of our scanning results on real-world objects of varying degrees of surface complexity. Introduction The projective geometry for a pair of pin-hole viewing devices is completely defined by their intrinsic calibration and their relative motion or extrinsic calibration in the form of matrices. For a Euclidean reconstruction, the geometry elements represented by the calibration matrices must be parameterised and estimated in some form. The use of a projector as the ‘second viewing’ device has led to numerous approaches to model and estimate its intrinsic parameters and relative motion with respect to the camera's 3D co-ordinate system. Proposed thesis work assimilates the benefits of projective geometry constructs such as Homography and the invariance of the cross-ratios to simplify the system calibration and the 3D estimation processes by an implicit modeling of the projector's intrinsic parameters and its relative motion. Though linear modeling of the projective geometry between a camera-projector view-pair captures the most essential aspects of the underlying geometry, it does not accommodate system non-linearities due to radiometric distortions of a projector device. We propose an approach that uses parametric splines to model the systematic errors introduced by radiometric non-linearities and thus correct for them. For 3D surfaces reconstructed as point-clouds, noise manifests itself as some high-frequency variations for the resulting mesh. Various pre and/or post processing techniques are proposed in the literature to model and minimize the effects of noise. We use simple bilateral filtering of the depth-map for the reconstructed surface to smoothen the surface while retaining its structural details. Modeling Projective Relations In our approach for calibrating the projective-geometric structure of a projector-camera view-pair, the frame of reference for measurements is attached to the camera. The camera is calibrated using a commonly used method. To calibrate the scanner system, one common approach is to project sinusoidal patterns onto the reference planes to generate reference phase maps. By relating the phase-information between the projector and image pixels, a dense mapping is obtained. However, this is an over-parameterisation of the calibration information. Since the reference object is a plane, we can use the projective relationships induced by a plane to implicitly calibrate the projector geometry. For the estimation of the three-dimensional structure of the imaged object, we utilise the invariance of cross-ratios along with the calibration information of two reference planes. Our formulation is also extensible to utilise more than two reference plane to compute more than one estimate of the location of an unknown surface point. Such estimates are amenable to statistical analysis which allows us to derive both the shape of an object and associate reliability scores to each estimated point location. Radiometric Correction Structured-light based 3D scanners commonly employ phase-shifted sinusoidal patterns to solve for the correspondence problem. For scanners using projective geometry between a camera and a projector, the projector's radiometric non-linearities introduce systematic errors in establishing correspondences. Such errors manifest as visual artifacts which become pronounced when fewer phase-shifted sinusoidal patterns are used. While these artifacts can be avoided by using a large number of phase-shifts, doing so also increases the acquisition time. We propose to model and rectify such systematic errors using parametric representations. Consequently, while some existing methods retain the complete reference phase maps to account for such distortions, our approach describes the deviations using a few model parameters. The proposed approach can be used to reduce the number of phase-shifted sinusoidal patterns required for codification while suppressing systematic artifacts. Additionally, our method avoids the 1D search steps that are needed when a complete reference phase map is used, thus reducing the computational load for 3D estimation. The effectiveness of our method is demonstrated with reconstruction of some geometric surfaces and a cultural figurine. Filtering Noise For a structured-light 3D scanner, various sources of noise in the environment and the devices lead to inaccuracies in estimating the codewords (phase map) for an unknown surface, during reconstruction. We examine the effects of such noise factors on our proposed methods for geometric and radiometric estimation. We present a quantitative evaluation for our proposed method by scanning the objects of known geometric properties or measures and then computing the deviations from the expected results. In addition, we evaluate the errors introduced due to inaccuracies in system calibration by computing the variance statistics from multiple estimates for the reconstructed 3D points, where each estimate is computed using a different pair of reference planes. Finally, we discuss the efficacy of certain filtering techniques in reducing the high-frequency surface noise when applied to: (a) the images of the unknown surface at a pre-processing stage, or (b) the respective phase (or depth) map at a post-processing stage. Conclusion In this thesis, we motivate the need for a procedurally simple and computationally less demanding approach for projector calibration. We present a method that uses homographies induced by a pair of reference planes to calibrate a structured-light scanner. By using the projective invariance of the cross-ratio, we solved for the 3D geometry of a scanned surface. We demonstrate the fact that 3D geometric information can be derived using our approach with accuracy on the order of 0.1 mm. Proposed method reduces the image acquisition time for calibration and the computational needs for 3D estimation. We demonstrate an approach to effectively model radiometric distortions for the projector using cubic splines. Our approach is shown to give significant improvement over the use of complete reference phase maps and its performance is comparable to that of a sate-of-the-art method, both quantitatively as well as qualitatively. In contrast with that method, proposed method is computationally less expensive, procedurally simpler and exhibits consistent performance even at relatively higher levels of noise in phase estimation. Finally, we use a simple bilateral filtering on the depth-map for the region-of-interest. Bilateral filtering provides the best trade-off between surface smoothing and the preservation of its structural details. Our filtering approach avoids computationally expensive surface normal estimation algorithms completely while improving surface fidelity.

This thesis falls into a field of user interface design. It focuses on tangible user interfaces which utilize a camera and projector to augment physical objects with a digital information. It also includes description of calibration of those devices. The primary object of this thesis is the implementation of an augmented user interface for application windows management. The system consists of a stationary camera, overhead projector and movable tangible objects - boards. The boards are equipped with fiducial markers, in order to be tracked in a camera image. The projector displays the conventional desktop onto the table and the tangible objects. For example, application windows can be projected onto some boards, while the windows move and rotate simultaneously with the boards.

The focus of this thesis is the cooperation of cameras and projectors in projection of data into a scene. It describes the means and theory necessary to achieve such cooperation, and suggests tasks for demonstration. A part of this project is also a program capable of using a camera and a projector to obtain necessary parameters of these devices. The program can demonstrate the quality of this calibration by projecting a pattern onto an object according to its current pose, as well as reconstruct the shape of an object with structured light. The thesis also describes some challenges and observations from development and testing of the program.

This thesis describes various game interfaces for human and gradually there are shown more modern and common available interfaces. Work closely describes how you can play games on a regular projector and sensors for gestures detection - Kinect and Leap Motion. This work focus on evaluation of aspects of table-game design with these components and also on display view calibration between projector and Leap Motion. Part of the project is devoted to describing the board game Catan, of which rules are taken to write down specification and options of player. According to this knowledge is then described the design of the play-set and the design of several interfaces of the game. Implementation is briefly described and final chapters covers forms of the testing methods of interfaces and testing evaluation from user perspective.