SILVA JÚNIOR, Erasmo Artur da. Método dinâmico para troca de representação em sistemas híbridos de renderização de multidões. 2013. 61 f. Dissertação (Mestrado em ciência da computação)- Universidade Federal do Ceará, Fortaleza-CE, 2013. / Submitted by Elineudson Ribeiro (elineudsonr@gmail.com) on 2016-07-11T13:53:21Z No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) / Approved for entry into archive by Rocilda Sales (rocilda@ufc.br) on 2016-07-15T13:37:49Z (GMT) No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) / Made available in DSpace on 2016-07-15T13:37:49Z (GMT). No. of bitstreams: 1 2013_dis_easilvajunior.pdf: 12555422 bytes, checksum: fafd7f37a684a97a47f66846c82081ed (MD5) Previous issue date: 2013 / Environments populated with crowds are employed in various applications, such as games, simulators and editors. Many of these environments require not only a realistic and detailed rendering, but it must run smoothly in real-time. This task easily exhausts the system’s resources, even considering the current state-of-the-art hardware. Therefore, crowd rendering in real-time remains a challenge in computer graphics. Approaches exploiting levels of detail, visibility culling and image-based rendering are presented in order to facilitate this task. The first two increase the efficiency of rendering, but sometimes are not enough to keep an interactive frame rate. Some researches on this subject focus on image-based rendering techniques, specifically with the use of impostors. In this work it is proposed a method that balances the computational demand of rendering job by varying the threshold’s distance of the representation switch between full geometry (mesh) and image-based(impostors) models in accordance with the available resources. / Ambientes providos de multidões são empregados em diversas aplicações, como jogos, simuladores e editores. Muitas destas aplicações não requerem somente a renderização de agentes animados de forma realística e detalhada, mas que seja executada suavemente em tempo real, tarefa que facilmente esgota os recursos do sistema (mesmo considerando hardware no estado da arte). Por conta disso,a renderização de multidões em tempo real permanece como um desafio dentro da computação gráfica. Abordagens explorando nível de detalhe, descarte por visibilidade e renderização baseada em imagens foram propostas no intuito de viabilizar esta tarefa. As duas primeiras aumentam a eficiência da renderização, mas as vezes não são suficientes para manter taxas de quadros por segundo interativas. Grande parte dos estudos acerca do tema se concentra em técnicas de renderização baseadas em imagem, especificamente com o emprego de impostores. Neste trabalho é proposto um método que faz o balanço da demanda computacional da renderização através da variação da distância do limiar onde ocorre a troca de representação entre os modelos de geometria completa (malhas) e os baseados em imagem (impostores) de acordo com os recursos disponíveis.

abstract: Toward the ambitious long-term goal of a fleet of cooperating Flexible Autonomous Machines operating in an uncertain Environment (FAME), this thesis addresses various control objectives for ground vehicles. There are two main objectives within this thesis, first is the use of visual information to control a Differential-Drive Thunder Tumbler (DDTT) mobile robot and second is the solution to a minimum time optimal control problem for the robot around a racetrack. One method to do the first objective is by using the Position Based Visual Servoing (PBVS) approach in which a camera looks at a target and the position of the target with respect to the camera is estimated; once this is done the robot can drive towards a desired position (x_ref, z_ref). Another method is called Image Based Visual Servoing (IBVS), in which the pixel coordinates (u,v) of markers/dots placed on an object are driven towards the desired pixel coordinates (u_ref, v_ref) of the corresponding markers. By doing this, the mobile robot gets closer to a desired pose (x_ref, z_ref, theta_ref). For the second objective, a camera-based and noncamera-based (v,theta) cruise-control systems are used for the solution of the minimum time problem. To set up the minimum time problem, optimal control theory is used. Then a direct method is implemented by discretizing states and controls of the system. Finally, the solution is obtained by modeling the problem in AMPL and submitting to the nonlinear optimization solver KNITRO. Simulation and experimental results are presented. The DDTT-vehicle used within this thesis has different components as summarized below: (1) magnetic wheel-encoders/IMU for inner-loop speed-control and outer-loop directional control, (2) Arduino Uno microcontroller-board for encoder-based inner-loop speed-control and encoder-IMU-based outer-loop cruise-directional-control, (3) Arduino motor-shield for inner-loop speed-control, (4) Raspberry Pi II computer-board for outer-loop vision-based cruise-position-directional-control, (5) Raspberry Pi 5MP camera for outer-loop cruise-position-directional control. Hardware demonstrations shown in this thesis are summarized: (1) PBVS without pan camera, (2) PBVS with pan camera, (3) IBVS with 1 marker/dot, (4) IBVS with 2 markers, (5) IBVS with 3 markers, (6) camera and (7) noncamera-based (v,theta) cruise control system for the minimum time problem. / Dissertation/Thesis / Masters Thesis Electrical Engineering 2016

Made available in DSpace on 2016-08-17T14:53:12Z (GMT). No. of bitstreams: 1 Irlandino Almeida.pdf: 795180 bytes, checksum: 9f8896e26b58665bad7d75b665940404 (MD5) Previous issue date: 2007-09-17 / We present a semi-automatic methodology for three-dimensional reconstruction using an image-based modeling environment. This is done through an initial camera calibration step that makes possible to the viewer, identify objects in the 2D acquired image and them get its position from the camera parameters. This methodology can be applied in the real word model reconstruction like electric and industry installations. / Apresentamos uma metodologia semi-automática para reconstrução tridimensional utilizando um ambiente de modelagem no qual o usuário, a partir de uma determinada imagem e de pontos nela selecionados com seu correspondente no espaço do mundo, realiza a calibração da câmera. Após este processo, a cena poderá ser composta a partir de objetos pré-estabelecidos fornecidos pelo ambiente de modelagem tridimensional baseado na imagem. Esta metodologia pode ser aplicada na reconstrução de modelos do mundo real, como instalações elétricas e industriais.

Image based modeling using visual images has undergone major development during the earlier parts of the 21th century. In this thesis a system for automated uncalibrated scene reconstruction using infrared images is implemented and tested. An automated reconstruction system could serve to simplify thermal inspection or as a demonstration tool. Thermal images will in general have lower resolution, less contrast and less high frequency content as compared to visual images. These characteristics of infrared images further complicates feature extraction and matching, key steps in the reconstruction process. In order to remedy the complication preprocessing methods are suggested and tested as well. Infrared modeling will also impose additional demands on the reconstruction as it is of importance to maintain thermal accuracy of the images in the product. Three main results are obtained from this thesis. Firstly, it is possible to obtain camera calibration and pose as well as a sparse point cloud reconstruction from an infrared image sequence using the suggested implementation. Secondly, correlation of thermal measurements from the images used to reconstruct three dimensional coordinates is presented and analyzed. Lastly, from the preprocessing evaluation it is concluded that the tested methods are not suitable. The methods will increase computational cost while improvements in the model are not proportional. / Bildbaserad modellering med visuella bilder har genomgått en stor utveckling under de tidigare delarna av 2000-talet. Givet en sekvens bestående av vanliga tvådimensionella bilder på en scen från olika perspektiv så är målet att rekonstruera en tredimensionell modell. I denna avhandling implementeras och testas ett system för automatiserad okalibrerad scenrekonstruktion från infraröda bilder. Okalibrerad rekonstruktion refererar till det faktum att parametrar för kameran, såsom fokallängd och fokus, är okända och enbart bilder används som indata till systemet. Ett stort användingsområde för värmekameror är inspektion. Temperaturskillnader i en bild kan indikera till exempel dålig isolering eller hög friktion. Om ett automatiserat system kan skapa en tredimensionell modell av en scen så kan det bidra till att förenkla inspektion samt till att ge en bättre överblick. Värmebilder kommer generellt att ha lägre upplösning, mindre kontrast och mindre högfrekvensinnehåll jämfört med visuella bilder. Dessa egenskaper hos infraröda bilder komplicerar extraktion och matchning av punkter i bilderna vilket är viktiga steg i rekonstruktionen. För att åtgärda komplikationen förbehandlas bilderna innan rekonstruktionen, ett urval av metoder för förbehandling har testats. Rekonstruktion med värmebilder kommer också att ställa ytterligare krav på rekonstruktionen, detta eftersom det är viktigt att bibehålla termisk noggrannhet från bilderna i modellen. Tre huvudresultat erhålls från denna avhandling. För det första är det möjligt att beräkna kamerakalibrering och position såväl som en gles rekonstruktion från en infraröd bildsekvens, detta med implementationen som föreslås i denna avhandling. För det andra presenteras och analyseras korrelationen för temperaturmätningar i bilderna som används för rekonstruktionen. Slutligen så visar den testade förbehandlingen inte en förbättring av rekonstruktionen som är propotionerlig med den ökade beräkningskomplexiteten.

Le travail présenté dans ce mémoire a pour cadre le dispositif de réalité mixte ray-on, conçu par la société on-situ. Ce dispositif, dédié à la mise en valeur du patrimoine architectural et en particulier d'édifices historiques, est installé sur le lieu de l'édifice et propose à l'utilisateur une vision uchronique de celui-ci. Le parti pris étant celui du photo-réalisme, deux pistes ont été suivies : l'amélioration du mélange réel virtuel par la reproduction de l'éclairage réel sur les objets virtuels, et la mise en place d'une méthode de segmentation d'image résiliente aux changements lumineux.Pour la reproduction de l'éclairage, une méthode de rendu basé-image est utilisée et associée à une capture haute dynamique de l'environnement lumineux. Une attention particulière est portée pour que ces deux phases soient justes photométriquement et colorimétriquement. Pour évaluer la qualité de la chaîne de reproduction de l'éclairage, une scène test constituée d'une mire de couleur calibrée est mise en place, et capturée sous de multiples éclairages par un couple de caméra, l'une capturant une image de la mire, l'autre une image de l'environnement lumineux. L'image réelle est alors comparée au rendu virtuel de la même scène, éclairée par cette seconde image.La segmentation résiliente aux changements lumineux a été développée à partir d'une classe d'algorithmes de segmentation globale de l'image, considérant celle-ci comme un graphe où trouver la coupe minimale séparant l'arrière plan et l'avant plan. L'intervention manuelle nécessaire à ces algorithmes a été remplacée par une pré-segmentation de moindre qualité à partir d'une carte de profondeur, cette pré-segmentation étant alors utilisée comme une graîne pour la segmentation finale. / The work described in this report has as a target the mixed reality device ray-on, developed by the on-situ company. This device, dedicated to cultural heritage and specifically architectural heritage, is meant to be installed on-site and shows the user an uchronic view of its surroundings. As the chosen stance is to display photo-realistic images, two trails were followed: the improvement of the real-virtual merging by reproducing accurately the real lighting on the virtual objects, and the development of a real-time segmentation method which is resilient to lighting changes.Regarding lighting reproduction, an image-based rendering method is used in addition to a high dynamic range capture of the lighting environment. The emphasis is put on the photometric and colorimetric correctness of these two steps. To measure the quality of the lighting reproduction chain, a test scene is set up with a calibrated color checker, captured by a camera while another camera is grabbing the lighting environment. The image of the real scene is then compared to the simulation of the same scene, enlightened by the light probe.Segmentation resilient to lighting changes is developed from a set of global image segmentation methods, which consider an image as a graph where a cut of minimal energy has to be found between the foreground and the background. These methods being semi-automatic, the manual part is replaced by a low resolution pre-segmentation based on the depthmap of the scene which is used as a seed for the final segmentation.

Nous présentons une approche de rendu à base d'images (IBR) qui permet, à partir de photos, de naviguer librement et générer des points de vue quelconques dans des scènes urbaines. Les approches précédentes dépendent des modèles 3D et donnent lieu à des rendus de qualité réduite avec beaucoup d'artefacts. Dans cette thèse, nous proposons une approximation basée sur l'image pour compenser le manque de précision de la géométrie 3D. Nous utilisons un warp d'image guidé par des cartes de profondeur quasi-denses qui donnent lieu à beaucoup moins d'artefacts. En se basant sur cette approche, nous avons développé une méthode entièrement automatique permettant de traiter les scènes complexes. Nous sur-segmentons les images d’entrées en superpixels qui limitent les occlusions sur les bords des objets. Nous introduisons la synthèse de profondeur pour créer une approximation de cette profondeur mal reconstruite dans certaines régions et calculons les warps sur les superpixels pour synthétiser le résultat final. Nous comparons nos résultats à de nombreuses approches récentes. Nous avons analysé les artefacts de l'IBR d'un point de vue perceptif en comparant les artefacts générés par le mélange de plusieurs images avec ceux des transitions temporelles brusques et avons élaboré une méthodologie pour la sélection d'un compromis idéal entre les deux. Nous avons également analysé les distorsions perspectives et avons développé un modèle quantitatif qui permet de prédire les distorsions en fonction des paramètres de capture et de visualisation. Comme application, nous avons mis en œuvre un système de réalité virtuelle qui utilise l'IBR à la place de l'infographie traditionnelle. / We present image-based rendering (IBR) approaches that allow free viewpoint walkthroughs of urban scenes using just a few photographs as input. Previous approaches depend upon 3D models which give artifacts as the quality of 3D model degrades. In this thesis, we propose image-based approximations to compensate for the lack of accurate 3D geometry. In the first project, we use discontinuous shape-preserving image warp guided by quasi-dense depth maps which gives far fewer rendering artifacts than previous approaches. We build upon this approach in the second project by developing a completely automated solution that is capable of handling more complex scenes. We oversegment input images into superpixels that capture all occlusion boundaries. We introduce depth synthesis to create approximate depth in very poorly reconstructed regions and compute shape-preserving warps on superpixels to synthesize the final result. We also compare our results to many recent approaches. We analyze IBR artifacts from a perceptual point of view. In the first study, we compare artifacts caused by blending multiple images with abrupt temporal transitions and develop guidelines for selecting the ideal tradeoff. We use vision science in another study to investigate perspective distortions and develop a quantitative model that predicts distortions as a function of capture and viewing parameters. We use guidelines from these experiments to motivate the design of our own IBR systems. We demonstrate the very first virtual reality system that uses IBR instead of traditional computer graphics. This drastically reduces the cost of modeling 3D scenes while producing highly realistic walkthroughs.

Le rendu photo-réaliste traditionnel exige un effort manuel et des calculs intensifs pour créer des scènes et rendre des images réalistes. C'est principalement pour cette raison que la création de contenus pour l’imagerie numérique de haute qualité a été limitée aux experts et le rendu hautement réaliste nécessite encore des temps de calcul significatifs. Le rendu basé image (IBR) est une alternative qui a le potentiel de rendre les applications de création et de rendu de contenus de haute qualité accessibles aux utilisateurs occasionnels, puisqu'ils peuvent générer des images photo-réalistes de haute qualité sans subir les limitations mentionnées ci-dessus. Nous avons identifié trois limitations importantes des méthodes actuelles de rendu basé image : premièrement, chaque algorithme possède des forces et faiblesses différentes, en fonction de la qualité de la reconstruction 3D et du contenu de la scène, et un seul algorithme ne permet souvent pas d’obtenir la meilleure qualité de rendu partout dans l’image. Deuxièmement, ces algorithmes présentent de forts artefacts lors du rendu d’objets manquants ou partiellement reconstruits. Troisièmement, la plupart des méthodes souffrent encore d'artefacts visuels significatifs dans les régions de l’image où la reconstruction est de faible qualité. Dans l'ensemble, cette thèse propose plusieurs améliorations significatives du rendu basé image aussi bien en termes de vitesse de rendu que de qualité d’image. Ces nouvelles solutions sont basées sur le rendu sélectif, la substitution de modèle basé sur l'apprentissage, et la prédiction et la correction des erreurs de profondeur. / Traditional photo-realistic rendering requires intensive manual and computational effort to create scenes and render realistic images. Thus, creation of content for high quality digital imagery has been limited to experts and highly realistic rendering still requires significant computational time. Image-Based Rendering (IBR) is an alternative which has the potential of making high-quality content creation and rendering applications accessible to casual users, since they can generate high quality photo-realistic imagery without the limitations mentioned above. We identified three important shortcomings of current IBR methods: First, each algorithm has different strengths and weaknesses, depending on 3D reconstruction quality and scene content and often no single algorithm offers the best image quality everywhere in the image. Second, such algorithms present strong artifacts when rendering partially reconstructed objects or missing objects. Third, most methods still result in significant visual artifacts in image regions where reconstruction is poor. Overall, this thesis addresses significant shortcomings of IBR for both speed and image quality, offering novel and effective solutions based on selective rendering, learning-based model substitution and depth error prediction and correction.

Abstract This work addresses the problems related to three-dimensional modeling of scenes and objects and model evaluation. The work is divided into four main parts. At first, the work concentrates on purely image-based reconstruction while the second part presents a modeling pipeline based on an active depth sensor. Then, the work introduces methods for producing surface meshes from point clouds, and finally, a novel approach for model evaluation is presented. In the first part, this work proposes a multi-view stereo (MVS) reconstruction method that takes a set of images as an input and outputs a model represented as a point cloud. The method is based on match propagation, where a set of initial corresponding points between images is expanded iteratively into larger regions by searching new correspondences in the spatial neighborhood of the existing ones. The expansion is implemented using a best-first strategy, where the most reliable match is always expanded first. The method produces comparable results with the state-of-the-art but significantly faster. In the second part, this work presents a method that merges a sequence of depth maps into a single non-redundant point cloud. In the areas, where the depth maps overlap, the method fuses points together by giving more weight to points which seem to be more reliable. The method overcomes its predecessor both in accuracy and robustness. In addition, this part introduces a method for depth camera calibration. The method develops on an existing calibration approach which was originally designed for the first generation Microsoft Kinect device. The third part of the thesis addresses the problem of converting the point clouds to surface meshes. The work briefly reviews two well-known approaches and compares their ability to produce sparse mesh models without sacrificing accuracy. Finally, the fourth part of this work describes the development of a novel approach for performance evaluation of reconstruction algorithms. In addition to the accuracy and completeness, which are the metrics commonly used in existing evaluation benchmarks, the method also takes the compactness of the models into account. The metric enables the evaluation of the accuracy-compactness trade-off of the models. / Tiivistelmä Tässä työssä käsitellään näkymän tai esineen kolmiulotteista mallintamista ja tulosten laadun arviointia. Työ on jaettu neljään osaan. Ensiksi keskitytään pelkästään valokuvia hyödyntävään mallinnukseen ja sitten esitellään menetelmä syvyyskamerapohjaiseen mallinnukseen. Kolmas osa kuvaa menetelmiä verkkomallien luomiseen pistepilvestä ja lopuksi esitellään menetelmä mallien laadun arviointiin. Ensimmäisessä osassa esitellään usean kuvan stereoon perustuva mallinnusmenetelmä, joka saa syötteenä joukon valokuvia ja tuottaa kuvissa näkyvästä kohteesta pistepilvimallin. Menetelmä perustuu vastinpisteiden laajennukseen, jossa kuvien välisiä pistevastaavuuksia laajennetaan iteratiivisesti suuremmiksi vastinalueiksi hakemalla uusia vastinpistepareja jo löydettyjen läheisyydestä. Laajennus käyttää paras ensin -menetelmää, jossa luotettavin pistevastaavuus laajennetaan aina ensin. Menetelmä tuottaa vertailukelpoisia tuloksia johtaviin menetelmiin verrattuna, mutta merkittävästi nopeammin. Toisessa osassa esitellään menetelmä, joka yhdistää joukon syvyyskameralla kaapattuja syvyyskarttoja yhdeksi pistepilveksi. Alueilla, jotka sisältävät syvyysmittauksia useasta syvyyskartasta, päällekkäiset mittaukset yhdistetään painottamalla luotettavammalta vaikuttavaa mittausta. Menetelmä on tarkempi kuin edeltäjänsä ja toimii paremmin kohinaisemmalla datalla. Lisäksi tässä osassa esitellään menetelmä syvyyskameran kalibrointiin. Menetelmä kehittää jo olemassa olevaa kalibrointityökalua, joka alun perin kehitettiin ensimmäisen sukupolven Microsoft Kinect laitteelle. Väitöskirjan kolmas osa käsittelee pintamallin luomista pistepilvestä. Työ esittelee kaksi hyvin tunnettua menetelmää ja vertailee niiden kykyä luoda harvoja, mutta edelleen tarkkoja malleja. Lopuksi esitellään uudenlainen menetelmä mallinnusmenetelmien arviointiin. Tarkkuuden ja kattavuuden lisäksi, jotka ovat yleisimmät arvioinnissa käytetyt metriikat, menetelmä ottaa huomioon myös mallin pistetiheyden. Metriikan avulla on mahdollista arvioida kompromissia mallin tarkkuuden ja tiheyden välillä.

Real-world radiance values can range over eight orders of magnitude from starlight to direct sunlight but few digital cameras capture more than three orders in a single Low Dynamic Range (LDR) image. We approach this problem using established High Dynamic Range (HDR) techniques in which multiple images are captured with different exposure times so that all portions of the scene are correctly exposed at least once. These images are then combined to create an HDR image capturing the full range of the scene. HDR capture introduces new challenges; movement in the scene creates faded copies of moving objects, referred to as ghosts. Many techniques have been introduced to handle ghosting, but typically they either address specific types of ghosting, or are computationally very expensive. We address ghosting by first detecting moving objects, then reducing their contribution to the final composite on a frame-by-frame basis. The detection of motion is addressed by performing change detection on exposure-normalized images. Additional special cases are developed based on a priori knowledge of the changing exposures; for example, if exposure is increasing every shot, then any decrease in intensity in the LDR images is a strong indicator of motion. Recent Superpixel over-segmentation techniques are used to refine the detection. We also propose a novel solution for areas that see motion throughout the capture, such as foliage blowing in the wind. Such areas are detected as always moving, and are replaced with information from a single input image, and the replacement of corrupted regions can be tailored to the scenario. We present our approach in the context of a panoramic tele-presence system. Tele-presence systems allow a user to experience a remote environment, aiming to create a realistic sense of "being there" and such a system should therefore provide a high quality visual rendition of the environment. Furthermore, panoramas, by virtue of capturing a greater proportion of a real-world scene, are often exposed to a greater dynamic range than standard photographs. Both facets of this system therefore stand to benefit from HDR imaging techniques. We demonstrate the success of our approach on multiple challenging ghosting scenarios, and compare our results with state-of-the-art methods previously proposed. We also demonstrate computational savings over these methods.

This work deals with the technical possibilities for automated optical inspection and the arrangements for monitoring the implementation of established elements. There are used methods of detection elements, including advanced algorithm processing. With the described methods was created program and each method was tested. The aim was on the clear description of the problem, the optimal design and processing program with objective results with the lowest number of false detection.