Since most people now have a high-performing computing device with an attached camera in their pocket, in the form of a smartphone, robotics and computer vision researchers are thrilled about the possibility this creates. Such devices have previously been used in robotics to create 3D maps of environments and objects by feeding the camera data to a 3D reconstruction algorithm. The big downside with smartphones is that their cameras use a different sensor than what is usually used in robotics, namely a rolling shutter camera.These cameras are cheaper to produce but are not as well suited for general 3D reconstruction algorithms as the global shutter cameras typically used in robotics research. One recent, accurate and performance effective 3D reconstruction method which could be used on a mobile device, if tweaked, is LSD-SLAM. This thesis uses the LSD-SLAM method developed for global shutter cameras and incorporates additional methods developed allow the usage of rolling shutter data.The developed method is evaluated by calculating numbers of failed 3D reconstructions before a successful one is obtained when using rolling shutter data.The result is a method which improves this metric with about 70\% compared to the unedited LSD-SLAM method.

No description available.

Les caméras sphériques et 360 deviennent populaires et sont utilisées notamment pour la création de vidéos immersives et la génération de contenu pour la réalité virtuelle. Elles sont souvent composées de plusieurs caméras grand-angles/fisheyes pointant dans différentes directions et rigidement liées les unes aux autres. Cependant, il n'est pas si simple de les calibrer complètement car ces caméras grand public sont rolling shutter et peuvent être mal synchronisées. Cette thèse propose des méthodes permettant de calibrer ces multi-caméras à partir de vidéos sans utiliser de mire de calibration. On initialise d'abord un modèle de multi-caméra grâce à des hypothèses appropriées à un capteur omnidirectionnel sans direction privilégiée : les caméras ont les mêmes réglages (dont la fréquence et l'angle de champs de vue) et sont approximativement équiangulaires. Deuxièmement, sachant que le module de la vitesse angulaire est le même pour deux caméras au même instant, nous proposons de synchroniser les caméras à une image près à partir des vitesses angulaires estimées par structure-from-motion monoculaire. Troisièmement, les poses inter-caméras et les paramètres intrinsèques sont estimés par structure-from-motion et ajustement de faisceaux multi-caméras avec les approximations suivantes : la multi-caméra est centrale, global shutter ; et la synchronisation précédant est imposée.Enfin, nous proposons un ajustement de faisceaux final sans ces approximations, qui raffine notamment la synchronisation (à précision sous-trame), le coefficient de rolling shutter et les autres paramètres (intrinsèques, extrinsèques, 3D). On expérimente dans un contexte que nous pensons utile pour des applications comme les vidéos 360 et la modélisation 3D de scènes : plusieurs caméras grand public ou une caméra sphérique fixée(s) sur un casque et se déplaçant le long d'une trajectoire de quelques centaines de mètres à quelques kilomètres. / 360 degree and spherical multi-cameras built by fixing together several consumer cameras become popular and are convenient for recent applications like immersive videos, 3D modeling and virtual reality. This type of cameras allows to include the whole scene in a single view.When the goal of our applications is to merge monocular videos together into one cylinder video or to obtain 3D informations from environment,there are several basic steps that should be performed beforehand.Among these tasks, we consider the synchronization between cameras; the calibration of multi-camera system including intrinsic and extrinsic parameters (i.e. the relative poses between cameras); and the rolling shutter calibration. The goal of this thesis is to develop and apply user friendly method. Our approach does not require a calibration pattern. First, the multi-camera is initialized thanks to assumptions that are suitable to an omnidirectional camera without a privileged direction:the cameras have the same setting (frequency, image resolution, field-of-view) and are roughly equiangular.Second, a frame-accurate synchronization is estimated from instantaneous angular velocities of each camera provided by monocular Structure-from-Motion.Third, both inter-camera poses and intrinsic parameters are refined using multi-camera Structure-from-Motion and bundle adjustment.Last, we introduce a bundle adjustment that estimates not only the usual parameters but also a subframe-accurate synchronization and the rolling shutter. We experiment in a context that we believe useful for applications (3D modeling and 360 videos):several consumer cameras or a spherical camera mounted on a helmet and moving along trajectories of several hundreds of meters or kilometers.

De nombreuses caméras CMOS modernes sont équipées de capteurs Rolling Shutter (RS). Ces caméras à bas coût et basse consommation permettent d’atteindre de très hautes fréquences d’acquisition. Dans ce mode d’acquisition, les lignes de pixels sont exposées séquentiellement du haut vers le bas de l'image. Par conséquent, les images capturées alors que la caméra et/ou la scène est en mouvement présentent des distorsions qui rendent les algorithmes classiques au mieux moins précis, au pire inutilisables en raison de singularités ou de configurations dégénérées. Le but de cette thèse est de revisiter la géométrie de la vision 3D avec des caméras RS en proposant des solutions pour chaque sous-tâche du pipe-line de Structure-from-Motion (SfM).Le chapitre II présente une nouvelle méthode de correction du RS en utilisant les droites. Contrairement aux méthodes existantes, qui sont itératives et font l’hypothèse dite Manhattan World (MW), notre solution est linéaire et n’impose aucune contrainte sur l’orientation des droites 3D. De plus, la méthode est intégrée dans un processus de type RANSAC permettant de distinguer les courbes qui sont des projections de segments droits de celles qui correspondent à de vraies courbes 3D. La méthode de correction est ainsi plus robuste et entièrement automatisée.Le chapitre III revient sur l'ajustement faisceaux ou bundle adjustment (BA). Nous proposons un nouvel algorithme basé sur une erreur de projection dans laquelle l’index de ligne des points projetés varie pendant l’optimisation afin de garder une cohérence géométrique contrairement aux méthodes existantes qui considère un index fixe (celui mesurés dans l’image). Nous montrons que cela permet de lever la dégénérescence dans le cas où les directions de scan des images sont trop proches (cas très communs avec des caméras embraquées sur un véhicule par exemple). Dans le chapitre VI nous étendons le concept d'homographie aux cas d’images RS en démontrant que la relation point-à-point entre deux images d’un nuage de points coplanaires pouvait s’exprimer sous la forme de 3 à 7 matrices de taille 3X3 en fonction du modèle de mouvement utilisé. Nous proposons une méthode linéaire pour le calcul de ces matrices. Ces dernières sont ensuite utilisées pour résoudre deux problèmes classiques en vision par ordinateur à savoir le calcul du mouvement relatif et le « mosaïcing » dans le cas RS.Dans le chapitre V nous traitons le problème de calcul de pose et de reconstruction multi-vues en établissant une analogie avec les méthodes utilisées pour les surfaces déformables telles que SfT (Structure-from-Template) et NRSfM (Non Rigid Structure-from-Motion). Nous montrons qu’une image RS d’une scène rigide en mouvement peut être interprétée comme une image Global Shutter (GS) d’une surface virtuellement déformée (par l’effet RS). La solution proposée pour estimer la pose et la structure 3D de la scène est ainsi composée de deux étapes. D’abord les déformations virtuelles sont d’abord calculées grâce à SfT ou NRSfM en assumant un modèle GS classique (relaxation du modèle RS). Ensuite, ces déformations sont réinterprétées comme étant le résultat du mouvement durant l’acquisition (réintroduction du modèle RS). L’approche proposée présente ainsi de meilleures propriétés de convergence que les approches existantes. / Many modern CMOS cameras are equipped with Rolling Shutter (RS) sensors which are considered as low cost, low consumption and fast cameras. In this acquisition mode, the pixel rows are exposed sequentially from the top to the bottom of the image. Therefore, images captured by moving RS cameras produce distortions (e.g. wobble and skew) which make the classic algorithms at best less precise, at worst unusable due to singularities or degeneracies. The goal of this thesis is to propose a general framework for modelling and solving structure from motion (SfM) with RS cameras. Our approach consists in addressing each sub-task of the SfM pipe-line (namely image correction, absolute and relative pose estimation and bundle adjustment) and proposing improvements.The first part of this manuscript presents a novel RS correction method which uses line features. Unlike existing methods, which uses iterative solutions and make Manhattan World (MW) assumption, our method R4C computes linearly the camera instantaneous-motion using few image features. Besides, the method was integrated into a RANSAC-like framework which enables us to detect curves that correspond to actual 3D straight lines and reject outlier curves making image correction more robust and fully automated.The second part revisits Bundle Adjustment (BA) for RS images. It deals with a limitation of existing RS bundle adjustment methods in case of close read-out directions among RS views which is a common configuration in many real-life applications. In contrast, we propose a novel camera-based RS projection algorithm and incorporate it into RSBA to calculate reprojection errors. We found out that this new algorithm makes SfM survive the degenerate configuration mentioned above.The third part proposes a new RS Homography matrix based on point correspondences from an RS pair. Linear solvers for the computation of this matrix are also presented. Specifically, a practical solver with 13 point correspondences is proposed. In addition, we present two essential applications in computer vision that use RS homography: plane-based RS relative pose estimation and RS image stitching. The last part of this thesis studies absolute camera pose problem (PnP) and SfM which handle RS effects by drawing analogies with non-rigid vision, namely Shape-from-Template (SfT) and Non-rigid SfM (NRSfM) respectively. Unlike all existing methods which perform 3D-2D registration after augmenting the Global Shutter (GS) projection model with the velocity parameters under various kinematic models, we propose to use local differential constraints. The proposed methods outperform stat-of-the-art and handles configurations that are critical for existing methods.

En informatique, la vision par ordinateur s’attache à extraire de l’information à partir de caméras. Les capteurs de celles-ci peuvent être produits avec la technologie CMOS que nous retrouvons dans les appareils mobiles en raison de son faible coût et d’un encombrement réduit. Cette technologie permet d’acquérir rapidement l’image en exposant les lignes de l’image de manière séquentielle. Cependant cette méthode produit des déformations dans l’image s’il existe un mouvement entre la caméra et la scène filmée. Cet effet est connu sous le nom de «Rolling Shutter» et de nombreuses méthodes ont tenté de corriger ces artefacts. Plutôt que de le corriger, des travaux antérieurs ont développé des méthodes pour extraire de l’information sur le mouvement à partir de cet effet. Ces méthodes reposent sur une extension de la modélisation géométrique classique des caméras pour prendre en compte l’acquisition séquentielle et le mouvement entre le capteur et la scène, considéré uniforme. À partir de cette modélisation, il est possible d’étendre le calcul de pose habituel (estimation de la position et de l’orientation de la scène par rapport au capteur) pour estimer aussi les paramètres du mouvement. Dans la continuité de cette démarche, nous présenterons une généralisation à des mouvements non-uniformes basée sur un lissage des dérivées des paramètres de mouvement. Ensuite nous présenterons une modélisation polynomiale du «Rolling Shutter» et une méthode d’optimisation globale pour l’estimation de ces paramètres. Correctement implémenté, cela permet de réaliser une mise en correspondance automatique entre le modèle tridimensionnel et l’image. Pour terminer nous comparerons ces différentes méthodes tant sur des données simulées que sur des données réelles et conclurons. / Computer Vision, a field of Computer Science, is about extracting information from cameras. Their sensors can be produced using the CMOS technology which is widely used on mobile devices due to its low cost and volume. This technology allows a fast acquisition of an image by sequentially exposin the scan-line. However this method produces some deformation in the image if there is a motion between the camera and the filmed scene. This effect is known as Rolling Shutter and various methods have tried to remove these artifacts. Instead of correcting it, previous works have shown methods to extract information on the motion from this effect. These methods rely on a extension of the usual geometrical model of cameras by taking into account the sequential acquisition and the motion, supposed uniform, between the sensor and the scene. From this model, it’s possible to extend the usual pose estimation (estimation of position and orientation of the camera in the scene) to also estimate the motion parameters. Following on from this approach, we will present an extension to non-uniform motions based on a smoothing of the derivatives of the motion parameters. Afterwards, we will present a polynomial model of the Rolling Shutter and a global optimisation method to estimate the motion parameters. Well implemented, this enables to establish an automatic matching between the 3D model and the image. We will conclude with a comparison of all these methods using either simulated or real data.

Shutter-less infrared cameras based on microbolometer focal plane arrays (FPAs) are the most widely used cameras in thermography, in particular in the fields of handheld devices and small distributed sensors. For acceptable measurement uncertainty values the disturbing influences of changing thermal ambient conditions have to be treated corresponding to temperature measurements of the thermal conditions inside the camera. We propose a compensation approach based on calibration measurements where changing external conditions are simulated and all correction parameters are determined. This allows to process the raw infrared data and to consider all disturbing influences. The effects on the pixel responsivity and offset voltage are considered separately. The responsivity correction requires two different, alternating radiation sources. This paper presents the details of the compensation procedure and discusses relevant aspects to gain low temperature measurement uncertainty.

<p>An integrated circuit for evaluation of APS technology has been implemented in a 0.35 um process. The APS features snapshot operation and the readout circuitry can carry out: CDS, DS, and analog summing all in one circuit that is fully programmable. The output from the chip is a differential analog signal, intended to be connected to a high-speed ADC on an evaluation board. The sensor is fully compatible with current IVP camera systems, hence, the evaluation board should be easy to design. </p><p>Several small code snippets that illustrate different modes of readout have been outlined, to aid the evaluation of the chip. It should be fairly straightforward to convert these code snippetsinto actual camera code. Furthermore, some code to illustrate a possible application and a faster mode of CDS have been indicated. </p><p>Six types of APs have been implemented. They differ regarding diode type and implementation of the sampling capacitor. Design instructions and models for hand calculation have been described. The models have in most cases been validated by simulations and it has been shown that a readout speed of 8 MHz is possible to obtain, even for a larger sensor than this test chip. The desired resolution of 8 bits cannot be obtained for high levels of illumination. However, for low levels of illumination a resolution as high as 10 bits is possible. </p><p>The chip layout has been validated to a large extent and should result in a fully functional chip, if manufactured. However, in the eventuality that IVP decides to manufacture this chip it is recommended to use the newer CAD tools, not available to the author at the time of implementation, to check the chip design for DRC and LVS errors.</p>

<p>An active optical shutter for infrared light (3-5 μm) has been designed, exploiting the phase transition in thermochromic vanadium dioxide (VO₂). A spin coating processing route for VO₂ films has been adapted to manufacture reproducible depositions onto sapphire (Al₂O₃) substrates. The VO₂ films have been characterized by X-ray powder diffraction (XRPD) and infrared spectroscopy (FTIR), showing 55 % transmittance in the open mode and 0.1 % in the closed mode.</p><p>The VO₂ film temperature determines the operating mode of the shutter, and a resistive circuit of gold was deposited on top of the film for heating purposes. Switching times from the open to the closed mode down to 15 ms have been measured.</p><p>This work is a part of a comprehensive project at the Swedish Defence Research Agency (FOI), aiming to improve active components for protection against lasers. The shutter within this work is at this stage an early prototype, and needs further development and complementary systems such as a control unit to be implemented in an optical system.</p>

<p>This report is the result of a master thesis at the Department of Mechanical Engineering, Division of Industrial and Economical Development, at Linköping University of Technology. The project was carried out on behalf of Fanerami AB, Mjölby and includes 20 weeks of full time schedule work.</p><p>Cabinets with vertical mounted roller shutters all share the same problem, namely weight balancing. A roller shutter has an equilibrium area where the sum of the different forces acting on the shutter equals zero. As the shutter leaves that area it will pick up speed as it falls towards the bottom. When the shutter hits the bottom of the cabinet there is a potential risk of getting fingers caught and in this project a solution to prevent this from happening was to be presented.</p><p>In this project, three concepts have been generated with the following different purposes:</p><p>•In concept 1 the risk of getting any fingers caught when the shutter closes was to be eliminated in an efficient and inexpensive way.</p><p>•In concept 2 the roller shutter had to be balanced at all time providing the possibility to keep the cabinet open in any position.</p><p>•The last and most challenging task was to create a remote controlled TV cabinet. The shutters were mounted horizontally and were to be opened simply by pushing the bottom either on the remote transmitter unit or the cabinet.</p><p>The first concept uses a small oil damper, often seen in kitchen cabinets and drawers. The roller shutter applies pressure on the damper via a lever that transforms the shutter motion along a certain path to a smaller motion in the direction of the damper. By doing so, a smaller damper with higher pressure can be used. Dampers are available with several different pressures and can be mounted in five different positions; therefore the concept can be adjusted to fit almost any cabinet.</p><p>Concept 2 consists of a thin cord winded around a wheel. The wheel is equipped with a spring that tightens when the cord is pulled in. The spring produces an equal and opposite force that holds the shutter in balance. As the wheel is mounted in the upper front of the cabinet, underneath the shutter, it will almost be invisible. The concept is easy to adjust to most cabinets by simply attach the shutter at different levels. In that way the spring will be tightened to equal the weight of the shutter.</p><p>The design of the TV cabinet has been a working progress throughout the whole project. A couple of prototypes were produced before the final concept. The mechanical solution is based on two pinion rackets placed one on each side of a rotating pinion powered by a motor through a planetary gearbox. To control the cabinet Electronic Solutions AB developed an electronic system that made it possible to open and close the shutters with a remote control or a switch on the cabinet.</p>

We describe the design and implementation of a high dynamic range (HDR) imaging system capable of capturing RGB color images with a dynamic range of 10,000,000 : 1 at 25 frames per second. We use a highly programmable camera unit with high throughput A/D conversion, data processing and data output. HDR acquisition is performed by multiple exposures in a continuous rolling shutter progression over the sensor. All the different exposures for one particular row of pixels are acquired head to tail within the frame time, which means that the time disparity between exposures is minimal, the entire frame time can be used for light integration and the longest expo- sure is almost the entire frame time. The system is highly configurable, and trade-offs are possible between dynamic range, precision, number of exposures, image resolution and frame rate.