The power and energy optimization of a device can lead to a reduced cost, smaller area, better temperature performance, and higher lifetime. Furthermore, in systems that have limited power budget, it allows running simultaneously more functionalities or using features that require higher power demand.Therefore, both from the user and the company perspective, the value of a product increases as the energy optimization improves. For nighttime surveillance video recording, it is common to use infrared illumination to light the targetscene, which draws a significant portion of the total camera energy consumption. This master thesis examines and discusses how stroboscopic infrared illumination can enhance the energy efficiency in videorecording cameras with rolling shutter image sensors. This report analyzes LED driver circuits, recommends methodologies, and sorts the most relevant parameters to help to dimension and design the illumination system for a light-strobing system. A promising field of use for this technique has been found to be the license-plate recognition (LPR) scenario, for which this thesis dedicates a chapter in this document. This project has been developed at AXIS Communications, where a prototype has been built for one of their network security cameras. The prototype has been tested for LPR for both strobing light systems and conventional IR lighting systems. The results obtained prove that the energy efficiency of the illumination system can be improved more than 95% when stroboscopic illumination is used. / Effektförbrukning och energioptimering av en produkt kan leda till lägre kostnad, mindre storlek, bättre temperaturprestanda och högre livslängd. I system med begränsad effektbudget möjliggör detta dessutom aktivering av fler funktioner samtidigt, eller användning av funktioner med högre strömförbrukning. Därmed gör energioptimering att produktens värde ökar både för användaren och för företaget som tillverkar den. För videoinspelning med övervakningskamera nattetid är det vanligt att använda infraröd belysning för att belysa scenen, vilket ofta förbrukar en betydande del av kamerans totala effektbudget. Detta examensarbete undersöker och diskuterar hur blixtrande (Eng. strobed) infraröd belysning kan förbättra energieffektiviteten vid videoinspelning med bildsensorer med rullande slutare. I denna rapport analyseras LED-drivkretsarna, metodik rekommenderas samt att de mest relevanta parametrarna för att dimensionera och designa ett belysningssystem baserad på strobed IR-belysning sorteras ut. Ett lovande användningsområde för denna teknik har visat sig vara LPR-scenariot (License Plate Recognition), vilket diskuteras i ett eget kapitel i denna rapport. Projektet har genomförts på AXIS Communications, där en prototyp har byggts baserat på en av dess nätverkskameror. Prototypen har utvärderats LPR-sammanhang med både strobed och konventionellt IR belysningssystem. De erhållna resultaten visar att energieffektiviteten hos belysningssystemet kan förbättras med mer än 95% när blixtrande belysning används.

Most mobile video-recording devices of today, e.g. cell phones and music players, make use of a Rolling Shutter camera. The camera captures video by recording every frame line-by-line from top to bottom leading to image distortion when either the target or camera is moving. Capturing video by hand also leads to visible frame-to-frame jitter. This thesis presents algorithms for estimation of camera orientations using accelerometer and gyroscope. These estimates can be used to reduce the image distortion caused by camera motion using image processing. In addition an adaptive low pass filtering algorithm used to produce a smooth camera motion is presented. Using the smooth motion the frame-to-frame jitter can be reduced. The algorithms are implemented on the iPod 4 and two output videos are evaluated in a blind experiment with 30 participants. Here, videos are compared to those of competing video stabilization software. The results indicate that the iPod 4 application performs equal or better than its competitors. Also the iPod 4 accelerometer and gyroscope are compared to high end reference sensors in terms of bias and variance. / Det är vanligt att dagens mobiltelefoner använder en kamera med rullande slutare för videoinspelning. Dessa kameror fångar varje bild rad för rad från topp till botten vilket leder till bilddistorsion när antingen målet eller kameran rör sig. Inspelade videor upplevs även som skakiga eftersom kameran ej är stilla under inspelningen. I detta examensarbete presenteras algoritmer för skattning av kamerans orientering med hjälp av accelerometer och gyroskop. Skattningarna används sedan för att reducera bilddistorsionen som uppstår då kameran rör sig. En adaptiv algoritm för lågpassfiltrering av kamerans rörelse presenteras. Den mjuka rörelsen som produceras används sedan för att reducera skakigheten i filmerna. Algoritmerna implementeras på iPod 4 och de resulterande filmerna utvärderas i ett blindtest med 30 deltagare. Filmerna jämförs med versioner stabiliserade av konkurrerande mjukvara. Resultaten visar att iPod-applikationen producerar ett lika bra eller bättre resultat än konkurrenterna. Accelerometern och gyroskopet på iPod 4 jämförs även med referenssensorer i den högre prisklassen.

Most mobile video-recording devices of today, e.g. cell phones and music players, make use of a rolling shutter camera. A rolling shutter camera captures video by recording every frame line-by-line from top to bottom of the image, leading to image distortions in situations where either the device or the target is moving. Recording video by hand also leads to visible frame-to-frame jitter. In this thesis, methods to decrease distortion caused by the motion of a video-recording device with a rolling shutter camera are presented. The methods are based on estimating the orientation of the camera from gyroscope and accelerometer measurements. The algorithms are implemented on the iPod Touch 4, and the resulting videos are compared to those of competing stabilization software, both commercial and free, in a series of blind experiments. The results from this user study shows that the methods presented in the thesis perform equal to or better than the others. / Mobiltelefoner, mp3-spelare och andra bärbara enheter som kan spela in film har ofta en kamera med rullande slutare. En sådan kamera fångar varje bild rad för rad, från topp till botten. Detta resulterar i distortion i bilden om antingen enheten eller objekt i bilden rör sig. Att filma för hand introducerar också skakighet i filmen. I det här examensarbetet presenteras metoder för att minska den distortion som uppstår till följd av att en enhet med rullande slutare förflyttas under inspelning av en film. Metoderna bygger på estimering av kamerans orientering, utgående från mätdata från gyroskop och accelerometer. Algoritmerna har implementerats på en iPod Touch 4 och de resulterande filmerna har jämförts med de från konkurrerande program i en serie blindtester. Resultaten från denna undersökning visar att metoderna som presenteras i examensarbetet är lika bra eller bättre än de övriga.

Almost all cell-phones and camcorders sold today are equipped with a CMOS (Complementary Metal Oxide Semiconductor) image sensor and there is also a general trend to incorporate CMOS sensors in other types of cameras. The sensor has many advantages over the more conventional CCD (Charge-Coupled Device) sensor such as lower power consumption, cheaper manufacturing and the potential for on-chip processing. Almost all CMOS sensors make use of what is called a rolling shutter. Compared to a global shutter, which images all the pixels at the same time, a rolling-shutter camera exposes the image row-by-row. This leads to geometric distortions in the image when either the camera or the objects in the scene are moving. The recorded videos and images will look wobbly (jello effect), skewed or otherwise strange and this is often not desirable. In addition, many computer vision algorithms assume that the camera used has a global shutter, and will break down if the distortions are too severe. In airborne remote sensing it is common to use push-broom sensors. These sensors exhibit a similar kind of distortion as a rolling-shutter camera, due to the motion of the aircraft. If the acquired images are to be matched with maps or other images, then the distortions need to be suppressed. The main contributions in this thesis are the development of the three dimensional models for rolling-shutter distortion correction. Previous attempts modelled the distortions as taking place in the image plane, and we have shown that our techniques give better results for hand-held camera motions. The basic idea is to estimate the camera motion, not only between frames, but also the motion during frame capture. The motion can be estimated using inter-frame image correspondences and with these a non-linear optimisation problem can be formulated and solved. All rows in the rolling-shutter image are imaged at different times, and when the motion is known, each row can be transformed to the rectified position. In addition to rolling-shutter distortions, hand-held footage often has shaky camera motion. It has been shown how to do efficient video stabilisation, in combination with the rectification, using rotation smoothing. In the thesis it has been explored how to use similar techniques as for the rolling-shutter case in order to correct push-broom images, and also how to rectify 3D point clouds from e.g. the Kinect depth sensor. / VGS

In space exploration optical imaging is one of the key measurements conducted, with a vast majority of missions heavily relying on optical data acquisition to examine alien worlds. One such endeavor is ESA’s F-class mission Comet Interceptor, a multi-element spacecraft expected to be launched in 2028. It consists of a primary platform and two sub-spacecraft, one of which carrying the Optical Periscopic Imager for Comets (OPIC). An accurate prediction of the generated imagery is of undeniable importance as mission planning and instrument design strongly depend on the real-world output quality of the camera system. In the case of OPIC, the collected image data will be used to reconstruct three dimensional models of targeted celestial bodies. Furthermore, the sub-spacecraft faces a risk of high velocity dust impacts, leading to a limited number of data samples to be broadcasted back to the primary spacecraft before collision. Testing image prioritization algorithms and reconstruction methods prior to mission start requires accurate computer-generated images. Camera sensors and lens systems are subjected to various optical distortions and aberrations that degrade the final image. Popular render engines model those effects to a certain degree only and as a result produce content that is looking too perfect. While more sophisticated software products exist, they often come with compatibility limitations and other drawbacks. This report discusses the most important optical aberrations, as well as their relevance for optical instruments in space applications with a particular focus on the Comet Interceptor mission. The main part of this work is however the implementation of a dedicated software tool that simulates a variety of optical aberrations complementing the basic camera model of the Blender render engine. While its functionality is mostly demonstrated for OPIC, the software is designed with a broad range of usage scenarios in mind.

Le travail de thèse a consisté, en priorité, à s’approprier les technologies d’intégration verticale en usage dans l’industrie pour réaliser des mémoires à plusieurs étages, et à en évaluer l’apport pour les capteurs à pixel CMOS (CPS). Cette approche s’appuie sur la capacité de l’industrie à interconnecter des puces amincies empilées les unes sur les autres. Elle ouvre la perspective d’associer plusieurs microcircuits superposés à un même pixel, en dépits de sa taille réduite. L’interconnexion est donc réalisée au niveau du pixel. Ce saut technologique permet de lever la majorité des obstacles à l’obtention de performances optimales des CPS. On peut en particulier combiner des puces réalisées dans des technologies CMOS très différentes, chacune optimale pour une fonctionnalité précise. La collection des charges du signal peut ainsi être réalisée dans une couche dédiée, les microcircuits de conditionnement analogique des signaux peuvent être concentrés dans une autre couche, une troisième couche pouvant héberger les parties numériques assurant la compression puis la transmission des signaux, etc. Ce progrès se traduit notamment par la possibilité de combiner haute résolution spatiale et lecture rapide, avec une amélioration probable de la tolérance aux rayonnements intenses.On s’affranchit de cette manière des limitations provenant des paramètres de fabrication des fondeurs, qui ne permettent pas à l’heure actuelle, de pleinement exploiter le potentiel des CPS à l’aide d’une technologie CMOS unique. / The thesis has been a priority as taking ownership of vertical integration technologies used in the industry to realize a multistage development, and to evaluate the contributions on CMOS pixel sensors (CPS). 3D integration technologies (3DIT) provide a way to mitigate this hampering correlation between speed and resolution, since they allow to staple layers of readout circuitry on top of the sensing layer, which results in a drastic increase of the functionalities located in (the shadow of) each pixel. A multi-layer structure allows for a higher spatial resolution because more and more transistors may be integrated vertically in a relatively small pixel. Moreover, bringing the components of the sensor closer to each other translates in a faster readout, owing to the reduction in the average length of the inner connecting wires. Vertical integration also opens up the possibility of combining different technologies best suited to each of the sensor main functionalities (signal sensing, analog and digital signal processing and transmission). It overcomes the limitations in this way from the foundry manufacturing parameters, which do not allow to fully exploit the potential ofCPS with a single CMOS technology. 3D-CPS are thus expected to overcome most of the limitations of standard 2DCPS, and are therefore suspected to over new perspectives for the innermost layer of the ILC vertex detector.

Ce travail contribue au programme de recherche et de développement d'un capteur CMOS à pixel qui pourrait satisfaire pleinement les spécifications du nouvel ITS (Inner Tracking System : trajectomètre interne) de l'expérience ALICE. Afin de briser les limites de la CPS de pointe, une technologie CMOS 0.18 µm à quatre puits a été explorée. Les capteurs fabriqués dans cette nouvelle technologie ont montré une meilleure tolérance aux radiations que les capteurs réalisés dans une technologie CMOS 0.35 µm plus ancienne. En outre, cette nouvelle technologie offre la possibilité d’implémenter des transistors de type P dans chaque pixel sans dégrader la capacité de collection de la diode. Il devient donc possible d’intégrer un discriminateur dans chaque pixel et obtenir un pixel à sortie binaire. En conséquence, la consommation sera largement réduite. De plus, le temps de traitement de la ligne peut être potentiellement réduit. Un premier prototype de petite taille, intitulé AROM-0, a été conçu et fabriqué afin d’étudier la faisabilité de la discrimination de signal dans un petit pixel. Dans ce prototype, chaque pixel de surface 22 × 33 µm2 contient une diode de détection, un préamplificateur et un discriminateur à tension d’offset compensée. La performance de bruit des différentes versions de pixels dans le capteur AROM-0 a été évaluée. Ensuite sera détaillé le développement des capteurs AROM-1. Ce sont les capteurs intermédiaires vers le capteur final proposé par notre groupe. Ils ont deux objectifs principaux, l’un est de valider les optimisations de conception du pixel et l’autre est de mettre en place une architecture du capteur évolutive intégrant l’intelligence nécessaire dans le circuit. Cette thèse présente en détail la conception et les résultats de mesure de ces capteurs AROM. / This work is part of the R&D program aimed for a CMOS pixel sensor (CPS) complying with the requirements of the upgrade of the inner tracking system (ITS) of the ALICE experiment. In order break the limitations of the state-of-the-art CPS, a 0.18 µm quadruple-well CMOS process was explored. Besides an enhanced radiation tolerance, with respect to the former sensors fabricated in a 0.35 µm process, the sensor based on this new process allows for full CMOS capability inside the pixel without degradation of the detection efficiency. Therefore, the signal discrimination, which was formerly performed at the column level, can be integrated inside the pixel. As a result, the readout speed and power consumption can be greatly improved as compared to the CPS with column-level discrimination. This work addresses the feasibility study of achieving the signal discrimination withina small pixel (i.e. 22 × 33 µm2), via the prototype named AROM-0. The pixel of AROM-0 contains a sensing diode, a pre-amplifier and an offset compensated discriminator. The noise performance of the various pixel versions implemented in AROM-0 was evaluated. The study was further pursued with the AROM-1 prototypes, incorporating the optimized pixel designs and the necessary on-chip intelligence to approach the final sensor we have proposed for the ALICE-ITS upgrade. This thesis presents in detail the design and the measurement results of these AROM sensors.