Most mobile video-recording devices of today, e.g. cell phones and music players, make use of a Rolling Shutter camera. The camera captures video by recording every frame line-by-line from top to bottom leading to image distortion when either the target or camera is moving. Capturing video by hand also leads to visible frame-to-frame jitter. This thesis presents algorithms for estimation of camera orientations using accelerometer and gyroscope. These estimates can be used to reduce the image distortion caused by camera motion using image processing. In addition an adaptive low pass filtering algorithm used to produce a smooth camera motion is presented. Using the smooth motion the frame-to-frame jitter can be reduced. The algorithms are implemented on the iPod 4 and two output videos are evaluated in a blind experiment with 30 participants. Here, videos are compared to those of competing video stabilization software. The results indicate that the iPod 4 application performs equal or better than its competitors. Also the iPod 4 accelerometer and gyroscope are compared to high end reference sensors in terms of bias and variance. / Det är vanligt att dagens mobiltelefoner använder en kamera med rullande slutare för videoinspelning. Dessa kameror fångar varje bild rad för rad från topp till botten vilket leder till bilddistorsion när antingen målet eller kameran rör sig. Inspelade videor upplevs även som skakiga eftersom kameran ej är stilla under inspelningen. I detta examensarbete presenteras algoritmer för skattning av kamerans orientering med hjälp av accelerometer och gyroskop. Skattningarna används sedan för att reducera bilddistorsionen som uppstår då kameran rör sig. En adaptiv algoritm för lågpassfiltrering av kamerans rörelse presenteras. Den mjuka rörelsen som produceras används sedan för att reducera skakigheten i filmerna. Algoritmerna implementeras på iPod 4 och de resulterande filmerna utvärderas i ett blindtest med 30 deltagare. Filmerna jämförs med versioner stabiliserade av konkurrerande mjukvara. Resultaten visar att iPod-applikationen producerar ett lika bra eller bättre resultat än konkurrenterna. Accelerometern och gyroskopet på iPod 4 jämförs även med referenssensorer i den högre prisklassen.

Most mobile video-recording devices of today, e.g. cell phones and music players, make use of a rolling shutter camera. A rolling shutter camera captures video by recording every frame line-by-line from top to bottom of the image, leading to image distortions in situations where either the device or the target is moving. Recording video by hand also leads to visible frame-to-frame jitter. In this thesis, methods to decrease distortion caused by the motion of a video-recording device with a rolling shutter camera are presented. The methods are based on estimating the orientation of the camera from gyroscope and accelerometer measurements. The algorithms are implemented on the iPod Touch 4, and the resulting videos are compared to those of competing stabilization software, both commercial and free, in a series of blind experiments. The results from this user study shows that the methods presented in the thesis perform equal to or better than the others. / Mobiltelefoner, mp3-spelare och andra bärbara enheter som kan spela in film har ofta en kamera med rullande slutare. En sådan kamera fångar varje bild rad för rad, från topp till botten. Detta resulterar i distortion i bilden om antingen enheten eller objekt i bilden rör sig. Att filma för hand introducerar också skakighet i filmen. I det här examensarbetet presenteras metoder för att minska den distortion som uppstår till följd av att en enhet med rullande slutare förflyttas under inspelning av en film. Metoderna bygger på estimering av kamerans orientering, utgående från mätdata från gyroskop och accelerometer. Algoritmerna har implementerats på en iPod Touch 4 och de resulterande filmerna har jämförts med de från konkurrerande program i en serie blindtester. Resultaten från denna undersökning visar att metoderna som presenteras i examensarbetet är lika bra eller bättre än de övriga.

Two related bodies of work are presented, both of which aim to further the rapid development of next generation concentrating photovoltaic systems using high efficiency multi junction solar cells. They are complementary since the characterization of commercial devices and the systematic application of design principles for future designs must progress in parallel in order to accelerate iterative improvements. First addressed, is the field characterization of state of the art concentrating photovoltaic systems. Performance modeling and root cause analysis of deviations from the modeling results are critical for bringing reliable high value products to the market. Two complementary tools are presented that facilitate acceleration of the development cycle. The “Dynamic real-time I V Curve Measurement System…” provides a live picture of the current-voltage characteristics of a CPV module. This provides the user with an intuitive understanding of how module performance responds under perturbation. The “Shutter technique for noninvasive individual cell characterization in sealed concentrating photovoltaic modules,” allows the user to probe individual cell characteristics within a sealed module. This facilitates non-invasive characterization of modules that are in situ. Together, these tools were used to diagnose the wide spread failure of epoxy connections between the carrier and the emitter of bypass diodes installed in sealed commercial modules. Next, the optimization principals that are used to choose energy yield maximizing bandgap combinations for multi-junction solar cells are investigated. It is well understood that, due to differences in the solar resource in different geographical locations, this is fundamentally a local optimization problem. However, until now, a robust methodology for determining the influences of geography and atmospheric content on the ideal design point has not been developed. This analysis is presented and the influence of changing environment on the representative spectra that are used to optimize bandgap combinations is demonstrated. Calculations are confirmed with ground measurements in Ottawa, Canada and the global trends are refined for this particular location. Further, as cell designers begin to take advantage of more flexible manufacturing processes, it is critical to know if and how optimization criteria must change for solar cells with more junctions. This analysis is expanded to account for the differences between cells with up to 8 subcell bandgaps. A number of software tools were also developed for the Sunlab during this work. A multi-junction solar cell model calibration tool was developed to determine the parameters that describe each subcell. The tool fits a two diode model to temperature dependent measurements of each subcell and provides the fitting parameters so that the performance of multi-junction solar cells composed of those subcells can be modeled for real world conditions before they are put on-sun. A multi-junction bandgap optimization tool was developed to more quickly and robustly determine the ideal bandgap combinations for a set of input spectra. The optimization process outputs the current results during iteration so that they may be visualized. Finally, software tools that compute annual energy yield for input multi-junction cell parameters were developed. Both a brute force tool that computes energy harvested at each time step, and an accelerated tool that first bins time steps into discrete bins were developed. These tools will continue to be used by members of the Sunlab.

Almost all cell-phones and camcorders sold today are equipped with a CMOS (Complementary Metal Oxide Semiconductor) image sensor and there is also a general trend to incorporate CMOS sensors in other types of cameras. The sensor has many advantages over the more conventional CCD (Charge-Coupled Device) sensor such as lower power consumption, cheaper manufacturing and the potential for on-chip processing. Almost all CMOS sensors make use of what is called a rolling shutter. Compared to a global shutter, which images all the pixels at the same time, a rolling-shutter camera exposes the image row-by-row. This leads to geometric distortions in the image when either the camera or the objects in the scene are moving. The recorded videos and images will look wobbly (jello effect), skewed or otherwise strange and this is often not desirable. In addition, many computer vision algorithms assume that the camera used has a global shutter, and will break down if the distortions are too severe. In airborne remote sensing it is common to use push-broom sensors. These sensors exhibit a similar kind of distortion as a rolling-shutter camera, due to the motion of the aircraft. If the acquired images are to be matched with maps or other images, then the distortions need to be suppressed. The main contributions in this thesis are the development of the three dimensional models for rolling-shutter distortion correction. Previous attempts modelled the distortions as taking place in the image plane, and we have shown that our techniques give better results for hand-held camera motions. The basic idea is to estimate the camera motion, not only between frames, but also the motion during frame capture. The motion can be estimated using inter-frame image correspondences and with these a non-linear optimisation problem can be formulated and solved. All rows in the rolling-shutter image are imaged at different times, and when the motion is known, each row can be transformed to the rectified position. In addition to rolling-shutter distortions, hand-held footage often has shaky camera motion. It has been shown how to do efficient video stabilisation, in combination with the rectification, using rotation smoothing. In the thesis it has been explored how to use similar techniques as for the rolling-shutter case in order to correct push-broom images, and also how to rectify 3D point clouds from e.g. the Kinect depth sensor. / VGS

Tato práce se zabývá výběrem vhodného obrazového snímače pro použití v kameře snímající rostliny ve vysokém rozlišení a návrhem vhodného obvodu pro propojení vybraného snímače (SONY IMX253) s vývojovou deskou Avnet MicroZed. Tato práce pojednává o jednotlivých parametrech obrazových snímačů podle kterých je vybírán vhodný obrazový snímač. Je vysvětlen proces výběru vhodného obrazového snímače a podrobněji popsány parametry vybraného snímače. Je naznačena problematika návrhu elektroniky a plošných spojů z hlediska požadavků vysokorychlostních obvodů a citlivých a specifických součástek jako je obrazový snímač. Je nastíněna konfigurace a programování obvodu Xilinx Zynq a nakonec je provedeno zjednodušené teoretické ověření funkčnosti navrženého modulu.

In space exploration optical imaging is one of the key measurements conducted, with a vast majority of missions heavily relying on optical data acquisition to examine alien worlds. One such endeavor is ESA’s F-class mission Comet Interceptor, a multi-element spacecraft expected to be launched in 2028. It consists of a primary platform and two sub-spacecraft, one of which carrying the Optical Periscopic Imager for Comets (OPIC). An accurate prediction of the generated imagery is of undeniable importance as mission planning and instrument design strongly depend on the real-world output quality of the camera system. In the case of OPIC, the collected image data will be used to reconstruct three dimensional models of targeted celestial bodies. Furthermore, the sub-spacecraft faces a risk of high velocity dust impacts, leading to a limited number of data samples to be broadcasted back to the primary spacecraft before collision. Testing image prioritization algorithms and reconstruction methods prior to mission start requires accurate computer-generated images. Camera sensors and lens systems are subjected to various optical distortions and aberrations that degrade the final image. Popular render engines model those effects to a certain degree only and as a result produce content that is looking too perfect. While more sophisticated software products exist, they often come with compatibility limitations and other drawbacks. This report discusses the most important optical aberrations, as well as their relevance for optical instruments in space applications with a particular focus on the Comet Interceptor mission. The main part of this work is however the implementation of a dedicated software tool that simulates a variety of optical aberrations complementing the basic camera model of the Blender render engine. While its functionality is mostly demonstrated for OPIC, the software is designed with a broad range of usage scenarios in mind.

This thesis analyzes the impact of and implements compensation for rolling shutter distortions in the state-of-the-art feature-based visual SLAM system ORB-SLAM3. The compensation method involves rectifying the detected features, and the evaluation was conducted on the "Rolling-Shutter Visual-Inertial Odometry Dataset" from TUM, which comprises of ten sequences recorded with side-by-side synchronized global and rolling shutter cameras in a single room.  The performance of ORB-SLAM3 on rolling shutter without the implemented rectification algorithms substantially decreased in terms of accuracy and robustness. The global shutter camera achieved centimeter or even sub-centimeter accuracy, while the rolling shutter camera's accuracy could reach the decimeter range in the more challenging sequences. Also, specific individual executions using a rolling shutter camera could not track the trajectory effectively, indicating a degradation in robustness. The effects of rolling shutter in inertial ORB-SLAM3 were even more pronounced with higher trajectory errors and outright failure to track in some sequences. This was the case even though using inertial measurements with the global shutter camera resulted in better accuracy and robustness compared to the non-inertial case.  The rectification algorithms implemented in this thesis yielded significant accuracy increases of up to a 7x relative improvement for the non-inertial case, which turned trajectory errors back to the centimeter scale from the decimeter one for the more challenging sequences. For the inertial case, the rectification scheme was even more crucial. It resulted in better trajectory accuracies, better than the non-inertial case for the less challenging sequences, and made tracking possible for the more challenging ones.

In the past decade, camera manufacturers have offered smaller form factors, smaller pixel sizes (leading to higher resolution images), and faster processing chips to increase the performance of consumer cameras. However, these conventional approaches have failed to capitalize on the spatio-temporal redundancy inherent in images, nor have they adequately provided a solution for finding $3$D point correspondences for cameras sampling different bands of the visible spectrum. In this thesis, we pose the following question---given the repetitious nature of image patches, and appropriate camera architectures, can statistical models be used to increase temporal, structural, or spectral resolution? While many techniques have been suggested to tackle individual aspects of this question, the proposed solutions either require prohibitively expensive hardware modifications and/or require overly simplistic assumptions about the geometry of the scene. We propose a two-stage solution to facilitate image reconstruction; 1) design a linear camera system that optically encodes scene information and 2) recover full scene information using prior models learned from statistics of natural images. By leveraging the tendency of small regions to repeat throughout an image or video, we are able to learn prior models from patches pulled from exemplar images. The quality of this approach will be demonstrated for two application domains, using low-speed video cameras for high-speed video acquisition and multi-spectral fusion using an array of cameras. We also investigate a conventional approach for finding 3D correspondence that enables a generalized assorted array of cameras to operate in multiple modalities, including multi-spectral, high dynamic range, and polarization imaging of dynamic scenes.

Le travail de thèse a consisté, en priorité, à s’approprier les technologies d’intégration verticale en usage dans l’industrie pour réaliser des mémoires à plusieurs étages, et à en évaluer l’apport pour les capteurs à pixel CMOS (CPS). Cette approche s’appuie sur la capacité de l’industrie à interconnecter des puces amincies empilées les unes sur les autres. Elle ouvre la perspective d’associer plusieurs microcircuits superposés à un même pixel, en dépits de sa taille réduite. L’interconnexion est donc réalisée au niveau du pixel. Ce saut technologique permet de lever la majorité des obstacles à l’obtention de performances optimales des CPS. On peut en particulier combiner des puces réalisées dans des technologies CMOS très différentes, chacune optimale pour une fonctionnalité précise. La collection des charges du signal peut ainsi être réalisée dans une couche dédiée, les microcircuits de conditionnement analogique des signaux peuvent être concentrés dans une autre couche, une troisième couche pouvant héberger les parties numériques assurant la compression puis la transmission des signaux, etc. Ce progrès se traduit notamment par la possibilité de combiner haute résolution spatiale et lecture rapide, avec une amélioration probable de la tolérance aux rayonnements intenses.On s’affranchit de cette manière des limitations provenant des paramètres de fabrication des fondeurs, qui ne permettent pas à l’heure actuelle, de pleinement exploiter le potentiel des CPS à l’aide d’une technologie CMOS unique. / The thesis has been a priority as taking ownership of vertical integration technologies used in the industry to realize a multistage development, and to evaluate the contributions on CMOS pixel sensors (CPS). 3D integration technologies (3DIT) provide a way to mitigate this hampering correlation between speed and resolution, since they allow to staple layers of readout circuitry on top of the sensing layer, which results in a drastic increase of the functionalities located in (the shadow of) each pixel. A multi-layer structure allows for a higher spatial resolution because more and more transistors may be integrated vertically in a relatively small pixel. Moreover, bringing the components of the sensor closer to each other translates in a faster readout, owing to the reduction in the average length of the inner connecting wires. Vertical integration also opens up the possibility of combining different technologies best suited to each of the sensor main functionalities (signal sensing, analog and digital signal processing and transmission). It overcomes the limitations in this way from the foundry manufacturing parameters, which do not allow to fully exploit the potential ofCPS with a single CMOS technology. 3D-CPS are thus expected to overcome most of the limitations of standard 2DCPS, and are therefore suspected to over new perspectives for the innermost layer of the ILC vertex detector.

Ce travail contribue au programme de recherche et de développement d'un capteur CMOS à pixel qui pourrait satisfaire pleinement les spécifications du nouvel ITS (Inner Tracking System : trajectomètre interne) de l'expérience ALICE. Afin de briser les limites de la CPS de pointe, une technologie CMOS 0.18 µm à quatre puits a été explorée. Les capteurs fabriqués dans cette nouvelle technologie ont montré une meilleure tolérance aux radiations que les capteurs réalisés dans une technologie CMOS 0.35 µm plus ancienne. En outre, cette nouvelle technologie offre la possibilité d’implémenter des transistors de type P dans chaque pixel sans dégrader la capacité de collection de la diode. Il devient donc possible d’intégrer un discriminateur dans chaque pixel et obtenir un pixel à sortie binaire. En conséquence, la consommation sera largement réduite. De plus, le temps de traitement de la ligne peut être potentiellement réduit. Un premier prototype de petite taille, intitulé AROM-0, a été conçu et fabriqué afin d’étudier la faisabilité de la discrimination de signal dans un petit pixel. Dans ce prototype, chaque pixel de surface 22 × 33 µm2 contient une diode de détection, un préamplificateur et un discriminateur à tension d’offset compensée. La performance de bruit des différentes versions de pixels dans le capteur AROM-0 a été évaluée. Ensuite sera détaillé le développement des capteurs AROM-1. Ce sont les capteurs intermédiaires vers le capteur final proposé par notre groupe. Ils ont deux objectifs principaux, l’un est de valider les optimisations de conception du pixel et l’autre est de mettre en place une architecture du capteur évolutive intégrant l’intelligence nécessaire dans le circuit. Cette thèse présente en détail la conception et les résultats de mesure de ces capteurs AROM. / This work is part of the R&D program aimed for a CMOS pixel sensor (CPS) complying with the requirements of the upgrade of the inner tracking system (ITS) of the ALICE experiment. In order break the limitations of the state-of-the-art CPS, a 0.18 µm quadruple-well CMOS process was explored. Besides an enhanced radiation tolerance, with respect to the former sensors fabricated in a 0.35 µm process, the sensor based on this new process allows for full CMOS capability inside the pixel without degradation of the detection efficiency. Therefore, the signal discrimination, which was formerly performed at the column level, can be integrated inside the pixel. As a result, the readout speed and power consumption can be greatly improved as compared to the CPS with column-level discrimination. This work addresses the feasibility study of achieving the signal discrimination withina small pixel (i.e. 22 × 33 µm2), via the prototype named AROM-0. The pixel of AROM-0 contains a sensing diode, a pre-amplifier and an offset compensated discriminator. The noise performance of the various pixel versions implemented in AROM-0 was evaluated. The study was further pursued with the AROM-1 prototypes, incorporating the optimized pixel designs and the necessary on-chip intelligence to approach the final sensor we have proposed for the ALICE-ITS upgrade. This thesis presents in detail the design and the measurement results of these AROM sensors.