Unmanned Aerial Vehicles (UAVs) equipped with lightweight, inexpensive cameras have grown in popularity by enabling new uses of UAV technology. However, the video retrieved from small UAVs is often unwatchable due to high frequency jitter. Beginning with an investigation of previous stabilization work, this thesis discusses the challenges of stabilizing UAV based video. It then presents a software based computer vision framework and discusses its use to develop a real-time stabilization solution. A novel approach of estimating intended video motion is then presented. Next, the thesis proceeds to extend previous target localization work by allowing the operator to easily identify targets—rather than relying solely on color segmentation—to improve reliability and applicability in real world scenarios. The resulting approach creates a low cost and easy to use solution for aerial video display and target localization.

Context: Video stabilization techniques have gained popularity for their permit to obtain high quality video footage even in non-optimal conditions. There have been significant works done on video stabilization by developing different algorithms. Most of the stabilization software displays the missing image areas in stabilized video. In the last few years hand-held video cameras have continued to grow in popularity, allowing everyone to easily produce personal video footage. Furthermore, with online video sharing resources being used by a rapidly increasing number of users, a large proportion of such video footage is shared with wide audiences. Sadly such videos often suffer from poor quality as frame vibration in video makes human perception not comfortable. In this research an attempt has been made to propose a robust video stabilization algorithm that stabilizes the videos effectively. Objectives: The main objective of our thesis work is to perform effective motion estimation using SIFT features to calculate the inter frame motion, allowing to find Global Motion Vectors and optimal motion compensation is to be achieved using adaptive fuzzy filter by removing the unwanted shakiness and preserve the panning leading to stabilized video. Methods: In this study three types of research questions are used- Experimentation and Literature review. To accomplish the goal of this thesis, experimentation is carried out for performing video stabilization. Motion estimation is done using feature based motion estimation using SIFT and GMVs are calculated. The intentional motion is filtered using Adaptive fuzzy filter to preserve panning and Motion compensation is performed to wrap the video to its stabilized position. MOS implies the mean scores of the subjective tests performed according to the recommendations of ITU-R BT.500-13 and ITU-T P.910 to analyze the results of our stabilized videos. Results: As a part of results from our work, we have successfully stabilized the videos of different resolutions from experimentation. Performance of our algorithm is found better using MOS. Conclusions: Video Stabilization can be achieved successfully by using SIFT features with pre conditions defined for feature matching and attempts are made to improve the video stabilization process.

Most mobile video-recording devices of today, e.g. cell phones and music players, make use of a Rolling Shutter camera. The camera captures video by recording every frame line-by-line from top to bottom leading to image distortion when either the target or camera is moving. Capturing video by hand also leads to visible frame-to-frame jitter. This thesis presents algorithms for estimation of camera orientations using accelerometer and gyroscope. These estimates can be used to reduce the image distortion caused by camera motion using image processing. In addition an adaptive low pass filtering algorithm used to produce a smooth camera motion is presented. Using the smooth motion the frame-to-frame jitter can be reduced. The algorithms are implemented on the iPod 4 and two output videos are evaluated in a blind experiment with 30 participants. Here, videos are compared to those of competing video stabilization software. The results indicate that the iPod 4 application performs equal or better than its competitors. Also the iPod 4 accelerometer and gyroscope are compared to high end reference sensors in terms of bias and variance. / Det är vanligt att dagens mobiltelefoner använder en kamera med rullande slutare för videoinspelning. Dessa kameror fångar varje bild rad för rad från topp till botten vilket leder till bilddistorsion när antingen målet eller kameran rör sig. Inspelade videor upplevs även som skakiga eftersom kameran ej är stilla under inspelningen. I detta examensarbete presenteras algoritmer för skattning av kamerans orientering med hjälp av accelerometer och gyroskop. Skattningarna används sedan för att reducera bilddistorsionen som uppstår då kameran rör sig. En adaptiv algoritm för lågpassfiltrering av kamerans rörelse presenteras. Den mjuka rörelsen som produceras används sedan för att reducera skakigheten i filmerna. Algoritmerna implementeras på iPod 4 och de resulterande filmerna utvärderas i ett blindtest med 30 deltagare. Filmerna jämförs med versioner stabiliserade av konkurrerande mjukvara. Resultaten visar att iPod-applikationen producerar ett lika bra eller bättre resultat än konkurrenterna. Accelerometern och gyroskopet på iPod 4 jämförs även med referenssensorer i den högre prisklassen.

Most mobile video-recording devices of today, e.g. cell phones and music players, make use of a rolling shutter camera. A rolling shutter camera captures video by recording every frame line-by-line from top to bottom of the image, leading to image distortions in situations where either the device or the target is moving. Recording video by hand also leads to visible frame-to-frame jitter. In this thesis, methods to decrease distortion caused by the motion of a video-recording device with a rolling shutter camera are presented. The methods are based on estimating the orientation of the camera from gyroscope and accelerometer measurements. The algorithms are implemented on the iPod Touch 4, and the resulting videos are compared to those of competing stabilization software, both commercial and free, in a series of blind experiments. The results from this user study shows that the methods presented in the thesis perform equal to or better than the others. / Mobiltelefoner, mp3-spelare och andra bärbara enheter som kan spela in film har ofta en kamera med rullande slutare. En sådan kamera fångar varje bild rad för rad, från topp till botten. Detta resulterar i distortion i bilden om antingen enheten eller objekt i bilden rör sig. Att filma för hand introducerar också skakighet i filmen. I det här examensarbetet presenteras metoder för att minska den distortion som uppstår till följd av att en enhet med rullande slutare förflyttas under inspelning av en film. Metoderna bygger på estimering av kamerans orientering, utgående från mätdata från gyroskop och accelerometer. Algoritmerna har implementerats på en iPod Touch 4 och de resulterande filmerna har jämförts med de från konkurrerande program i en serie blindtester. Resultaten från denna undersökning visar att metoderna som presenteras i examensarbetet är lika bra eller bättre än de övriga.

Automatic video quality assessment has many potential use cases in today’s video-filledsociety, for example, when trying to find highlights in a video. This thesis studies the possibilityof extracting the best segments from a video automatically based on five selected metrics:sharpness, colorfulness, contrast, stability, and aesthetics. Multiple different methods from eachmetric category were compared against each other using datasets with subjective ratings ofimages from KADID-10k and videos from LIVE-Qualcomm. The best method from eachcategory was combined into a single video quality score. The combination was done through aweighted sum, obtained from a least-square fit on the subjective scores of a training dataset(KonViD-1k). The segment of a video with the highest average quality score was chosen as thehighlight. The video quality score achieved Spearman correlations of 0.67 and 0.7 whenevaluated on two validation datasets (KonViD-150k-B and LIVE-VQC). In conclusion, themetrics work well, but are currently too slow for the intended target platform (mobile devices)and thus future work should focus on improving their performance.

Video acquired using a camera mounted on a mini Unmanned Air Vehicle (mUAV) may be very helpful in Wilderness Search and Rescue and many other applications but is commonly plagued with limited spatial and temporal field of views, distractive jittery motions, disorienting rotations, and noisy and distorted images. These problems collectively make it very difficult for human viewers to identify objects of interest as well as infer correct orientations throughout the video. In order to expand the temporal and spatial field of view, stabilize, and better orient users of noisy and distorted mUAV video, a method is proposed of estimating in software and in real time the relative motions of each frame to the next by tracking a small subset of features within each frame to the next. Using these relative motions, a local Euclidean mosaic of the video can be created and a curve can be fit to the video's accumulative motion path to stabilize the presentations of both the video and the local Euclidean mosaic. The increase in users' abilities to perform common search-and-rescue tasks of identifying objects of interest throughout the stabilized and locally mosaiced mUAV video is then evaluated. Finally, a discussion of remaining limitations is presented along with some possibilities for future work.

Cette thèse aborde le problème de la calibration d’un ensemble de capteurscomposé d’une centrale inertielle, d’un magnétomètre et d’une caméra, avecpour objectif leur intégration sur un système très compact : un mini-drone.Cette étude expose tout d’abord les contraintes imposées par l’application surle choix des capteurs et les solutions envisagées notamment pour résoudre leproblème de la synchronisation des mesures. Après avoir étudié les techniquesde calibration existantes, une méthode permettant la calibration de l’ensembledes capteurs (accéléromètre, gyromètre, magnétomètre et caméra) est présentée.La solution proposée permet également d’estimer les changements de repèresentre les différents capteurs. Elle a la particularité de ne nécessiter l’emploid’aucun matériel particulier. D’autre part, l’intégration de ces capteurs dans unsystème aussi compact soulève de nouvelles difficultés. Dans ces conditions, leschamps magnétiques créés par les actionneurs du drone perturbent les mesuresdu magnétomètre se trouvant à proximité. Une nouvelle méthode est proposéeafin d’estimer et de compenser dynamiquement ces perturbations magnétiquesen fonction de l’état des actionneurs du drone. Enfin, deux applications dusystème comprenant une centrale inertielle et une caméra sont présentées :la construction de mosaïques d’images géo-référencées et la stabilisation devidéos. Ces deux applications exploitent les mesures des capteurs inertiels afind’effectuer un traitement en temps réel pour un coût calculatoire très faible. / This thesis deal with the issue of the calibration of a group of sensor composedof an inertial unit, a magnetometer and a camera. It aims at integratingthem into a very compact system : a mini-drone. First of all, this study outlinesthe constraints imposed by the application on the choice of the sensors andthe solutions considered to solve the measures synchronization issue. Afterstudying existing calibration techniques, a method for the calibration of allthe sensors (accelerometer, gyroscope, magnetometer and camera) is presented.The proposed solution allows to estimate the frame transformation between thedifferent sensors. It has the advantage of not requiring the use of any specialequipment. Furthermore, the integration of these sensors into a compact systemraises new difficulties. Under these conditions, the magnetic fields created bythe drone actuators disrupt magnetometer measurements. A new method isproposed to estimate and compensate for these magnetic disturbances. Thecompensation is dynamically adapted based on the state of the drone actuators.Finally, two applications of the system including an inertial unit and a cameraare presented : the construction of geo-referenced images mosaic and videostabilization. Both applications use measurements of inertial sensors and precisecalibration to perform a real-time processing for a very low computational cost.