This thesis investigates the use of monocular computer vision for odometry, specificallyemploying optical flow techniques. The goal is to develop and evaluate avisual odometry system that accurately estimates the trajectory and rotation of acamera in real-time. The system utilizes a camera and the Lucas-Kanade methodto capture high-frame-rate images, detect and track features within these images,and calculate the camera’s motion based on the movement of these features.Odometry is essential for estimating the position and orientation of moving objects,such as robots or vehicles, over time. Traditional methods rely on wheel encodersand Inertial Measurement Units (IMU), but visual odometry leverages visual datato enhance accuracy and robustness, without the risk of slippage and change inwheel diameter from loads. Furthermore a visual odometry system, like the oneused in this project, is not affected by occlusion.In this project, a camera was set up and calibrated, followed by the implementationof feature detection using the Shi-Tomasi corner detection algorithm. TheLucas-Kanade method was then applied to estimate optical flow, and an affinetransformation was used to compute the translation and rotation of the camera.The system’s performance was evaluated based on accuracy, computational efficiency,and robustness to noise.The results demonstrate that the visual odometry system can effectively track thecamera’s motion with a high degree of accuracy. But with limitations in speed. Thediscussion highlights potential applications in autonomous navigation and areas forfuture improvement, such as integrating additional sensors and enhancing featuredetection algorithm. / Denna rapport undersöker användningen av monokulärt datorseende förodometri, med särskilt fokus på optisk flödesteknik. Målet är att utveckla ochutvärdera ett visuellt odometrisystem som noggrant uppskattar en kameras positionoch rotation i realtid. Systemet använder en kamera och Lucas-Kanademetodenför att fånga bilder i hög frekvens, upptäcka och spåra pixelpunkter inomdessa bilder samt beräkna kamerans rörelse baserat på dessa punkters förflyttning.Odometri är viktigt för att uppskatta position och orientering av rörliga objekt,såsom robotar eller fordon, över tid. Traditionella metoder förlitar sig på hjulenkodersoch Inetrial Measurement Units (IMU), men visuell odometri utnyttjarvisuella data för att förbättra noggrannheten och robustheten, utan risken förslirning och förändringar i hjuldiametern på grund av belastningar. Dessutompåverkas inte ett visuellt odometrisystem, som det som används i detta projekt,av ocklusion.I detta projekt installerades och kalibrerades en kamera, följt av implementeringav funktionsdetektion med hjälp av Shi-Tomasi hörndetektionsalgoritm.Därefter tillämpades Lucas-Kanade-metoden för att uppskatta optiskt flöde, ochen affin transformation användes för att beräkna kamerans förflyttning och rotation.Systemets prestanda utvärderades utifrån noggrannhet, beräkningseffektivitetoch robusthet mot brus.Resultaten visar att det visuella odometrisystemet effektivt kan spåra kameransrörelse med hög noggrannhet. Med begränsning beroende på hastighet. Diskussionenbelyser potentiella tillämpningar inom autonom navigering och områden förframtida förbättringar, såsom integrering av ytterligare sensorer och förbättringav feature detection algoritmen.

We present a system that estimates the motion of a robot relying solely on images from onboard omnidirectional camera (visual odometry). Compared to other visual odometry hardware, ours is unusual in utilizing high resolution, low frame-rate (1 to 3 Hz) omnidirectional camera mounted on a robot that is propelled using continuous tracks. We focus on high precision estimates in scenes, where objects are far away from the camera. This is achieved by utilizing omnidirectional camera that is able to stabilize the motion estimates between camera frames that are known to be ill-conditioned for narrow field of view cameras. We employ feature based-approach for estimation camera motion. Given our hardware, possibly high ammounts of camera rotation between frames can occur. Thus we use techniques of feature matching rather than feature tracking.

We present a system that estimates the motion of a robot relying solely on images from onboard omnidirectional camera (visual odometry). Compared to other visual odometry hardware, ours is unusual in utilizing high resolution, low frame-rate (1 to 3 Hz) omnidirectional camera mounted on a robot that is propelled using continuous tracks. We focus on high precision estimates in scenes, where objects are far away from the camera. This is achieved by utilizing omnidirectional camera that is able to stabilize the motion estimates between camera frames that are known to be ill-conditioned for narrow field of view cameras. We employ feature based-approach for estimation camera motion. Given our hardware, possibly high ammounts of camera rotation between frames can occur. Thus we use techniques of feature matching rather than feature tracking.

<div>For either entertainment or documenting purposes, people are starting to record their life using egocentric cameras, mounted on either a person or a vehicle. Our target is to improve the utility of these egocentric videos. </div><div><br></div><div>For egocentric videos with an entertainment purpose, we aim to enhance the viewing experience to improve overall enjoyment. We focus on First-Person Videos (FPVs), which are recorded by wearable cameras. People record FPVs in order to share their First-Person Experience (FPE). However, raw FPVs are usually too shaky to watch, which ruins the experience. We explore the mechanism of human perception and propose a biometric-based measurement called the Viewing Experience (VE) score, which measures both the stability and the First-person Motion Information (FPMI) of a FPV. This enables us to further develop a system to stabilize FPVs while preserving their FPMI. Experimental results show that our system is robust and efficient in measuring and improving the VE of FPVs.</div><div><br></div><div>For egocentric videos whose goal is documentation, we aim to build a system that can centrally collect, compress and manage the videos. We focus on Dash Camera Videos (DCVs), which are used by people to document the route they drive each day. We proposed a system that can classify videos according to the route they drove using GPS information and visual information. When new DCVs are recorded, their bit-rate can be reduced by jointly compressing them with videos recorded on the similar route. Experimental results show that our system outperforms other similar solutions and the standard HEVC particularly in varying illumination.</div><div><br></div><div>The First-Person Video viewing experience topic and the Dashcam Video compression topic are two representations of applications rely on Visual Odometers (VOs): visual augmentation and robotic perception. Different applications have different requirement for VOs. And the performance of VOs are also influenced by many different factors. To help our system and other users that also work on similar applications, we further propose a system that can investigate the performance of different VOs under various factors. The proposed system is shown to be able to provide suggestion on selecting VOs based on the application.</div>

O problema da localização consiste em estimar a posição de um robô com relação a algum referencial externo e é parte essencial de sistemas de navegação de robôs e veículos autônomos. A localização baseada em odometria visual destaca-se em relação a odometria de encoders na obtenção da rotação e direção do movimento do robô. Esse tipo de abordagem é também uma escolha atrativa para sistemas de controle de veículos autônomos em ambientes urbanos, onde a informação visual é necessária para a extração de informações semânticas de placas, semáforos e outras sinalizações. Neste contexto este trabalho propõe o desenvolvimento de um sistema de odometria visual utilizando informação visual de uma câmera monocular baseado em reconstrução 3D para estimar o posicionamento do veículo. O problema da escala absoluta, inerente ao uso de câmeras monoculares, é resolvido utilizando um conhecimento prévio da relação métrica entre os pontos da imagem e pontos do mundo em um mesmo plano. / The localization problem consists of estimating the position of the robot with regards to some external reference and it is an essential part of robots and autonomous vehicles navigation systems. Localization based on visual odometry, compared to encoder based odometry, stands out at the estimation of rotation and direction of the movement. This kind of approach is an interesting choice for vehicle control systems in urban environment, where the visual information is mandatory for the extraction of semantic information contained in the street signs and marks. In this context this project propose the development of a visual odometry system based on structure from motion using visual information acquired from a monocular camera to estimate the vehicle pose. The absolute scale problem, inherent with the use of monocular cameras, is achieved using som previous known information regarding the metric relation between image points and points lying on a same world plane.

Extrair informação de profundidade a partir de imagens bidimensionais é um importante problema na área de visão computacional. Diversas aplicações se beneficiam desta classe de algoritmos tais como: robótica, a indústria de entretenimento, aplicações médicas para diagnóstico e confecção de próteses e até mesmo exploração interplanetária. Esta aplicação pode ser dividida em duas etapas interdependentes: a estimação da posição e orientação da câmera no momento em que a imagem foi gerada, e a estimativa da estrutura tridimensional da cena. Este trabalho foca em técnicas de visão computacional usadas para estimar a trajetória de um veículo equipado com uma câmera, problema conhecido como odometria visual. Para obter medidas objetivas de eficiência e precisão, e poder comparar os resultados obtidos com o estado da arte, uma base de dados de alta precisão, bastante utilizada pela comunidade científica foi utilizada. No curso deste trabalho novas técnicas para rastreamento de detalhes, estimativa de posição de câmera, cálculo de posição 3D de pontos e recuperação de escala são propostos. Os resultados alcançados superam os mais bem ranqueados trabalhos na base de dados escolhida até o momento da publicação desta tese. / Recovering three-dimensional information from bi-dimensional images is an important problem in computer vision that finds several applications in our society. Robotics, entertainment industry, medical diagnose and prosthesis, and even interplanetary exploration benefit from vision based 3D estimation. The problem can be divided in two interdependent operations: estimating the camera position and orientation when each image was produced, and estimating the 3D scene structure. This work focuses on computer vision techniques, used to estimate the trajectory of a vehicle equipped camera, a problem known as visual odometry. In order to provide an objective measure of estimation efficiency and to compare the achieved results to the state-of-the-art works in visual odometry a high precision popular dataset was selected and used. In the course of this work new techniques for image feature tracking, camera pose estimation, point 3D position calculation and scale recovery are proposed. The achieved results outperform the best ranked results in the popular chosen dataset.

Un grand nombre d'applications de robotique embarquées pourrait bénéficier d'une détection explicite des objets mobiles. A ce jour, la majorité des approches présentées repose sur la classification, ou sur une analyse structurelle de la scène (la V-Disparité est un bon exemple de ces approches). Depuis quelques années, nous sommes témoins d'un intérêt croissant pour les méthodes faisant collaborer activement l'analyse structurelle et l'analyse du mouvement. Ces deux processus sont en effet étroitement liés. Dans ce contexte, nous proposons, à travers de travail de thèse, deux approches différentes. Si la première fait appel à l'intégralité de l'information stéréo/mouvement, la seconde se penche sur le cas des capteurs monoculaires, et permet de retrouver une information partielle.La première approche présentée consiste en un système innovation d'odométrie visuelle. Nous avons en effet démontré que le problème d'odométrie visuelle peut être posé de façon linéaire, alors que l'immense majorité des auteurs sont contraint de faire appel à des méthodes d'optimisation non-linéaires. Nous avons également montré que notre approche permet d'atteindre, voire de dépasser le niveau de performances présenté par des système matériels haut de gamme (type centrale inertielle). A partir de ce système d'odométrie visuelle, nous définissons une procédure permettant de détecter les objets mobiles. Cette procédure repose sur une compensation de l'influence de l'égo-mouvement, puis une mesure du mouvement résiduel. Nous avons ensuite mené une réflexion de fond sur les limitations et les sources d'amélioration de ce système. Il nous est apparu que les principaux paramètres du système de vision (base, focale) ont un impact de premier plan sur les performances du détecteur. A notre connaissance, cet impact n'a jamais été décrit dans la littérature. Il nous semble cependant que nos conclusions peuvent constituer un ensemble de recommandations utiles à tout concepteur de système de vision intelligent.La seconde partie de ce travail porte sur les systèmes de vision monoculaire, et plus précisément sur le concept de C-Vélocité. Alors que la V-Disparité a défini une transformée de la carte de disparité permettant de mettre en avant certains plans de l'image, la C-Vélocité défini une transformée du champ de flot optique, et qui utilise la position du FoE, qui permet une détection facile de certains plans spécifiques de l'image. Dans ce travail, nous présentons une modification de la C-Vélocité. Au lieu d'utiliser un a priori sur l'égo-mouvement (la position du FoE) afin d'inférer la structure de la scène, nous utilisons un a priori sur la structure de la scène afin de localiser le FoE, donc d'estimer l'égo-mouvement translationnel. Les premiers résultats de ce travail sont encourageants et nous permettent d'ouvrir plusieurs pistes de recherches futures. / Many embedded robotic applications could benefit from an explicit detection of mobile objects. To this day, most approaches rely on classification, or on some structural scene analysis (for instance, V-Disparity). During the last few years, we've witnessed a growing interest for collaboration methods, that use actively btw structural analysis and motion analysis. These two processes are, indeed, closely related. In this context, we propose, through this study, two novel approaches that address this issue. While the first one use information from stereo and motion, the second one focuses on monocular systems, and allows us to retrieve a partial information.The first presented approach consists in a novel visual odometry system. We have shown that, even though the wide majority of authors tackle the visual odometry problem as non-linear, it can be shown to be purely linear. We have also shown that our approach achieves performances, as good as, or even better than the ones achieved by high-end IMUs. Given this visual odometry system, we then define a procedure allowing us to detect mobile objects. This procedure relies on a compensation of the ego-motion and a measure of the residual motion. We then lead a reflexion on the causes of limitation and the possible sources of improvement of this system. It appeared that the main parameters of the vision system (baseline, focal length) have a major impact on the performances of our detector. To the best of our knowledge, this impact had never been discussed, prior to our study. However, we think that our conclusion could be used as a set of recommendations, useful for every designer of intelligent vision system.the second part of this work focuses on monocular systems, and more specifically on the concept of C-Velocity. When V-Disparity defined a disparity map transform, allowing an easy detection of specific planes, C-Velocity defines a transform of the optical flow field, using the position of the FoE, allowing an easy detection of specific planes. Through this work, we present a modification of the C-Velocity concept. Instead of using a priori knowledge of the ego-motion (the position of the FoE) in order to determine the scene structure, we use a prior knowledge of the scene structure in order to localize the FoE, thus the translational ego-motion. the first results of this work are promising, and allow us to define several future works.

We present a system that estimates the motion of a robot relying solely on images from onboard omnidirectional camera (visual odometry). Compared to other visual odometry hardware, ours is unusual in utilizing high resolution, low frame-rate (1 to 3 Hz) omnidirectional camera mounted on a robot that is propelled using continuous tracks. We focus on high precision estimates in scenes, where objects are far away from the camera. This is achieved by utilizing omnidirectional camera that is able to stabilize the motion estimates between camera frames that are known to be ill-conditioned for narrow field of view cameras and the fact that low frame-rate of the imaging system allows us to focus computational resources on utilizing high resolution images. We employ feature based-approach for estimation camera motion. Given our hardware, possibly high ammounts of camera rotation between frames can occur. Thus we use techniques of feature matching rather than feature tracking.

Visual Simultaneous Localization And Mapping (SLAM) allows for three dimensionalreconstruction from a camera’s output and simultaneous positioning of the camera withinthe reconstruction. With use cases ranging from autonomous vehicles to augmentedreality, the SLAM field has garnered interest both commercially and academically. A SLAM system performs odometry as it estimates the camera’s movement throughthe scene. The incremental estimation of odometry is not error free and exhibits driftover time with map inconsistencies as a result. Detecting the return to a previously seenplace, a loop, means that this new information regarding our position can be incorporatedto correct the trajectory retroactively. Loop detection can also facilitate relocalization ifthe system loses tracking due to e.g. heavy motion blur. This thesis proposes an odometric system making use of bundle adjustment within akeyframe based stereo SLAM application. This system is capable of detecting loops byutilizing the algorithm FAB-MAP. Two aspects of this system is evaluated, the odometryand the capability to relocate. Both of these are evaluated using the EuRoC MAV dataset,with an absolute trajectory RMS error ranging from 0.80 m to 1.70 m for the machinehall sequences. The capability to relocate is evaluated using a novel methodology that intuitively canbe interpreted. Results are given for different levels of strictness to encompass differentuse cases. The method makes use of reprojection of points seen in keyframes to definewhether a relocalization is possible or not. The system shows a capability to relocate inup to 85% of all cases when a keyframe exists that can project 90% of its points intothe current view. Errors in estimated poses were found to be correlated with the relativedistance, with errors less than 10 cm in 23% to 73% of all cases. The evaluation of the whole system is augmented with an evaluation of local imagedescriptors and pose estimation algorithms. The descriptor SIFT was found to performbest overall, but demanding to compute. BRISK was deemed the best alternative for afast yet accurate descriptor. Conclusions that can be drawn from this thesis is that FAB-MAP works well fordetecting loops as long as the addition of keyframes is handled appropriately.

In this master thesis a visual odometry system is implemented and explained. Visual odometry is a technique, which could be used on autonomous vehicles to determine its current position and is preferably used indoors when GPS is notworking. The only input to the system are the images from a stereo camera and the output is the current location given in relative position. In the C++ implementation, image features are found and matched between the stereo images and the previous stereo pair, which gives a range of 150-250 verified feature matchings. The image coordinates are triangulated into a 3D-point cloud. The distance between two subsequent point clouds is minimized with respect to rigid transformations, which gives the motion described with six parameters, three for the translation and three for the rotation. Noise in the image coordinates gives reconstruction errors which makes the motion estimation very sensitive. The results from six experiments show that the weakness of the system is the ability to distinguish rotations from translations. However, if the system has additional knowledge of how it is moving, the minimization can be done with only three parameters and the system can estimate its position with less than 5 % error.