La surveillance visuelle des objets dynamiques dans les carrefours routiers a été un sujet de recherche majeur au sein des communautés de vision par ordinateur et de transports intelligents, ces dernières années. De nombreux projets ont été menés afin d’améliorer la sécurité dans le contexte très particulier des carrefours. Notre analyse approfondie de l’état de l’art révèle que la majorité des systèmes en bord de voie, utilisent la vision monoculaire. Dans cette thèse, nous présentons un systèmenon-intrusif, de stéréovision-fisheye à large entraxe. Le dispositif proposé est particulièrement adapté aux carrefours ruraux ou sans signalisation. Notre objectif principal est la localisation des véhicules afin de reconstruire leurs trajectoires. Pour ce faire, l’estimation de la calibration extrinsèque entre les caméras est nécessaire afin d’effectuer des analyses à l’échelle métrique. Cette tâche s’avère très complexe dans notre configuration de déploiement. En effet la grande distance entre les caméras, la différence de vue et la forte présence de végétation, rendent inapplicables les méthodes de calibration qui requièrent la mise en correspondance d’images de mires. Il est donc nécessaire d’avoir une solution indépendante de la géométrie de la scène. Ainsi, nous proposons une méthode automatique reposant sur l’idée que les véhicules mobiles peuvent être utilisés comme objets dynamiques de calibration. Il s’agit d’une approche de type Structure à partir du Mouvement, découplée en l’estimation de la rotation extrinsèque à partir de points de fuite, suivie du calcul de la translation extrinsèque à l’échelle absolue par mise en correspondance de plans virtuels. Afin de généraliser notre méthode, nous adoptons le modèle de caméra sphérique sous l’hypothèse d’un mouvement plan. Des expérimentations conduites en laboratoire, puis dans des carrefours en Normandie, permettent de valider notre approche. Les paramètres extrinsèques sont alors directement exploités pour la trajectographie métrique des véhicules, en vue d’évaluer le risque et procéder à un diagnostic des intersections rurales. / Visual surveillance of dynamic objects at road intersections has been an active research topic in the computer vision and intelligent transportations systems communities, over the past decades. Several projects have been carried out in order to enhance the safety of drivers in the special context of intersections. Our extensive review of related studies revealedthat most roadside systems are based on monocular vision and provide output results generally in the image domain. In this thesis, we introduce a non-intrusive, wide-baseline stereoscopic system composed of fisheye cameras, perfectly suitable for rural or unsignalized intersections. Our main goal is to achieve vehicle localization and metric trajectory estimation in the world frame. For this, accurate extrinsic calibration is required to compute metric information. But the task is quite challenging in this configuration, because of the wide-baseline, the strong view difference between the cameras, and the important vegetation. Also, pattern-based methods are hardly feasible without disrupting the traffic. Therefore, we propose a points-correspondence-free solution. Our method is fully-automatic and based on a joint analysis of vehicles motion and appearance, which areconsidered as dynamic calibration objects. We present a Structure-from-Motion approach decoupled into the estimation of the extrinsic rotation from vanishing points, followed by the extrinsic translation at scale from a virtual-plane matching strategy. For generalization purposes we adopt the spherical camera model under the assumption of planar motion. Extensive experiments both in the lab and at rural intersections in Normandy allow to validate our work, leading to accurate vehicle motion analysis for risk assessment and safety diagnosis at rural intersections.

Cette recherche vise à la modélisation 3D depuis un système de vision fisheye embarqué, utilisée pour une application GNSS dans le cadre du projet Predit CAPLOC. La propagation des signaux satellitaires en milieu urbain est soumise à des réflexions sur les structures, altérant la précision et la disponibilité de la localisation. L’ambition du projet est (1) de définir un système de vision omnidirectionnelle capable de fournir des informations sur la structure 3D urbaine et (2) de montrer qu’elles permettent d’améliorer la localisation.Le mémoire expose les choix en (1) calibrage automatique, (2) mise en correspondance entre images, (3) reconstruction 3D ; chaque algorithme est évalué sur images de synthèse et réelles. De plus, il décrit une manière de corriger les réflexions des signaux GNSS depuis un nuage de points 3D pour améliorer le positionnement. En adaptant le meilleur de l’état de l’art du domaine, deux systèmes sont proposés et expérimentés. Le premier est un système stéréoscopique à deux caméras fisheye orientées vers le ciel. Le second en est l’adaptation à une unique caméra.Le calibrage est assuré à travers deux étapes : l’algorithme des 9 points adapté au modèle « équisolide » couplé à un RANSAC, suivi d’un affinement par optimisation Levenberg-Marquardt. L’effort a été porté sur la manière d’appliquer la méthode pour des performances optimales et reproductibles. C’est un point crucial pour un système à une seule caméra car la pose doit être estimée à chaque nouvelle image.Les correspondances stéréo sont obtenues pour tout pixel par programmation dynamique utilisant un graphe 3D. Elles sont assurées le long des courbes épipolaires conjuguées projetées de manière adaptée sur chaque image. Une particularité est que les distorsions ne sont pas rectifiées afin de ne pas altérer le contenu visuel ni diminuer la précision. Dans le cas binoculaire il est possible d’estimer les coordonnées à l’échelle. En monoculaire, l’ajout d’un odomètre permet d’y arriver. Les nuages successifs peuvent être calés pour former un nuage global en SfM.L’application finale consiste dans l’utilisation du nuage 3D pour améliorer la localisation GNSS. Il est possible d’estimer l’erreur de pseudodistance d’un signal après multiples réflexions et d’en tenir compte pour une position plus précise. Les surfaces réfléchissantes sont modélisées grâce à une extraction de plans et de l’empreinte des bâtiments. La méthode est évaluée sur des paires d’images fixes géo-référencées par un récepteur bas-coût et un récepteur GPS RTK (vérité terrain). Les résultats montrent une amélioration de la localisation en milieu urbain. / This research deals with 3D modelling from an embedded fisheye vision system, used for a GNSS application as part of CAPLOC project. Satellite signal propagation in urban area implies reflections on structures, impairing localisation’s accuracy and availability. The project purpose is (1) to define an omnidirectional vision system able to provide information on urban 3D structure and (2) to demonstrate that it allows to improve localisation.This thesis addresses problems of (1) self-calibration, (2) matching between images, (3) 3D reconstruction ; each algorithm is assessed on computer-generated and real images. Moreover, it describes a way to correct GNSS signals reflections from a 3D point cloud to improve positioning. We propose and evaluate two systems based on state-of-the-art methods. First one is a stereoscopic system made of two sky facing fisheye cameras. Second one is the adaptation of the former to a single camera.Calibration is handled by a two-steps process: the 9-point algorithm fitted to “equisolid” model coupled with a RANSAC, followed by a Levenberg-Marquardt optimisation refinement. We focused on the way to apply the method for optimal and repeatable performances. It is a crucial point for a system composed of only one camera because the pose must be estimated for every new image.Stereo matches are obtained for every pixel by dynamic programming using a 3D graph. Matching is done along conjugated epipolar curves projected in a suitable manner on each image. A distinctive feature is that distortions are not rectified in order to neither degrade visual content nor to decrease accuracy. In the binocular case it is possible to estimate full-scale coordinates.In the monocular case, we do it by adding odometer information. Local clouds can be wedged in SfM to form a global cloud.The end application is the usage of the 3D cloud to improve GNSS localisation. It is possible to estimate and consider a signal pseudodistance error after multiple reflections in order to increase positioning accuracy. Reflecting surfaces are modelled thanks to plane and buildings trace fitting. The method is evaluated on fixed image pairs, georeferenced by a low-cost receiver and a GPS RTK receiver (ground truth). Study results show the localisation improvement ability in urban environment.

The impacts of climate change on temperate forests are predicted to accelerate, with widespread implications for forest biodiversity and function. Remote sensing has provided insights into regional patterns of vegetation dynamics, and experimental studies have demonstrated impacts of specific changes on individual species. However, forests are diverse and complex ecosystems. To understand how different species in different forests respond to interacting environmental pressures, widespread ground-based monitoring is needed. The only practical way to achieve this is through the involvement of non-professional researchers, i.e., with citizen science. However, many techniques used to identify subtle changes in forests require expensive equipment and professional expertise. This thesis aimed to identify practical methods for citizen scientists to collect useful data on forest ecosystem dynamics in relation to climate change. Methods for monitoring tree phenology and canopy-understorey interactions were the main focus, as tree phenology exerts strong control on understorey light and forest biodiversity, and is already responding to climate change. The response of understorey vegetation to canopy closure in four woodlands from a single region of England (Devon) was examined in detail. These geographically close woodlands differed considerably in their composition and seasonal dynamics. The spring period was particularly important for herb-layer development, and small variations in canopy openness had important effects on herb-layer cover and composition. This work highlights the need to monitor a range of different woodlands at the regional scale, with sufficient resolution to pick up small but crucial differences through time. Citizen scientists could help to collect such data by monitoring herb-layer cover and changes in the abundance of key species, alongside monitoring the overstorey canopy. The spring leaf phenology of four canopy trees (ash, beech, oak and sycamore) were monitored intensively in one woodland using a range of methods: counts, percentage estimates and photography. First budburst and leaf expansion dates were compared with estimates of leaf expansion timing and rate, derived from time-series data using logistic growth models. Frequently used first-event dates were potentially misleading due to high variation in leaf development rates within and between species. Percentage estimates and counts produced similar estimates of leaf expansion timing and rate. A photo-derived greenness index produced similar estimates of timing, but not rate, and was compromised by practical issues of photographing individual crowns in closed canopy woodland. Citizen science should collect time-series data instead of frequently-used first event dates―visual observations offer the most practical way to do this, but further work is needed to test reliability with citizen scientists. Given high intra- and inter-species variation in tree phenology, whole forest canopies need to be monitored to infer canopy closure timing. Canopy openness was assessed using sophisticated hemispherical photography and a range of low-cost alternatives, across four Devon woodlands over a year. Visual estimates and ordinary photography were too coarse to identify fine-scale variation in canopies. Smartphone fisheye photography analysed with free software was identified as a reliable surrogate for estimating relative, though not absolute, canopy openness. The method has high potential as a citizen science tool, as different phone models and users gave similar canopy openness estimates. In a detailed follow-up study, smartphone fisheye photography, hemispherical photography and visual observations of leaf expansion were used every other day to characterise spring canopy development. Logistic growth models estimated canopy closure timing and rate. Visual observations identified much earlier canopy development than either photographic method. Smartphone fisheye photography performed comparably to hemispherical photography. There is good potential for practical application of smartphone fisheye photography, as similar canopy closure estimates were gained from photos taken once every two weeks. The research in this thesis identifies a range of methods suitable for widespread monitoring of forest ecosystem dynamics in relation to climate change. Developing a smartphone app for automatic analysis and submission of canopy images will be an important next step to enabling widespread use. A pilot project is underway to begin testing methods with citizen scientists. Further research into data quality with citizen scientists is needed before the methods can be rolled out widely with confidence.

L'objectif principal de ce travail de thèse est l'élaboration d'un système de vision binoculaire mettant en oeuvre deux caméras de types différents. Le système étudié est constitué d'une caméra de type omnidirectionnelle associée à une caméra PTZ. Nous appellerons ce couple de caméras un système de vision hybride. L'utilisation de ce type de capteur fournit une vision globale de la scène à l'aide de la caméra omnidirectionnelle tandis que l'usage de la caméra mécanisée permet une fovéation, c'est-à-dire l'acquisition de détails, sur une région d'intérêt détectée depuis l'image panoramique.Les travaux présentés dans ce manuscrit ont pour objet, à la fois de permettre le suivi d'une cible à l'aide de notre banc de caméras mais également de permettre une reconstruction 3D par stéréoscopie hybride de l'environnement nous permettant d'étudier le déplacement du robot équipé du capteur. / The primary goal of this thesis is to elaborate a binocular vision system using two different types of camera. The system studied here is composed of one omnidirectional camera coupled with a PTZ camera. This heterogeneous association of cameras having different characteristics is called a hybrid stereo-vision system. The couple composed of these two cameras combines the advantages given by both of them, that is to say a large field of view and an accurate vision of a particular Region of interest with an adjustable level of details using the zoom. In this thesis, we are presenting multiple contributions in visual tracking using omnidirectional sensors, PTZ camera self calibration, hybrid vision system calibration and structure from motion using a hybrid stereo-vision system.

In the field of autonomous vehicles, Advanced Driver Assistance Systems (ADAS)play a key role. Their applications vary from aiding with critical safety systems to assisting with trivial parking scenarios. To optimize the use of resources, trivial ADAS applications are often limited to make use of low-cost sensors. As a result, sensors such as Cameras and UltraSonics are preferred over LiDAR (Light Detection and Ranging) and RADAR (RAdio Detection And Ranging) in assisting the driver with parking. In a parking scenario, to ensure the safety of people in and around the car, the sensors need to detect objects around the car in real-time. With the advancements in Deep Learning, Deep Neural Networks (DNN) are becoming increasingly effective in detecting objects with real-time performance. Therefore, the thesis aims to investigate the viability of Deep Neural Networks using Fisheye cameras to detect pedestrians around the car. To achieve the objective, an experiment was conducted on a test vehicle equipped with multiple Fisheye cameras. Three Deep Neural Networks namely, YOLOv3 (You Only Look Once), its faster variant Tiny-YOLOv3 ND ResNet-50 were chosen to detect pedestrians. The Networks were trained on Fisheye image dataset with the help of transfer learning. After training, the models were also compared to pre-trained models that were trained to detect pedestrians on normal images. Our experiments have shown that the YOLOv3 variants have performed well but with a difficulty of localizing the pedestrians. The ResNet model has failed to generate acceptable detections and thus performed poorly. The three models produced detections with a real-time performance for a single camera but when scaled to multiple cameras, the detection speed was not on par. The YOLOv3 variants could detect pedestrians successfully on dewarped fish-eye images but the pipeline still needs a better dewarping algorithm to lessen the distortion effects. Further, the models need to be optimized in order to generate detections with real-time performance on multiple cameras and also to fit the model on an embedded system.

Previous research has found that programmers spend a significant fraction of their time navigating between different source code locations and that much of that time is spent returning to previously viewed code. Other work has identified the ability to juxtapose arbitrary pieces of code as cognitively important. However, modern IDEs have inherited a user interface design in which, usually, only one source file is displayed at a time, with the result that users must switch back and forth from one file to another. Taking advantage of the increasing availability of large displays, we propose a new interaction paradigm in which an IDE presents parts of multiple source files side by side, using the Mylyn degree-of-interest function to dynamically allocate screen space to them on the basis of degree-of-interest to the current development task. We demonstrate the feasibility of this paradigm with a prototype implementation built on the Eclipse IDE and note that it was used by the author over a period of months in the development of the prototype itself. Additionally, we present two case studies which quantify the potential reduction in navigation and demonstrate the simplicity of the approach and its ability to capture complete concerns on screen. These case studies suggest that the approach has the potential to reduce the time that programmers spend navigating by as much as 50%.

Knowledge workers are frequently called upon to perform deep, critical reading involving a heightened level of interaction with the reading media and other tools. This process, known as active reading, entails highlighting, commenting upon, and flipping through a text, in addition to other actions. While paper is traditionally seen as the ideal medium for active reading, computers have recently become comparable to paper through replicating the latter’s affordances. But even paper is not a panacea; it offers an inflexible document representation that supports some things well, such as embellishment, but supports others very poorly, like comparison and large scale annotation. In response to this, I developed a prototype system, called LiquidText, to embody a flexible, high degree-of-freedom visual representation that seeks to alleviate some of the problems in paper and paper-like representations. To provide efficient control of this representation, LiquidText runs on a multi-finger touch and gesture based platform. To guide the development of this system, I conducted a formative study of current active reading practice. I investigated knowledge workers’ active reading habits, perceptions, and the problems they face with current reading media. I also inquired into what they would like in a future active reading environment. I used these results in conjunction with multiple design iterations and formative system evaluations to refine LiquidText for use in a summative study. The summative study assessed, through a controlled, laboratory evaluation, LiquidText’s impact on 1) the subjective experience of active reading, 2) the process of active reading, and 3) the outputs resulting from active reading. Generally, the study found a strong participant preference for LiquidText, and a focus on the creation of a summary of the original document as part of the reading process. On average, reading outputs were not significantly better or worse with LiquidText, but some conditions were observed that may help identify the subset of people for whom LiquidText will result in an improvement.

Previous research has found that programmers spend a significant fraction of their time navigating between different source code locations and that much of that time is spent returning to previously viewed code. Other work has identified the ability to juxtapose arbitrary pieces of code as cognitively important. However, modern IDEs have inherited a user interface design in which, usually, only one source file is displayed at a time, with the result that users must switch back and forth from one file to another. Taking advantage of the increasing availability of large displays, we propose a new interaction paradigm in which an IDE presents parts of multiple source files side by side, using the Mylyn degree-of-interest function to dynamically allocate screen space to them on the basis of degree-of-interest to the current development task. We demonstrate the feasibility of this paradigm with a prototype implementation built on the Eclipse IDE and note that it was used by the author over a period of months in the development of the prototype itself. Additionally, we present two case studies which quantify the potential reduction in navigation and demonstrate the simplicity of the approach and its ability to capture complete concerns on screen. These case studies suggest that the approach has the potential to reduce the time that programmers spend navigating by as much as 50%.

Målet med detta examensarbete är att ta fram ett gränsnitt för navigering i stora datamängder samt undersöka eventuella styrkor och svagheter hos detta gränsnitt. Bakgrunden till projektet är att företaget Analog Software utvecklar ett system som automatiskt skall klassificera objekt i en datamängd och sedan presentera denna klassificerade datamängd. Gränsnittet skall stödja en informationssökningsstrategi som benämns ”browsing”. En teknik för visualisering av information som varit framgångsrik är den så kallade ”Fisheye-tekniken” som baseras på en analogi med en vidvinkellins. Tekniken visar lokal information med hög detaljrikedom samtidigt som den visar global information med låg detaljrikedom. Ett flertal gränssnitt har använt sig av denna teknik för visualisering av information. I detta arbete har ett textbaserat Fisheye-gränssnitt tagits fram och testat med avseende på två olika funktioner. De två funktionerna är ”anomalier” och ”stöd för bokmärken”. Hypoteserna var: 1. Om anomalier fanns i datastrukturen så skulle en Fisheye-teknik förstärka dessa. 2. En bokmärkesfunktion skall ej behövas då Fisheye-tekniken erbjuder global kontext och byte av fokus går snabbt. Två experiment utfördes med 28 försökspersoner. I experiment 1 använde hälften av försökspersonerna ett standargränssnitt och hälften ett Fisheye-gränssnitt. De fick navigera i en datamängd sorterad på två olika sätt och ange om det upplevde sorteringen som naturlig eller konstig. I experimet 2 ingick samma försökspersoner som i experiment 1. I detta experiment fick hälften navigera med ett Fisheye-gränsnitt med bokmärkesfunktion och hälften med ett Fisheye-gränssnitt utan bokmärkesfunktion. De skulle navigera i en datamängd och svara på ett antal frågor. Resultaten visade att Hypotes 1 kunde förkastas. Försökspersonerna som använde sig av Fisheye-gränsittet upplevde datamängden att vara mer naturlig än den grupp som använt sig av ett standardgränssnitt. För hypotes 2 kunde inga säkra slutsatser dra på grund av att experimentet inte lyckades mäta det som avsågs att mätas.

Integrated camera systems for increasing safety and maneuverability are becoming increasingly common for heavy vehicles. One problem with heavy vehicles today is that there are blind spots where the driver has no or very little view. There is a great demand on increasing the safety and helping the driver to get a better view of his surroundings. This can be achieved by a sophisticated camera system, using cameras with wide field of view, that could cover dangerous blind spots. This master thesis aims to investigate and develop a prototype solution for a camera system consisting of two fisheye cameras. The solution covers both hardware choices and software development including camera calibration and image stitching. Two different fisheye camera calibration toolboxes are compared and their results discussed, with the aim to find the most suitable for this application. The result from the two toolboxes differ in performance, and the result from only one of the toolboxes is sufficient for image stitching.