This Master’s thesis studies the possibility of using image processing as a tool to facilitate vine management, in particular shoot counting and assessment of the grapevine canopy. Both are areas where manual inspection is done today. The thesis presents methods of capturing images and segmenting different parts of a vine. It also presents and evaluates different approaches on how shoot counting can be done. Within canopy assessment, the emphasis is on methods to estimate canopy density. Other possible assessment areas are also discussed, such as canopy colour and measurement of canopy gaps and fruit exposure. An example of a vine assessment system is given.

Den här rapporten beskriver och utvärderar ett antal algoritmer för multisensordatafusion av radar och IR/TV-data på rådatanivå. Med rådatafusion menas att fusionen ska ske innan attribut- eller objektextrahering. Attributextrahering kan medföra att information går förlorad som skulle kunna förbättra fusionen. Om fusionen sker på rådatanivå finns mer information tillgänglig och skulle kunna leda till en förbättrad attributextrahering i ett senare steg. Två tillvägagångssätt presenteras. Den ena metoden projicerar radarbilden till IR-vyn och vice versa. Fusionen utförs sedan på de par av bilder med samma dimensioner. Den andra metoden fusionerar de två ursprungliga bilderna till en volym. Volymen spänns upp av de tre dimensionerna representerade i ursprungsbilderna. Metoden utökas också genom att utnyttja stereoseende. Resultaten visar att det kan vara givande att utnyttja stereoseende då den extra informationen underlättar fusionen samt ger en mer generell lösning på problemet. / This thesis describes and evaluates a number of algorithms for multi sensor fusion of radar and IR/TV data. The fusion is performed on raw data level, that is prior to attribute extraction. The idea is that less information will be lost compared to attribute level fusion. Two methods are presented. The first method transforms the radar image to the IR-view and vice versa. The images sharing the same dimension are then fused together. The second method fuses the original images to a three dimensional volume. Another version is also presented, where stereo vision is used. The results show that stereo vision can be used with good performance and gives a more general solution to the problem.

A novel video observational method for the space-time stereoscopic reconstruction of dynamic surfaces representable as graphs, such as ocean waves, is developed. Variational optimization algorithms combining image processing, computer vision and partial differential equations are designed to address the problem of the recovery of the shape of an object's surface from sequences of synchronized multi-view images. Several theoretical and numerical paths are discussed to solve the problem. The variational stereo method developed in this thesis has several advantages over existing 3-D reconstruction algorithms. Our method follows a top-down approach or object-centered philosophy in which an explicit model of the target object in the scene is devised and then related to image measurements. The key advantages of our method are the coherence (smoothness) of the reconstructed surface caused by a coherent object-centered design, the robustness to noise due to a generative model of the observed images, the ability to handle surfaces with smooth textures where other methods typically fail to provide a solution, and the higher resolution achieved due to a suitable graph representation of the object's surface. The method provides competitive results with respect to existing variational reconstruction algorithms. However, our method is based upon a simplified but complete physical model of the scene that allows the reconstruction process to include physical properties of the object's surface that are otherwise difficult to take into account with existing reconstruction algorithms. Some initial steps are taken toward incorporating the physics of ocean waves in the stereo reconstruction process. The developed method is applied to empirical data of ocean waves collected at an off-shore oceanographic platform located off the coast of Crimea, Ukraine. An empirically-based physical model founded upon current ocean engineering standards is used to validate the results. Our findings suggest that this remote sensing observational method has a broad impact on off-shore engineering to enrich the understanding of sea states, enabling improved design of off-shore structures. The exploration of ways to incorporate dynamical properties, such as the wave equation, in the reconstruction process is discussed for future research.

<p>In the research of developing arbitrary and unique virtual views from a real- world scene, a prototype of an interactive relief texture mapping system capable of processing video using dynamic image-based rendering, is developed in this master thesis. The process of deriving depth from recorded video using binocular stereopsis is presented, together with how the depth information is adjusted to be able to manipulate the orientation of the original scene. When the scene depth is known, the recorded organic and dynamic objects can be seen from viewpoints not available in the original video.</p>

<p>This Master’s thesis studies the possibility of using image processing as a tool to facilitate vine management, in particular shoot counting and assessment of the grapevine canopy. Both are areas where manual inspection is done today. The thesis presents methods of capturing images and segmenting different parts of a vine. It also presents and evaluates different approaches on how shoot counting can be done. Within canopy assessment, the emphasis is on methods to estimate canopy density. Other possible assessment areas are also discussed, such as canopy colour and measurement of canopy gaps and fruit exposure. An example of a vine assessment system is given.</p>

Cette thèse aborde le problème de la localisation d'un robot mobile autonome en environnements naturels. La première partie du mémoire s'intéresse aux méthodes algorithmiques pouvant fournir une estimation de position et une classification de ces différentes méthodes en trois grandes catégories est proposée. La première classe présentée est dite " locale ". Elle repose sur un niveau d'abstraction très faible et utilise des données " brutes " ; la position du robot est calculée de façon incrémentale, par cumul de déplacements élémentaires. L'odométrie appartient à cette classe et ce document présente une analyse de ses performances. Une méthode originale d'estimation visuelle des déplacements est également proposée. Elle utilise la stéreo-vision et effectue le suivi de pixels dans une séquence d'images vidéo pour en déduire des déplacements élémentaires. Cette méthode permet de pallier certains inconvénients de l'odométrie, notamment en terrains accidentés. Les positions produites par les méthodes locales sont cependant sujettes à des dérives inexorables. Il est donc nécessaire d'utiliser des méthodes de la seconde classe --- dite " globale ". Celles-ci permettent, dans certaines circonstances, de réduire la dérive des méthodes locales (on trouve notamment dans cette catégorie les méthodes de localisation sur amers). Une méthode basée sur des cartes d'élévations est présentée. Cette représentation permet un recalage en position par minimisation d'une distance entre une image 3D locale et le modèle. De plus, grâce à une technique particulière de construction, une mémorisation de la trajectoire du robot permet de rétro-propager des modifications sur certaines positions et de garantir ainsi une (meilleure) cohérence spatiale du modèle. La dernière catégorie, regroupant les méthodes de localisation par rapport à un modèle initial ou les méthodes " absolues ", ne sont pas abordées dans ce document. La deuxième partie du mémoire analyse les problèmes posés par l'intégration des diverses méthodes à bord d'un robot dans le but de doter celui-ci d'un éventail de fonctionnalités, éventuellement redondantes. Une démonstration de navigation, réalisée grâce au robot Lama du groupe RIA du LAAS, est détaillée. L'intégration a été réalisée à travers le formalisme défini dans l'architecture LAAS (architecture pour les systèmes autonomes) et a abouti à la précision, au sein de cette architecture, du cadre nécessaire aux fonctionnalités de localisation. Notamment, les problèmes du datage des données et de fusion des informations de position sont pris en compte et une réflexion sur les transferts de données est proposée.

Computational stereo estimates 3D structure by analyzing visual changes between two or more passive images of a scene that are captured from different viewpoints. It is a key enabler for ubiquitous autonomous systems, large-scale surveying, virtual reality, and improved techniques for compression, tracking, and object recognition. The fact that computational stereo is an under-constrained inverse problem causes many challenges. Its computational and memory requirements are high. Typical heuristics and assumptions, used to constrain solutions or reduce computation, prevent treatment of key realities such as reflection, translucency, ambient lighting changes, or moving objects in the scene. As a result, a general solution is lacking. Stochastic models are common in computational stereo, but stochastic algorithms are severely under-represented. In this dissertation I present two stochastic algorithms and demonstrate their advantages over deterministic approaches. I first present the Quality-Efficient Stochastic Sampling (QUESS) approach. QUESS reduces the number of match quality function evaluations needed to estimate dense stereo correspondences. This facilitates the use of complex quality metrics or metrics that take unique values at non-integer disparities. QUESS is shown to outperform two competing approaches, and to have more attractive memory and scaling properties than approaches based on exhaustive sampling. I then present a second novel approach based on the Hough transform and extend it with distributed ray tracing (DRT). DRT is a stochastic anti-aliasing technique common to computer rendering but which has not been used in computational stereo. I demonstrate that the DRT-enhanced approach outperforms the unenhanced approach, a competing variation that uses re-accumulation in the Hough domain, and another baseline approach. DRT’s advantages are particularly strong for reduced image resolution and/or reduced accumulator matrix resolution. In support of this second approach, I develop two novel variations of the Hough transform that use DRT, and demonstrate that they outperform competing variations on a traditional line segment detection problem. I generalize these two examples to draw broader conclusions, suggest future work, and call for a deeper exploration by the community. Both practical and academic gaps in the state of the art can be reduced by a renewed exploration of stochastic computational stereo techniques. / text

Το αντικείμενο της εργασίας είναι η τρισδιάστατη ανακατασκευή ενός αντικειμένου ή ενός χώρου, μέσα από τουλάχιστο δύο φωτογραφίες του. Το θέμα της εργασίας είναι μέρος του τομέα της Υπολογιστικής Όρασης, που έχει μεγάλη άνθιση τα τελευταία χρόνια λόγω των πολλών εφαρμογών, όπου η γνώση της τρισδιάστατης δομής ενός αντικειμένου ή ενός χώρου κρίνεται απαραίτητη. Βέβαια, συνέβαλε σε αυτό και η ραγδαία εξέλιξη των ηλεκτρονικών υπολογιστών, με αποτέλεσμα να είναι δυνατή η ακριβής και ποιοτική απεικόνιση σύνθετων τρισδιάστατων σκηνών σε πραγματικό χρόνο, μέσω κατάλληλων αλγορίθμων. Η τρισδιάστατη ανακατασκευή ενός αντικειμένου ή ενός χώρου, από φωτογραφίες ή βίντεο αποτελεί ένα ενδιαφέρον και εντυπωσιακό θέμα με πολλές εφαρμογές και πολύ ενθαρρυντικά αποτελέσματα. Αυτά αποτέλεσαν ουσιαστικά και το έναυσμα για την ενασχόλησή μου με τον τομέα της Υπολογιστικής Όρασης και την επιλογή του θέματος της εργασίας. Οι εφαρμογές που αναπτύσσονται στα πλαίσια της εργασίας παρέχουν οπτικά ευχάριστα αποτελέσματα και έχουν μεγάλη προσαρμοστικότητα και ευελιξία στης διάφορες συνθήκες φωτογράφησης ή λήψης βίντεο. Το σημαντικό, λοιπόν, είναι ότι δεν χρειάζεται απαραίτητα περιβάλλον εργαστηρίου για την λήψη των δεδομένων, δηλαδή των εικόνων. Προκύπτουν καλά αποτελέσματα ακόμα και με εικόνες που λήφθηκαν μέσω μίας απλής φορητής φωτογραφικής κάμερας, χωρίς τρίποδα για στήριξη. Αρκεί απλά να δημιουργήσουμε τις προϋποθέσεις για μικρή κίνηση της κάμερας μεταξύ των λήψεων των εικόνων. Στην παρούσα εργασία παρουσιάζονται και εξετάζονται διεξοδικά όλα τα θέματα που αφορούν την τρισδιάστατη οπτικοποίηση των αντικειμένων. Αρχικά, στις παραγράφους 2.1 έως 3.2, αναλύεται η θεωρία των δύο κυριότερων μεθόδων της "Δομής και Κίνησης" και της "Στερεοσκοπικής Όρασης" . Στην συνέχεια στις παραγράφους 3.3 και 3.4 αναπτύσσεται η μεθοδολογία που ακολουθείται από τις εφαρμογές της Δομής και Κίνησης , ενώ της Στερεοσκοπική Όρασης αναπτύσσεται στην 3.5 . Στην μέθοδο της Δομής και Κίνησης περικλείονται δύο περιπτώσεις. Η πρώτη είναι η μη βαθμονομημένη περίπτωση και η δεύτερη είναι η βαθμονομημένη. Στην δεύτερη, λοιπόν, προηγείται η βαθμονόμηση της κάμερας, οπότε είναι γνωστές εκ των προτέρων οι παράμετροι της κάμερας . Η υλοποίηση των αλγορίθμων γίνεται στο παράρτημα, με την βοήθεια του περιβάλλοντος αριθμητικής υπολογιστικής της προγραμματιστικής γλώσσας Matlab. Τέλος, στο τέταρτο κεφάλαιο, δίνονται κάποια παραδείγματα ανακατασκευών που αποδεικνύουν την αποτελεσματικότητα των αλγορίθμων της υλοποίησης. Τόσο η θεωρία, όσο και οι αλγόριθμοι που παρουσιάζονται στην παρούσα εργασία καλύπτουν πλήρως τις απαραίτητες γνώσεις για την υλοποίηση της τρισδιάστατης αναπαράστασης. Συσσωρεύτηκαν πληροφορίες από δύο μεθόδους, δηλαδή της Δομής και Κίνησης αλλά και της Στερεοσκοπικής Όρασης, οι οποίες συνδυασμένες δίνουν βέλτιστα και αρτιότερα αποτελέσματα. Πρωταρχικός στόχος της εργασίας είναι η ανάδειξη των δυνατοτήτων που παρέχουν οι συγκεκριμένες μέθοδοι. Από την άλλη μεριά, η υλοποίηση των δύο μεθόδων και κατά συνέπεια των αλγόριθμων, αποτελούν μία αρκετά καλή βάση για περαιτέρω ανάπτυξη και προώθηση για μελλοντική έρευνα στον εν λόγω τομέα. Ούτως ή άλλως, τα τελευταία χρόνια, η ερεύνα που αφορά την Υπολογιστική Όραση έχει αποδώσει και έχει δημιουργήσει ικανοποιητικότατα αποτελέσματα. Οπότε στο μέλλον αναμένονται ισχυρότεροι αλγόριθμοι, βελτιώσεις αλλά και πολλές εφαρμογές στους εξελισσόμενους τομείς της ηλεκτρονικής και όχι μόνο. / The purpose of this thesis is the three-dimensional reconstruction of an object or a space, through at least two photos. The theme is part of the field of Computer Vision, which has known great development in the recent years due to the many applications, where the knowledge of the three-dimensional structure of an object or a space is necessary. Of course to this development contributed also the rapid evolution of computers, making possible the accurate and high quality display of complex three-dimensional scenes in real time, through appropriate algorithms. The three-dimensional reconstruction of an object or a space, using photos or video, is an interesting and impressive subject with many applications and very encouraging results. This was basically what intrigued me to involve with the field of Computer Vision and choose the topic of this thesis. The applications in this thesis provide visually pleasant results and have great adaptability and flexibility in various conditions of shooting and making of videos. What is important, therefore, is that a laboratory environment to obtain the data, meaning images, is not necessary. The results are satisfactory even with pictures taken by a simple handheld camera, without the use of a tripod. It’s enough if we just create the proper conditions for a small camera movement between the shots. The present thesis presents and discusses thoroughly all the subjects related with the three-dimensional visualization of objects. First in paragraphs 2.1 to 3.2, is analyzed the theory of the two most important methods, of "Structure and Motion" and of "Stereo Vision". Then in paragraphs 3.3 and 3.4 is discussed the methodology followed by the applications of Structure and Motion , and this of Stereo Vision is discussed in paragraph 3.5. The method of Structure and Motion encloses two cases. The first is the non-calibrated case and the second is the calibrated case. In the calibrated case the calibration of the camera comes first, so the parameters of the camera are known in advance. The implementation of the algorithms is in the Annex, with the help of the numerical computing environment of the programming language Matlab. Finally, in chapter four, are given same examples of reconstructions that demonstrate the effectiveness of the algorithms of implementation. Both the theory and the algorithms presented in this thesis cover fully the necessary knowledge for the materialization of the three-dimensional representation. The information are is accumulated by two methods, this of Structure and Motion and this of Stereo Vision, which combined give the best and most complete results. Primary objective of this thesis is to highlight the possibilities offered by these methods. On the other hand, the implementation of these two methods and thus the algorithms is a good basis for further development and promotion for future research in this field. Anyway, in recent years, the research on Computer Vision has given great results. So in the future are expected stronger algorithms, improvements but also many applications relating with the developing sector of electronics and more.

Syfte – Syftet för den här studien är att utveckla en metod för att mäta diametern på travat timmers ändträytor i en bild med förvridet perspektiv där ytorna befinner sig på olika djup i förhållande till varandra, för att effektivisera mätning inom skogsindustrin. Metod – Denna studie genomfördes i samarbete med Cind AB, och arbetet var uppdelat i två faser med Design Science Research som forskningsmetod. I Fas 1 rektifierades bilder på ändträytorna manuellt och i Fas 2 utnyttjades ett punktmoln för att uppskatta rektifieringsplanet. Detta gjordes i en stereokamerarigg i skala 1:25 på 139 stockar. Samtliga stockar mättes digitalt i de rektifierade bilderna och manuellt med ett digitalt skjutmått. Ett konfidensintervall för differensen beräknades fram för att bedöma mätnoggrannheten. Resultat – Konfidensintervallet för Fas 1 tyder på att metoden har potential då rektifieringsplanet placeras korrekt, vilket Fas 2 visar är en svår och komplex uppgift. Slutsatser – Den utvecklade metodens mätnoggrannhet uppnådde inte studiens mål på 5% felmarginal. Det skulle dock vara möjligt att mäta ändträytor med god noggrannhet om punktmolnet har tillräckligt hög kvalitet. Begränsningar – Mjukvaran som använder punktmolnet för att rektifiera bilderna är en modifierad version av Cinds proprietära produkt. Datamängden som används i studien samlas endast in via Cinds testrigg. / Purpose – The purpose of this study is to develop a method for measuring the diameter of piled logs on a truck in a picture that has skewed perspective and where the end surfaces are at different depths in relation to each other. The intent of this method is to further streamline log measurement in the logging industry. Method – This study was conducted in collaboration with Cind AB, and the work was split in two phases with Design Science Research as research method. In Phase 1, images with log end surfaces were rectified manually, and in Phase 2 a point cloud was used to estimate the rectification plane. This was done with a stereo camera rig in scale 1:25 on a total of 139 logs. All logs were digitally measured in the rectified images and manually measured with a digital caliper. A confidence interval for the difference was calculated to assess the measurement accuracy. Findings – The confidence interval from Phase 1 indicates that the developed method has potential when the rectification plane is placed correctly, which Phase 2 shows is a difficult and complex task. Conclusions – The developed method did not reach the desired measurement accuracy of 5% margin of error, which means that the goal of the study was not achieved. It would be possible to measure the end surfaces of logs with high precision if the point cloud is of a sufficiently high quality. Limitations – The software that utilizes point cloud information to rectify the images is a modified version of Cind’s proprietary product. The dataset that is used in this study is collected solely through Cind’s test rig.

abstract: Using stereo vision for 3D reconstruction and depth estimation has become a popular and promising research area as it has a simple setup with passive cameras and relatively efficient processing procedure. The work in this dissertation focuses on locally adaptive stereo vision methods and applications to different imaging setups and image scenes. Solder ball height and substrate coplanarity inspection is essential to the detection of potential connectivity issues in semi-conductor units. Current ball height and substrate coplanarity inspection tools are expensive and slow, which makes them difficult to use in a real-time manufacturing setting. In this dissertation, an automatic, stereo vision based, in-line ball height and coplanarity inspection method is presented. The proposed method includes an imaging setup together with a computer vision algorithm for reliable, in-line ball height measurement. The imaging setup and calibration, ball height estimation and substrate coplanarity calculation are presented with novel stereo vision methods. The results of the proposed method are evaluated in a measurement capability analysis (MCA) procedure and compared with the ground-truth obtained by an existing laser scanning tool and an existing confocal inspection tool. The proposed system outperforms existing inspection tools in terms of accuracy and stability. In a rectified stereo vision system, stereo matching methods can be categorized into global methods and local methods. Local stereo methods are more suitable for real-time processing purposes with competitive accuracy as compared with global methods. This work proposes a stereo matching method based on sparse locally adaptive cost aggregation. In order to reduce outlier disparity values that correspond to mis-matches, a novel sparse disparity subset selection method is proposed by assigning a significance status to candidate disparity values, and selecting the significant disparity values adaptively. An adaptive guided filtering method using the disparity subset for refined cost aggregation and disparity calculation is demonstrated. The proposed stereo matching algorithm is tested on the Middlebury and the KITTI stereo evaluation benchmark images. A performance analysis of the proposed method in terms of the I0 norm of the disparity subset is presented to demonstrate the achieved efficiency and accuracy. / Dissertation/Thesis / Doctoral Dissertation Electrical Engineering 2017