National Technical University of Athens (NTUA)
A local stereo matching algorithm for dense 3D scene reconstruction
Abstract
dc:descriptionThis thesis investigates problems and algorithms on stereo-matching for the purposes of 3D reconstruction in a variety of applications. Stereo-matching can be part, for instance, of multi-view algorithms used to produce 3D models and orthomaps from aerial images, either from digital aerial cameras or UAV cameras, or architectural reconstructions. Nevertheless, it is also often applied to single stereo-pairs since many modern applications rely on stereo-matching to obtain the shape of the surrounding world, including robotics, autonomous navigation, augmented reality. Considering such applications, stereo-algorithms need, among other characteristics, to adapt well to new domains, offer accurate results, be computationally efficient and run on low-powered devices. The main scope here has been to thoroughly study existing methods and approaches for the core components of matching algorithms, namely cost computation, aggregation and optimization, as well as confidence measures and disparity map refinements, understand their limits and limitations, and propose in each step possible novelties and improvements concerning the quality of the final 3D reconstructed scene. First, the variety of active and passive (image-based) methods for 3D reconstruction are briefly reviewed to correctly place stereo-matching in the broader area of surface reconstruction approaches; also, fundamental concepts for calibrating and creating a rectified stereo-pair as a standard prerequisite for stereo-matching algorithms are given. Several stereo-algorithms have been studied and evaluated in an in-depth review to acquire a firm understanding of concepts and methods in the overall area of stereo-matching. Different cost functions have been implemented or adapted, as well as image filters, confidence measures and matching constraints. Also, aggregation methods, both simple and more elaborate, have been implemented and experimented with to better comprehend and evaluate their performance, also with respect to global methods. Based on this, an end-to-end stereo-algorithm was developed, including certain novelties in each of its individual components. A combination of cost measures is used to define the overall cost based on census transformation on gradients which produced improved results, especially in sub-pixel disparity estimation and under radiometric differences in the stereo-pair, under different aggregation and optimization schemes. As an aggregation scheme, an improved version of cross-based matching, which defines the geometric and radiometric proximity based on a linear threshold, has been proposed; this yielded better results than the original definition and reduced the heuristic thresholds needed to define it. It is noticed that local methods, despite advantages in speed, computational complexity, and disparity edge retrieval, perform poorly on strongly slanted surfaces. To this end, the computed cost volume is filtered via a geometrically constrained smoothing kernel, based on a 3D Gaussian kernel but modified to serve the ordering and uniqueness constraints. This approach significantly improved the disparities in slanted surfaces at the low cost of a convolutional operation. This technique is generic and can be used in conjunction with any stereo algorithm. The above methods have been integrated within a hierarchical matching scheme to tackle issues related to high-resolution images, namely computational efficiency and accuracy via reducing the disparity search space. The cross-based support regions have been intuitively exploited for this. Some extra investigations have been conducted on semi-global matching and the different approaches to define the penalties on disparity changes and on global optimization (via MRFs) combined with local aggregation. Different disparity map refinement steps have also been implemented to evaluate the improvement on the final results, and a robust scheme of consecutive refinement steps is proposed mixing existing approaches and new techniques, some of which exploit the cross-based regions.The proposed techniques were assembled in an algorithm that performs reasonably well, with quantitative evaluation criteria, to various scenes. The evaluation and the comparison to other methods were based on typical datasets available in standard publicly available evaluation platforms, and on datasets that can better fit the purposes of typical photogrammetric applications.
Degree
thesis:*- Grantor dc:publisher
- National Technical University of Athens (NTUA)
- Year dc:date
- 2022
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Stentoumis, Christos
- Στεντούμης, Χρήστος
Subjects
dc:subject × 15- Φωτογραμμετρία
- Όραση υπολογιστών
- Συνταύτιση
- Ψηφιακή φωτογραμμετρία
- Photogrammetry
- Digital photogrammetry
- Computer vision
- Stereo vision
- Stereo matching
- Άλλες Επιστήμες Μηχανικού και Τεχνολογίες
- Επιστήμες Μηχανικού και Τεχνολογία
- Μηχανική και Τεχνολογίες, άλλοι τομείς
- Other Engineering and Technologies
- Engineering and Technology
- Engineering and Technologies, miscellaneous
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/51135