Back to search

Democritus University of Thrace (DUTH)

Επιτάχυνση εκτέλεσης προηγμένων αλγορίθμων επεξεργασίας εικόνας και βίντεο σε επαναδιαμορφούμενο υλικό (FPGA), για εφαρμογές ρομποτικής όρασης

Abstract

dc:description

A fundamental problem in computer vision is finding correspondences between similar images. Image features extraction and their use to establish correspondences improve the matching process. In the literature, several feature extraction algorithms have been proposed. One of the most reliable and robust algorithms is SIFT (Scale Invariant Feature Transform). However, SIFT is computationally demanding and it is rarely used in real-time applications. In this PHD thesis, an FPGA-based architecture for real-time SIFT extraction and matching is presented. The SIFT detector and descriptor modules are fully pipelined in pixel basis. The proposed SIFT matcher was implemented with the same pipelined design concept. In order to remove the false matches, RANSAC (random sample consensus) algorithm is applied in the last step. RANSAC is an iterative and computationally demanding algorithm. Although its execution is time consuming, it is parallelized in a significant degree and its completion requires only a few clock cycles. In conclusion, the architecture includes all the necessary steps from the image capture, up to the establishment of the correct correspondences. The architecture includes various customized controllers, which have been designed to minimize the required resources from the FPGA device. The main controller is the frame grabber which captures the incoming frames from the CMOS image sensor. It consists of the I2C controller, the Bayer decoder as well as the VGA controller which is used for visual verification. The I2C controller configures the CMOS image sensor, while the Bayer decoder extracts pixel intensities from Bayer encoding. Furthermore, controllers such as SPI master and FIFO-to-USB have been developed in order to communicate with external devices. The last but not the least controller is the SDR SDRAM controller, which has been designed in order to store image results to external RAM. Regarding the SIFT detector, two main originalities have been presented. Firstly, an improved method for the Gaussian scale space construction is proposed. In the literature, the cascade scheme is usually met, in which every Gaussian image is produced from the previous one by applying convolution with a kernel of fixed standard deviation. In the proposed architecture, convolutions are applied to the initial image with progressively increased standard deviation. In this way, the memory resources required from the FPGA are reduced. Moreover, when more than four scales are demanded, the reuse of previously calculated Gaussians is feasible so that more efficient resource management can be achieved. The proposed convolution scheme also saves hardware multipliers in comparison with the widely used separable Gaussian scheme. Those improvements in resource usage do not degrade SIFT performance. A significant contribution of this PHD thesis to the state of the art constitutes the fully pipelined architecture of the SIFT extraction. The proposed scheme is capable to extract one descriptor vector per clock cycle. As pixel data from CMOS sensor is read in a streaming manner, the architecture extracts descriptors in a streaming manner too. The same principle is followed in the matcher scheme. The descriptor of each detected feature in the current frame is compared with feature descriptors of the previous frame and the match, if exists, is identified in one clock cycle. It should be also noticed that RANSAC algorithm has been parallelized in a significant degree. RANSAC is usually executed iteratively and considerable time is required for its completion. In the literature, we meet papers in which only a few random samples are selected to compute the transformation matrix, in order to reduce the processing time. However, this leads to accuracy reduction in false correspondences removal process. In the proposed RANSAC implementation, real-time requirements are met even if all random samples are used to calculate the transformation matrix. When all random samples are used, then the best accuracy in transformation matrix computation can be derived. The proposed architecture achieves significant acceleration in the execution of the SIFT algorithm. The descriptor processing time was estimated 40ns, considering use of Cyclone IV FPGA technology. That is 50 times faster than the current state of the art papers. In order to produce one correspondence, the matcher scheme requires processing time 80ns. As a result, the processing rate of the complete SIFT detector/descriptor/matcher is 40 fps, when it is applied to images with resolution of 640 x 480 pixels. The aforementioned processing rate meets real-time requirements.

Degree

thesis:*
Grantor dc:publisher
Democritus University of Thrace (DUTH)
Year dc:date
2017

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Vourvoulakis, Ioannis
  • Βουρβουλάκης, Ιωάννης

Subjects

dc:subject × 15

Rights

Language dc:language
gre

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/40918

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Vourvoulakis, Ioannis; Βουρβουλάκης, Ιωάννης. Επιτάχυνση εκτέλεσης προηγμένων αλγορίθμων επεξεργασίας εικόνας και βίντεο σε επαναδιαμορφούμενο υλικό (FPGA), για εφαρμογές ρομποτικής όρασης. Democritus University of Thrace (DUTH), 2017. http://hdl.handle.net/10442/hedi/40918