Democritus University of Thrace (DUTH)
Επιτάχυνση εκτέλεσης προηγμένων αλγορίθμων επεξεργασίας εικόνας και βίντεο σε επαναδιαμορφούμενο υλικό (FPGA), για εφαρμογές ρομποτικής όρασης
Abstract
dc:descriptionA fundamental problem in computer vision is finding correspondences between similar images. Image features extraction and their use to establish correspondences improve the matching process. In the literature, several feature extraction algorithms have been proposed. One of the most reliable and robust algorithms is SIFT (Scale Invariant Feature Transform). However, SIFT is computationally demanding and it is rarely used in real-time applications. In this PHD thesis, an FPGA-based architecture for real-time SIFT extraction and matching is presented. The SIFT detector and descriptor modules are fully pipelined in pixel basis. The proposed SIFT matcher was implemented with the same pipelined design concept. In order to remove the false matches, RANSAC (random sample consensus) algorithm is applied in the last step. RANSAC is an iterative and computationally demanding algorithm. Although its execution is time consuming, it is parallelized in a significant degree and its completion requires only a few clock cycles. In conclusion, the architecture includes all the necessary steps from the image capture, up to the establishment of the correct correspondences. The architecture includes various customized controllers, which have been designed to minimize the required resources from the FPGA device. The main controller is the frame grabber which captures the incoming frames from the CMOS image sensor. It consists of the I2C controller, the Bayer decoder as well as the VGA controller which is used for visual verification. The I2C controller configures the CMOS image sensor, while the Bayer decoder extracts pixel intensities from Bayer encoding. Furthermore, controllers such as SPI master and FIFO-to-USB have been developed in order to communicate with external devices. The last but not the least controller is the SDR SDRAM controller, which has been designed in order to store image results to external RAM. Regarding the SIFT detector, two main originalities have been presented. Firstly, an improved method for the Gaussian scale space construction is proposed. In the literature, the cascade scheme is usually met, in which every Gaussian image is produced from the previous one by applying convolution with a kernel of fixed standard deviation. In the proposed architecture, convolutions are applied to the initial image with progressively increased standard deviation. In this way, the memory resources required from the FPGA are reduced. Moreover, when more than four scales are demanded, the reuse of previously calculated Gaussians is feasible so that more efficient resource management can be achieved. The proposed convolution scheme also saves hardware multipliers in comparison with the widely used separable Gaussian scheme. Those improvements in resource usage do not degrade SIFT performance. A significant contribution of this PHD thesis to the state of the art constitutes the fully pipelined architecture of the SIFT extraction. The proposed scheme is capable to extract one descriptor vector per clock cycle. As pixel data from CMOS sensor is read in a streaming manner, the architecture extracts descriptors in a streaming manner too. The same principle is followed in the matcher scheme. The descriptor of each detected feature in the current frame is compared with feature descriptors of the previous frame and the match, if exists, is identified in one clock cycle. It should be also noticed that RANSAC algorithm has been parallelized in a significant degree. RANSAC is usually executed iteratively and considerable time is required for its completion. In the literature, we meet papers in which only a few random samples are selected to compute the transformation matrix, in order to reduce the processing time. However, this leads to accuracy reduction in false correspondences removal process. In the proposed RANSAC implementation, real-time requirements are met even if all random samples are used to calculate the transformation matrix. When all random samples are used, then the best accuracy in transformation matrix computation can be derived. The proposed architecture achieves significant acceleration in the execution of the SIFT algorithm. The descriptor processing time was estimated 40ns, considering use of Cyclone IV FPGA technology. That is 50 times faster than the current state of the art papers. In order to produce one correspondence, the matcher scheme requires processing time 80ns. As a result, the processing rate of the complete SIFT detector/descriptor/matcher is 40 fps, when it is applied to images with resolution of 640 x 480 pixels. The aforementioned processing rate meets real-time requirements.
Degree
thesis:*- Grantor dc:publisher
- Democritus University of Thrace (DUTH)
- Year dc:date
- 2017
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Vourvoulakis, Ioannis
- Βουρβουλάκης, Ιωάννης
Subjects
dc:subject × 15- Επεξεργασία εικόνας
- Εξαγωγή χαρακτηριστικών εικόνας
- Ενσωματωμένα συστήματα βασισμένα σε FPGA
- Εφαρμογές πραγματικού χρόνου
- Ρομποτική όραση
- Image processing
- Image features extraction
- FPGA-based embedded systems
- Real-time applications
- Robotic vision
- SIFT
- Επιστήμες Μηχανικού και Τεχνολογία
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Engineering and Technology
- Electrical Engineering, Electronic Engineering, Information Engineering
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/40918