In this thesis, contributions are presented in the areas of shape parsing for view-based object recognition and spatio-temporal feature description for action recognition. A probabilistic model for parsing shapes into several distinguishable parts for accurate shape recognition is presented. This approach is based on robust geometric features that permit high recognition accuracy. As the second contribution in this thesis, a binary spatio-temporal feature descriptor is presented. Recent work shows that binary spatial feature descriptors are effective for increasing the efficiency of object recognition, while retaining comparable performance to state of the art descriptors. An extension of these approaches to action recognition is presented, facilitating huge gains in efficiency due to the computational advantage of computing a bag-of-words representation with the Hamming distance. A scene's motion and appearance is encoded with a short binary string. Exploiting the binary makeup of this descriptor greatly increases the efficiency while retaining competitive recognition performance.

In this thesis, contributions are presented in the areas of shape parsing for view-based object recognition and spatio-temporal feature description for action recognition. A probabilistic model for parsing shapes into several distinguishable parts for accurate shape recognition is presented. This approach is based on robust geometric features that permit high recognition accuracy. As the second contribution in this thesis, a binary spatio-temporal feature descriptor is presented. Recent work shows that binary spatial feature descriptors are effective for increasing the efficiency of object recognition, while retaining comparable performance to state of the art descriptors. An extension of these approaches to action recognition is presented, facilitating huge gains in efficiency due to the computational advantage of computing a bag-of-words representation with the Hamming distance. A scene's motion and appearance is encoded with a short binary string. Exploiting the binary makeup of this descriptor greatly increases the efficiency while retaining competitive recognition performance.

Context. Embedded platforms GPUs are reaching a level of perfor-mance comparable to desktop hardware. Therefore it becomes inter-esting to apply Computer Vision techniques to modern smartphones.The platform holds different challenges, as energy use and heat gen-eration can be an issue depending on load distribution on the device. Objectives. We evaluate the viability of a feature detector and de-scriptor on the Xperia Z3. Specifically we evaluate the the pair basedon real-time execution, heat generation and performance. Methods. We implement the feature detection and feature descrip-tor pair Harris-Hessian/FREAK for GPU execution using OpenCL,focusing on embedded platforms. We then study the heat generationof the application, its execution time and compare our method to twoother methods, FAST/BRISK and ORB, to evaluate the vision per-formance. Results. Execution time data for the Xperia Z3 and desktop GeForceGTX660 is presented. Run time temperature values for a run ofnearly an hour are presented with correlating CPU and GPU ac-tivity. Images containing comparison data for BRISK, ORB andHarris-Hessian/FREAK is shown with performance data and discus-sion around notable aspects. Conclusion. Execution times on Xperia Z3 is deemed insufficientfor real-time applications while desktop execution shows that there isfuture potential. Heat generation is not a problem for the implemen-tation. Implementation improvements are discussed to great lengthfor future work. Performance comparisons of Harris-Hessian/FREAKsuggest that the solution is very vulnerable to rotation, but superiorin scale variant images. Generally appears suitable for near duplicatecomparisons, delivering much greater number of keypoints. Finally,insight to OpenCL application development on Android is given

In this thesis, contributions are presented in the areas of shape parsing for view-based object recognition and spatio-temporal feature description for action recognition. A probabilistic model for parsing shapes into several distinguishable parts for accurate shape recognition is presented. This approach is based on robust geometric features that permit high recognition accuracy. As the second contribution in this thesis, a binary spatio-temporal feature descriptor is presented. Recent work shows that binary spatial feature descriptors are effective for increasing the efficiency of object recognition, while retaining comparable performance to state of the art descriptors. An extension of these approaches to action recognition is presented, facilitating huge gains in efficiency due to the computational advantage of computing a bag-of-words representation with the Hamming distance. A scene's motion and appearance is encoded with a short binary string. Exploiting the binary makeup of this descriptor greatly increases the efficiency while retaining competitive recognition performance.

This work deals with recognition of hand-drawn objects traced by children with mental disorders. The aim is to classify object’s geometrical primitives into classes so then can be plotted along with the idealized shape of the input object. Level of mental retardation is determined by the variance of the input (drawn) object from idealized shape of the object (artwork).

The aims of this research work are to develop a feature detection, description, and matching system for low-resource applications. This work was motivated by the need for a vision sensor to assist the flight of a quad-rotor UAV. This application presented a real-world challenge of autonomous drift stabilization using vision sensors. The initial solution implemented a basic feature detector and matching system on an FPGA. The research then pursued ways to improve the vision system. Research began with color feature detection, and the Color Difference of Gaussians feature detector was developed. CDoG provides better results than gray scale DoG and does not require any additional processing than gray scale if implemented in a parallel architecture. The CDoG Scale-Invariant Feature Transform modification was developed which provided color feature detection and description to the gray scale SIFT descriptor. To demonstrate the benefits of color information, the CDSIFT algorithm was applied to a real application: library book inventory. While color provides added benefit to the CDSIFT descriptor, CDSIFT descriptors are still computationally intractable for a low-resource hardware implementation. Because of these shortcomings, this research focused on developing a new feature descriptor. The BAsis Sparse-coding Inspired Similarity (BASIS) descriptor was developed with low-resource systems in mind. BASIS utilizes sparse coding to provide a generic description of feature characterstics. The BASIS descriptor provided improved accuracy over SIFT, and similar accuracy to SURF on the task of aerial UAV frame-to-frame feature matching. However, basis dictionaries are non-orthogonal and can contain redundant information. In addition to a feature descriptor, an FPGA-based feature correlation (or matching) system needed to be developed. TreeBASIS was developed to answer this need and address the redundancy issues of BASIS. TreeBASIS utilizes a vocabulary tree to drastically reduce descriptor computation time and descriptor size. TreeBASIS also obtains a higher level of accuracy than SIFT, SURF, and BASIS on the UAV aerial imagery task. Both BASIS and TreeBASIS were implemented in VHDL and are well suited for low-resource FPGA applications. TreeBASIS provides a complete feature detection, description, and correlation system-on-a-chip for low-resource FPGA vision systems.

Local feature detection and its applications in different problems are quite popular in vision research. In order to analyze a scene, its invariant features, which are distinguishable in many views of this scene, are used in pose estimation, object detection and augmented reality. However, required performance metrics might change according to the application type / in general, the main metrics are accepted as accuracy and computational complexity. The contributions in this thesis provide improving these metrics and can be divided into three parts, as local feature detection, local feature description and description matching in different views of the same scene. In this thesis an efficient feature detection algorithm with sufficient repeatability performance is proposed. This detection method is convenient for real-time applications. For local description, a novel local binary pattern outperforming state-of-the-art binary pattern is proposed. As a final task, a fuzzy decision tree method is presented for approximate nearest neighbor search. In all parts of the system, computational efficiency is considered and the algorithms are designed according to limited processing time. Finally, an overall system capable of matching different views of the same scene has been proposed and executed in a mobile platform. The results are quite promising such that the presented system can be used in real-time applications, such as augmented reality, object retrieval, object tracking and pose estimation.

Feature detection and matching is an important step in many object tracking and detection algorithms. This paper discusses methods to improve upon previous work on the SYnthetic BAsis feature descriptor (SYBA) algorithm, which describes and compares image features in an efficient and discreet manner. SYBA utilizes synthetic basis images overlaid on a feature region of interest (FRI) to generate binary numbers that uniquely describe the feature contained within the FRI. These binary numbers are then used to compare against feature values in subsequent images for matching. However, in a non-ideal environment the accuracy of the feature matching suffers due to variations in image scale, and rotation. This paper introduces a new version of SYBA which processes FRI’s such that the descriptions developed by SYBA are rotation and scale invariant. To demonstrate the improvements of this robust implementation of SYBA called rSYBA, included in this paper are applications that have to cope with high amounts of image variation. The first detects objects along an oil pipeline by transforming and comparing frame-by-frame two surveillance videos recorded at two different times. The second shows camera pose plotting for a ground based vehicle using monocular visual odometry. The third generates panoramic images through image stitching and image transforms. All applications contain large amounts of image variation between image frames and therefore require a significant amount of correct feature matches to generate acceptable results.

With the drone industry's recent explosive advancement, aerial photography is becoming increasingly important for an array of applications ranging from construction to agriculture. A drone flyover can give a better overview of regions that are difficult to navigate, and is often significantly faster, cheaper and more accurate than man-made sketches and other alternatives. With this increased use comes a growing need for image processing methods to help in analyzing captured photographs. This thesis presents a method for automatic location detection in aerial photographs using databases of aerial photographs and satellite images. The proposed pipeline is based on an initial round of tests, performed by using existing feature detection, description and matching algorithms on aerial photographs with a high degree of similarity. After which further modifications and improvements were implemented to make the method functional also for handling aerial photographs with a high level of inherent differences, e.g., viewpoint changes, different camera- and lens parameters, temporary objects and weather effects. The method is shown to yield highly accurate results in geographical regions containing features with a low level of ambiguity, and where factors like viewpoint difference are not too extreme. In particular, the method has been most successful in cities and some types of farmland, producing very good results compared to methods based on camera parameters and GPS-location, which have been common in automatic location detection previously. Knowledge of these parameters is not necessary when applying the method, making it applicable more generally and also independently of the precision of the instruments used to determine said parameters.  Furthermore, the approach is extended for automatic processing of video streams. With lack of available ground truth data, no definite conclusions about absolute accuracy of the method can be drawn for this use case. But it is nevertheless clear that processing speeds can be greatly improved by making use of the fact that subsequent video snapshots have a large graphical overlap. And it can indeed also be said that, for the tested video stream, using a type of extrapolation can greatly reduce the risk of graphical noise making location detection impossible for any given snapshot. / Då drönarindustrin växer så det knakar, har flygfoton blivit allt viktigare för en rad applikationer i vårt samhälle. Att flyga över ett svårnavigerat område med en drönare kan ge bättre översikt och är ofta snabbare, billigare och mer precist än skisser eller andra alternativa översiktsmetoder. Med denna ökade användning kommer också ett ökat behov av automatisk bildprocessering för att hjälpa till i analysen av dessa fotografier. Denna avhandling presenterar en metod för automatisk positionsbedömning av flygfoton, med hjälp av databaser med flygfoton och satellitfoton. Den presenterade metoden är baserad på inledande tester av existerande feature detection, feature description och feature matching algoritmer på ett något förenklat problem, där givna foton är väldigt grafiskt lika. Efter detta implementerades ytterligare modifikationer och förbättringar för att göra metoden mer robust även för bilder med en hög nivå av grafisk diskrepans, exempelvis skillnad i synvinkel, kamera- och linsparametrar, temporära objekt och vädereffekter. Den föreslagna metoden ger nöjaktiga resultat i geografiska regioner med en proportionellt stor mängd grafiska särdrag som enkelt kan särskiljas från varandra och där den grafiska diskrepansen inte är allt för stor. Särskilt goda resultat ses i bland annat städer och vissa typer av jordbruksområden, där metoden kan ge betydligt bättre resultat än metoder baserade på kända kameraparametrar och fotografens GPS-positionering, vilket har varit ett vanligt sätt att utföra denna typ av automatisk positionsbestämning tidigare. Dessutom är den presenterade metoden ofta enklare att applicera, då precisionen för diverse mätinstrument som annars måste användas när fotot tas inte spelar in alls i metodens beräkningar. Dessutom har metoden utökats för automatisk processering av videoströmmar. På grund av bristfälligt referensdata kan inga definitiva slutsatser dras angående metodens precision för detta användningsområde. Men det är ändå tydligt att beräkningstiden kan minskas drastiskt genom att använda faktumet att två påföljande ögonblicksbilder har ett stort grafiskt överlapp. Genom att använda en sorts extrapolering kan inverkan från grafiskt brus också minskas, brus som kan göra positionsbestämning omöjligt för en given ögonblicksbild.

Finding salient features in an image, and matching them to their corresponding features in another image is an important step for many vision-based applications. Feature description plays an important role in the feature matching process. A robust feature descriptor must works with a number of image deformations and should be computationally efficient. For resource-limited systems, floating point and complex operations such as multiplication and square root are not desirable. This research first introduces a robust and efficient feature descriptor called PRObability (PRO) descriptor that meets these requirements without sacrificing matching accuracy. The PRO descriptor is further improved by incorporating only affine features for matching. While performing well, PRO descriptor still requires larger descriptor size, higher offline computation time, and more memory space than other binary feature descriptors. SYnthetic BAsis (SYBA) descriptor is developed to overcome these drawbacks. SYBA is built on the basis of a new compressed sensing theory that uses synthetic basis functions to uniquely encode or reconstruct a signal. The SYBA descriptor is designed to provide accurate feature matching for real-time vision applications. To demonstrate its performance, we develop algorithms that utilize SYBA descriptor to localize the soccer ball in a broadcast soccer game video, track ground objects for unmanned aerial vehicle, and perform motion analysis, and improve visual odometry accuracy for advanced driver assistance systems. SYBA provides high feature matching accuracy with computational simplicity and requires minimal computational resources. It is a hardware-friendly feature description and matching algorithm suitable for embedded vision applications.