El objetivo de esta tesina es estudiar, desarrollar, analizar y comparar distintas técnicas de aprendizaje automático aplicables al reconocimiento automático de gestos dinámicos. Para ello, se definió un modelo de gestos a reconocer, se generó una base de datos de prueba con gestos llamadas LNHG, y se estudiaron e implementaron clasificadores basados en máquinas de vectores de soporte (SVM), redes neuronales feedfoward (FF) y redes neuronales competitivas (CPN), utilizando representaciones locales y globales para caracterizar los gestos. Además, se propone un nuevo modelo de reconocimiento de gestos, el clasificador neuronal competitivo (CNC). Los gestos a reconocer son movimientos de la mano, con invariancia a la velocidad, la rotación, la escala y la traslación. La captura de la información referida a los gestos para generar la base de datos se realizó mediante el dispositivo Kinect y su SDK correspondiente, que reconoce las partes del cuerpo y determina sus posiciones en tiempo real. Los clasificadores se entrenaron con dichos datos para poder determinar si una secuencia de posiciones de la mano es un gesto. Se implementó una librería de clasificadores con los métodos mencionados anteriormente, junto con las transformaciones para llevar una secuencia de posiciones a una representación adecuada para el reconocimiento. Se realizaron experimentos con la base de datos LNHG, compuesta de gestos que representan dígitos y letras, y con un base de datos de otro autor con gestos típicos de interacción, obteniendo resultados satisfactorios.

Résumé : L’expansion des images sur le Web a provoqué le besoin de mettre en œuvre des méthodes de classement d’images précises pour plusieurs applications notamment la cybersécurité. L’extraction des caractéristiques est une étape primordiale dans la procédure du classement des images vu son impact direct sur la performance de la catégorisation finale des images et de leur classement. L’objectif de cette étude est d’analyser l’état de l’art des différents espaces de caractéristiques pour évaluer leur efficacité dans le contexte de la reconnaissance de forme pour les applications de cybersécurité. Les expériences ont montré que les descripteurs de caractéristiques HOG et GIST ont une performance élevée. Par contre, cette dernière se dégrade face aux transformations géométriques des objets dans les images. Afin d’obtenir des systèmes de classement d’image plus fiables basés sur ces descripteurs, nous proposons deux méthodes. Dans la première méthode (PrMI) nous nous concentrons sur l’amélioration de la propriété d’invariance du système de classement par tout en maintenant la performance du classement. Dans cette méthode, un descripteur invariant par rapport à la rotation dérivé de HOG est utilisé (RIHOG) dans une technique de recherche "top-down" pour le classement des images. La méthode (PrMI) proposée donne non seulement une robustesse face aux transformations géométriques des objets, mais aussi une performance élevée similaire à celle de HOG. Elle est aussi efficace en terme de coût de calcul avec une complexité de l’ordre de O(n). Dans la deuxième méthode proposée (PrMII), nous nous focalisons sur la performance du classement en maintenant la propriété d’invariance du système de classement. Les objets sont localisés d’une façon invariante aux changement d’échelle dans l’espace de caractéristiques de covariance par région. Ensuite elles sont décrites avec les descripteurs HOG et GIST. Cette méthode procure une performance de classement meilleure en comparaison avec les méthodes implémentées dans l’étude et quelques méthodes CBIR expérimentées sur les données Caltech-256 dans les travaux antérieurs. // Abstract : The tremendous growth of accessible online images (Web images), provokes the need to perform accurate image ranking for applications like cyber-security. Fea­ture extraction is an important step in image ranking procedures due to its direct impact on final categorization and ranking performance. The goal of this study is to analyse the state of the art feature spaces in order to evaluate their efficiency in the abject recognition context and image ranking framework for cyber-security applications. Experiments show that HOG and GIST feature descriptors exhibit high ranking performance. Whereas, these features are not rotation and scale invariant. In order to obtain more reliable image ranking systems based on these feature spaces, we proposed two methods. In the first method (PrMI) we focused on improving the invariance property of the ranking system while maintaining the ranking perfor­mance. In this method, a rotation invariant feature descriptor is derived from HOC (RIHOC). This descriptor is used in a top-down searching technique to caver the scale variation of the abjects in the images. The proposed method (PrMI) not only pro­ vides robustness against geometrical transformations of objects but also provides high ranking performance close to HOC performance. It is also computationally efficient with complexity around O(n). In the second proposed method (PrMII) we focused on the ranking performance while maintaining the invariance property of the ranking system. Objects are localized in a scale invariant fashion under a Region Covariance feature space, then they are described using HOC and CIST features. Finally to ob­ tain better evaluation over the performance of proposed method we compare it with existing research in the similar domain(CBIR) on Caltech-256. Proposed methods provide highest ranking performance in comparison with implemented methods in this study, and some of the CBIR methods on Caltech-256 dataset in previous works.

In this thesis, contributions are presented in the areas of shape parsing for view-based object recognition and spatio-temporal feature description for action recognition. A probabilistic model for parsing shapes into several distinguishable parts for accurate shape recognition is presented. This approach is based on robust geometric features that permit high recognition accuracy. As the second contribution in this thesis, a binary spatio-temporal feature descriptor is presented. Recent work shows that binary spatial feature descriptors are effective for increasing the efficiency of object recognition, while retaining comparable performance to state of the art descriptors. An extension of these approaches to action recognition is presented, facilitating huge gains in efficiency due to the computational advantage of computing a bag-of-words representation with the Hamming distance. A scene's motion and appearance is encoded with a short binary string. Exploiting the binary makeup of this descriptor greatly increases the efficiency while retaining competitive recognition performance.

Background: The visual system may process spatial frequency information in a low-to-high, coarse-to-fine sequence. In particular, low and high spatial frequency information may be processed via different pathways during object recognition, with LSF information projected rapidly to frontal areas and HSF processed later in visual ventral areas. In an electroencephalographic study, we examined the time course of information processing for images filtered to contain different ranges of spatial frequencies. Participants viewed either high spatial frequency (HSF), low spatial frequency (LSF), or unfiltered, broadband (BB) images of objects or nonobject textures, classifying them as showing either man-made or natural objects, or nonobjects. Event-related potentials (ERPs) and evoked and total gamma band activity (eGBA and tGBA) recorded using the electroencephalogram were compared for object and nonobject images across the different spatial frequency ranges. Results: The visual P1 showed independent modulations by object and spatial frequency, while for the N1 these factors interacted. The P1 showed more positive amplitudes for objects than nonobjects, and more positive amplitudes for BB than for HSF images, which in turn evoked more positive amplitudes than LSF images. The peak-to-peak N1 showed that the N1 was much reduced for BB non-objects relative to all other images, while HSF and LSF nonobjects still elicited as negative an N1 as objects. In contrast, eGBA was influenced by spatial frequency and not objecthood, while tGBA showed a stronger response to objects than nonobjects. Conclusions: Different pathways are involved in the processing of low and high spatial frequencies during object recognition, as reflected in interactions between objecthood and spatial frequency in the visual N1 component. Total gamma band seems to be related to a late, probably highlevel representational process.

In very few mobile robotic applications stereo vision based navigation and mapping is used because dealing with stereo images is very hard and very time consuming. Despite all the problems, stereo vision still becomes one of the most important resources of knowing the world for a mobile robot because imaging provides much more information than most other sensors. Real robotic applications are very complicated because besides the problems of finding how the robot should behave to complete the task at hand, the problems faced while controlling the robot&rsquo / s internal parameters bring high computational load. Thus, finding the strategy to be followed in a simulated world and then applying this on real robot for real applications is preferable. In this study, we describe an algorithm for object recognition and cognitive map formation using stereo image data in a 3D virtual world where 3D objects and a robot with active stereo imaging system are simulated. Stereo imaging system is simulated so that the actual human visual system properties are parameterized. Only the stereo images obtained from this world are supplied to the virtual robot. By applying our disparity algorithm, depth map for the current stereo view is extracted. Using the depth information for the current view, a cognitive map of the environment is updated gradually while the virtual agent is exploring the environment. The agent explores its environment in an intelligent way using the current view and environmental map information obtained up to date. Also, during exploration if a new object is observed, the robot turns around it, obtains stereo images from different directions and extracts the model of the object in 3D. Using the available set of possible objects, it recognizes the object.

Computer vision tasks such as three dimensional (3D) registration, 3D modelling, and 3D object recognition are becoming more and more useful in industry, and have application such as reverse CAD engineering, and robot navigation. Each of these applications use correspondence algorithms as part of their processes. Correspondence algorithms are required to compute accurate mappings between artificial surfaces that represent actual objects or scenes. In industry, inaccurate correspondence is related to factors such as expenses in time and labour, and also safety. Therefore, it is essential to select an appropriate correspondence algorithm for a given surface matching task. However, current research in the area of surface correspondence is hampered by an abundance of applications specific algorithms, and no uniform terminology of consistent model for selecting and/or comparing algorithms. This dissertation presents a correspondence framework for surface matching algorithms. The framework is a conceptual model that is implementable. It is designed to assist in the analysis, comparison, development, and implementation of correspondence algorithms, which are essential tasks when selecting or creating an algorithm for a particular application. The primary contribution of the thesis is the correspondence framework presented as a conceptual model for surface matching algorithms. The model provides a systematic method for analysing, comparing, and developing algorithms. The dissertation demonstrates that by dividing correspondence computation into five stages: region definition, feature extraction, feature representation, local matching, and global matching, the task becomes smaller and more manageable. It also shows that the same stages of different algorithms are directly comparable. Furthermore, novel algorithms can be created by simply connecting compatible stages of different algorithms. Finally, new ideas can be synthesised by creating only the stages to be tested, without developing a while new correspondence algorithm. The secondary contribution that is outlined is the correspondence framework presented as a software design tool for surface matching algorithms. The framework is shown to reduce the complexity of implementing existing algorithms within the framework. This is done by encoding algorithms in a stage-wise procedure, whereby an algorithm is separated into the five stages of the framework. The software design tool is shown to validate the integrity of restructuring existing algorithms within it, and also provide an efficient basis for creating new algorithms. The third contribution that is made is the specification of a quality metric for algorithms comparison. The metric is used to assess the accuracy of the outcomes of a number of correspondence algorithms, which are used to match a wide variety of input surface pairs. The metric is used to demonstrate that each algorithm is application specific, and highlight the types of surfaces that can be matched by each algorithm. Thus, it is shown that algorithms that are implemented within the framework can be selected for particular surface correspondence tasks. The final contribution made is this dissertation is the expansion of the correspondence framework beyond the surface matching domain. The correspondence framework is maintained in its original form, and is used for image matching algorithms. Existing algorithms from three image matching applications are implemented and modified using the framework. It is shown how the framework provides a consistent means and uniform terminology for developing both surface and image matching algorithms. In summary, this thesis presents a correspondence framework for surface matching algorithms. The framework is general, encompassing a comprehensive set of algorithms, and flexible, expanding beyond surface matching to major image matching applications.

This thesis explores neural activity associated with processing of rotated alphanumeric characters, focusing particularly on linear and quadratic trend components of orientation-dependent activity. Choice of these components was driven by results of reaction-time (RT) studies; judging whether characters are normal or backward (parity task) typically elicit RTs that are linearly related to character disorientation, implying mental rotation of the characters to the upright, while judging whether they are letters or digits (categorisation task) elicits RTs related nonlinearly to disorientation, combining both linear and quadratic component, but not indicative of mental rotation. In Experiment 1 neural activity was monitored using fMRI while participants performed these tasks. In the next two experiments, neural processing was monitored with high-density EEG. In Experiment 2 participants performed the same two tasks, while in Experiment 3 they performed the category task and red-blue colour judgements. In Experiment 1, linear increases in fMRI activation were elicited only by the parity task and were observed in the posterior portion of the dorsal intraparietal sulcus and lateral and medial pre-supplementary motor areas, suggesting a fronto-parietal network underlying mental rotation. Experiment 2 showed that linear increases in parietal negativity between 350 and 710 ms only evident in the parity task, again indicating that mental rotation is only elicited by that task. Contrary to previous evidence, Experiment 2 indicated that both hemispheres may be involved in mental rotation, but rotation is faster in the right hemisphere than in the left hemisphere. Experiment 1 also showed that effects of orientation common to both tasks were best characterised by a quadratic trend, and were restricted to the supramarginal gyrus. This activation was interpreted as representing orientation-dependent shape recognition. Experiments 2 and 3 also revealed orientation-dependent neural activity at three distinct stages prior to mental rotation. First, on the P1 component, there was a difference between oblique and vertical orientations, suggesting the extraction of orientation based on axis of elongation. Next, orientation affected the N1 component, with longer latencies and larger amplitudes with misorientation, and smaller effects for inversion than for intermediate angular rotations. Finally, changes in orientation affected the P2 component differently for the parity and category tasks, probably signalling the perception of orientation relative to a parity-defined memory representation, and serving as a preparation for mental rotation. These experiments identify both the orientation-specific neural processing that occurs prior to the onset of mental rotation, and the subsequent neural correlates of mental rotation itself. / Top Achiever Doctoral Scholarship, University of Auckland Doctoral Scholarship, The New Zealand Neurological Foundation, University of Auckland Research Fund (Project numbers: 3607199, 3605876 3604420)

This thesis explores neural activity associated with processing of rotated alphanumeric characters, focusing particularly on linear and quadratic trend components of orientation-dependent activity. Choice of these components was driven by results of reaction-time (RT) studies; judging whether characters are normal or backward (parity task) typically elicit RTs that are linearly related to character disorientation, implying mental rotation of the characters to the upright, while judging whether they are letters or digits (categorisation task) elicits RTs related nonlinearly to disorientation, combining both linear and quadratic component, but not indicative of mental rotation. In Experiment 1 neural activity was monitored using fMRI while participants performed these tasks. In the next two experiments, neural processing was monitored with high-density EEG. In Experiment 2 participants performed the same two tasks, while in Experiment 3 they performed the category task and red-blue colour judgements. In Experiment 1, linear increases in fMRI activation were elicited only by the parity task and were observed in the posterior portion of the dorsal intraparietal sulcus and lateral and medial pre-supplementary motor areas, suggesting a fronto-parietal network underlying mental rotation. Experiment 2 showed that linear increases in parietal negativity between 350 and 710 ms only evident in the parity task, again indicating that mental rotation is only elicited by that task. Contrary to previous evidence, Experiment 2 indicated that both hemispheres may be involved in mental rotation, but rotation is faster in the right hemisphere than in the left hemisphere. Experiment 1 also showed that effects of orientation common to both tasks were best characterised by a quadratic trend, and were restricted to the supramarginal gyrus. This activation was interpreted as representing orientation-dependent shape recognition. Experiments 2 and 3 also revealed orientation-dependent neural activity at three distinct stages prior to mental rotation. First, on the P1 component, there was a difference between oblique and vertical orientations, suggesting the extraction of orientation based on axis of elongation. Next, orientation affected the N1 component, with longer latencies and larger amplitudes with misorientation, and smaller effects for inversion than for intermediate angular rotations. Finally, changes in orientation affected the P2 component differently for the parity and category tasks, probably signalling the perception of orientation relative to a parity-defined memory representation, and serving as a preparation for mental rotation. These experiments identify both the orientation-specific neural processing that occurs prior to the onset of mental rotation, and the subsequent neural correlates of mental rotation itself. / Top Achiever Doctoral Scholarship, University of Auckland Doctoral Scholarship, The New Zealand Neurological Foundation, University of Auckland Research Fund (Project numbers: 3607199, 3605876 3604420)

This thesis explores neural activity associated with processing of rotated alphanumeric characters, focusing particularly on linear and quadratic trend components of orientation-dependent activity. Choice of these components was driven by results of reaction-time (RT) studies; judging whether characters are normal or backward (parity task) typically elicit RTs that are linearly related to character disorientation, implying mental rotation of the characters to the upright, while judging whether they are letters or digits (categorisation task) elicits RTs related nonlinearly to disorientation, combining both linear and quadratic component, but not indicative of mental rotation. In Experiment 1 neural activity was monitored using fMRI while participants performed these tasks. In the next two experiments, neural processing was monitored with high-density EEG. In Experiment 2 participants performed the same two tasks, while in Experiment 3 they performed the category task and red-blue colour judgements. In Experiment 1, linear increases in fMRI activation were elicited only by the parity task and were observed in the posterior portion of the dorsal intraparietal sulcus and lateral and medial pre-supplementary motor areas, suggesting a fronto-parietal network underlying mental rotation. Experiment 2 showed that linear increases in parietal negativity between 350 and 710 ms only evident in the parity task, again indicating that mental rotation is only elicited by that task. Contrary to previous evidence, Experiment 2 indicated that both hemispheres may be involved in mental rotation, but rotation is faster in the right hemisphere than in the left hemisphere. Experiment 1 also showed that effects of orientation common to both tasks were best characterised by a quadratic trend, and were restricted to the supramarginal gyrus. This activation was interpreted as representing orientation-dependent shape recognition. Experiments 2 and 3 also revealed orientation-dependent neural activity at three distinct stages prior to mental rotation. First, on the P1 component, there was a difference between oblique and vertical orientations, suggesting the extraction of orientation based on axis of elongation. Next, orientation affected the N1 component, with longer latencies and larger amplitudes with misorientation, and smaller effects for inversion than for intermediate angular rotations. Finally, changes in orientation affected the P2 component differently for the parity and category tasks, probably signalling the perception of orientation relative to a parity-defined memory representation, and serving as a preparation for mental rotation. These experiments identify both the orientation-specific neural processing that occurs prior to the onset of mental rotation, and the subsequent neural correlates of mental rotation itself. / Top Achiever Doctoral Scholarship, University of Auckland Doctoral Scholarship, The New Zealand Neurological Foundation, University of Auckland Research Fund (Project numbers: 3607199, 3605876 3604420)

This thesis explores neural activity associated with processing of rotated alphanumeric characters, focusing particularly on linear and quadratic trend components of orientation-dependent activity. Choice of these components was driven by results of reaction-time (RT) studies; judging whether characters are normal or backward (parity task) typically elicit RTs that are linearly related to character disorientation, implying mental rotation of the characters to the upright, while judging whether they are letters or digits (categorisation task) elicits RTs related nonlinearly to disorientation, combining both linear and quadratic component, but not indicative of mental rotation. In Experiment 1 neural activity was monitored using fMRI while participants performed these tasks. In the next two experiments, neural processing was monitored with high-density EEG. In Experiment 2 participants performed the same two tasks, while in Experiment 3 they performed the category task and red-blue colour judgements. In Experiment 1, linear increases in fMRI activation were elicited only by the parity task and were observed in the posterior portion of the dorsal intraparietal sulcus and lateral and medial pre-supplementary motor areas, suggesting a fronto-parietal network underlying mental rotation. Experiment 2 showed that linear increases in parietal negativity between 350 and 710 ms only evident in the parity task, again indicating that mental rotation is only elicited by that task. Contrary to previous evidence, Experiment 2 indicated that both hemispheres may be involved in mental rotation, but rotation is faster in the right hemisphere than in the left hemisphere. Experiment 1 also showed that effects of orientation common to both tasks were best characterised by a quadratic trend, and were restricted to the supramarginal gyrus. This activation was interpreted as representing orientation-dependent shape recognition. Experiments 2 and 3 also revealed orientation-dependent neural activity at three distinct stages prior to mental rotation. First, on the P1 component, there was a difference between oblique and vertical orientations, suggesting the extraction of orientation based on axis of elongation. Next, orientation affected the N1 component, with longer latencies and larger amplitudes with misorientation, and smaller effects for inversion than for intermediate angular rotations. Finally, changes in orientation affected the P2 component differently for the parity and category tasks, probably signalling the perception of orientation relative to a parity-defined memory representation, and serving as a preparation for mental rotation. These experiments identify both the orientation-specific neural processing that occurs prior to the onset of mental rotation, and the subsequent neural correlates of mental rotation itself. / Top Achiever Doctoral Scholarship, University of Auckland Doctoral Scholarship, The New Zealand Neurological Foundation, University of Auckland Research Fund (Project numbers: 3607199, 3605876 3604420)