Structured representation of input physical patterns as a set of local features has been useful for a veriety of robotics and human computer interaction (HCI) applications. It enables a stable understanding of the variable inputs. However, this representation does not fit the conventional machine learning algorithms and distance metrics because they assume vector inputs. To learn from input patterns with variable structure is thus challenging. To address this problem, I propose a general and systematic method to design distance metrics between structured inputs that can be used in conventional learning algorithms. Based on the observation of the stability in the geometric distributions of local features over the physical patterns across similar inputs, this is done combining the local similarities and the conformity of the geometric relationship between local features. The produced distance metrics, called “parametric kernels”, are positive semi-definite and require almost linear time to compute. To demonstrate the general applicability and the efficacy of this approach, I designed and applied parametric kernels to handwritten character recognition, on-line face recognition, and object detection from laser range finder sensor data. Parametric kernels achieve recognition rates competitive to state-of-the-art approaches in these tasks. / text

Many automotive safety applications in modern cars make use of cameras and object detection to analyze the surrounding environment. Pedestrians, animals and other vehicles can be detected and safety actions can be taken before dangerous situations arise. To detect occurrences of the different objects, these systems are traditionally trained to learn a classification model using a set of images that carry labels corresponding to their content. To obtain high performance with a variety of object appearances, the required amount of data is very large. Acquiring unlabeled images is easy, while the manual work of labeling is both time-consuming and costly. Semi-supervised learning refers to methods that utilize both labeled and unlabeled data, a situation that is highly desirable if it can lead to improved accuracy and at the same time alleviate the demand of labeled data. This has been an active area of research in the last few decades, but few studies have investigated the performance of these algorithms in larger systems. In this thesis, we investigate if and how semi-supervised learning can be used in a large-scale pedestrian detection system. With the area of application being automotive safety, where real-time performance is of high importance, the work is focused around boosting classifiers. Results are presented on a few publicly available UCI data sets and on a large data set for pedestrian detection captured in real-life traffic situations. By evaluating the algorithms on the pedestrian data set, we add the complexity of data set size, a large variety of object appearances and high input dimension. It is possible to find situations in low dimensions where an additional set of unlabeled data can be used successfully to improve a classification model, but the results show that it is hard to efficiently utilize semi-supervised learning in large-scale object detection systems. The results are hard to scale to large data sets of higher dimensions as pair-wise computations are of high complexity and proper similarity measures are hard to find.

This thesis is devoted to the problems of defining and developing the basic building blocks of an automated surveillance system. As its initial step, a background-modeling algorithm is described for segmenting moving objects from the background, which is capable of adapting to dynamic scene conditions, as well as determining shadows of the moving objects. After obtaining binary silhouettes for targets, object association between consecutive frames is achieved by a hypothesis-based tracking method. Both of these tasks provide basic information for higher-level processing, such as activity analysis and object identification. In order to recognize the nature of an event occurring in a scene, hidden Markov models (HMM) are utilized. For this aim, object trajectories, which are obtained through a successful track, are written as a sequence of flow vectors that capture the details of instantaneous velocity and location information. HMMs are trained with sequences obtained from usual motion patterns and abnormality is detected by measuring the distance to these models. Finally, MPEG-7 visual descriptors are utilized in a regional manner for object identification. Color structure and homogeneous texture parameters of the independently moving objects are extracted and classifiers, such as Support Vector Machine (SVM) and Bayesian plug-in (Mahalanobis distance), are utilized to test the performance of the proposed person identification mechanism. The simulation results with all the above building blocks give promising results, indicating the possibility of constructing a fully automated surveillance system for the future.

Visual target tracking problem consists of two topics: Obtaining targets from camera measurements and target tracking. Even though it has been studied for more than 30 years, there are still some problems not completely solved. Especially in the case of multiple targets, association of measurements to targets, creation of new targets and deletion of old ones are among those. What is more, it is very important to deal with the occlusion and crossing targets problems suitably. We believe that a slightly modified version of multiple hypothesis tracking can successfully deal with most of the aforementioned problems with sufficient success. Distance, track size, track color, gate size and track history are used as parameters to evaluate the hypotheses generated for measurement to track association problem whereas size and color are used as parameters for occlusion problem. The overall tracker has been fine tuned over some scenarios and it has been observed that it performs well over the testing scenarios as well. Furthermore the performance of the tracker is analyzed according to those parameters in both association and occlusion handling situations.

In the master thesis about object detection(OD) using deep convolutional neural network(DCNN), the area of OD is being tested when being applied to infrared images(IR). In this thesis the, goal is to use both long wave infrared(LWIR) images and short wave infrared(SWIR) images taken from an airplane in order to train a DCNN to detect runways, Precision Approach Path Indicator(PAPI) lights, and approaching lights. The purpose for detecting these objects in IR images is because IR light transmits better than visible light under certain weather conditions, for example, fog. This system could then help the pilot detect the runway in bad weather. The RetinaNet model architecture was used and modified in different ways to find the best performing model. The models contain parameters that are found during the training process but some parameters, called hyperparameters, need to be determined in advance. A way to automatically find good values of these hyperparameters was also tested. In hyperparameter optimization, the Bayesian optimization method proved to create a model with equally good performance as the best performance acieved by the author using manual hyperparameter tuning. The OD system was implemented using Keras with Tensorflow backend and received a high perfomance (mAP=0.9245) on the test data. The system manages to detect the wanted objects in the images but is expected to perform worse in a general situation since the training data and test data are very similar. In order to further develop this system and to improve performance under general conditions more data is needed from other airfields and under different weather conditions.

La détection de personnes dans les vidéos est un défi bien connu du domaine de la vision par ordinateur avec un grand nombre d'applications telles que le développement de systèmes de surveillance visuels. Même si les détecteurs monoculaires sont plus simples à mettre en place, ils sont dans l’incapacité de gérer des scènes complexes avec des occultations, une grande densité de personnes ou des scènes avec beaucoup de profondeur de champ menant à une grande variabilité dans la taille des personnes. Dans cette thèse, nous étudions la détection de personnes multi-vues et notamment l'utilisation de cartes d'occupation probabilistes créées en fusionnant les différentes vues grâce à la connaissance de la géométrie du système. La détection à partir de ces cartes d'occupation amène cependant des fausses détections (appelées « fantômes ») dues aux différentes projections. Nous proposons deux nouvelles techniques afin de remédier à ce phénomène et améliorer la détection des personnes. La première utilise une déconvolution par un noyau dont la forme varie spatialement tandis que la seconde est basée sur un principe de validation d’hypothèse. Ces deux approches n'utilisent volontairement pas l'information temporelle qui pourra être réintroduite par la suite dans des algorithmes de suivi. Les deux approches ont été validées dans des conditions difficiles présentant des occultations, une densité de personnes plus ou moins élevée et de fortes variations dans les réponses colorimétriques des caméras. Une comparaison avec d'autres méthodes de l’état de l'art a également été menée sur trois bases de données publiques, validant les méthodes proposées pour la surveillance d'une gare et d'un aéroport / People detection is a well-studied open challenge in the field of Computer Vision with applications such as in the visual surveillance systems. Monocular detectors have limited ability to handle occlusion, clutter, scale, density. Ubiquitous presence of cameras and computational resources fuel the development of multi-camera detection systems. In this thesis, we study the multi-camera people detection; specifically, the use of multi-view probabilistic occupancy maps based on the camera calibration. Occupancy maps allow multi-view geometric fusion of several camera views. Detection with such maps create several false detections and we study this phenomenon: ghost pruning. Further, we propose two novel techniques in order to improve multi-view detection based on: (a) kernel deconvolution, and (b) occupancy shape modeling. We perform non-temporal, multi-view reasoning in occupancy maps to recover accurate positions of people in challenging conditions such as of occlusion, clutter, lighting, and camera variations. We show improvements in people detections across three challenging datasets for visual surveillance including comparison with state-of-the-art techniques. We show the application of this work in exigent transportation scenarios i.e. people detection for surveillance at a train station and at an airport

The goals of this work are a design and implementation of a new vision system, integrated with the waterjet machine. This system combines two commercial webcams applied on an industrial dedicated platform. A main purpose of the vision system is to detect the position and rotation of a workpiece placed on the machine table. The used object recognition algorithm consists of edge detection, standard math processing functions and noise filters. The Hough transform technique is used to extract lines and their intersections of a workpiece. Metric rectification method is used, in order to obtain a top view of the workspace and to adjust an image coordinate system, accordingly to the waterjet machine coordinates. In situ calibration procedures of the booth webcams are developed and implemented. Experimental results of the proposed new vision system prototype confirm required performance and precision of the element detection.

Radiologists are often plagued by limited time for completing their work, with an ever increasing workload. A picture archiving and communication system (PACS) is a platform for daily image reviewing that improves their work environment, and on that platform for example spinal MR images can be reviewed. When reviewing spinal images a radiologist wants vertebrae labels, and in Sectra's PACS platform there is a good opportunity for implementing an automatic method for spinal labeling. In this thesis a method for performing automatic spinal labeling, called a vertebrae classifier, is presented. This method should remove the need for radiologists to perform manual spine labeling, and could be implemented in Sectra's PACS software to improve radiologists overall work experience.Spine labeling is the process of marking vertebrae centres with a name on a spinal image. The method proposed in this thesis for performing that process was developed using a machine learning approach for vertebrae detection in sagittal MR images. The developed classifier works for both the lumbar and the cervical spine, but it is optimized for the lumbar spine. During the development three different methods for the purpose of vertebrae detection were evaluated. Detection is done on multiple sagittal slices. The output from the detection is then labeled using a pictorial structure based algorithm which uses a trained model of the spine to correctly assess correct labeling. The suggested method achieves 99.6% recall and 99.9% precision for the lumbar spine. The cervical spine achieves slightly worse performance, with 98.1% for both recall and precision. This result was achieved by training the proposed method on 43 images and validated with 89 images for the lumbar spine. The cervical spine was validated using 26 images. These results are promising, especially for the lumbar spine. However, further evaluation is needed to test the method in a clinical setting. / Radiologer får bara mindre och mindre tid för att utföra sina arbetsuppgifter, då arbetsbördan bara blir större. Ett picture archiving and communication system (PACS) är en platform där radiologer kan undersöka medicinska bilder, däribland magnetic resonance (MR) bilder av ryggraden. När radiologerna tittar på dessa bilder av ryggraden vill de att kotorna ska vara markerade med sina namn, och i Sectra's PACS platform finns det en bra möjlighet för att implementera en automatisk metod för att namnge ryggradens kotor på bilden. I detta examensarbete presenteras en metod för att automatiskt markera alla kotorna utifrån saggitala MR bilder. Denna metod kan göra så att radiologer inte längre behöver manuellt markera kotor, och den skulle kunna implementeras i Sectra's PACS för att förbättra radiologernas arbetsmiljö. Det som menas med att markera kotor är att man ger mitten av alla kotor ett namn utifrån en MR bild på ryggraden. Metoden som presenteras i detta arbete kan utföra detta med hjälp av ett "machine learning" arbetssätt. Metoden fungerar både för övre och nedre delen av ryggraden, men den är optimerad för den nedre delen. Under utvecklingsfasen var tre olika metoder för att detektera kotor evaluerade. Resultatet från detektionen är sedan använt för att namnge alla kotor med hjälp av en algoritm baserad på pictorial structures, som använder en tränad model för att kunna evaluera vad som bör anses vara korrekt namngivning. Metoden uppnår 99.6% recall och 99.9% precision för nedre ryggraden. För övre ryggraden uppnås något sämre resultat, med 98.1% vad gäller både recall och precision. Detta resultat uppnådes då metoden tränades på 43 bilder och validerades på 89 bilder för nedre ryggraden. För övre ryggraden användes 26 stycken bilder. Resultaten är lovande, speciellt för den nedre delen. Dock måste ytterligare utvärdering göras för metoden i en klinisk miljö.

La thèse s'inscrit dans l'étude de la structuration des forêts à partir des propriétés de la canopée telles que décrites par la distribution spatiale ou la taille des houppiers des arbres dominants. L'approche suivie est fondée sur la théorie des Processus Ponctuels Marqués (PPM) qui permet de modéliser ces houppiers comme des disques sur images considérées comme un espace 2D. Le travail a consisté à évaluer le potentiel des PPM pour détecter automatiquement les houppiers d'arbres dans des images optiques de très résolution spatiale acquises sur des forêts de mangroves et des plantations d'Eucalyptus. Pour les mangroves, nous avons également travaillé sur des images simulées de réflectance et des données Lidar. Différentes adaptations (paramétrage, modèles d'énergie) de la méthode de PPM ont été testées et comparées grâce à des indices quantitatifs de comparaison entre résultats de la détection et références de positionnement issues du terrain, de photo-interprétation ou de maquettes forestières.Dans le cas des mangroves, les tailles de houppier estimées par détection restent cohérentes avec les sorties des modèles allométriques disponibles. Les résultats thématiques indiquent que la détection par PPM permet de cartographier dans une jeune plantation d'Eucalyptus la densité locale d'arbres dont la taille des houppiers est proche de la résolution spatiale de l'image (0.5m). Cependant, la qualité de la détection diminue quand le couvert se complexifie. Ce travail dresse plusieurs pistes de recherche tant mathématique, comme la prise en compte des objets de forme complexe, que thématiques, comme l'apport des informations forestières à des échelles pertinentes pour la mise au point de méthodes de télédétection. / This PhD work aims at providing information on the forest structure through the analysis of canopy properties as described by the spatial distribution and the crown size of dominant trees. Our approach is based on the Marked Point Processes (MPP) theory, which allows modeling tree crowns observed in remote sensing images by discs belonging a two dimensional space. The potential of MPP to detect the trees crowns automatically is evaluated by using very high spatial resolution optical satellite images of both Eucalyptus plantations and mangrove forest. Lidar and simulated reflectance images are also analyzed for the mangrove application. Different adaptations (parameter settings, energy models) of the MPP method are tested and compared through the development of quantitative indices that allow comparison between detection results and tree references derived from the field, photo-interpretation or the forest mockups.In the case of mangroves, the estimated crown sizes from detections are consistent with the outputs from the available allometric models. Other results indicate that tree detection by MPP allows mapping, the local density of trees of young Eucalyptus plantations even if crown size is close to the image spatial resolution (0.5m). However, the quality of detection by MPP decreases with canopy closeness. To improve the results, further work may involve MPP detection using objects with finer shapes and forest data measurements collected at the tree plant scale.

Bien que devenant un environnement de plus en plus riche technologiquement, la salle opératoire reste un endroit où la sécurité des patients n'est pas assurée à 100% comme le montre le nombre toujours conséquent d'erreurs chirurgicales. La nécessité de développer des systèmes intelligents au bloc opératoire apparait comme croissante. Un des éléments clés pour ce type de système est la reconnaissance du processus chirurgical, passant par une identification précise des outils chirurgicaux utilisés. L'objectif de cette thèse a donc porté sur la détection en temps-réel des outils chirurgicaux dans des vidéos 2D provenant de microscopes. Devant l'absence de jeux de données de référence, qui plus est dans un contexte neurochirurgical, la première contribution de la thèse a donc été la création d'un nouvel ensemble d'images de chirurgies du cerveau et du rachis cervical, mis à disposition en ligne. Comme seconde contribution, deux approches différentes ont été proposées permettant de détecter des outils chirurgicaux via des techniques d'analyse d'image. Tout d'abord, le SquaresChnFtrs adapté, basé sur une des méthodes de détection de piétons les plus performantes de la littérature. Notre deuxième méthode, le ShapeDetector, est une approche à deux niveaux n'utilisant aucune contrainte ou hypothèse a priori sur le nombre, la position, ou la forme des outils dans l'image. Par rapport aux travaux précédents du domaine, nous avons choisi de représenter les détections potentielles par des polygones plutôt que par des rectangles, obtenant ainsi des détections plus précises. Pour intégration dans des systèmes médicaux, une optimisation de la vitesse de calcul a été effectuée via un usage optimal du CPU, du GPU, et de méthodes ad-hoc. Pour des vidéos de résolution 612x480 pixels, notre ShapeDetector est capable d'effectuer les détections à une vitesse maximale de 8 Hz. Pour la validation de nos méthodes, nous avons pris en compte trois paramètres: la position globale, la position de l'extrémité, et l'orientation des détections. Les méthodes ont été classées et comparées avec des méthodes de référence compétitives. Pour la détection des tubes d'aspiration, nous avons obtenu un taux de manqué de 15% pour un taux de faux positifs par image de 0.1, comparé à un taux de manqué de 55% pour le SquaresChnFtrs adapté. L'orientation future du travail devra porter sur l'intégration des informations 3D, l'amélioration de la couche de labélisation sémantique, et la classification des outils chirurgicaux. Pour finir, un enrichissement du jeu de données et des annotations de plus haute précision seront nécessaires. / Despite modern-life technological advances and tremendous progress made in surgical techniques including MIS, today's OR is facing many challenges remaining yet to be addressed. The development of CAS systems integrating the SPM methodology was born as a response from the medical community, with the long-term objective to create surgical cockpit systems. Being able to identify surgical tools in use is a key component for systems relying on the SPM methodology. Towards that end, this thesis work has focused on real-time surgical tool detection from microscope 2D images. From the review of the literature, no validation data-sets have been elected as benchmarks by the community. In addition, the neurosurgical context has been addressed only once. As such, the first contribution of this thesis work consisted in the creation of a new surgical tool data-set, made freely available online. Two methods have been proposed to tackle the surgical tool detection challenge. First, the adapted SquaresChnFtrs, evolution of one of the best computer vision state-of-the-art approach for pedestrian detection. Our second contribution, the ShapeDetector, is fully data-driven and performs detection without the use of prior knowledge regarding the number, shape, and position of tools in the image. Compared to previous works, we chose to represent candidate detections with bounding polygons instead of bounding boxes, hence providing more fitting results. For integration into medical systems, we performed different code optimization through CPU and GPU use. Speed gain and accuracy loss from the use of ad-hoc optimization strategies have been thoroughly quantified to find an optimal trade-off between speed and accuracy. Our ShapeDetector is running in-between 5 and 8Hz for 612x480 pixel video sequences.We validated our approaches using a detailed methodology covering the overall tool location, tip position, and orientation. Approaches have been compared and ranked conjointly with a set of competitive baselines. For suction tube detections, we achieved a 15% miss-rate at 0.1 FPPI, compared to a 55% miss-rate for the adapted SquaresChnFtrs. Future works should be directed toward the integration of 3D feature extraction to improve detection performance but also toward the refinement of the semantic labelling step. Coupling the tool detection task to the tool classification in one single framework should be further investigated. Finally, increasing the data-set in diversity, number of tool classes, and detail of annotations is of interest.