In the post September 11 era, the demand for security has increased in virtually all parts of the society. The need for increased security originates from the emergence of new threats which differ from the traditional ones in such a way that they cannot be easily defined and are sometimes unknown or hidden in the “noise” of daily life. When the threats are known and definable, methods based on situation recognition can be used find them. However, when the threats are hard or impossible to define, other approaches must be used. One such approach is data-driven anomaly detection, where a model of normalcy is built and used to find anomalies, that is, things that do not fit the normal model. Anomaly detection has been identified as one of many enabling technologies for increasing security in the society. In this thesis, the problem of how to detect anomalies in the surveillance domain is studied. This is done by a characterisation of the surveillance domain and a literature review that identifies a number of weaknesses in previous anomaly detection methods used in the surveillance domain. Examples of identified weaknesses include: the handling of contextual information, the inclusion of expert knowledge and the handling of joint attributes. Based on the findings from this study, a new anomaly detection method is proposed. The proposed method is evaluated with respect to detection performance and computational cost on a number datasets, recorded from real-world sensors, in different application areas of the surveillance domain. Additionally, the method is also compared to two other commonly used anomaly detection methods. Finally, the method is evaluated on a dataset with anomalies developed together with maritime subject matter experts. The conclusion of the thesis is that the proposed method has a number of strengths compared to previous methods and is suitable foruse in operative maritime command and control systems. / Christoffer Brax forskar också vid högskolan i Skövde, Informatics Research Centre / Christoffer Brax also does research at the University of Skövde, Informatics Research Centre

We propose a framework for detecting and tracking multiple interacting objects from a single, static, uncalibrated camera. The number of objects is variable and unknown, and object-class-specific models are not available. We use background subtraction results as measurements for object detection and tracking. Given these constraints, the main challenge is to associate pixel measurements with (possibly interacting) object targets. We first track clusters of pixels, and note when they merge or split. We then build an inference graph, representing relations between the tracked clusters. Using this graph and a generic object model based on spatial connectedness and coherent motion, we label the tracked clusters as whole objects, fragments of objects or groups of interacting objects. The outputs of our algorithm are entire tracks of objects, which may include corresponding tracks from groups of objects during interactions. Experimental results on multiple video sequences are shown.

La détection d'objets en mouvement représente une étape très importante pour de nombreuses applications telles que l'analyse du comportement humain pour la surveillance visuelle, la reconnaissance d'action par modèle, le suivi du trafic routier, etc. La soustraction d'arrière-plan est une approche populaire, mais difficile étant donnée qu'elle doit surmonter de nombreux obstacles, comme l'évolution dynamique du fond, les variations de luminosité, les occlusions, etc. Dans les travaux présentés, nous nous sommes intéressés à ce problème de segmentation objets/fond, avec une modélisation floue de type-2 pour gérer l'imprécision du modèle et des données. La méthode proposée modélise l'état de chaque pixel à l'aide d'un modèle de mélange de gaussiennes imprécis et évolutif, qui est exploité par plusieurs classifieurs flous pour finalement estimer la classe du pixel à chaque image. Plus précisément, cette décision prend en compte l'historique de son évolution, mais aussi son voisinage spatial et ses éventuels déplacements dans les images précédentes. Puis nous avons comparé la méthode proposée avec d'autres méthodes proches, notamment des méthodes basées sur un modèle de mélanges gaussiens, des méthodes basées floues, ou de type ACP. Cette comparaison nous a permis de situer notre méthode par rapport à l'existant et de proposer quelques perspectives à ce travail. / Moving objects detection is a very important step for many applications such as human behavior analysis surveillance, model-based action recognition, road traffic monitoring, etc. Background subtraction is a popular approach, but difficult given that it must overcome many obstacles, such as dynamic background changes, brightness variations, occlusions, and so on. In the presented works, we focused on this problem of objects/background segmentation, using a type-2 fuzzy modeling to manage the inaccuracy of the model and the data. The proposed method models the state of each pixel using an imprecise and scalable Gaussian mixture model, which is exploited by several fuzzy classifiers to ultimately estimate the pixel class at each image. More precisely, this decision takes into account the history of its evolution, but also its spatial neighborhood and its possible displacements in the preceding images. Then we compared the proposed method with other close methods, including methods based on a gaussian mixture model, fuzzy based methods, or ACP type methods. This comparison allowed us to assess its good performances, and to propose some perspectives to this work.

Visual target tracking problem consists of two topics: Obtaining targets from camera measurements and target tracking. Even though it has been studied for more than 30 years, there are still some problems not completely solved. Especially in the case of multiple targets, association of measurements to targets, creation of new targets and deletion of old ones are among those. What is more, it is very important to deal with the occlusion and crossing targets problems suitably. We believe that a slightly modified version of multiple hypothesis tracking can successfully deal with most of the aforementioned problems with sufficient success. Distance, track size, track color, gate size and track history are used as parameters to evaluate the hypotheses generated for measurement to track association problem whereas size and color are used as parameters for occlusion problem. The overall tracker has been fine tuned over some scenarios and it has been observed that it performs well over the testing scenarios as well. Furthermore the performance of the tracker is analyzed according to those parameters in both association and occlusion handling situations.

Moving object detection techniques can be divided into two categories based on the type of the camera which is either static or active. Methods of static cameras can detect moving objects according to the variable regions on the video frame. However, the same method is not suitable for active cameras. The task of moving object detection for active cameras generally needs more complex algorithms and unique solutions. The aim of this thesis work is real time detection and tracking of moving objects with an active camera. For this purpose, feature based algorithms are implemented due to the computational efficiency of these kinds of algorithms and SURF (Speeded Up Robust Features) is mainly used for these algorithms. An algorithm is developed in C++ environment and OpenCV library is frequently used. The developed algorithm is capable of detecting and tracking moving objects by using a PTZ (Pan-Tilt-Zoom) camera at a frame rate of approximately 5 fps and with a resolution of 640x480.

Visual target tracking problem consists of two topics: Obtaining targets from camera measurements and target tracking. Even though it has been studied for more than 30 years, there are still some problems not completely solved. Especially in the case of multiple targets, association of measurements to targets, creation of new targets and deletion of old ones are among those. What is more, it is very important to deal with the occlusion and crossing targets problems suitably. We believe that a slightly modified version of multiple hypothesis tracking can successfully deal with most of the aforementioned problems with sufficient success. Distance, track size, track color, gate size and track history are used as parameters to evaluate the hypotheses generated for measurement to track association problem whereas size and color are used as parameters for occlusion problem. The overall tracker has been fine tuned over some scenarios and it has been observed that it performs well over the testing scenarios as well. Furthermore the performance of the tracker is analyzed according to those parameters in both association and occlusion handling situations.

In this study, primary steps of a visual surveillance system are presented: moving object detection and tracking of these moving objects. Background subtraction has been performed to detect the moving objects in the video, which has been taken from a static camera. Four methods, frame differencing, running (moving) average, eigenbackground subtraction and mixture of Gaussians, have been used in the background subtraction process. After background subtraction, using some additional operations, such as morphological operations and connected component analysis, the objects to be tracked have been acquired. While tracking the moving objects, active contour models (snakes) has been used as one of the approaches. In addition to this method / Kalman tracker and mean-shift tracker are other approaches which have been utilized. A new approach has been proposed for the problem of tracking multiple targets. We have implemented this method for single and multiple camera configurations. Multiple cameras have been used to augment the measurements. Homography matrix has been calculated to find the correspondence between cameras. Then, measurements and tracks have been associated by the new tracking method.

Video surveillance has long been in use to monitor security sensitive areas such as banks, department stores, crowded public places and borders. The rise in computer speed, availability of cheap large-capacity storage devices and high speed network infrastructure enabled the way for cheaper, multi sensor video surveillance systems. In this thesis, the problem of tracking multiple targets with multiple cameras has been discussed. Cameras have been located so that they have overlapping fields of vision. A dynamic background-modeling algorithm is described for segmenting moving objects from the background, which is capable of adapting to dynamic scene changes and periodic motion, such as illumination change and swaying of trees. After segmentation of foreground scene, the objects to be tracked have been acquired by morphological operations and connected component analysis. For the purpose of tracking the moving objects, an active contour model (snakes) is one of the approaches, in addition to a Kalman tracker. As the main tracking algorithm, a rule based tracker has been developed first for a single camera, and then extended to multiple cameras. Results of used and proposed methods are given in detail.

This thesis describes the development of methods for automatically obtaining coarse gaze direction estimates for pedestrians in surveillance video. Gaze direction estimates are beneficial in the context of surveillance as an indicator of an individual's intentions and their interest in their surroundings and other people. The overall task is broken down into two problems. The first is that of tracking large numbers of pedestrians in low resolution video, which is required to identify the head regions within video frames. The second problem is to process the extracted head regions and estimate the direction in which the person is facing as a coarse estimate of their gaze direction. The first approach for head tracking combines image measurements from HOG head detections and KLT corner tracking using a Kalman filter, and can track the heads of many pedestrians simultaneously to output head regions with pixel-level accuracy. The second approach uses Markov-Chain Monte-Carlo Data Association (MCMCDA) within a temporal sliding window to provide similarly accurate head regions, but with improved speed and robustness. The improved system accurately tracks the heads of twenty pedestrians in 1920x1080 video in real-time and can track through total occlusions for short time periods. The approaches for gaze direction estimation all make use of randomised decision tree classifiers. The first develops classifiers for low resolution head images that are invariant to hair and skin colours using branch decisions based on abstract labels rather than direct image measurements. The second approach addresses higher resolution images using HOG descriptors and novel Colour Triplet Comparison (CTC) based branches. The final approach infers custom appearance models for individual scenes using weakly supervised learning over large datasets of approximately 500,000 images. A Conditional Random Field (CRF) models interactions between appearance information and walking directions to estimate gaze directions for head image sequences.

L’objectif du travail de cette thèse est la contribution au développement de nouvelles techniques dans le domaine dessystèmes de détection sans marqueur pour une utilisation dans trois domaines vitaux de la santé en utilisant des capteursinnovants et peu coûteux. Pour la réalisation de nos objectifs nous avons eu recours principalement à de l’électroniqueembarquées et du traitement du signal en utilisant le capteur Kinect. Des résultats encourageants ont été obtenus et sontprésentés tout au long de cette thèse. Dans la première partie de ce travail, nous présentons un nouveau système desurveillance visuelle sans marqueur en temps réel pour détecter et suivre les personnes âgées et surveiller leurs activitésdans leur environnement intérieur en utilisant un réseau de capteurs Kinect. Le système identifie également l’événementde chute des personnes âgées sous surveillance. Dans la deuxième partie nous utilisons également le capteur Kinectmais cette fois ci pour la détection sans marqueur des mouvements de la tête d’un patient lors d’un examen utilisant LaTomographie par Emission de Positons (CT/PET) du cerveau. Ce travail est basé sur la compensation de la dégradationde l’image TEP due aux mouvements de la tête du patient. Pour nos essais un cobaye dit « fantôme » a été réalisé,les résultats sur le fantôme sont prometteur ce qui a donné lieu à un test sur un vrai patient volontaire. Les résultatsfinaux montrent l’efficacité de ce nouveau système. La troisième partie du travail présente la mise en oeuvre d’un nouveausystème intelligent pour contrôler un fauteuil roulant électrique par des mouvements spéciaux de la tête toujours sansmarqueur. Un algorithme adapté est conçu pour détecter en continu les degrés des mouvements du visage en utilisant lecapteur Kinect. Fautes de fauteuil roulant électrique, le système a été testé sur un véhicule radio commandé. / The objective of the work of this thesis is the contribution in developing novel technical methods in the field of marker-lesssensing systems for use in three vital health areas by using new inexpensive sensors. Several scientific areas are involvedin achieving our objective such as; electronics and signal processing by using the Kinect sensor. Encouraging results wereachieved as presented throughout this thesis. In the first part of this work we present a new real-time marker-less visualsurveillance system for detecting and tracking seniors and monitoring their activities in the indoor environment by usingnetwork of Kinect sensors. The system also identifies the fall event with the elderly. In the second part, we present anew approach for a marker-less movement detection system for influential head movements in the brain Positron EmissionTomography imaging (CT/PET) by employing the Kinect sensor. This work addresses the compensation of the PET imagedegradation due to subject’s head movements. A developed particular phantom and volunteer studies were carried out.The experimental results show the effectiveness of this new system. The third part of the work presents the design andimplementation of a new smart system for controlling an electric wheelchair by special mark-less head movements. Anadaptable algorithm is designed to continuously detect the rotation degrees of the face pose using the Kinect sensor inreal-time that are interpreted as controlling signals through a hardware interface for the electric wheelchair actuators.