The purpose of this dissertation is to present our agent-based human tracking framework, and to evaluate the results of our work in light of the previous research in the same field. Our agent-based approach departs from a process-centric model where the agents are bound to specific processes, and introduces a novel model by which agents are bound to the objects or sub-objects being recognized or tracked. The hierarchical agent-based model allows the system to handle a variety of cases, such as single people or multiple people in front of single or stereo cameras. We employ the job-market model for agents' communication. In this dissertation, we will present several experiments in detail, which demonstrate the effectiveness of the agent-based tracking system. Per our research, the agents are designed to be autonomous, self-aware entities that are capable of communicating with other agents to perform tracking within agent coalitions. Each agent with high-level abstracted knowledge seeks evidence for its existence from the low-level features (e.g. motion vector fields, color blobs) and its peers (other agents representing body-parts with which it is compatible). The power of the agent-based approach is its flexibility by which the domain information may be encoded within each agent to produce an overall tracking solution. / Ph. D.

Conventional surveillance systems present video to a user from more than one camera on a single display. Such a display allows the user to observe different part of the scene, or to observe the same part of the scene from different viewpoints. Each video is usually labeled by a fixed textual annotation displayed under the video segment to identify the image. With the growing number of surveillance cameras set up and the expanse of surveillance area, the conventional split-screen display approach cannot provide intuitive correspondence between the images acquired and the areas under surveillance. Such a system has a number of inherent flaws¡GLower relativity of split videos¡BThe difficulty of tracking new activities¡BLow resolution of surveillance videos¡BThe difficulty of total surveillance¡FIn order to improve the above defects, the ¡§Immersive Surveillance for Total Situational Awareness¡¨ use computer graphic technique to construct 3D model of buildings on the 2D satellite-images, the users can construct the floor platform by defining the information of each floor or building and the position of each camera. This information is combined to construct 3D surveillance scene, and the images acquired by surveillance cameras are pasted into the constructed 3D model to provide intuitively visual presentation. The users could also walk through the scene by a fixed-frequency , self-defined business model to perform a virtual surveillance. Multi-camera Human Tracking on Realtime 3D Immersive Surveillance System based on the ¡§Immersive Surveillance for Total Situational Awareness,¡¨ 1. Salient object detection¡GThe System converts videos to corresponding image sequences and analyze the videos provided by each camera. In order to filter out the foreground pixels, the background model of each image is calculated by pixel-stability-based background update algorithm. 2. Nighttime image fusion¡GUse the fuzzy enhancement method to enhance the dark area in nighttime image, and also maintain the saturation information. Then apply the Salient object detection Algorithm to extract salient objects of the dark area. The system divides fusion results into 3 parts: wall, ceiling, and floor, then pastes them as materials into corresponding parts of 3D scene. 3. Multi-camera human tracking¡GApply connected component labeling to filter out small area and save each block¡¦s infomation. Use RGB-weight percentage information in each block and 5-state status (Enter¡BLeave¡BMatch¡BOcclusion¡BFraction) to draw out the trajectory of each person in every camera¡¦s field of view on the 3D surveillance scene. Finally, fuse every camera together to complete the multi-camera realtime people tracking. Above all, we can track every human in our 3D immersive surveillance system without watching out each of thousand of camera views.

Human activity regions (HARs) are human-centric semantic partitions where observing and/or interacting with humans is likely in indoor environments. HARs are useful for achieving successful human-robot interaction, such as in safe navigation around a building or to know where to be able to assist humans in their activities. In this thesis, a system is designed for generating HARs automatically based on data recorded by robots. This approach to generating HARs is to cluster the areas that are commonly associated with frequent human presence. In order to detect human positions, we employ state-of-the-art perception techniques. The environment that the robot patrols is assumed to be an indoor environment such as an office. We show how we can generate HARs in correct regions by clustering human position data. The experimental evaluations show that we can do so in different indoor environments, with data acquired from different sensors and that the system can handle noise. / Mänskliga aktivitetsregioner, HARs (Human Activity Regions) är människocentreraderegioner som ger en semantisk partitionering av inomhusmiljöer. HARs är användbara för att uppnå väl fungerande människarobot- interaktioner. I denna avhandling utformas ett system för att generera HARs automatiskt baserat på data från robotar. Detta görs genom att klustra observationer av människor för att på så vis få fram de områden som är associerade med frekvent mänsklig närvaro. Experiment visar att systemet kan hantera data som registrerats av olika sensorer i olika inomhusmiljöer och att det är robust. Framförallt genererar systemet en pålitlig partitionering av miljön.

Analyzing human motion is important in a number of ways. An athlete constantly needs to evaluate minute details about his or her motion pattern. In physical rehabilitation, the doctor needs to evaluate how well a patient is rehabilitating from injuries. Some systems are being developed in order to identify people only based on their gait. Automatic interpretation of sign language is another area that has received much attention. While all these applications can be considered useful in some sense, the analysis of human motion can also be used for pure entertainment. For example, by filming a sport activity from one view, it is possible to create a 3D reconstruction of this motion, that can be rendered from a view where no camera was originally placed. Such a reconstruction system can be enjoyable for the TV audience. It can also be useful for the computer-game industry. This thesis presents ideas and new methods on how such reconstructions can be obtained. One of the main purposes of this thesis is to identify a number of qualitative constraints that strongly characterizes a certain class of motion. These qualitative constraints provide enough information about the class so that every motion satisfying the constraints will "look nice" and appear, according to a human observer, to belong to the class. Further, the constraints must not be too restrictive; a large variation within the class is necessary. It is shown how such qualitative constraints can be learned automatically from a small set of examples. Another topic that will be addressed concerns analysis of motion in terms of quality assessment as well as classification. It is shown that in many cases, 2D projections of a motion carries almost as much information about the motion as the original 3D representation. It is also shown that single-view reconstruction of 2D data for the purpose of analysis is generally not useful. Using these facts, a prototype of a "virtual coach" that is able to track and analyze image data of human action is developed. Potentials and limitations of such a system are discussed in the the thesis. / QC 20100601

No description available.

With the recent developments in Computer Vision and Pattern Recognition, surveillance applications are equipped with the capabilities of event/activity understanding and interpretation which usually require recognizing humans in real world scenes. Real world scenes such as airports, streets and train stations are complex because they involve many people, complicated occlusions and cluttered backgrounds. Although complex real world scenes exist, human detectors have the capability to locate pedestrians accurately even in complex scenes and visual trackers have the capability to track targets in cluttered environments. The integration of visual object detection and tracking, which are the fundamental features of available surveillance applications, is one of the solutions for multi-human tracking problem in crowded scenes which is studied in this thesis. In this thesis, human body part detectors, which are capable of detecting human heads and human upper body parts, are trained with Support Vector Machines (SVM) by using Histogram of Oriented Gradients (HOG), which is one of the state-of-the-art descriptor for human detection. The training process is elaborated by investigating the effects of the parameters of the HOG descriptor. The human heads and upper body parts are searched in the region of interests (ROI) computed by detecting motion. In addition, these human body part detectors are integrated with a multi-human tracker which solves the data association problem with the Multi Scan Markov Chain Monte Carlo Data Association (MCMCDA) algorithm. Associated measurements of human upper body part locations are used for state correction for each track. State estimation is done through Kalman Filter. The performance of detectors are evaluated using MIT Pedestrian dataset and INRIA Human dataset.

This thesis presents a multi-feature histogram approach to track a person in thermal vision. Illumination variation is a primary constraint in the performance of object tracking in visible spectrum. Thermal infrared (IR) sensor, which measures the heat energy emitted from an object, is less sensitive to illumination variations. Therefore, thermal vision has immense advantage in object tracking in varying illumination conditions. Kernel based approaches such as mean shift tracking algorithm which uses a single feature histogram for object representation, has gained popularity in the field of computer vision due its efficiency and robustness to track non-rigid object in significant complex background. However, due to low resolution of IR images the gray level intensity information is not sufficient enough to give a strong cue for object representation using histogram. Multi-feature histogram, which is the combination of the gray level intensity information and edge information, generates an object representation which is more robust in thermal vision. The objective of this research is to develop a robust human tracking system which can autonomously detect, identify and track a person in a complex thermal IR scene. In this thesis the tracking procedure has been adapted from the well-known and efficient mean shift tracking algorithm and has been modified to enable fusion of multiple features to increase the robustness of the tracking procedure in thermal vision. In order to identify the object of interest before tracking, rapid human detection in thermal IR scene is achieved using Adaboost classification algorithm. Furthermore, a computationally efficient body pose recognition method is developed which uses Hu-invariant moments for matching object shapes. An experimental setup consisting of a Forward Looking Infrared (FLIR) camera, mounted on a Pioneer P3-DX mobile robot platform was used to test the proposed human tracking system in both indoor and uncontrolled outdoor environments. The performance evaluation of the proposed tracking system on the OTCBVS benchmark dataset shows improvement in tracking performance in comparison to the traditional mean-shift tracking algorithm. Moreover, experimental results in different indoor and outdoor tracking scenarios involving different appearances of people show tracking is robust under cluttered background, varying illumination and partial occlusion of target object. / Electrical and Computer Engineering

This thesis presents a framework for a mobile robot to escort a human to their destination successfully and efficiently. The proposed technique uses accurate path prediction incorporating human intention to locate the robot in front of the human while walking. Human intention is inferred by the head pose, an effective past-proven implicit indicator of intention, and fused with conventional physics-based motion prediction. The human trajectory is estimated and predicted using a particle filter because of the human's nonlinear and non-Gaussian behavior, and the robot control action is determined from the predicted human pose allowing for anticipative autonomous escorting. Experimental analysis shows that the incorporation of the proposed human intention model reduces human position prediction error by approximately 35% when turning. Furthermore, experimental validation with an omnidirectional mobile robotic platform shows escorting up to 50% more accurate compared to the conventional techniques, while achieving 97% success rate. / Master of Science / This thesis presents a method for a mobile robot to escort a human to their destination successfully and efficiently. The proposed technique uses human intention to predict the walk path allowing the robot to be in front of the human while walking. Human intention is inferred by the head direction, an effective past-proven indicator of intention, and is combined with conventional motion prediction. The robot motion is then determined from the predicted human position allowing for anticipative autonomous escorting. Experimental analysis shows that the incorporation of the proposed human intention reduces human position prediction error by approximately 35% when turning. Furthermore, experimental validation with an mobile robotic platform shows escorting up to 50% more accurate compared to the conventional techniques, while achieving 97% success rate. The unique escorting interaction method proposed has applications such as touch-less shopping cart robots, exercise companions, collaborative rescue robots, and sanitary transportation for hospitals.

Η παρούσα διπλωματική εργασία αφορά την ανίχνευση και παρακολούθηση ανθρώπινων μορφών σε ακολουθίες βίντεο με μεθόδους μηχανικής όρασης. Οι ακολουθίες αυτές θεωρούμε πως έχουν ληφθεί από στατική κάμερα σε εσωτερικό ή εξωτερικό χώρο. Πιο συγκεκριμένα, το εν λόγω πρόβλημα υποδιαιρείται σε τρία κυρίως μέρη τα οποία μελετώνται, αναλύονται και υλοποιούνται σε ξεχωριστά κεφάλαια. Ξεκινάμε με το κομμάτι κατάτμησης κίνησης, συνεχίζουμε με την ταξινόμηση αντικειμένων ώστε να αναγνωριστούν οι άνθρωποι ανάμεσα στις κινούμενες οντότητες και τελειώνουμε με την παρακολούθηση των ανθρώπινων σιλουετών για καταγραφή της πορείας τους όση ώρα βρίσκονται στο πλάνο. Οι αλγόριθμοι που αναπτύχθηκαν λειτούργησαν ικανοποιητικά κάτω από διάφορες συνθήκες και τα αποτελέσματά τους μπορούν να περάσουν ως είσοδοι σε μια πληθώρα εφαρμογών υψηλότερου επιπέδου με σκοπό την αναγνώριση ανθρώπινης δραστηριότητας και την κατανόηση συμπεριφοράς. / The purpose of this thesis is to deal with the problem of human tracking in video sequences. We have divided the problem in three parts: motion segmentation, human tracking and object classification. Finally we have dedicate a whole chapter to optical flow techniques and the relevant methods that can be employed to solve the same problem.

No description available.