Anti-Terrorism has been a global issue and video surveillance has become increasingly popular in public places e.g. banks, airports, public squares, casinos, etc. However, when encountered with the crowd environment, conventional surveillance technologies will have difficulties in understanding human behaviors in crowded environment. / Firstly, I developed a learning-based algorithm for people counting task in crowded environment. The main difference between this method and traditional ones is that it adopts separated blobs as the input of the people number estimator. The blobs are selected according to their features after background estimation and calibration by tracking. After this, each selected blob in the scene is trained to predict the number of persons in the blob and the people number estimator is formed by combining trained sub-estimators according to a pre-defined rule. / In the last part, I discussed the method to analyze the crowd motion from a different angle: by video energies. I mainly use the defined energies to identify the human crowd density and human abnormal behaviors in the crowd. I define two categories of video energies based on intensity variation and motion features and adopt two surveillance methods for the two energies accordingly. Using wavelet analysis of the energy curves, I obtained a result which shows that both methods can be used to deal with crowd modeling and real-time surveillance satisfactorily. / In this thesis, I address the problem of crowd surveillance and present the methodology of how to model and monitor the crowd. The methodology is mainly based on motion features of crowd under human constrains. By utilizing this methodology, dynamic velocity field is extracted and later used for learning. Thereafter, learning technology based on appropriate features will enable the system to classify the crowd motion and behaviors. In this thesis, I tried four topics in crowd modeling and the contributions are in the following areas, namely, (1) robust people counting in crowded environment, (2) the detection and identification of abnormal behaviors in crowded environment, (3) modeling crowd behaviors via human motion constrains, and (4) modeling crowd behaviors using crowd energy. / Secondly, I introduced a human abnormal behavior identification system in the crowd based on optical flow features. Optical flow calculation is applied to obtain the velocity field of the raw images and the corresponding optical flows in the foreground are selected and processed. Then, the optical flows are encoded by support vector machine to identify the abnormal behaviors of humans in crowded environments. Experimental results show that this method can handle some places where it is very crowded while the traditional methods can not. / The work in this thesis has provided a theoretical framework for crowd modeling research and also proposed corresponding algorithms to understand crowd behaviors. Moreover, it has potential applications in areas such as security monitoring in public regions, and pedestrian fluxes control, etc. / Thirdly, I discussed how crowd modeling using human motion constrains is realized and the quantitative evaluation is given. I declare that the human motion patterns can be added to increase the accuracy and robustness of abnormal behavior identification. In more detail, I applied Bayesian rules to optimize the optical flow calculation result. I also declare that the motion pattern of crowd is similar with that of water when the environment become very crowded and corresponding rules are applied. / Ye, Weizhong. / "May 2008." / Adviser: Yangsheng Xu. / Source: Dissertation Abstracts International, Volume: 70-03, Section: A, page: 0724. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2008. / Includes bibliographical references (p. 75-85). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.

A learning-based approach to detect abnormal audio information is presented, which can be applied to audio surveillance systems that work alone or as supplements to video surveillance systems. / An automatic surveillance system is also presented that can generate a density map with multi-resolution cells and calculate the density distribution of the image by using texture analysis technique. Hosed on the estimated density distribution, the SVM method is used to solve the classification problem of detecting abnormal situations caused by changes in density distribution. / Anti-terrorism has become a global issue, and surveillance has become increasingly popular in public places such as elevators, banks, airports, and casinos. With traditional surveillance systems, human observers inspect the monitor arrays. However, with screen arrays becoming larger as the number of cameras increases, human observers may feel burdened, lose concentration, and make mistakes, which may be significant in such crucial positions as security posts. To solve this problem, I have developed an intelligent surveillance system that can understand human actions in real-time. / I have built a low-cost PC-based real-time video surveillance system that can model and analyze human real-time actions based on learning by demonstration. By teaching the system the difference between normal and abnormal human actions, the computational action models built inside the trained machines can automatically identify whether newly observed behavior requires security interference. The video surveillance system can detect the following abnormal behavior in a crowded environment using learning algorithms: (1) running people in a crowded environment; (2) falling down movements when most people are walking or standing; and (3) a person carrying an abnormally long bar in a square. Even a person running and waving a hand in a very crowded environment can be detected using an optical flow algorithm. / I have developed a real-time face detection and classification system in which the classification problem is defined as differentiating and is used to classify the front of a face as Asian or non-Asian. I combine the selected principal component analysis (PCA) and independent component analysis (ICA) features into a support vector machine (SVM) classifier to achieved a good classification rate. The system can also be used for other binary classifications of face images, such as gender and age classification without much modification. / This thesis establishes a framework for video, audio, and crowd surveillance, and successfully implements it on a mobile surveillance robot. The work is of significance in understanding human behavior and the detection of abnormal events, and has potential applications in areas such as security monitoring in household and public spaces. / To test my algorithms, the video and audio surveillance technology are implemented on a mobile platform to develop a household surveillance robot. The robot can detect a moving target and track it across a large field of vision using a pan/tilt camera platform, and can detect abnormal behavior in a cluttered environment; such as a person suddenly running or falling down on the floor. When abnormal audio information is detected, a camera on the robot is triggered to further confirm the occurrence of the abnormal event. / Wu, Xinyu. / "May 2008." / Adviser: Yangsheng Xu. / Source: Dissertation Abstracts International, Volume: 70-03, Section: B, page: 1915. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2008. / Includes bibliographical references (p. 101-109). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.

published_or_final_version / Electrical and Electronic Engineering / Master / Master of Philosophy

Visual surveillance in dynamic scenes, especially for human activities, is one of the current challenging research topics in computer vision. It is a key technology to fight against terrorism and crime to ensure public safety. The motivation of this thesis is to design an efficient human visual tracking system for video surveillance deployed in complex environments. In video surveillance, detection of moving objects is the first step to analyze the video streams. And motion segmentation is one of popular approaches to do it. In this thesis, we propose a motion segmentation method to overcome the problem of motion blurring. The task of human tracking is key to the effective use of more advanced technologies, like activity recognition and behavior understanding. However, human tracking routines often fail either due to human's arbitrary movements or occlusions by other objects. To overcome human's arbitrary movement, we propose a new Silhouette Chain Shift model for human detection and tracking. To track human under occlusions, firstly each frame is represented by a scene energy which consists of all the moving objects. Then the process of tracking is converted to a process of minimizing the proposed scene energy. Findings from the thesis contribute to improve the performance of human visual tracking system and therefore improve security in areas under surveillance. / published_or_final_version / Computer Science / Doctoral / Doctor of Philosophy

Image sequences with humans and human activities are everywhere. With the amount of produced and distributed data increasing at an unprecedented rate, there has been a lot of interest in building systems that can understand and interpret the visual data, and in particular detect and recognise human actions. Dictionary based approaches learn a dictionary from descriptors extracted from the videos in the first stage and a classifier or a detector in the second stage. The major drawback of such an approach is that the dictionary is learned in an unsupervised manner without considering the task (classification or detection) that follows it. In this work we develop task dependent(supervised) dictionaries for action recognition and localization, i.e., dictionaries that are best suited for the subsequent task. In the first part of the work, we propose a supervised max-margin framework for linear and non-linear Non-Negative Matrix Factorization (NMF). To achieve this, we impose max-margin constraints within the formulation of NMF and simultaneously solve for the classifier and the dictionary. The dictionary (basis matrix) thus obtained maximizes the margin of the classifier in the low dimensional space (in the linear case) or in the high dimensional feature space (in the non-linear case). In the second part the work, we develop methodologies for action localization. We first propose a dictionary weighting approach where we learn local and global weights for the dictionary by considering the localization information of the training sequences. We next extend this approach to learn a task-dependent dictionary for action localization that incorporates the localization information of the training sequences into dictionary learning. The results on publicly available datasets show that the performance of the system is improved by using the supervised information while learning dictionary.

The ability to analyse and predict the movement of people in crowded scenarios can be of fundamental importance for tracking across multiple cameras and interaction localisation. In this thesis, we propose a person re-identification method that takes into account the spatial location of cameras using a plan of the locale and the potential paths people can follow in the unobserved areas. These potential paths are generated using two models. In the first, people’s trajectories are constrained to pass through a set of areas of interest (landmarks) in the site. In the second we integrate a goal-driven approach to the Social Force Model (SFM), initially introduced for crowd simulation. SFM models the desire of people to reach specific interest points (goals) in a site, such as exits, shops, seats and meeting points while avoiding walls and barriers. Trajectory propagation creates the possible re-identification candidates, on which association of people across cameras is performed using spatial location of the candidates and appearance features extracted around a person’s head. We validate the proposed method in a challenging scenario from London Gatwick airport and compare it to state-of-the-art person re-identification methods. Moreover, we perform detection and tracking of interacting people in a framework based on SFM that analyses people’s trajectories. The method embeds plausible human behaviours to predict interactions in a crowd by iteratively minimising the error between predictions and measurements. We model people approaching a group and restrict the group formation based on the relative velocity of candidate group members. The detected groups are then tracked by linking their centres of interaction over time using a buffered graph-based tracker. We show how the proposed framework outperforms existing group localisation techniques on three publicly available datasets.

Chan, Fai. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2008. / Includes bibliographical references (leaves 104-108). / Abstracts in English and Chinese. / Chapter 1. --- Introduction --- p.14 / Chapter 1.1. --- Background --- p.14 / Chapter 1.1.1. --- Introduction to Intelligent Surveillance System (ISS) --- p.14 / Chapter 1.1.2. --- Typical architecture of Surveillance System --- p.17 / Chapter 1.1.3. --- Single-camera vs Multi-camera Surveillance System --- p.17 / Chapter 1.1.4. --- Intelligent Surveillance System with Face Recognition (ISFAR) --- p.20 / Chapter 1.1.5. --- Minimal requirements for automatic Face Recognition --- p.21 / Chapter 1.2. --- Motivation --- p.22 / Chapter 1.3. --- Major Contributions --- p.26 / Chapter 1.3.1. --- A unified design framework for IS FAR --- p.26 / Chapter 1.3.2. --- Prototyping of IS FAR (ISFARO) --- p.29 / Chapter 1.3.3. --- Evaluation of ISFARO --- p.29 / Chapter 1.4. --- Thesis Organization --- p.30 / Chapter 2. --- Related Works --- p.31 / Chapter 2.1. --- Distant Human Identification (DHID) --- p.31 / Chapter 2.2. --- Distant Targets Identification System --- p.33 / Chapter 2.3. --- Virtual Vision System with Camera Scheduling --- p.35 / Chapter 3. --- A unified design framework for IS FAR --- p.37 / Chapter 3.1. --- Camera system modeling --- p.40 / Chapter 3.1.1. --- Stereo Triangulation (Human face location estimation) --- p.40 / Chapter 3.1.2. --- Camera system calibration --- p.42 / Chapter 3.2. --- Human face detection --- p.44 / Chapter 3.3. --- Human face tracking --- p.46 / Chapter 3.4. --- Human face correspondence --- p.50 / Chapter 3.4.1. --- Information consistency in stereo triangulation --- p.51 / Chapter 3.4.2. --- Proposed object correspondent algorithm --- p.52 / Chapter 3.5. --- Human face location and velocity estimation --- p.57 / Chapter 3.6. --- Human-Camera Synchronization --- p.58 / Chapter 3.6.1. --- Controlling a PTZ Camera for capturing human facial images --- p.60 / Chapter 3.6.2. --- Mathematical Formulation of the Human Face Capturing Problem --- p.61 / Chapter 4. --- Prototyping of lSFAR (ISFARO) --- p.64 / Chapter 4.1. --- Experiment Setup --- p.64 / Chapter 4.2. --- Speed of the PTZ camera 一 AXIS 213 PTZ --- p.67 / Chapter 4.3. --- Performance of human face detection and tracking --- p.68 / Chapter 4.4. --- Performance of human face correspondence --- p.72 / Chapter 4.5. --- Performance of human face location estimation --- p.74 / Chapter 4.6. --- Stability test of the Human-Camera Synchronization model --- p.75 / Chapter 4.7. --- Performance of ISFARO in capturing human facial images --- p.76 / Chapter 4.8. --- System Profiling of ISFARO --- p.79 / Chapter 4.9. --- Summary --- p.79 / Chapter 5. --- Improvements to ISFARO --- p.80 / Chapter 5.1. --- System Dynamics oflSFAR --- p.80 / Chapter 5.2. --- Proposed improvements to ISFARO --- p.82 / Chapter 5.2.1. --- Semi-automatic camera system calibration --- p.82 / Chapter 5.2.2. --- Velocity estimation using Kalman filter --- p.83 / Chapter 5.2.3. --- Reduction in PTZ camera delay --- p.87 / Chapter 5.2.4. --- Compensation of image blurriness due to motion from human --- p.89 / Chapter 5.3. --- Experiment Setup --- p.91 / Chapter 5.4. --- Performance of human face location estimation --- p.91 / Chapter 5.5. --- Speed of the PTZ Camera - SONY SNC RX-570 --- p.93 / Chapter 5.6. --- Performance of human face velocity estimation --- p.95 / Chapter 5.7. --- Performance of improved ISFARO in capturing human facial images --- p.99 / Chapter 6. --- Conclusions --- p.101 / Chapter 7. --- Bibliography --- p.104

Video surveillance systems are playing an increasing role in preventing and investigating crime, protecting public safety, and safeguarding national security. In a typical surveillance installation, a human operator has to constantly monitor a large array of video feeds for suspicious behaviour. As the number of cameras increases, information overload makes manual surveillance increasingly difficult, adding to other confounding factors like human fatigue and boredom. The objective of an intelligent vision-based surveillance system is to automate the monitoring and event detection components of surveillance, alerting the operator only when unusual behaviour or other events of interest are detected. While most traditional methods for trajectory-based unusual behaviour detection rely on low-level trajectory features, this thesis improves a recently introduced approach that makes use of higher-level features of intentionality. Individuals in a scene are modelled as intentional agents instead of simply objects. Unusual behaviour detection then becomes a task of determining whether an agent's trajectory is explicable in terms of learned spatial goals. The proposed method extends the original goal-based approach in three ways: first, the spatial scene structure is learned in a training phase; second, a region transition model is learned to describe normal movement patterns between spatial regions; and third, classification of trajectories in progress is performed in a probabilistic framework using particle filtering. Experimental validation on three published third-party datasets demonstrates the validity of the proposed approach.

This thesis proposes a merging mechanism for multiple video surveillances in TCP networks. Merging video streams not only can benefit network administration but also reduce the waste of bandwidth. In this thesis, we design a Video-Merging Gateway (VMG) between cameras and control center to merge two video streams transmitted from cameras and received by control center. In the merging mechanism, we develop two modes: Interleave and Overlay. Interleave mode includes two operation types: Single Frame and Proportional. The former merges video streams by interleaving frames one by one from two cameras, and the latter merges video streams according to an FPS (frame per second) ratio between two cameras. Overlay mode vertically displays two video streams in separate frames on the web browser. We implement VMG on a Linux platform. In the interleave mode, we recalculate both the sequence number and the Ack number of a video packet, and create Ack packet for dropped frames while merging two TCP video streams. In the overlay mode, we modify the decoding messages in the frames and separate data between two video streams to avoid decoding errors. Finally, we analyze the complexity of merging algorithms. By carefully determining the timing for responding the created Ack based on Retransmission Time Out (RTO), packet retransmition can be avoided. In addition, we found out that the number of instructions to execute the algorithm is increased by multiple integers along with the picture sizes under interleave mode. As for overlay mode, the number of instructions is increased linearly along with the payload length and the total amount of data and Ack packets.

Video surveillance systems are playing an increasing role in preventing and investigating crime, protecting public safety, and safeguarding national security. In a typical surveillance installation, a human operator has to constantly monitor a large array of video feeds for suspicious behaviour. As the number of cameras increases, information overload makes manual surveillance increasingly difficult, adding to other confounding factors like human fatigue and boredom. The objective of an intelligent vision-based surveillance system is to automate the monitoring and event detection components of surveillance, alerting the operator only when unusual behaviour or other events of interest are detected. While most traditional methods for trajectory-based unusual behaviour detection rely on low-level trajectory features, this thesis improves a recently introduced approach that makes use of higher-level features of intentionality. Individuals in a scene are modelled as intentional agents instead of simply objects. Unusual behaviour detection then becomes a task of determining whether an agent's trajectory is explicable in terms of learned spatial goals. The proposed method extends the original goal-based approach in three ways: first, the spatial scene structure is learned in a training phase; second, a region transition model is learned to describe normal movement patterns between spatial regions; and third, classification of trajectories in progress is performed in a probabilistic framework using particle filtering. Experimental validation on three published third-party datasets demonstrates the validity of the proposed approach.