Anti-Terrorism has been a global issue and video surveillance has become increasingly popular in public places e.g. banks, airports, public squares, casinos, etc. However, when encountered with the crowd environment, conventional surveillance technologies will have difficulties in understanding human behaviors in crowded environment. / Firstly, I developed a learning-based algorithm for people counting task in crowded environment. The main difference between this method and traditional ones is that it adopts separated blobs as the input of the people number estimator. The blobs are selected according to their features after background estimation and calibration by tracking. After this, each selected blob in the scene is trained to predict the number of persons in the blob and the people number estimator is formed by combining trained sub-estimators according to a pre-defined rule. / In the last part, I discussed the method to analyze the crowd motion from a different angle: by video energies. I mainly use the defined energies to identify the human crowd density and human abnormal behaviors in the crowd. I define two categories of video energies based on intensity variation and motion features and adopt two surveillance methods for the two energies accordingly. Using wavelet analysis of the energy curves, I obtained a result which shows that both methods can be used to deal with crowd modeling and real-time surveillance satisfactorily. / In this thesis, I address the problem of crowd surveillance and present the methodology of how to model and monitor the crowd. The methodology is mainly based on motion features of crowd under human constrains. By utilizing this methodology, dynamic velocity field is extracted and later used for learning. Thereafter, learning technology based on appropriate features will enable the system to classify the crowd motion and behaviors. In this thesis, I tried four topics in crowd modeling and the contributions are in the following areas, namely, (1) robust people counting in crowded environment, (2) the detection and identification of abnormal behaviors in crowded environment, (3) modeling crowd behaviors via human motion constrains, and (4) modeling crowd behaviors using crowd energy. / Secondly, I introduced a human abnormal behavior identification system in the crowd based on optical flow features. Optical flow calculation is applied to obtain the velocity field of the raw images and the corresponding optical flows in the foreground are selected and processed. Then, the optical flows are encoded by support vector machine to identify the abnormal behaviors of humans in crowded environments. Experimental results show that this method can handle some places where it is very crowded while the traditional methods can not. / The work in this thesis has provided a theoretical framework for crowd modeling research and also proposed corresponding algorithms to understand crowd behaviors. Moreover, it has potential applications in areas such as security monitoring in public regions, and pedestrian fluxes control, etc. / Thirdly, I discussed how crowd modeling using human motion constrains is realized and the quantitative evaluation is given. I declare that the human motion patterns can be added to increase the accuracy and robustness of abnormal behavior identification. In more detail, I applied Bayesian rules to optimize the optical flow calculation result. I also declare that the motion pattern of crowd is similar with that of water when the environment become very crowded and corresponding rules are applied. / Ye, Weizhong. / "May 2008." / Adviser: Yangsheng Xu. / Source: Dissertation Abstracts International, Volume: 70-03, Section: A, page: 0724. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2008. / Includes bibliographical references (p. 75-85). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.

A learning-based approach to detect abnormal audio information is presented, which can be applied to audio surveillance systems that work alone or as supplements to video surveillance systems. / An automatic surveillance system is also presented that can generate a density map with multi-resolution cells and calculate the density distribution of the image by using texture analysis technique. Hosed on the estimated density distribution, the SVM method is used to solve the classification problem of detecting abnormal situations caused by changes in density distribution. / Anti-terrorism has become a global issue, and surveillance has become increasingly popular in public places such as elevators, banks, airports, and casinos. With traditional surveillance systems, human observers inspect the monitor arrays. However, with screen arrays becoming larger as the number of cameras increases, human observers may feel burdened, lose concentration, and make mistakes, which may be significant in such crucial positions as security posts. To solve this problem, I have developed an intelligent surveillance system that can understand human actions in real-time. / I have built a low-cost PC-based real-time video surveillance system that can model and analyze human real-time actions based on learning by demonstration. By teaching the system the difference between normal and abnormal human actions, the computational action models built inside the trained machines can automatically identify whether newly observed behavior requires security interference. The video surveillance system can detect the following abnormal behavior in a crowded environment using learning algorithms: (1) running people in a crowded environment; (2) falling down movements when most people are walking or standing; and (3) a person carrying an abnormally long bar in a square. Even a person running and waving a hand in a very crowded environment can be detected using an optical flow algorithm. / I have developed a real-time face detection and classification system in which the classification problem is defined as differentiating and is used to classify the front of a face as Asian or non-Asian. I combine the selected principal component analysis (PCA) and independent component analysis (ICA) features into a support vector machine (SVM) classifier to achieved a good classification rate. The system can also be used for other binary classifications of face images, such as gender and age classification without much modification. / This thesis establishes a framework for video, audio, and crowd surveillance, and successfully implements it on a mobile surveillance robot. The work is of significance in understanding human behavior and the detection of abnormal events, and has potential applications in areas such as security monitoring in household and public spaces. / To test my algorithms, the video and audio surveillance technology are implemented on a mobile platform to develop a household surveillance robot. The robot can detect a moving target and track it across a large field of vision using a pan/tilt camera platform, and can detect abnormal behavior in a cluttered environment; such as a person suddenly running or falling down on the floor. When abnormal audio information is detected, a camera on the robot is triggered to further confirm the occurrence of the abnormal event. / Wu, Xinyu. / "May 2008." / Adviser: Yangsheng Xu. / Source: Dissertation Abstracts International, Volume: 70-03, Section: B, page: 1915. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2008. / Includes bibliographical references (p. 101-109). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts in English and Chinese. / School code: 1307.

Image sequences with humans and human activities are everywhere. With the amount of produced and distributed data increasing at an unprecedented rate, there has been a lot of interest in building systems that can understand and interpret the visual data, and in particular detect and recognise human actions. Dictionary based approaches learn a dictionary from descriptors extracted from the videos in the first stage and a classifier or a detector in the second stage. The major drawback of such an approach is that the dictionary is learned in an unsupervised manner without considering the task (classification or detection) that follows it. In this work we develop task dependent(supervised) dictionaries for action recognition and localization, i.e., dictionaries that are best suited for the subsequent task. In the first part of the work, we propose a supervised max-margin framework for linear and non-linear Non-Negative Matrix Factorization (NMF). To achieve this, we impose max-margin constraints within the formulation of NMF and simultaneously solve for the classifier and the dictionary. The dictionary (basis matrix) thus obtained maximizes the margin of the classifier in the low dimensional space (in the linear case) or in the high dimensional feature space (in the non-linear case). In the second part the work, we develop methodologies for action localization. We first propose a dictionary weighting approach where we learn local and global weights for the dictionary by considering the localization information of the training sequences. We next extend this approach to learn a task-dependent dictionary for action localization that incorporates the localization information of the training sequences into dictionary learning. The results on publicly available datasets show that the performance of the system is improved by using the supervised information while learning dictionary.

The ability to analyse and predict the movement of people in crowded scenarios can be of fundamental importance for tracking across multiple cameras and interaction localisation. In this thesis, we propose a person re-identification method that takes into account the spatial location of cameras using a plan of the locale and the potential paths people can follow in the unobserved areas. These potential paths are generated using two models. In the first, people’s trajectories are constrained to pass through a set of areas of interest (landmarks) in the site. In the second we integrate a goal-driven approach to the Social Force Model (SFM), initially introduced for crowd simulation. SFM models the desire of people to reach specific interest points (goals) in a site, such as exits, shops, seats and meeting points while avoiding walls and barriers. Trajectory propagation creates the possible re-identification candidates, on which association of people across cameras is performed using spatial location of the candidates and appearance features extracted around a person’s head. We validate the proposed method in a challenging scenario from London Gatwick airport and compare it to state-of-the-art person re-identification methods. Moreover, we perform detection and tracking of interacting people in a framework based on SFM that analyses people’s trajectories. The method embeds plausible human behaviours to predict interactions in a crowd by iteratively minimising the error between predictions and measurements. We model people approaching a group and restrict the group formation based on the relative velocity of candidate group members. The detected groups are then tracked by linking their centres of interaction over time using a buffered graph-based tracker. We show how the proposed framework outperforms existing group localisation techniques on three publicly available datasets.

This thesis proposes a merging mechanism for multiple video surveillances in TCP networks. Merging video streams not only can benefit network administration but also reduce the waste of bandwidth. In this thesis, we design a Video-Merging Gateway (VMG) between cameras and control center to merge two video streams transmitted from cameras and received by control center. In the merging mechanism, we develop two modes: Interleave and Overlay. Interleave mode includes two operation types: Single Frame and Proportional. The former merges video streams by interleaving frames one by one from two cameras, and the latter merges video streams according to an FPS (frame per second) ratio between two cameras. Overlay mode vertically displays two video streams in separate frames on the web browser. We implement VMG on a Linux platform. In the interleave mode, we recalculate both the sequence number and the Ack number of a video packet, and create Ack packet for dropped frames while merging two TCP video streams. In the overlay mode, we modify the decoding messages in the frames and separate data between two video streams to avoid decoding errors. Finally, we analyze the complexity of merging algorithms. By carefully determining the timing for responding the created Ack based on Retransmission Time Out (RTO), packet retransmition can be avoided. In addition, we found out that the number of instructions to execute the algorithm is increased by multiple integers along with the picture sizes under interleave mode. As for overlay mode, the number of instructions is increased linearly along with the payload length and the total amount of data and Ack packets.

The thesis represents the research in the domain of modern video tracking systems and presents the details of the implementation of such a system. Video surveillance is a high point of interest and it relies on robust systems that interconnect several critical modules: data acquisition, data processing, background modeling, foreground detection and multiple object tracking. The present work analyzes different state of the art methods that are suitable for each module. The emphasis of the thesis is on the background subtraction stage, as the final accuracy and performance of the person tracking dramatically dependent on it. The experimental results show the performance of four different foreground detection algorithms, including two variations of self-organizing feature maps for background modeling, a machine learning technique. The undertaken work provides a comprehensive view of the actual state of the research in the foreground detection field and multiple object tracking and offers solution for common problems that occur when tracking in complex scenes. The chosen data set for experiments covers extremely different and complex scenes (outdoor environments) that allow a detailed study of the appropriate approaches and emphasize the weaknesses and strengths of each algorithm. The proposed system handles problems like: dynamic backgrounds, illumination changes, camouflage, cast shadows, frequent occlusions and crowded scenes. The tracking obtains a maximum Multiple Object Tracking Accuracy of 92,5% for the standard video sequence MWT and a minimum of 32,3% for an extremely difficult sequence that challenges every method.

Bürger weisen eine positive Einstellung gegenüber Videoüberwachung auf, obwohl sie sehr wenig über Videoüberwachung nachdenken, wenig über die Zahl und Verteilung der Videokameras in ihrer Stadt wissen, Videoüberwachung nicht mit ihren Bürgerrechten in Beziehung bringen sowie dem Staat „blind\\\\\\\\\\\\\\\" vertrauen. Klocke resümiert: Das Unwissen über die Kamerawirklichkeit ist als ein Anzeichen für bürgerrechtliche Unmotiviertheit und mangelnde Freiheitssensibilität anzusehen. Daraus ergibt sich die Forschungsfrage dieses Aufsatzes, welche darauf abzielt nicht die Einstellung zur Videoüberwachung, sondern die (geringe) Aufmerksamkeit gegenüber Videoüberwachung zu erklären: Warum sind Menschen gleichgültig gegenüber Videoüberwachung, obwohl dadurch eines ihrer Grundrechte beeinträchtigt wird?

If “a picture is worth a thousand words,” then a video may be worth a thousand pictures. Videos have been increasingly used in multiple applications, including surveillance, teleconferencing, learning and experience sharing. Since a video captures a scene from a particular viewpoint, it can often be understood better if presented within a larger spatial context. We call such interactive visualizations that combine videos with their spatial context "Contextualized Videos". Over recent years, multiple innovative Contextualized Video interfaces have been proposed to taking advantage of the latest computer graphics and video processing technologies. These interfaces opened a huge design space with numerous design possibilities, each with its own benefits and limitations. To avoid piecemeal understanding of the design space, this dissertation systematically designs and evaluates Contextualized Video interfaces based on a taxonomy of tasks that can potentially benefit from Contextualized Videos. This dissertation first formalizes a design space. New designs are created incrementally along the four major dimensions of the design space. These designs are then empirically compared through a series of controlled experiments using multiple tasks. The tasks are carefully selected from a task taxonomy, which helps to avoid piecemeal understanding of the effect of the designs. Our design practices and empirical evaluations result in a set of design guidelines on how to choose proper designs according to the characteristics of the tasks and the users. Finally, we demonstrate how to apply the design guidelines to prototype a complex interface for a specific video surveillance application. / Ph. D.

Reconstructing the 3D geometry of scenes based on monocular image sequences is a long-standing problem in computer vision. Structure from motion (SfM) aims at a data-driven approach without requiring a priori models of the scene. When the scene is rigid, SfM is a well understood problem with solutions widely used in industry. However, if the scene is non-rigid, monocular reconstruction without additional information is an ill-posed problem and no satisfactory solution has yet been found. Current non-rigid SfM (NRSfM) methods typically aim at modelling deformable motion globally. Additionally, most of these methods focus on cases where deformable motion is seen as small variations from a mean shape. In turn, these methods fail at reconstructing highly deformable objects such as a flag waving in the wind. Additionally, reconstructions typically consist of low detail, sparse point-cloud representation of objects. In this thesis we aim at reconstructing highly deformable surfaces by modelling them locally. In line with a recent trend in NRSfM, we propose a piecewise approach which reconstructs local overlapping regions independently. These reconstructions are merged into a global object by imposing 3D consistency of the overlapping regions. We propose our own local model – the Quadratic Deformation model – and show how patch division and reconstruction can be formulated in a principled approach by alternating at minimizing a single geometric cost – the image re-projection error of the reconstruction. Moreover, we extend our approach to dense NRSfM, where reconstructions are preformed at the pixel level, improving the detail of state of the art reconstructions. Finally we show how our principled approach can be used to perform simultaneous segmentation and reconstruction of articulated motion, recovering meaningful segments which provide a coarse 3D skeleton of the object.

行人再識別的主要任務是匹配不交叉的監控攝像頭中觀測到的行人。隨著監控攝像頭的普遍，這是一個非常重要的任務。並且，它是其他很多任務的重要子任務，例如跨攝像頭的跟蹤。行人再識別的難度存在於不同攝像頭中觀測到的同一個人會有很大的變化。這些變化來自於觀察角度的不同，光照的不同，和行人姿態的變化等等。在本文中，我們希望從如下的方面來重新思考並解決這個問題。 / 首先，我們發現當待匹配集合增大的時候，匹配的難度大幅度增加。在實際應用中，我們可以通過時間上的推演來減少待匹配集合的大小，簡化行人再識別這個問題。現有通過機器學習的方法來解決這個問題的算法基本會假設一個全局固定的度量。我們的方法來自提出於對於不同的待匹配集合應該有不同的度量的新觀點。因此，我們把這個問題重新定義在一個遷移學習的框架下。給定一個較大的訓練集合，我們通過訓練集合的樣本與當前的查詢集合和待匹配集合的相似程度，重新對訓練集合進行加權。這樣，我們提出一個加權的最大化邊界的度量學習方法，而這個度量較全訓練集共享的整體度量更加的具體。 / 我們進一步發現，在兩個不同的鏡頭中，物體形態的變換很難通過一個單一模型來進行描述。為了解決這一個問題，我們提出一個混合專家模型，要將圖片的空間進行進一步細化。我們的算法將剖分圖形空間和在每個細分後的空間中學習一個跨鏡頭的變換來將特征進行對齊。測試時，新樣本會與現有的“專家“模型進行匹配，選擇合適的變換。 我們通過一個稀疏正則項和最小信息損失正則項來進行約束。 / 在對上面各種方法的分析中，我們發現提取特征和訓練模型總是分開進行。一個更好的方法是將模型的訓練和特征提取同時進行。為此，我們希望能夠使用卷積神經網絡 來實現這個目標。通過精心設計網絡結構，底層網絡能夠通過兩組一一對應的特征來描 述圖像的局部信息。而這種信息對於匹配人的顏色紋理等特徵更加適合。在較高的層我 們希望學習到人在空間上的位移來判斷局部的位移是符合於人在不同攝像頭中的位移。 通過這些信息，我們的模型來決定這兩張圖片是否來自于同一個人。 / 在以上三個部分中，我們都同最先進的度量學習和其他基于特征設計的行人再識別方法進行比較。我們在不同的數據集上均取得了較為優秀的效果。我們進一步建立了一 個大規模的數據集，這個數據集包含更多的視角、更多的人且每個人在不同的視角下有 更多的圖片。 / Person re-identification is to match persons observed in non-overlapping camera views with visual features. This is an important task in video surveillance by itself and serves as metatask for other problems like inter-camera tracking. Challenges lie in the dramatic intra-person variation introduced by viewpoint change, illumination change and pose variation etc. In this thesis, we are trying to tackle this problem in the following aspects: / Firstly, we observe that the ambiguity increases with the number of candidates to be distinguished. In real world scenario, temporal reasoning is available and can simplify the problem by pruning the candidate set to be matched. Existing approaches adopt a fixed metric for matching all the subjects. Our approach is motivated by the insight that different visual metrics should be optimally learned for different candidate sets. The problem is further formulated under a transfer learning framework. Given a large training set, the training samples are selected and re-weighted according to their visual similarities with the query sample and its candidate set. A weighted maximum margin metric is learned and transferred from a generic metric to a candidate-set-specific metric. / Secondly, we observe that the transformations between two camera views may be too complex to be uni-modal. To tackle this, we propose to partition the image space and formulate the problem into a mixture of expert framework. Our algorithm jointly partitions the image spaces of two camera views into different configurations according to the similarity of cross-view transforms. The visual features of an image pair from different views are locally aligned by being projected to a common feature space and then matched with softly assigned metrics which are locally optimized. The features optimal for recognizing identities are different from those for clustering cross-view transforms. They are jointly learned by utilizing sparsity-inducing norm and information theoretical regularization. / In all the above analysis, feature extraction and learning models are separately designed. A better idea is to directly learn features from training samples and those features can be applied to directly train a discriminative models. We propose a new model where feature extraction is jointly learned with a discriminative convolutional neural network. Local filters at the bottom layer can well extract the information useful for matching persons across camera views like color and texture. Higher layers will capture the spatial shift of those local patches. Finally, we will test whether the shift patterns of those local patches conform to the intra-camera variation of the same person. / In all three parts, comparisons with the state-of-the-art metric learning algorithms and person re-identification methods are carried out and our approach shows the superior performance on public benchmark dataset. Furthermore, we are building a much larger dataset that addresses the real-world scenario which contains much more camera views, identities, and images perview. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Li, Wei. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2013. / Includes bibliographical references (leaves 63-68). / Abstracts also in Chinese. / Acknowledgments --- p.iii / Abstract --- p.vii / Contents --- p.xii / List of Figures --- p.xiv / List of Tables --- p.xv / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Person Re-Identification --- p.1 / Chapter 1.2 --- Challenge in Person Re-Identification --- p.2 / Chapter 1.3 --- Literature Review --- p.4 / Chapter 1.3.1 --- Feature Based Person Re-Identification --- p.4 / Chapter 1.3.2 --- Learning Based Person Re-Identification --- p.7 / Chapter 1.4 --- Thesis Organization --- p.8 / Chapter 2 --- Tranferred Metric Learning for Person Re-Identification --- p.10 / Chapter 2.1 --- Introduction --- p.10 / Chapter 2.2 --- Related Work --- p.12 / Chapter 2.2.1 --- Transfer Learning --- p.12 / Chapter 2.3 --- Our Method --- p.13 / Chapter 2.3.1 --- Visual Features --- p.13 / Chapter 2.3.2 --- Searching and Weighting Training Samples --- p.13 / Chapter 2.3.3 --- Learning Adaptive Metrics by Maximizing Weighted Margins --- p.15 / Chapter 2.4 --- Experimental Results --- p.17 / Chapter 2.4.1 --- Dataset Description --- p.17 / Chapter 2.4.2 --- Generic Metric Learning --- p.18 / Chapter 2.4.3 --- Transferred Metric Learning --- p.19 / Chapter 2.5 --- Conclusions and Discussions --- p.21 / Chapter 3 --- Locally Aligned Feature Transforms for Person Re-Identification --- p.23 / Chapter 3.1 --- Introduction --- p.23 / Chapter 3.2 --- Related Work --- p.24 / Chapter 3.2.1 --- Localized Methods --- p.25 / Chapter 3.3 --- Model --- p.26 / Chapter 3.4 --- Learning --- p.27 / Chapter 3.4.1 --- Priors --- p.27 / Chapter 3.4.2 --- Objective Function --- p.29 / Chapter 3.4.3 --- Training Model --- p.29 / Chapter 3.4.4 --- Multi-Shot Extension --- p.30 / Chapter 3.4.5 --- Discriminative Metric Learning --- p.31 / Chapter 3.5 --- Experiment --- p.32 / Chapter 3.5.1 --- Identification with Two Fixed Camera Views --- p.33 / Chapter 3.5.2 --- More General Camera Settings --- p.37 / Chapter 3.6 --- Conclusions --- p.38 / Chapter 4 --- Deep Neural Network for Person Re-identification --- p.39 / Chapter 4.1 --- Introduction --- p.39 / Chapter 4.2 --- Related Work --- p.43 / Chapter 4.3 --- Introduction of the New Dataset --- p.44 / Chapter 4.4 --- Model --- p.46 / Chapter 4.4.1 --- Architecture Overview --- p.46 / Chapter 4.4.2 --- Convolutional and Max-Pooling Layer --- p.48 / Chapter 4.4.3 --- Patch Matching Layer --- p.49 / Chapter 4.4.4 --- Maxout Grouping Layer --- p.52 / Chapter 4.4.5 --- Part Displacement --- p.52 / Chapter 4.4.6 --- Softmax Layer --- p.53 / Chapter 4.5 --- Training Strategies --- p.54 / Chapter 4.5.1 --- Data Augmentation and Balancing --- p.55 / Chapter 4.5.2 --- Bootstrapping --- p.55 / Chapter 4.6 --- Experiment --- p.56 / Chapter 4.6.1 --- Model Specification --- p.56 / Chapter 4.6.2 --- Validation on Single Pair of Cameras --- p.57 / Chapter 4.7 --- Conclusion --- p.58 / Chapter 5 --- Conclusion --- p.60 / Chapter 5.1 --- Conclusion --- p.60 / Chapter 5.2 --- Future Work --- p.61 / Bibliography --- p.63