No description available.

This thesis is focused on human detection in video streams using Convolutional Neural Networks (CNNs). In recent years, CNNs have become common methods in various computer vision problems, and image detection is one popular application. The performance of CNNs on the detection problem has undergone a rapid increase in both accuracy and speed. In this thesis, we focus on a specific sub-domain of detection: human detection. Furthermore, it makes the problem more challenging as the data extracted from video streams captured by a head-mounted camera and therefore include difficult view points and strong motion blur. Considering both accuracy and speed, we choose two models with typical structures--You Only Look Once (YOLO) and Single Shot MultiBox Detector (SSD)--to experiment how robust the models perform on human domain with motion blur, and how the differences between the structures may influence the results. Several experiments are carried out in this thesis. With a better design of structure, SSD outperforms YOLO in various aspects. It is further proved as we fine-tuned YOLO and SSD300 on human data in Pascal VOC 2012 trainval dataset, showing the efficiency of SSD with fewer classes trained. As for motion blur problem, it is shown in the experiments that SSD300 has good ability to learn blurred patterns. The structure of SSD300 is further tested with regard to the design of default boxes and its performance on different scales and locations. The results show that the SSD model has a superior performance on online detection in video streams, but with a more customized structure it has potential to achieve even better results. / Detta examensarbete undersöker problemet att detektera människor i videströmmar med hjälp av convolutional neural networks (CNNs). Under de senaste åren har CNNs ökat i användning, vilket medfört stora förbättringar i noggrannhet och beräkningshastighet. CNN är nu en populär metod i olika datorseende- och bildigenkänningsproblem. I detta projekt fokuserar vi på en specifik subdomän: detektion av människor. Problemet försvåras ytterligare av att vår videodata är inspelad från en huvudmonterad kamera. Detta medför att vårt system behöver hantera ovanliga betraktningsvinklar och rörelseoskärpa. Efter att ha tagit hänsyn till beräkningshastighet och detektionskvalitet har vi valt att undersöka två olika CNN-modeller: You Only Look Once (YOLO) och Single Shot MultiBox Detector (SSD). Experimenten har designats för att visa hur robusta metoderna är på att detektera människor i bilder med rörelseoskärpa. Vi har också undersökt hur modifikationer på nätverksstrukturer kan påverka slutresultaten. Flera experiment har gjorts i detta projekt. Vi visar att SSD ger bättre resultat än YOLO i många avseenden, vilket beror på att SSD har en bättre designad nätverksstruktur. Genom att utföra fin-anpassning av YOLO och SSD på bildkollektionen i Pascal VOC 2012 kan vi visa att SSD fungerar bra även när vi tränar på färre objektklasser. SSD300 har också god förmåga att lära mönster som påverkats av oskärpa. Vi analyserar även hur valet av position och skalor av de predefinierade sökområdenen påverkar resultaten från SSD300. Resultaten visar att SSD-modellen presterar överlägset i realtidsdetektion i videoströmmar. Genom att anpassa strukturerna ytterligare finns potential att uppnå ännu bättre resultat.

Visual analysis of dense crowds is particularly challenging due to large number of individuals, occlusions, clutter, and fewer pixels per person which rarely occur in ordinary surveillance scenarios. This dissertation aims to address these challenges in images and videos of extremely dense crowds containing hundreds to thousands of humans. The goal is to tackle the fundamental problems of counting, detecting and tracking people in such images and videos using visual and contextual cues that are automatically derived from the crowded scenes. For counting in an image of extremely dense crowd, we propose to leverage multiple sources of information to compute an estimate of the number of individuals present in the image. Our approach relies on sources such as low confidence head detections, repetition of texture elements (using SIFT), and frequency-domain analysis to estimate counts, along with confidence associated with observing individuals, in an image region. Furthermore, we employ a global consistency constraint on counts using Markov Random Field which caters for disparity in counts in local neighborhoods and across scales. We tested this approach on crowd images with the head counts ranging from 94 to 4543 and obtained encouraging results. Through this approach, we are able to count people in images of high-density crowds unlike previous methods which are only applicable to videos of low to medium density crowded scenes. However, the counting procedure just outputs a single number for a large patch or an entire image. With just the counts, it becomes difficult to measure the counting error for a query image with unknown number of people. For this, we propose to localize humans by finding repetitive patterns in the crowd image. Starting with detections from an underlying head detector, we correlate them within the image after their selection through several criteria: in a pre-defined grid, locally, or at multiple scales by automatically finding the patches that are most representative of recurring patterns in the crowd image. Finally, the set of generated hypotheses is selected using binary integer quadratic programming with Special Ordered Set (SOS) Type 1 constraints. Human Detection is another important problem in the analysis of crowded scenes where the goal is to place a bounding box on visible parts of individuals. Primarily applicable to images depicting medium to high density crowds containing several hundred humans, it is a crucial pre-requisite for many other visual tasks, such as tracking, action recognition or detection of anomalous behaviors, exhibited by individuals in a dense crowd. For detecting humans, we explore context in dense crowds in the form of locally-consistent scale prior which captures the similarity in scale in local neighborhoods with smooth variation over the image. Using the scale and confidence of detections obtained from an underlying human detector, we infer scale and confidence priors using Markov Random Field. In an iterative mechanism, the confidences of detections are modified to reflect consistency with the inferred priors, and the priors are updated based on the new detections. The final set of detections obtained are then reasoned for occlusion using Binary Integer Programming where overlaps and relations between parts of individuals are encoded as linear constraints. Both human detection and occlusion reasoning in this approach are solved with local neighbor-dependent constraints, thereby respecting the inter-dependence between individuals characteristic to dense crowd analysis. In addition, we propose a mechanism to detect different combinations of body parts without requiring annotations for individual combinations. Once human detection and localization is performed, we then use it for tracking people in dense crowds. Similar to the use of context as scale prior for human detection, we exploit it in the form of motion concurrence for tracking individuals in dense crowds. The proposed method for tracking provides an alternative and complementary approach to methods that require modeling of crowd flow. Simultaneously, it is less likely to fail in the case of dynamic crowd flows and anomalies by minimally relying on previous frames. The approach begins with the automatic identification of prominent individuals from the crowd that are easy to track. Then, we use Neighborhood Motion Concurrence to model the behavior of individuals in a dense crowd, this predicts the position of an individual based on the motion of its neighbors. When the individual moves with the crowd flow, we use Neighborhood Motion Concurrence to predict motion while leveraging five-frame instantaneous flow in case of dynamically changing flow and anomalies. All these aspects are then embedded in a framework which imposes hierarchy on the order in which positions of individuals are updated. The results are reported on eight sequences of medium to high density crowds and our approach performs on par with existing approaches without learning or modeling patterns of crowd flow. We experimentally demonstrate the efficacy and reliability of our algorithms by quantifying the performance of counting, localization, as well as human detection and tracking on new and challenging datasets containing hundreds to thousands of humans in a given scene.

A growing number of applications involving autonomous mobile robots will require their navigation across environments in which spaces are shared with humans. In those situations, the robot’s actions are socially acceptable if they reflect the behaviours that humans would generate in similar conditions. Therefore, the robot must perceive people in the environment and correctly react based on their actions and relevance to its mission. In order to give a push forward to human-robot interaction, the proposed research is focused on efficient robot motion algorithms, covering all the tasks needed in the whole process, such as obstacle detection, human motion tracking and prediction, socially aware navigation, etc. The final framework presented in this thesis is a robust and efficient solution enabling the robot to correctly understand the human intentions and consequently perform safe, legible, and socially compliant actions. The thesis retraces in its structure all the different steps of the framework through the presentation of the algorithms and models developed, and the experimental evaluations carried out both with simulations and on real robotic platforms, showing the performance obtained in real–time in complex scenarios, where the humans are present and play a prominent role in the robot decisions. The proposed implementations are all based on insightful combinations of traditional model-based techniques and machine learning algorithms, that are adequately fused to effectively solve the human-aware navigation. The specific synergy of the two methodology gives us greater flexibility and generalization than the navigation approaches proposed so far, while maintaining accuracy and reliability which are not always displayed by learning methods.

The networks of surveillance cameras installed in public places and private territories continuously record video data with the aim of detecting and preventing unlawful activities. This enhances the importance of video content analysis applications, either for real time (i.e. analytic) or post-event (i.e. forensic) analysis. In this thesis, the primary focus is on four key aspects of video content analysis, namely; 1. Moving object detection and recognition, 2. Correction of colours in the video frames and recognition of colours of moving objects, 3. Make and model recognition of vehicles and identification of their type, 4. Detection and recognition of text information in outdoor scenes. To address the first issue, a framework is presented in the first part of the thesis that efficiently detects and recognizes moving objects in videos. The framework targets the problem of object detection in the presence of complex background. The object detection part of the framework relies on background modelling technique and a novel post processing step where the contours of the foreground regions (i.e. moving object) are refined by the classification of edge segments as belonging either to the background or to the foreground region. Further, a novel feature descriptor is devised for the classification of moving objects into humans, vehicles and background. The proposed feature descriptor captures the texture information present in the silhouette of foreground objects. To address the second issue, a framework for the correction and recognition of true colours of objects in videos is presented with novel noise reduction, colour enhancement and colour recognition stages. The colour recognition stage makes use of temporal information to reliably recognize the true colours of moving objects in multiple frames. The proposed framework is specifically designed to perform robustly on videos that have poor quality because of surrounding illumination, camera sensor imperfection and artefacts due to high compression. In the third part of the thesis, a framework for vehicle make and model recognition and type identification is presented. As a part of this work, a novel feature representation technique for distinctive representation of vehicle images has emerged. The feature representation technique uses dense feature description and mid-level feature encoding scheme to capture the texture in the frontal view of the vehicles. The proposed method is insensitive to minor in-plane rotation and skew within the image. The capability of the proposed framework can be enhanced to any number of vehicle classes without re-training. Another important contribution of this work is the publication of a comprehensive up to date dataset of vehicle images to support future research in this domain. The problem of text detection and recognition in images is addressed in the last part of the thesis. A novel technique is proposed that exploits the colour information in the image for the identification of text regions. Apart from detection, the colour information is also used to segment characters from the words. The recognition of identified characters is performed using shape features and supervised learning. Finally, a lexicon based alignment procedure is adopted to finalize the recognition of strings present in word images. Extensive experiments have been conducted on benchmark datasets to analyse the performance of proposed algorithms. The results show that the proposed moving object detection and recognition technique superseded well-know baseline techniques. The proposed framework for the correction and recognition of object colours in video frames achieved all the aforementioned goals. The performance analysis of the vehicle make and model recognition framework on multiple datasets has shown the strength and reliability of the technique when used within various scenarios. Finally, the experimental results for the text detection and recognition framework on benchmark datasets have revealed the potential of the proposed scheme for accurate detection and recognition of text in the wild.

During the last decade the human body detection and tracking has been a very extensive research eld within the computer vision. There are many potential applications of people tracking such as security-monitoring, anthropomorphic analysis or biometrics. In this thesis we present an algorithm and related software to detect human bodies in an indoor environment. It is part of a wider project which aims to estimate the human height. The purposed algorithm performs in real-time to detect people. The algorithm is developed using the free OpenCV library in C++ programming language. As far as this algorithm is rst part of a wider system, our software gives two outputs. The principal one is the coordinates of the detected object. With the coordinates, the aforementioned measuring system will be able to calculate the height by itself. The other output is the video sequence with the detected person bounded by a rectangle, wich provides visual feedback to the user. This software is able to communicate with Matlab Engine. It is important since the subsequent height estimation system works in Matlab®.

The problem addressed in this thesis is the detection, localisation and tracking of human workers from mobile industrial machinery using a customised vision system developed at Örebro University. Coined the RefleX Vision System, its hardware configuration and computer vision algorithms were specifically designed for real-world industrial scenarios where workers are required to wear protective high-visibility garments with retro-reflective markers. The demand for robust industry-purpose human sensing methods originates from the fact that many industrial environments represent work spaces that are shared between humans and mobile machinery. Typical examples of such environments include construction sites, surface and underground mines, storage yards and warehouses. Here, accidents involving mobile equipment and human workers frequently result in serious injuries and fatalities. Robust sensor-based detection of humans in the surrounding of mobile equipment is therefore an active research topic and represents a crucial requirement for safe vehicle operation and accident prevention in increasingly automated production sites. Addressing the described safety issue, this thesis presents a collection of papers which introduce, analyse and evaluate a novel vision-based method for detecting humans equipped with protective high-visibility garments in the neighbourhood of manned or unmanned industrial vehicles. The thesis provides a comprehensive discussion of the numerous aspects regarding the design of the hardware and the computer vision algorithms that constitute the vision system. An active nearinfrared camera setup that is customised for the robust perception of retroreflective markers builds the basis for the sensing method. Using its specific input, a set of computer vision and machine learning algorithms then perform extraction, analysis, classification and localisation of the observed reflective patterns, and eventually detection and tracking of workers with protective garments. Multiple real-world challenges, which existing methods frequently struggle to cope with, are discussed throughout the thesis, including varying ambient lighting conditions and human body pose variation. The presented work has been carried out with a strong focus on industrial applicability, and therefore includes an extensive experimental evaluation in a number of different real-world indoor and outdoor work environments.

No description available.

This paper deals with study of human movement and using that in identification. In the first part of my work are explained characteristics of human movement and factors which take effect on these characteristics. Practical part is dealing with design of program which is solving mentioned problems. The input of program is created by video sequence with lateral movement of human. The program is finding coordinates of lower limbs joints. From this coordinates are created locomotion characteristics used for human identification. Matching of time behaviors is based on correlation.