Inertial wearable sensors constitute a booming industry. They are self contained, low powered and highly miniaturized. They allow for remote or self monitoring of health-related parameters. When used to obtain 3-D position, velocity and orientation information, research has shown that it is possible to draw conclusion about issues such as fall risk, Parkinson disease and gait assessment. A key issues in extracting information from accelerometers and gyroscopes is the fusion of their noisy data to allow accurate assessment of the disease. This, so far, is an unsolved problem. Typically, a Kalman filter or its nonlinear, non-Gaussian version are implemented for estimating attitude â?? which in turn is critical for position estimation. However, sampling rates and large state vectors required make them unacceptable for the limited-capacity batteries of low-cost wearable sensors. The low-computation cost complementary filter has recently been re-emerging as the algorithm for attitude estimation. We employ it with a heuristic drift elimination method that is shown to remove, almost entirely, the drift caused by the gyroscope and hence generate a fairly accurate attitude and drift-eliminated position estimate. Inertial sensor data is obtained from the 10-axis SP-10C sensor, attached to a wearable insole that is inserted in the shoe. Data is obtained from walking in a structured indoor environment in Votey Hall.

Multi-pedestrian tracking (MPT) is the task of localizing and following the trajectory of pedestrians in a sequence. Using an MPT algorithm is an important part in preventing pedestrian-vehicle collisions in Automated Driving (AD) and Advanced Driving Assistance Systems (ADAS). It has benefited greatly from the advances in computer vision and machine learning in the last decades. Using a pedestrian detector, the tracking consists of associating the detections between frames and maintaining pedestrian identities throughout the sequence. This can be a challenging task due to occlusions, missed detections and complex scenes. The number of pedestrians is unknown, and it varies with time. Finding new methods for improving MPT is an active research field and there are many approaches found in the literature. This work focuses on improving the detection-to-track association, the data association, with the help of extracted color features for each pedestrian. Utilizing the recent improvements in object detection this work shows that classical color features still is relevant in pedestrian tracking for real time applications with limited computational resources. The appearance is not only used in the data association but also integrated in a new proposed method to avoid tracking errors due to missed detections. The results show that even with simple models the color appearance can be used to improve the tracking results. Evaluation on the commonly used Multi-Object Tracking-benchmark shows an improvement in the Multi-Object Tracking Accuracy and identity switches, while keeping other measures essentially unchanged.

A análise de comportamento coletivo e rastreamento de pedestres apresentam diversas aplicações, especialmente em sistemas de vigilância inteligente. Neste trabalho é proposta uma solução compreensiva com objetivo de atingir rastreamento de pedestre e reconhecimento de atividade coletiva de maneira robusta baseada na utilização de câmeras calibradas. Primeiramente, com o objetivo de remover a necessidade de calibração manual, nós apresentamos um método de calibração automática que explora detectores de pedestres e remoção de fundo para calibragem baseada em otimização não-linear. Adicionalmente, nós propomos a utilização da matriz de calibração para gerar candidatos coerentes com a geometria de cena em detectores de pedestres. Nossa abordagem tem como objetivo diminuir o intervalo de escalas comumente utilizado em detectores baseados em janelas deslizantes, gerando um número menor de extrações de atributos e reduzindo o número de falsos positivos na detecção. Em seguida, nós propomos um método de rastreamento de múltiplos pedestres utilizando câmeras calibradas. Nossa abordagem explora histogramas de cor para rastrear os pequenas regiões (patches) de cada alvo. Os vetores de deslocamento obtidos através do pareamento de atributos de aparência são combinados com um vetor obtido através de um preditor de movimento em coordenadas de mundo. Adicionalmente, nós incluímos informações originárias de detectores de pedestres para aumentar a acurácia do sistema e sua habilidade de recuperação a falhas. Por fim, nós propomos uma abordagem hierárquica de duas camadas para o problema de reconhecimento de atividade coletiva baseada no uso de classificadores Random Forests. No primeiro nível da técnica proposta, nós utilizamos distâncias entre pares de pessoas e suas respectivas velocidades relativas para classificar interações de pares. Estas interações são combinadas com a dinâmica do formato do grupo observado (e sua respectiva velocidade) para o reconhecimento de atividades coletivas. Os experimentos realizados neste trabalho demonstram a qualidade de nossas abordagens em sequências de vídeos disponíveis publicamente. Nossos resultados mostram serem competitivos quando comparados com técnicas do estado da arte e, particularmente, apresentam uma boa generalização entre diferentes cenários de captura de vídeo. / Collective behavior detection and pedestrian tracking present many applications, specially in surveillance systems. In this dissertation, we proposed a complete pipeline for achieving robust tracking and collective behavior recognition based on calibrated static cameras. To remove the necessity of manual calibration, we first present a fully automatic self-calibration system that explores pedestrian detection results and background removal at non-consecutive frames in order to calibrate a static camera using a non-linear cost function. We also propose the use of camera calibration to generate geometrically coherent candidates for pedestrian detection. Our approach aims to reduce the scale range typically used in sliding-window techniques, which leads to less feature extractions and decreased number of false positives. Then, we propose a multi-target pedestrian tracking algorithm using a calibrated static camera. The tracking approach explores color histograms to track patches of each target. Obtained displacement vectors are combined with the expected motion of pedestrians in the world coordinate system. The proposed tracker also incorporates pedestrian detector results to improve the system’s accuracy and its ability to recover from failure. Finally, we propose a two-layered approach for collective behavior recognition based on Random Forests classifiers. In the first level, we use inter-personal distances and relative speeds computed in the world coordinate system to classify asymmetrical pair interactions. Those interactions are combined with group shape dynamics and mean velocity to recognize the collective behavior. We devise a set of experiments to attest the quality of our approaches using publicly available datasets. Results have shown to be competitive against state-of-the-art techniques, and particularly of good generalization across different databases.

A análise de comportamento coletivo e rastreamento de pedestres apresentam diversas aplicações, especialmente em sistemas de vigilância inteligente. Neste trabalho é proposta uma solução compreensiva com objetivo de atingir rastreamento de pedestre e reconhecimento de atividade coletiva de maneira robusta baseada na utilização de câmeras calibradas. Primeiramente, com o objetivo de remover a necessidade de calibração manual, nós apresentamos um método de calibração automática que explora detectores de pedestres e remoção de fundo para calibragem baseada em otimização não-linear. Adicionalmente, nós propomos a utilização da matriz de calibração para gerar candidatos coerentes com a geometria de cena em detectores de pedestres. Nossa abordagem tem como objetivo diminuir o intervalo de escalas comumente utilizado em detectores baseados em janelas deslizantes, gerando um número menor de extrações de atributos e reduzindo o número de falsos positivos na detecção. Em seguida, nós propomos um método de rastreamento de múltiplos pedestres utilizando câmeras calibradas. Nossa abordagem explora histogramas de cor para rastrear os pequenas regiões (patches) de cada alvo. Os vetores de deslocamento obtidos através do pareamento de atributos de aparência são combinados com um vetor obtido através de um preditor de movimento em coordenadas de mundo. Adicionalmente, nós incluímos informações originárias de detectores de pedestres para aumentar a acurácia do sistema e sua habilidade de recuperação a falhas. Por fim, nós propomos uma abordagem hierárquica de duas camadas para o problema de reconhecimento de atividade coletiva baseada no uso de classificadores Random Forests. No primeiro nível da técnica proposta, nós utilizamos distâncias entre pares de pessoas e suas respectivas velocidades relativas para classificar interações de pares. Estas interações são combinadas com a dinâmica do formato do grupo observado (e sua respectiva velocidade) para o reconhecimento de atividades coletivas. Os experimentos realizados neste trabalho demonstram a qualidade de nossas abordagens em sequências de vídeos disponíveis publicamente. Nossos resultados mostram serem competitivos quando comparados com técnicas do estado da arte e, particularmente, apresentam uma boa generalização entre diferentes cenários de captura de vídeo. / Collective behavior detection and pedestrian tracking present many applications, specially in surveillance systems. In this dissertation, we proposed a complete pipeline for achieving robust tracking and collective behavior recognition based on calibrated static cameras. To remove the necessity of manual calibration, we first present a fully automatic self-calibration system that explores pedestrian detection results and background removal at non-consecutive frames in order to calibrate a static camera using a non-linear cost function. We also propose the use of camera calibration to generate geometrically coherent candidates for pedestrian detection. Our approach aims to reduce the scale range typically used in sliding-window techniques, which leads to less feature extractions and decreased number of false positives. Then, we propose a multi-target pedestrian tracking algorithm using a calibrated static camera. The tracking approach explores color histograms to track patches of each target. Obtained displacement vectors are combined with the expected motion of pedestrians in the world coordinate system. The proposed tracker also incorporates pedestrian detector results to improve the system’s accuracy and its ability to recover from failure. Finally, we propose a two-layered approach for collective behavior recognition based on Random Forests classifiers. In the first level, we use inter-personal distances and relative speeds computed in the world coordinate system to classify asymmetrical pair interactions. Those interactions are combined with group shape dynamics and mean velocity to recognize the collective behavior. We devise a set of experiments to attest the quality of our approaches using publicly available datasets. Results have shown to be competitive against state-of-the-art techniques, and particularly of good generalization across different databases.

A análise de comportamento coletivo e rastreamento de pedestres apresentam diversas aplicações, especialmente em sistemas de vigilância inteligente. Neste trabalho é proposta uma solução compreensiva com objetivo de atingir rastreamento de pedestre e reconhecimento de atividade coletiva de maneira robusta baseada na utilização de câmeras calibradas. Primeiramente, com o objetivo de remover a necessidade de calibração manual, nós apresentamos um método de calibração automática que explora detectores de pedestres e remoção de fundo para calibragem baseada em otimização não-linear. Adicionalmente, nós propomos a utilização da matriz de calibração para gerar candidatos coerentes com a geometria de cena em detectores de pedestres. Nossa abordagem tem como objetivo diminuir o intervalo de escalas comumente utilizado em detectores baseados em janelas deslizantes, gerando um número menor de extrações de atributos e reduzindo o número de falsos positivos na detecção. Em seguida, nós propomos um método de rastreamento de múltiplos pedestres utilizando câmeras calibradas. Nossa abordagem explora histogramas de cor para rastrear os pequenas regiões (patches) de cada alvo. Os vetores de deslocamento obtidos através do pareamento de atributos de aparência são combinados com um vetor obtido através de um preditor de movimento em coordenadas de mundo. Adicionalmente, nós incluímos informações originárias de detectores de pedestres para aumentar a acurácia do sistema e sua habilidade de recuperação a falhas. Por fim, nós propomos uma abordagem hierárquica de duas camadas para o problema de reconhecimento de atividade coletiva baseada no uso de classificadores Random Forests. No primeiro nível da técnica proposta, nós utilizamos distâncias entre pares de pessoas e suas respectivas velocidades relativas para classificar interações de pares. Estas interações são combinadas com a dinâmica do formato do grupo observado (e sua respectiva velocidade) para o reconhecimento de atividades coletivas. Os experimentos realizados neste trabalho demonstram a qualidade de nossas abordagens em sequências de vídeos disponíveis publicamente. Nossos resultados mostram serem competitivos quando comparados com técnicas do estado da arte e, particularmente, apresentam uma boa generalização entre diferentes cenários de captura de vídeo. / Collective behavior detection and pedestrian tracking present many applications, specially in surveillance systems. In this dissertation, we proposed a complete pipeline for achieving robust tracking and collective behavior recognition based on calibrated static cameras. To remove the necessity of manual calibration, we first present a fully automatic self-calibration system that explores pedestrian detection results and background removal at non-consecutive frames in order to calibrate a static camera using a non-linear cost function. We also propose the use of camera calibration to generate geometrically coherent candidates for pedestrian detection. Our approach aims to reduce the scale range typically used in sliding-window techniques, which leads to less feature extractions and decreased number of false positives. Then, we propose a multi-target pedestrian tracking algorithm using a calibrated static camera. The tracking approach explores color histograms to track patches of each target. Obtained displacement vectors are combined with the expected motion of pedestrians in the world coordinate system. The proposed tracker also incorporates pedestrian detector results to improve the system’s accuracy and its ability to recover from failure. Finally, we propose a two-layered approach for collective behavior recognition based on Random Forests classifiers. In the first level, we use inter-personal distances and relative speeds computed in the world coordinate system to classify asymmetrical pair interactions. Those interactions are combined with group shape dynamics and mean velocity to recognize the collective behavior. We devise a set of experiments to attest the quality of our approaches using publicly available datasets. Results have shown to be competitive against state-of-the-art techniques, and particularly of good generalization across different databases.

In this thesis, a novel Pedestrian Protection System (PPS), composed of the Pedestrian Detection System (PDS) and the Pedestrian Tracking System (PTS), was proposed. The PPS is a supplementary application for the Advanced Driver Assistance System, which is used to avoid collisions between vehicles and pedestrians. The Pedestrian Detection System (PDS) is used to detect pedestrians from near to far ranges with the feature-classi er-based detection method (HOG + SVM). To achieve pedestrian detection from near to far ranges, a novel structure was proposed. The structure of our PDS consists of two cameras (called CS and CL separately). The CS is equipped with a short focal length lens to detect pedestrians in near-to-mid range; and, the CL is equipped with a long focal length lens to detect pedestrians in mid-to-far range. To accelerate the processing speed of pedestrian detection, the parallel computing capacity of GPU was utilized in the PDS. The synchronization algorithm is also introduced to synchronize the detection results of CS and CL. Based on the novel pedestrian detection structure, the detection process can reach a distance which is more than 130 meters away without decreasing detection accuracy. The detection range can be extended more than 100 meters without decreasing the processing speed of pedestrian detection. Afterwards, an algorithm to eliminate duplicate detection results is proposed to improve the detection accuracy. The Pedestrian Tracking System (PTS) is applied following the Pedestrian Detection System. The PTS is used to track the movement trajectory of pedestrians and to predict the future motion and movement direction. A C + + class (called pedestrianTracking class, which is short for PTC) was generated to operate the tracking process for every detected pedestrian. The Kalman lter is the main algorithm inside the PTC. During the operation of PPS, the nal detection results of each frame from PDS will be transmitted to the PTS to enable the tracking process. The new detection results will be used to update the existing tracking results in the PTS. Moreover, if there is a newly detected pedestrian, a new process will be generated to track the pedestrian in the PTS. Based on the tracking results in PTS, the movement trajectory of pedestrians can be obtained and their future motion and movement direction can be predicted. Two kinds of alerts are generated based on the predictions: warning alert and dangerous alert. These two alerts represent di erent situations; and, they will alert drivers to the upcoming situations. Based on the predictions and alerts, the collisions can be prevented e ectively. The safety of pedestrians can be guaranteed.

Multiple-object tracking is a task within the field of computer vision. As the name stated, the task consists of tracking multiple objects in the video, an algorithm that completes such task are called trackers. Many of the existing trackers require supervision, meaning that the location and identity of each object which appears in the training data must be labeled. The procedure of generating these labels, usually through manual annotation of video material, is highly resource-consuming. On the other hand, different from well-known labeled Multiple-object tracking datasets, there exist a massive amount of unlabeled video with different objects, environments, and video specifications. Using such unlabeled video can therefore contribute to cheaper and more diverse datasets. There have been numerous attempts on unsupervised object tracking, but most rely on evaluating the tracker performance on a labeled dataset. The reason behind this is the lack of an evaluation method for unlabeled datasets. This project explores unsupervised pedestrian tracking on video taken from a stationary camera over a long duration. On top of a simple baseline tracker, two methods are proposed to extend the baseline to increase its performance. We then propose an evaluation method that works for unlabeled video, which we use to evaluate the proposed methods. The evaluation method consists of the trajectory completion rate and the number of ID switches. The trajectory completion rate is a novel metric proposed for pedestrian tracking. Pedestrians generally enter and exit the scene for video taken by a stationary camera in specific locations. We define a complete trajectory as a trajectory that goes from one area to another. The completion rate is calculated by the number of complete trajectories over all trajectories. Results showed that the two proposed methods had increased the trajectory completion rate on top of the original baseline performance. Moreover, both proposed methods did so without significantly increasing the number of ID switches. / Spårning av flera objekt är en uppgift inom området datorseende. Som namnet angav består uppgiften av att spåra flera objekt i videon, en algoritm som slutför en sådan uppgift kallas trackers. Många av de befintliga spårarna kräver övervakning, vilket innebär att platsen och identiteten för varje objekt som visas i träningsdata måste märkas. Proceduren för att generera dessa etiketter, vanligtvis genom manuell anteckning av videomaterial, är mycket resurskrävande. Å andra sidan, till skillnad från välkända märkta uppsättningar för spårning av flera objekt, finns det en enorm mängd omärkt video med olika objekt, miljöer och videospecifikationer. Att använda sådan omärkt video kan därför bidra till billigare och mer varierande datauppsättningar. Det har gjorts många försök med oövervakad objektspårning, men de flesta förlitar sig på att utvärdera spårningsprestandan på en märkt dataset. Anledningen till detta är avsaknaden av en utvärderingsmetod för omärkta datamängder. Detta projekt utforskar oövervakad fotgängarspårning på video som tagits från en stillastående kamera under lång tid. Utöver en enkel baslinjespårare föreslås två metoder för att utöka baslinjen för att öka dess prestanda. Vi föreslår sedan en utvärderingsmetod som fungerar för omärkt video, som vi använder för att utvärdera de föreslagna metoderna. Utvärderingsmetoden består av banans slutförandegrad och antalet ID-växlar. Banans slutförandegrad är ett nytt mått som föreslås för spårning av fotgängare. Fotgängare går vanligtvis in och lämnar scenen för video tagna med en stillastående kamera på specifika platser. Vi definierar en komplett bana som en bana som går från ett område till ett annat. Färdigställandegraden beräknas av antalet kompletta banor över alla banor. Resultaten visade att de två föreslagna metoderna hade ökat graden av fullbordande av banan utöver den ursprungliga baslinjeprestandan. Dessutom gjorde båda de föreslagna metoderna det utan att nämnvärt öka antalet ID-växlar.

This thesis addresses the problem of tracking multiple ground targets whose motion is dependent on one another. Multiple approaches which integrate the social force based motion model into different filtering algorithms are proposed. The social force concept has previously been used to model pedestrian motion where the interactions among pedestrians are described using social forces. First, the social force based motion model integrated into the Probability Hypothesis Density (PHD) framework is proposed. Two different implementations, namely, the Sequential Monte Carlo (SMC) technique and the Gaussian Mixture (GM) technique, are derived to implement the proposed Social Force PHD (SF-PHD) filter in ground target tracking scenarios. Next, a social-force-based motion model integrated into the stacked Kalman filter (stacked SF-KF) is developed and its multiple model (stacked IMM-SF-KF) variant is derived. Then, the assumption used in the proposed algorithms, that the actual values of the social force parameters are known, is not valid at all times and the assumption is relaxed. Hence, simultaneous parameter estimation techniques for the social force parameters during the tracking are proposed. Three approaches based on the state augmentation method, the Expectation Maximization (EM) method and the maximum likelihood method are derived. The maximum likelihood method can be implemented offline or online, depending on the requirement. The traditional Posterior Cramer Rao Lower Bound (PCRLB), which is the inverse of the Fisher information matrix, gives a bound on the optimal achievable accuracy of the estimated state of a target with independent motion. Subsequently, a modified performance measure based on the PCRLB for targets whose motion is dependent on each other is derived to validate the performance of the proposed algorithms. Finally, the PCRLB that accounts for unknown interactions is derived to validate the proposed simultaneous parameter estimation techniques. Simulated and real data are used to show the performance of the proposed algorithms and simultaneous parameter estimation techniques compared to the algorithms in the literature. / Thesis / Doctor of Philosophy (PhD) / This thesis addresses the problem of tracking multiple ground targets whose motion is dependent on one another. In target tracking literature, it is commonly assumed that a target’s motion follows a nearly constant velocity, constant turn or a constant acceleration model independent of the motion of other targets. But the actual behavior of a ground target may be more intricate than that and it is often affected by the motion of other targets, obstacles in the surrounding and its intended destination. Hence, a more sophisticated motion modeling technique, which integrates the various factors that affect the motion of ground targets, is needed. In this thesis, multiple approaches which integrate the social force based motion model into different filtering algorithms are proposed. The social force concept has previously been used to model pedestrian motion where the interactions among pedestrians are described using social forces. First, the social force based motion model integrated into the Probability Hypothesis Density (PHD) framework is proposed. Two different implementations, namely, the Sequential Monte Carlo (SMC) technique and the Gaussian Mixture (GM) technique, are derived to implement the proposed Social Force PHD (SF-PHD) filter in ground target tracking scenarios. Next, a social-force-based motion model integrated into the stacked Kalman filter (stacked SF-KF) is developed and its multiple model (stacked IMM-SF-KF) variant is derived. Then, the assumption used in the proposed algorithms, that the actual values of the social force parameters are known, is not valid at all times and the assumption is relaxed. Hence, simultaneous parameter estimation techniques for the social force parameters during the tracking are proposed. Three approaches based on the state augmentation method, the Expectation Maximization (EM) method and the maximum likelihood method are derived. The maximum likelihood method can be implemented offline or online, depending on the requirement. The traditional Posterior Cramer Rao Lower Bound (PCRLB), which is the inverse of the Fisher information matrix, gives a bound on the optimal achievable accuracy of the estimated state of a target with independent motion. Subsequently, a modified performance measure based on the PCRLB for targets whose motion is dependent on each other is derived to validate the performance of the proposed algorithms. Finally, the PCRLB that accounts for unknown interactions is derived to validate the proposed simultaneous parameter estimation techniques. Simulated and real data are used to show the performance of the proposed algorithms and simultaneous parameter estimation techniques compared to the algorithms in the literature.

No description available.

Multiple person detection and tracking is challenging problem with high application potential. The difficulty of the problem is caused mainly by complexity of scene and large variations in articulation and appearance of person. The aim of this work is to design and implement system capable of detecting and tracking people in video from static mono-camera. For this purpose, an online method for tracking has been proposed based on tracking-by-detection approach. The method combines detection, tracking and fusion of responses to achieve accurate results. The implementation was evaluated on available dataset and the results show that it is suitable to use for this task. A method for motion segmentation was proposed and implemented to improve the tracking results. Furthermore, implementation of detector based on histogram of oriented gradients was accelerated by taking advantage of graphics processing unit (GPU).