This thesis is related to the broad subject of automatic motion detection and analysis in videosurveillance image sequence. Besides, proposing the new unique solution, some of the previousalgorithms are evaluated, where some of the approaches are noticeably complementary sometimes.In real time surveillance, detecting and tracking multiple objects and monitoring their activities inboth outdoor and indoor environment are challenging task for the video surveillance system. Inpresence of a good number of real time problems limits scope for this work since the beginning. Theproblems are namely, illumination changes, moving background and shadow detection.An improved background subtraction method has been followed by foreground segmentation, dataevaluation, shadow detection in the scene and finally the motion detection method. The algorithm isapplied on to a number of practical problems to observe whether it leads us to the expected solution.Several experiments are done under different challenging problem environment. Test result showsthat under most of the problematic environment, the proposed algorithm shows the better qualityresult.

An automated vision system performs critical tasks in video surveillance, while decreasing costs and increasing efficiency. It can provide high quality scene monitoring without the limitations of human distraction and fatigue. Advances in embedded processors, wireless networks, and imager technology have enabled computer vision systems to be deployed pervasively in stationary surveillance monitors, hand-held devices, and vehicular sensors. However, the size, weight, power, and cost requirements of these platforms present a great challenge in developing real-time systems. This dissertation explores the development of background modeling algorithms for surveillance on embedded platforms. Our contributions are as follows: - An efficient pixel-based adaptive background model, called multimodal mean, which produces results comparable to the widely used mixture of Gaussians multimodal approach, at a much reduced computational cost and greater control of occluded object persistence. - A novel and efficient chromatic clustering-based background model for embedded vision platforms that leverages the color uniformity of large, permanent background objects to yield significant speedups in execution time. - A multi-scale temporal model for midground analysis which provides a means to "tune-in" to changes in the scene beyond the standard background/foreground framework, based on user-defined temporal constraints. Multimodal mean reduces instruction complexity with the use of fixed integer arithmetic and periodic long-term adaptation that occurs once every d frames. When combined with fixed thresholding, it performs 6.2 times faster than the mixture of Gaussians method while using 18% less storage. Furthermore, fixed thresholding compares favorably to standard deviation thresholding with a percentage difference in error less than five percent when used on scenes with stable lighting conditions and modest multimodal activity. The chromatic clustering-based approach to optimized background modeling takes advantage of the color distributions in large permanent background objects, such as a road, building, or sidewalk, to speedup execution time. It abstracts their colors to a small color palette and suppresses their adaptation during processing. When run on a representative embedded platform it reduces storage usage by 58% and increases runtime execution by 45%. Multiscale temporal modeling for midground analysis presents a unified approach for scene analysis that can be applied to several application domains. It extends scene analysis from the standard background/foreground framework to one that includes a temporal midground object saliency window that is defined by the user. When applied to stationary object detection, the midground model provides accurate results at low sampling frame rates (~ 1 fps) while using only 18 Mbytes of storage and 15 Mops/sec processing throughput.

The performance of intelligent video surveillance systems is always degraded under complicated scenarios, like dynamic changing backgrounds and extremely bad weathers. Dynamic changing backgrounds make the foreground/background segmentation, which is often the first step in vision-based algorithms, become unreliable. Bad weathers, such as foggy scenes, not only degrade the visual quality of the monitoring videos, but also seriously affect the accuracy of the vision-based algorithms. In this thesis, a fast and robust texture-based background modeling technique is first presented for tackling the problem of foreground/background segmentation under dynamic backgrounds. An adaptive multi-modal framework is proposed which uses a novel texture feature known as scale invariant local states (SILS) to model an image pixel. A pattern-less probabilistic measurement (PLPM) is also derived to estimate the probability of a pixel being background from its SILS. Experimental results show that texture-based background modeling is more robust than illumination-based approaches under dynamic backgrounds and lighting changes. Furthermore, the proposed background modeling technique can run much faster than the existing state-of-the-art texture-based method, without sacrificing the output quality. Two fast adaptive defogging techniques, namely 1) foreground decremental preconditioned conjugate gradient (FDPCG), and 2) adaptive guided image filtering are next introduced for removing the foggy effects on video scenes. These two methods allow the estimation of the background transmissions to converge over consecutive video frames, and then background-defog the video sequences using the background transmission map. Results show that foreground/background segmentation can be improved dramatically with such background-defogged video frames. With the reliable foreground/ background segmentation results, the foreground transmissions can then be recovered by the proposed 1) foreground incremental preconditioned conjugate gradient (FIPCG), or 2) on-demand guided image filtering. Experimental results show that the proposed methods can effectively improve the visual quality of surveillance videos under heavy fog and bad weathers. Comparing with state-of-the-art image defogging methods, the proposed methods are shown to be much more efficient. / published_or_final_version / Computer Science / Doctoral / Doctor of Philosophy

Thesis (M.S.)--Rochester Institute of Technology, 2008. / Typescript. Includes bibliographical references (leaves 124-127).

Abstract The last decade was marked by a set of harmful events ranging from economical crises to organized crime, acts of terror and natural catastrophes. This has led to a paradigm transformation concerning security. Millions of surveillance cameras have been deployed, which led to new challenges, as the systems and operations behind those cameras could not cope with the rapid growth in number of video cameras and systems. Looking at today’s control rooms, often hundreds or even thousands of cameras are displayed, overloading security officers with irrelevant information. The purpose of this research was the creation of a novel video surveillance system with automated analysis mechanisms which enable security authorities and their operators to cope with this information flood. By automating the process, video surveillance was transformed into a proactive information system. The progress in technology as well as the ever increasing demand in security have proven to be an enormous driver for security technology research, such as this study. This work shall contribute to the protection of our personal freedom, our lives, our property and our society by aiding the prevention of crime and terrorist attacks that diminish our personal freedom. In this study, design science research methodology was utilized in order to ensure scientific rigor while constructing and evaluating artifacts. The requirements for this research were sought in close cooperation with high-level security authorities and prior research was studied in detail. The created construct, the “Intelligent Video Surveillance System”, is a distributed, highly-scalable software framework, that can function as a basis for any kind of high-performance video surveillance system, from installations focusing on high-availability to flexible cloud-based installation that scale across multiple locations and tens of thousands of cameras. First, in order to provide a strong foundation, a modular, distributed system architecture was created, which was then augmented by a multi-sensor analysis process. Thus, the analysis of data from multiple sources, combining video and other sensors in order to automatically detect critical events, was enabled. Further, an intelligent mobile client, the video surveillance local control, which addressed remote access applications, was created. Finally, a wireless self-contained surveillance system was introduced, a novel smart camera concept that enabled ad hoc and mobile surveillance. The value of the created artifacts was proven by evaluation at two real-world sites: An international airport, which has a large-scale installation with high-security requirements, and a security service provider, offering a multitude of video-based services by operating a video control center with thousands of cameras connected. / Tiivistelmä Viime vuosikymmen tunnetaan vahingollisista tapahtumista alkaen talouskriiseistä ja ulottuen järjestelmälliseen rikollisuuteen, terrori-iskuihin ja luonnonkatastrofeihin. Tämä tilanne on muuttanut suhtautumista turvallisuuteen. Miljoonia valvontakameroita on otettu käyttöön, mikä on johtanut uusiin haasteisiin, koska kameroihin liittyvät järjestelmät ja toiminnot eivät pysty toimimaan yhdessä lukuisien uusien videokameroiden ja järjestelmien kanssa. Nykyajan valvontahuoneissa voidaan nähdä satojen tai tuhansien kameroiden tuottavan kuvaa ja samalla runsaasti tarpeetonta informaatiota turvallisuusvirkailijoiden katsottavaksi. Tämän tutkimuksen tarkoitus oli luoda uusi videovalvontajärjestelmä, jossa on automaattiset analyysimekanismit, jotka mahdollistavat turva-alan toimijoiden ja niiden operaattoreiden suoriutuvan informaatiotulvasta. Automaattisen videovalvontaprosessin avulla videovalvonta muokattiin proaktiiviseksi tietojärjestelmäksi. Teknologian kehitys ja kasvanut turvallisuusvaatimus osoittautuivat olevan merkittävä ajuri turvallisuusteknologian tutkimukselle, kuten tämä tutkimus oli. Tämä tutkimus hyödyttää yksittäisen ihmisen henkilökohtaista vapautta, elämää ja omaisuutta sekä yhteisöä estämällä rikoksia ja terroristihyökkäyksiä. Tässä tutkimuksessa suunnittelutiedettä sovellettiin varmistamaan tieteellinen kurinalaisuus, kun artefakteja luotiin ja arvioitiin. Tutkimuksen vaatimukset perustuivat läheiseen yhteistyöhön korkeatasoisten turva-alan viranomaisten kanssa, ja lisäksi aiempi tutkimus analysoitiin yksityiskohtaisesti. Luotu artefakti - ’älykäs videovalvontajärjestelmä’ - on hajautettu, skaalautuva ohjelmistoviitekehys, joka voi toimia perustana monenlaiselle huipputehokkaalle videovalvontajärjestelmälle alkaen toteutuksista, jotka keskittyvät saatavuuteen, ja päättyen joustaviin pilviperustaisiin toteutuksiin, jotka skaalautuvat useisiin sijainteihin ja kymmeniin tuhansiin kameroihin. Järjestelmän tukevaksi perustaksi luotiin hajautettu järjestelmäarkkitehtuuri, jota laajennettiin monisensorianalyysiprosessilla. Siten mahdollistettiin monista lähteistä peräisin olevan datan analysointi, videokuvan ja muiden sensorien datan yhdistäminen ja automaattinen kriittisten tapahtumien tunnistaminen. Lisäksi tässä työssä luotiin älykäs kännykkäsovellus, videovalvonnan paikallinen kontrolloija, joka ohjaa sovelluksen etäkäyttöä. Viimeksi tuotettiin langaton itsenäinen valvontajärjestelmä – uudenlainen älykäs kamerakonsepti – joka mahdollistaa ad hoc -tyyppisen ja mobiilin valvonnan. Luotujen artefaktien arvo voitiin todentaa arvioimalla ne kahdessa reaalimaailman ympäristössä: kansainvälinen lentokenttä, jonka laajamittaisessa toteutuksessa on korkeat turvavaatimukset, ja turvallisuuspalveluntuottaja, joka tarjoaa moninaisia videopohjaisia palveluja videovalvontakeskuksen avulla käyttäen tuhansia kameroita.

Video surveillance applications have experienced an increase in demand over the last decade. Surveillance systems can easily be found in places such as commercial offices, banks and traffic intersections, parks and recreational areas. Surveillance applications have the potential to be implemented on a WiMAX (Worldwide Interoperability for Microwave Access) network. Moreover, WiMAX devices have been used widely in the market and WiMAX-based video surveillance products have also been available. As a radio technology, WiMAX is a wireless broadband system that offers greater capacity than WiFi networks and wider coverage than cellular networks. The acceptance of WiMAX in the market, the availability of WiMAX products and its technology excellence, contribute to the possibility of implementing it for surveillance application. However, since WiMAX is designed to accommodate various applications with different quality of service (QoS) requirements, dedicated surveillance network implementation of WiMAX may not achieve optimum performance, as all Subscriber Stations (SSs) generate the same QoS requirements. In the medium access (MAC) layer, this thesis proposes a bandwidth allocation scheme that considers the QoS uniformity of the traffic sources. The proposed bandwidth allocation scheme comprises a simplified bandwidth allocation architecture, a packet-aware bandwidth request mechanism and packet-aware scheduling algorithms. The simplified architecture maximizes resources in the Base Station (BS), deactivates unnecessary services and minimizes the processing delay. The proposed bandwidth request mechanism reduces bandwidth grant and transmission delays. The proposed scheduling algorithms prioritize bandwidth granting access to a request that contains important packet(s). The proposed methods in the MAC layer are designed to be applied to existing devices in the market, without the necessity to change hardware. The transport protocol should be able to deliver video with sufficient quality while maintaining low delay connectivity. The proposed transport layer protocol is therefore designed to improve the existing user datagram protocol (UDP) performance by retransmitting packet loss selectively to increase the received video quality, and utilizing MAC support to achieve low delay connectivity. In order to overcome the limitations of the lower layers, this thesis employs a rateless code instead of transport layer redundancy in the application layer. Moreover, this thesis proposes post-decoding error concealment techniques as the last means to overcome packet loss. To evaluate the performances of the proposed methods, simulations are carried out using NS-2 simulator on Linux platform. The proposed methods are compared to existing works to measure their effectiveness. To facilitate the implementation of the transport layer protocols in practical scenarios, UDP packet modification is applied for each transport layer protocol.

Die vorliegende Arbeit basiert auf der Sekundäranalyse eines Datensatzes, welcher im Stadtgebiet Leipzig im Rahmen eines Forschungspraktikums erhoben wurde. Die Autoren folgen modernen sozialwissenschaftlichen Standards, indem sie theoretisch abgeleitete Hypothesen empirisch prüfen. Besonders hervorhebenswert ist dabei, dass sie ein komplexes Mehrebenenmodell zur Erklärung von Einstellungen zum Überwachen und Strafen theoretisch versiert entwickeln. Beides, Überwachen und Strafen, sind hochbrisante aktuelle Themen in der öffentlichen Diskussion. Gerade in diesen sensiblen Sicherheitsbereichen erweist sich die Bevölkerungsmeinung als ausgesprochen instabil. Immer wieder gelingt es über Mediendramaturgien beachtliche Teile der öffentlichen Meinung für die populistische Konstruktion der Erhöhung der Strafhärte zu mobilisieren. Beeinflussungen der Kriminalitätsfurcht erweisen sich dabei als besonders effektiv. Über die affektive Komponente gelingt es nicht selten auch die kognitive, die Urteilsbildung zu beeinflussen. Hinsichtlich der Einstellungen zum Überwachen ist die Situation ebenfalls nicht besser. Zumindest kurzfristig haben demoskopische Untersuchungen gezeigt, dass ein gewisser Teil der Bevölkerung bereit ist, demokratische Rechte zur Erhöhung der öffentlichen Sicherheit wenn nicht aufzugeben so doch zumindest merklich einschränken zu lassen.

Manual video analysis is apparently a tedious task. An efficient solution is of highly importance to automate the process and to assist operators. A major goal of video analysis is understanding and recognizing human activities captured by surveillance cameras, a very challenging problem; the activities can be either individual or interactional among multiple objects. It involves extraction of relevant spatial and temporal information from visual images. Most video analytics systems are constrained by specific environmental situations. Different domains may require different specific knowledge to express characteristics of interesting events. Spatial-temporal trajectories have been utilized to capture motion characteristics of activities. The focus of this dissertation is on how trajectories are utilized in assist in developing video analytic system in the context of surveillance. The research as reported in this dissertation begins real-time highway traffic monitoring and dynamic traffic pattern analysis and in the end generalize the knowledge to event and activity analysis in a broader context. The main contributions are: the use of the graph-theoretic dominant set approach to the classification of traffic trajectories; the ability to first partition the trajectory clusters using entry and exit point awareness to significantly improve the clustering effectiveness and to reduce the computational time and complexity in the on-line processing of new trajectories; A novel tracking method that uses the extended 3-D Hungarian algorithm with a Kalman filter to preserve the smoothness of motion; a novel camera calibration method to determine the second vanishing point with no operator assistance; and a logic reasoning framework together with a new set of context free LLEs which could be utilized across different domains. Additional efforts have been made for three comprehensive surveillance systems together with main contributions mentioned above.

Aujourd’hui, du fait de l’essor des nouvelles technologies, l’image a pris une place prépondérante dans notre société. Si son évolution au sein du grand public est significatif, le droit a lui aussi eu à connaître du développement de cet outil. L’avènement de l’image a touché l’ensemble des branches du droit, y compris le droit pénal et, plus particulièrement, la procédure pénale. Pour autant, dire qu’elle serait désormais présente au niveau de l’ensemble de cette procédure semble presque relever de la méprise, tant les images de cette dernière sont rares. On en vient alors à s’interroger sur la place réellement occupée aujourd’hui par cet outil protéiforme au sein de ce vaste ensemble. L’image englobe en réalité l’ensemble du procès pénal pris dans sa définition la plus large, des premiers stades de l’enquête, jusqu’au prononcé d’un verdict de culpabilité. Plus encore, l’image déborde ce cadre par sa présence en amont de la commission d’une infraction et en aval de l’audience. C’est par la multiplicité de ses formes que l’image se retrouve à toutes les étapes de la procédure. Toutefois, la diversité des usages de l’image n’est pas la seule raison à son utilisation récurrente. À cela s’ajoute sa faculté à servir différentes finalités. Les différentes formes que peut revêtir l’image lui permet de servir des ambitions et des usages différents, parfois complémentaires, d’autres fois très différents. Néanmoins, se dessine une distinction fondamentale entre une image à vocation probatoire et une image servant la bonne administration de la justice. Présente tout au long du procès pénal, il est manifeste que l’image va se voir confrontée à l’ensemble des grands principes qui gouvernent la matière pénale, particulièrement dans son aspect procédural. Dans une période où une réforme globale de la procédure pénale est sans cesse mise en avant, l’étude d’une notion transversale, au service à la fois de la manifestation de la vérité et de la bonne administration de la justice, pourrait s’avérer nécessaire si d’avenir une évolution devait intervenir. L’image servirait alors de fil d’Ariane sur le « chemin menant à la peine » / Today, with the rise of new technologies, the image is playing a leading role in our society. If it has been incrementally utilised by the general public, the field of law has also increasingly developed and put this tool to use. The advent of the image has impacted all areas of law, including criminal law and more specifically criminal proceedings. However, it would be wrong to say that it is currently present throughout the whole procedure, where its appearances are surprisingly rare. This leads us to question the place and role of this multiform tool within this wide system. In reality, the image covers all aspects of the criminal trial in its entirety, from the first stage of the investigation to the delivery of the judgement. And even more, as the image goes beyond this framework, by its upstream presence before a criminal act is committed, and downstream from the hearing. In this way, the image appears in a multiplicity of forms throughout all steps of the procedure. However, the diversity of its uses is not the only explanation to its recurring presence, as it also has the ability to serve different purposes. The different forms of an image allow for a variety of uses and purposes, sometimes complementary, but other times very diverse. We can notice, nevertheless, a fundamental distinction between an image serving probationary purposes and used for the proper administration of justice. Present throughout the criminal proceedings, the image will clearly be confronted with all the main principles that govern criminal matters, particularly in its procedural aspect. At a time when a comprehensive reform of criminal proceedings is constantly put forward, the study of a transversal notion, simultaneously serving the establishment of the truth and the good administration of justice, may be necessary if the system needed to evolve. The image would then be considered as the breadcrumb trail on the « path leading to the sentence »

Accurate motion capture of deformable objects from monocular video sequences is a challenging Computer Vision problem with immense applicability to domains ranging from virtual reality, animation to image guided surgery. Existing dense motion capture methods rely on expensive setups with multiple calibrated cameras,structured light, active markers or prior scene knowledge learned from a large 3D dataset. In this thesis, we propose an end-to-end pipeline for 3D reconstruction of deformable scenes from a monocular video sequence. Our method relies on a two step pipeline in which temporally consistent video registration is followed by a dense non-rigid structure from motion approach. We present a data-driven method to reconstruct non-rigid smooth surfaces densely, using only a single video as input, without the need for any prior models or shape templates. We focus on the well explored low-rank prior for deformable shape reconstruction and propose its convex relaxation to introduce the first variational energy minimisation approach to non-rigid structure from motion. To achieve realistic dense reconstruction of sparsely textured surfaces, we incorporate an edge preserving spatial smoothness prior into the low-rank factorisation framework and design a single variational energy to address the non-rigid structure from motion problem. We also discuss the importance of long-term 2D trajectories for several vision problems and explain how subspace constraints can be used to exploit the redundancy present in the motion of real scenes for dense video registration. To that end, we adopt a variational optimisation approach to design a robust multi-frame video registration algorithm that combines a robust subspace prior with a total variation spatial regulariser. Throughout this thesis, we advocate the use of GPU-portable and scalable energy minimisation algorithms to progress towards practical dense non-rigid 3D motion capture from a single video in the presence of occlusions and illumination changes.