[Truncated abstract] Video sequence synchronization is necessary for any computer vision application that integrates data from multiple simultaneously recorded video sequences. With the increased availability of video cameras as either dedicated devices, or as components within digital cameras or mobile phones, a large volume of video data is available as input for a growing range of computer vision applications that process multiple video sequences. To ensure that the output of these applications is correct, accurate video sequence synchronization is essential. Whilst hardware synchronization methods can embed timestamps into each sequence on-the-fly, they require specialized hardware and it is necessary to set up the camera network in advance. On the other hand, computer vision-based software synchronization algorithms can be used to post-process video sequences recorded by cameras that are not networked, such as common consumer hand-held video cameras or cameras embedded in mobile phones, or to synchronize historical videos for which hardware synchronization was not possible. The current state-of-the-art software algorithms vary in their input and output requirements and camera configuration assumptions. ... Next, I describe an approach that synchronizes two video sequences where an object exhibits ballistic motions. Given the epipolar geometry relating the two cameras and the imaged ballistic trajectory of an object, the algorithm uses a novel iterative approach that exploits object motion to rapidly determine pairs of temporally corresponding frames. This algorithm accurately synchronizes videos recorded at different frame rates and takes few iterations to converge to sub-frame accuracy. Whereas the method presented by the first algorithm integrates tracking data from all frames to synchronize the sequences as a whole, this algorithm recovers the synchronization by locating pairs of temporally corresponding frames in each sequence. Finally, I introduce an algorithm for synchronizing two video sequences recorded by stationary cameras with unknown epipolar geometry. This approach is unique in that it recovers both the frame rate ratio and the frame offset of the two sequences by finding matching space-time interest points that represent events in each sequence; the algorithm does not require object tracking. RANSAC-based approaches that take a set of putatively matching interest points and recover either a homography or a fundamental matrix relating a pair of still images are well known. This algorithm extends these techniques using space-time interest points in place of spatial features, and uses nested instances of RANSAC to also recover the frame rate ratio and frame offset of a pair of video sequences. In this thesis, it is demonstrated that each of the above algorithms can accurately recover the frame rate ratio and frame offset of a range of real video sequences. Each algorithm makes a contribution to the body of video sequence synchronization literature, and it is shown that the synchronization problem can be solved using a range of approaches.

[Truncated abstract] This research addresses endemic problems in the fields of computer graphics and simulation such as jittery motion, spatial scalability, rendering problems such as z-buffer tearing, the repeatability of physics dynamics and numerical error in positional systems. Designers of simulation and computer graphics software tend to map real world navigation rules onto the virtual world, expecting to see equivalent virtual behaviour. After all, if computers are programmed to simulate the real world, it is reasonable to expect the virtual behaviour to correspond. However, in computer simulation many behaviours and other computations show measurable problems inconsistent with realworld experience, particularly at large distances from the virtual world origin. Many of these problems, particularly in rendering, can be imperceptible, so users may be oblivious to them, but they are measurable using experimental methods. These effects, generically termed spatial jitter in this thesis, are found in this study to stem from floating point error in positional parameters such as spatial coordinates. This simulation error increases with distance from the coordinate origin and as the simulation progresses through the pipeline. The most common form of simulation error relevant to this study is spatial error which is found by this thesis to not be calculated, as may be expected, using numerical relative error propagation rules but using the rules of geometry. ... The thesis shows that the thinking behind real-world rules, such as for navigation, has to change in order to properly design for optimal fidelity simulation. Origincentric techniques, formulae, terms, architecture and processes are all presented as one holistic solution in the form of an optimised simulation pipeline. The results of analysis, experiments and case studies are used to derive a formula for relative spatial error that accounts for potential pathological cases. A formula for spatial error propagation is then derived by using the new knowledge of spatial error to extend numerical relative error propagation mathematics. Finally, analytical results are developed to provide a general mathematical expression for maximum simulation error and how it varies with distance from the origin and the number of mathematical operations performed. We conclude that the origin centric approach provides a general and optimal solution to spatial jitter. Along with changing the way one thinks about navigation, process guidelines and formulae developed in the study, the approach provides a new paradigm for positional computing. This paradigm can improve many aspects of computer simulation in areas such as entertainment, visualisation for education, industry, science, or training. Examples are: spatial scalability, the accuracy of motion, interaction and rendering; and the consistency and predictability of numerical computation in physics. This research also affords potential cost benefits through simplification of software design and code. These cost benefits come from some core techniques for minimising position dependent error, error propagation and also the simplifications and from new algorithms that flow naturally out of the core solution.

In construction sites, tracking project-related entities such as construction equipment, materials, and personnel provides useful information for productivity measurement, progress monitoring, on-site safety enhancement, and activity sequence analysis. Radio frequency technologies such as Global Positioning Systems (GPS), Radio Frequency Identification (RFID) and Ultra Wide Band (UWB) are commonly used for this purpose. However, on large-scale congested sites, deploying, maintaining and removing such systems can be costly and time-consuming because radio frequency technologies require tagging each entity to track. In addition, privacy issues can arise from tagging construction workers, which often limits the usability of these technologies on construction sites. A vision-based approach that can track moving objects in camera views can resolve these problems. The purpose of this research is to investigate the vision-based tracking system that holds promise to overcome the limitations of existing radio frequency technologies for large-scale, congested sites. The proposed method use videos from static cameras. Stereo camera system is employed for tracking of construction entities in 3D. Once the cameras are fixed on the site, intrinsic and extrinsic camera parameters are discovered through camera calibration. The method automatically detects and tracks interested objects such as workers and equipment in each camera view, which generates 2D pixel coordinates of tracked objects. The 2D pixel coordinates are converted to 3D real-world coordinates based on calibration. The method proposed in this research was implemented in .NET Framework 4.0 environment, and tested on the real videos of construction sites. The test results indicated that the methods could locate construction entities with accuracy comparable to GPS.

No description available.

Whenever a new sensor or system comes online, engineers and analysts responsible for processing the measured data turn first to methods that are tried and true on existing systems. This is a natural, if not wholly logical approach, and is exactly what has happened in the advent of hyperspectral imagery (HSI) exploitation. However, a closer look at the assumptions made by the approaches published in the literature has not been undertaken. This thesis analyzes three key aspects of HSI exploitation: statistical data modeling, covariance estimation from training data, and dimension reduction. These items are part of standard processing schemes, and it is worthwhile to understand and quantify the impact that various assumptions for these items have on target detectability and detection statistics. First, the accuracy and applicability of the standard Gaussian (i.e., Normal) model is evaluated, and it is shown that the elliptically contoured t-distribution (EC-t) sometimes offers a better statistical model for HSI data. A finite mixture approach for EC-t is developed in which all parameters are estimated simultaneously without a priori information. Then the effects of making a poor covariance estimate are shown by including target samples in the training data. Multiple test cases with ground targets are explored. They show that the magnitude of the deleterious effect of covariance contamination on detection statistics depends on algorithm type and target signal characteristics. Next, the two most widely used dimension reduction approaches are tested. It is demonstrated that, in many cases, significant dimension reduction can be achieved with only a minor loss in detection performance. In addition, a concise development of key HSI detection algorithms is presented, and the state-of-the-art in adaptive detectors is benchmarked for land mine targets. Methods for detection and identification of airborne gases using hyperspectral imagery are discussed, and this application is highlighted as an excellent opportunity for future work.

This thesis presents methods and a system for video search over the internet or the intranet. The objective is to design a real time and automated video clustering and search system that provides users of the search engine the most relevant videos available that are responsive to a query at a particular moment in time, and supplementary information that may also be useful. The thesis highlights methods to mitigate the effect of the semantic gap faced by current content based video search approaches. A context-sensitive video ranking scheme is used, wherein the context is generated in an automated manner.

This thesis aims to contribute to the area of visual tracking, which is the process of identifying an object of interest through a sequence of successive images. The thesis explores kernel-based statistical methods, which map the data to a higher dimensional space. A pre-image framework is provided to find the mapping from the embedding space to the input space for several manifold learning and dimensional learning algorithms. Two algorithms are developed for visual tracking that are robust to noise and occlusions. In the first algorithm, a kernel PCA-based eigenspace representation is used. The de-noising and clustering capabilities of the kernel PCA procedure lead to a robust algorithm. This framework is extended to incorporate the background information in an energy based formulation, which is minimized using graph cut and to track multiple objects using a single learned model. In the second method, a robust density comparison framework is developed that is applied to visual tracking, where an object is tracked by minimizing the distance between a model distribution and given candidate distributions. The superior performance of kernel-based algorithms comes at a price of increased storage and computational requirements. A novel method is developed that takes advantage of the universal approximation capabilities of generalized radial basis function neural networks to reduce the computational and storage requirements for kernel-based methods.

Video coding standards such as H.264 AVC (Advanced Video Coding) rely on predictive coding to achieve high compression efficiency. Predictive coding consists of predicting each frame using preceding frames. However, predictive coding incurs a cost when transmitting over unreliable networks: frames are no longer independent and the loss of data in one frame may affect future frames. In this thesis, we study the effectiveness of Flexible Macroblock Ordering (FMO) in mitigating the effect of errors on the decoded video and propose solutions to improve the error concealment on H.264 decoders. After introducing the subject matter, we present the H.264 profiles and briefly determine their intended applications. Then we describe FMO and justify its usefulness for transmission over lossy networks. More precisely, we study the cost in terms of overheads and the improvements it offers in visual quality for damaged video frames. The unavailability of FMO in most H.264 profiles leads us to design a lossless FMO removal scheme, which allows the playback of FMO-encoded video on non FMO-compliant decoders. Then, we describe the process of removing the FMO structure but also underline some limitations that prevent the application of the scheme. Finally, we assess the induced overheads and propose a model to predict these overheads when FMO Type 1 is employed. Eventually, we develop a new error concealment method to enhance video quality without relying on channel feedback. This method is shown to be superior to existing methods, including those from the JM reference software and can be applied to compensate for the limitations of the scheme proposed FMO-removal scheme. After introducing our new method, we evaluate its performance and compare it to some classical algorithms.

The field of computer vision focuses on the goal of developing techniques to exploit and extract information from underlying data that may represent images or other multidimensional data. In particular, two well-studied problems in computer vision are the fundamental tasks of 2D image segmentation and 3D pose estimation from a 2D scene. In this thesis, we first introduce two novel methodologies that attempt to independently solve 2D image segmentation and 3D pose estimation separately. Then, by leveraging the advantages of certain techniques from each problem, we couple both tasks in a variational and non-rigid manner through a single energy functional. Thus, the three theoretical components and contributions of this thesis are as follows: Firstly, a new distribution metric for 2D image segmentation is introduced. This is employed within the geometric active contour (GAC) framework. Secondly, a novel particle filtering approach is proposed for the problem of estimating the pose of two point sets that differ by a rigid body transformation. Thirdly, the two techniques of image segmentation and pose estimation are coupled in a single energy functional for a class of 3D rigid objects. After laying the groundwork and presenting these contributions, we then turn to their applicability to real world problems such as visual tracking. In particular, we present an example where we develop a novel tracking scheme for 3-D Laser RADAR imagery. However, we should mention that the proposed contributions are solutions for general imaging problems and therefore can be applied to medical imaging problems such as extracting the prostate from MRI imagery

As more and more multimedia applications are integrated in mobile devices, a significant amount of energy is devoted to digital signal processing (DSP). Thus, reducing energy consumption for DSP systems has become an important design goal for battery operated mobile devices. Since supply voltage scaling is one of the most effective methods to reduce power/energy consumption, this study examines aggressive voltage scaling to achieve significant energy savings by allowing some output quality degradation for error tolerant image processing system. The objective of proposed research is to explore ultra-low energy image processing system design methodologies based on efficient accuracy (quality)-energy tradeoffs. This dissertation presents several new analyses and techniques to achieve significant energy savings without noticeable quality degradation under aggressive voltage scaling. In the first, this work starts from accurate error analysis and a model based on input sequence dependent delay estimation. Based on the analysis, we explain the dependence of voltage scalability on input image types, which may be used for input dependent adaptive control for optimal accuracy-energy tradeoffs. In addition, this work includes the system-level analysis of the impact of aggressive voltage scaling on overall energy consumption and a low-cost technique to reduce overall energy consumption. Lastly, this research exploits an error concealment technique to improve the efficiency of accuracy-energy tradeoffs. For an image compression system, the technique minimizes the impact of delay errors on output quality while allowing very low voltage operations for significant energy reduction.