No description available.

No description available.

No description available.

Matting, which aims to separate the foreground object from the background of an image, is an important problem in computer vision. Most existing methods rely on auxiliary information such as trimaps or scibbles to alleviate the difficulty arising from the underdetermined nature of the matting problem. However, such methods tend to be sensitive to the quality of auxiliary information, and are unsuitable for real-time deployment. In this paper, we propose a novel Attention-based Multi-scale Matting Network (AMMNet), which can estimate the alpha matte from a given RGB image without resorting to any auxiliary information. The proposed AMMNet consists of three (sub-)networks: 1) a multi-scale neural network designed to provide the semantic information of the foreground object, 2) a Unet-like network for attention mask generation, and 3) a Convolutional Neural Network (CNN) customized to integrate high- and low-level features extracted by the first two (sub-)networks. The AMMNet is generic in nature and can be trained end-to-end in a straightforward manner. The experimental results indicate that the performance of AMMNet is competitive against the state-of-the-art matting methods, which either require additional side information or are tailored to images with a specific type of content (e.g., portrait). / Thesis / Master of Applied Science (MASc)

One of the major problems in computer vision involves dealing with uncertain information. Occlusion, dissimilar views, insufficient illumination, insufficient resolution, and degradation give rise to imprecise data. At the same time, incomplete or local knowledge of the scene gives rise to imprecise interpretation rules. Uncertainty arises at different processing levels of computer vision either because of the imprecise data or because of the imprecise interpretation rules. It is natural to build computer vision systems that incorporate uncertainty reasoning. The Dempster-Shafer (D-S) theory of evidence is appealing for coping with uncertainty hierarchically. However, very little work has been done to apply D-S theory to practical vision systems because some important problems are yet to be resolved. / Ph. D.

This metaverse and computer vision-powered application is designed to serve people with low vision or a visual impairment, ranging from adults to old age. Specifically, we hope to improve the situational awareness of users in a scene by narrating the visual content from their point of view. The user would be able to understand the information through auditory channels as the system would narrate the scene's description using speech technology. This could increase the accessibility of visual-spatial information for the users in a metaverse and later in the physical world. This solution is designed and developed considering the hypothesis that if we enable the narration of a scene's visual content, we can increase the understanding and access to that scene. This study paves the way for VR technology to be used as a training and exploration tool not limited to blind people in generic environments, but applicable to specific domains such as military, healthcare, or architecture and planning. We have run a user study and evaluated our hypothesis about which set of algorithms will perform better for a specific category of tasks - like search or survey - and evaluated the narration algorithms by the user's ratings of naturalness, correctness and satisfaction. The tasks and algorithms have been discussed in detail in the chapters of this thesis. / Master of Science / The solution is built using an object detection algorithm and virtual environments which run on the web browser using X3DOM. The solution would help improve situational awareness for normal people as well as for low vision individuals through speech. On a broader scale, we seek to contribute to accessibility solutions. We have designed four algorithms which will help user to understand the scene information through auditory channels as the system would narrate the scene's description using speech technology. The idea would increase the accessibility of visual-spatial information for the users in a metaverse and later in the physical world.

The demand and interest in serving Computer Vision applications at the Edge, where Edge Devices generate vast quantities of data, clashes with the reality that many Devices are largely unable to process their data in real time. While computational offloading, not to the Cloud but to nearby Edge Nodes, offers convenient acceleration for these applications, such systems are not without their constraints. As Edge networks may be unreliable or wireless, offloading quality is sensitive to communication bottlenecks. Unlike seemingly unlimited Cloud resources, an Edge Node, serving multiple clients, may incur delays due to resource contention. This project describes relevant Computer Vision workloads and how an effective offloading framework must adapt to the constraints that impact the Quality of Service yet have not been effectively nor properly addressed by previous literature. We design an offloading controller, based on closed-loop control theory, that enables Devices to maximize their throughput by appropriately offloading under variable conditions. This approach ensures a Device can utilize the maximum available offloading bandwidth. Finally, we constructed a realistic testbed and conducted measurements to demonstrate the superiority of our offloading controller over previous techniques. / Master of Science / Devices like security cameras and some Internet of Things gadgets produce valuable real-time video for AI applications. A field within AI research called Computer Vision aims to use this visual data to compute a variety of useful workloads in a way that mimics the human visual system. However, many workloads, such as classifying objects displayed in a video, have large computational demands, especially when we want to keep up with the frame rate of a real-time video. Unfortunately, these devices, called Edge Devices because they are located far from Cloud datacenters at the edge of the network, are notoriously weak for Computer Vision algorithms, and, if running on a battery, will drain it quickly. In order to keep up, we can offload the computation of these algorithms to nearby servers, but we need to keep in mind that the bandwidth of the network might be variable and that too many clients connected to a single server will overload it. A slow network or an overloaded server will incur delays which slow processing throughput. This project describes relevant Computer Vision workloads and how an effective offloading framework that effectively adapts to these constraints has not yet been addressed by previous literature. We designed an offloading controller that measures feedback from the system and adapts how a Device offloads computation, in order to achieve the best possible throughput despite variable conditions. Finally, we constructed a realistic testbed and conducted measurements to demonstrate the superiority of our offloading controller over previous techniques.

The extraction and analysis of human gait characteristics using image sequences and the subsequent classification of these characteristics are currently an intense area of research. Recently, the focus of this research area has turned to the realm of computer vision as an unobtrusive way of performing this analysis. With such systems becoming more common, a gait analysis system that will quickly and accurately determine if a subject is walking normally becomes more valuable. Such a system could be used as a preprocessing step in a more sophisticated gait analysis system or could be used for rehabilitation purposes. In this thesis a system is proposed which utilizes a novel fusion of spatial computer vision operations as well as motion in order to accurately and efficiently determine if a subject moving through a scene is walking normally or abnormally. Specifically this system will yield a classification of the type of motion being observed, whether it is a human walking normally or some other kind of motion taking place within the frame. Experimental results will show that the system provides accurate detection of normal walking and can distinguish abnormalities as subtle as limping or walking with a straight leg reliably. / Master of Science

This thesis describes a mosaicking system designed to generate image mosaics that facilitate size estimation of 3-dimensional objects by improving data obtained with a multi-sensor video camera. The multi-sensor camera is equipped with a pulse laser-rangefinder and internally mounted inclinometers that measure instrument orientation about three axes. Using orientation data and video data, mosaics are constructed to reduce orientation data errors by augmenting orientation data with image information. Mosaicking is modeled as a 7-step refinement process: 1) an initial mosaic is constructed using orientation information obtained from the camera's inclinometers; 2) mosaics are refined by using coarse-to-fine processing to minimize an energy metric and, consequently, align overlapping video frames; 3) pair-wise mosaicking errors are detected, and removed, using an energy-based confidence metric; 4) mosaic accuracy is refined via color analysis; 5) mosaic accuracy is refined by estimating an affine transformation to align overlapping frames; 6) affine transformation approximations between overlapping video frames are used to reduce image noise through super-resolution; 7) original orientation data are corrected given the refined orientations of images within the mosaic. The mosaicking system has been tested using objects of known size and orientation accuracy has been improved by 86% for these cases. / Master of Science

This work focuses on automated classification of railway ties based on their condition using aerial imagery. Four approaches are explored and compared to achieve this goal - handcrafted features, HOG features, transfer learning and proposed CNN architecture. Mean test accuracy per class and Quadratic Weighted Kappa score are used as performance metrics, particularly suited for the ordered classification in this work. Transfer learning approach outperforms the handcrafted features and HOG features by a significant margin. The proposed CNN architecture caters to the unique nature of the railway tie images and their defects. The performance of this approach is superior to the handcrafted and HOG features. It also shows a significant reduction in the number of parameters as compared to the transfer learning approach. Data augmentation boosts the performance of all approaches. The problem of label noise is also analyzed. The techniques proposed in this work will help in reducing the time, cost and dependency on experts involved in traditional railway tie inspections and will facilitate efficient documentation and planning for maintenance of railway ties. / Master of Science / Railway tracks and their components need to be frequently inspected for defects or design non-compliances. Manual inspections involve long hours, high cost and dependency on the availability of experts. Previous efforts to automate the inspection of the railway track inspections are focused towards either other components of the railway track or towards using custom designed ground vehicles. This work presents four approaches to automate the inspection of wooden railway ties by categorizing them into one of three categories based on their condition. Images of the track are taken by an aerial vehicle, in which the track is left untouched. The techniques of computer vision and machine learning used in this work outperform the baselines. The efforts are directed towards making the algorithm learn from the labeled data. The labeled data is also artificially enlarged and enriched, which boosts the performance of the classifiers. The performance metrics used to evaluate the classification approaches are particularly suited for the task at hand. The problem of inconsistency in labeling between two human labelers is analyzed. Potential further directions for research are also identified.