Thesis (Ph. D.)--University of Hong Kong, 2010. / Includes bibliographical references (p. 139-144). Also available in print.

Thesis (M. Phil.)--University of Hong Kong, 2007. / Title proper from title frame. Also available in printed format.

Thesis (Ph. D.)--University of Hong Kong, 2009. / Includes bibliographical references (leaves 70-74). Also available in print.

Thesis (Ph. D.)--Carleton University, 2004. / Includes bibliographical references (p. 148-154). Also available in electronic format on the Internet.

Thesis (M. Phil.)--University of Hong Kong, 2010. / Includes bibliographical references (leaves 54-59). Also available in print.

Thesis (M.C.S.) Carleton University, 2005. / Includes bibliographical references (p. 102-106). Also available in electronic format on the Internet.

From the last decade, studies on human facial emotion recognition revealed that computing models based on regression modelling can produce applicable performance. In this study, an automatic facial expression real-time system was built and tested. The method is used in this study has been used widely in different areas such as Local Binary Pattern method, which has been used in many research projects in machine vision, and the K-Nearest Neighbour algorithm is method utilized for regression modelling. In this study, these two techniques has been used and implemented on the FPGA for the first time, on the side and joined together to great the model in such way to display a continues and automatic emotional state detection model on the monitor. To evaluate the effectiveness of the classifier technique for human emotion recognition from video, the model was designed and tested on MATLAB environment and then MATLAB Simulink environment that is capable of recognizing continuous facial expression in real time with a rate of 1 frame per second and implemented on a desktop PC. It has been evaluated in a testing dataset and the experimental results were promising with the accuracy of 51.28%. The datasets and labels used in this study are made from videos which, recorded twice from 5 participants while watching a video. In order to implement it in real-time in faster frame rate, the facial expression recognition system was built on FPGA. The model was built on Atlys™ Spartan-6 FPGA Development Board. It can perform continuously emotional state recognition in real time at a frame rate of 30 with the accuracy of 47.44%. A graphic user interface was designed to display the participant video in real time and also two dimensional predict labels of the emotion at the same time. This is the first time that automatic emotional state detection has been successfully implemented on FPGA by using LBP and K-NN techniques in such way to display a continues and automatic emotional state detection model on the monitor.

<p>Sewer pipelines are an essential component of wastewater infrastructure and serve as the primary means for transporting wastewater to treatment plants. In the face of increasing demands and declining budgets, municipalities across the US face unprecedented challenges in maintaining current service levels of the 800,000 miles of public sewer pipes. Inadequate maintenance of sewer pipes leads to inflow and infiltration, sanitary sewer overflows, and sinkholes, which threaten human health and are expensive to correct. Accurate condition information from sewers is essential for planning maintenance, repair, and rehabilitation activities and ensuring the longevity of sewer systems. Currently, this information is obtained through visual closed-circuit television (CCTV) inspections and deterioration modeling of sewer pipelines. CCTV inspection facilitates the identification of defects in pipe walls whereas deterioration modeling estimates the remaining service life of pipes based on their current condition. However, both methods have drawbacks that limit their effective usage for sewer condition assessment. For instance, CCTV inspections tend to be labor intensive, costly, and time consuming, with the accuracy of collected data depending on the operator’s experience and skill level. Current deterioration modeling approaches are unable to incorporate spatial information about pipe deterioration, such as the relative locations, densities, and clustering of defects, which play a crucial role in pipe failure. This study attempts to leverage recent advances in deep learning and data mining to address these limitations of CCTV inspection and deterioration modeling and consists of three objectives. </p> <p> </p> <p>The first objective of this study seeks to develop algorithms for automated defect interpretation, to improve the speed and consistency of sewer CCTV inspections. The development, calibration, and testing of the algorithms in this study followed an iterative approach that began with the development of a defect classification system using a 5-layer convolutional neural network (CNN) and evolved into a two-step defect classification and localization framework, which combines a the ResNet34 CNN and Faster R-CNN object detection model. This study also demonstrates the use of a feature visualization technique, called class activation mapping (CAM), as a diagnostic tool to improve the accuracy of CNNs in defect classification tasks—thereby representing a crucial first step in using CNN interpretation techniques to develop improved models for sewer defect identification. </p> <p> </p> <p>Extending upon the development of automated defect interpretation algorithms, the second objective of this study attempts to facilitate autonomous navigation of sewer CCTV robots. To overcome Global Positioning System (GPS) signal unavailability inside underground pipes, this study developed a vision-based algorithm that combines deep learning-based object detection with optical flow for estimating the orientation of sewer CCTV cameras. This algorithm can enable inspection robots to estimate their trajectories and make corrective actions while autonomously traversing pipes. Hence, considered together, the first two objectives of this study pave the way for future inspection technologies that combine automated defect interpretation with autonomous navigation of sewer CCTV robots.</p> <p> </p> <p>The third and final objective of this study seeks to develop a novel methodology that incorporates spatial information about defects (such as their locations, densities, and co-occurrence characteristics) when assessing sewer deterioration. A methodology called Defect Cluster Analysis (DCA) was developed in order to mine sewer inspection reports and identify pipe segments that contain clusters of defects (i.e., multiple defects in proximity). Additionally, an approach to mine co-occurrence characteristics among defects is also introduced (i.e., identification of defects which occur frequently together). Together the two approaches (i.e., DCA and co-occurrence mining) address a key limitation of existing deterioration modeling approaches (i.e., the lack of consideration to spatial information about defects)—thereby leading to the generation of new insights into pipeline rehabilitation decision-making. </p> <p> </p> <p>The algorithms and approaches presented in this dissertation have the potential to improve the speed, accuracy, and consistency of assessing sewer pipeline deterioration, leading to better prioritization strategies for maintenance, repair, and rehabilitation. The automated defect interpretation algorithms proposed in this study can be used to assign the subjective and error-prone task of defect identification to computer processes, thereby enabling human operators to focus on decision-making aspects, such as deciding whether to repair or rehabilitate a pipe. Automated interpretation of sewer CCTV videos could also facilitate re-evaluation of historical sewer inspection videos, which would be infeasible if performed manually. The information gleaned from re-evaluating these videos could generate insights into pipe deterioration, leading to improved deterioration models. The algorithms for autonomous navigation could enable the development of completely autonomous inspection platforms that utilize unmanned aerial vehicles (UAVs) or similar technologies to facilitate rapid assessment of sewers. Furthermore, these technologies could be integrated into wireless sensor networks, paving the way for real-time condition monitoring of sewer infrastructure. The DCA approach could be used as a diagnostic tool to identify specific sections in a pipeline system that have a high propensity for failure due to the existence of multiple defects in proximity. When combined with contextual information (e.g., soil properties, water table levels, and presence of large trees), DCA could provide insights about the likelihood of void formation due to sand infiltration. The DCA approach could also be used to periodically determine how the distribution of defects and their clustering progresses with time and when examined alongside contextual data (e.g., soil properties, water table levels, presence of trees) could reveal trends in pipeline deterioration. </p>

<p> </p> <p>Satellite imagery is becoming increasingly available due to a large number of commercial satellite operators. Many fields use satellite images, including meteorology, forestry, natural disaster analysis, and agriculture. These images can be changed or tampered with image manipulation tools that can cause issues in many applications. Manipulation detection techniques designed for images captured by ``consumer cameras'' tend to fail when used on satellite images. In this thesis we examine methods for detecting splices where an object or area is inserted into a satellite image. Three semi-supervised one-class methods are proposed for the detection and localization of manipulated images. A supervised and supervised fusion approach are also describe to detect spliced forgeries. The semi-supervised one-class method does not require any prior knowledge of the type of manipulations that an adversary could insert in the satellite imagery. First, a new method known as Satellite SVDD (Sat-SVDD) which adapts the Deep SVDD technique is described. Another semi-supervised one-class one-class detection technique based on deep belief networks (DBN) for splicing detection and localization is then discussed. Multiple configurations of the Deep Belief network were compared to other common one-class classification methods. Finally, a semi-supervised one-class technique that uses a Vision Transformer to detect spliced areas within satellite images is introduced. The supervised method does not require prior knowledge of the type of manipulations inserted into the satellite imagery. A supervised method known as Nested Attention U-Net, to detect spliced. The supervised fusion approach known as Sat U-Net fuses the results of two exiting forensic splicing localization methods to increase their overall accuracy. Sat U-Net is a U-Net based architecture exploiting several Transformers to enhance the splicing detection performance. Sat U-Net fuses the outputs of two semi-supervised one-class splicing detection methods, Gated PixelCNN Ensemble and Vision Transformer, to produce a heatmap highlighting the manipulated image region. The supervised fusion approach trained on images from one satellite can be lightly retrained on few images from another satellite to detect spliced regions. In this thesis I introduce five datasets of manipulated satellite images that contain spliced objects. Three of the datasets contains images with spliced objects generated by a generative adversarial network (GAN).</p>

<div><div><div> How can we transform an agricultural vehicle into an autonomous weeding robot? A robot that can run autonomously through a vegetable field, classify multiple types of weeds from real-time video feed and then spray specific herbicides based of previously classified weeds. In this research, we answer some of the theoretical and practical challenges regarding the transformation of an agricultural vehicle into an autonomous weeding robot. How can we transform an agricultural vehicle into an autonomous weeding robot? A robot that can run autonomously through a vegetable field, classify multiple types of weeds from real-time video feed and then spray specific herbicides based of previously classified weeds. In this research, we answer some of the theoretical and practical challenges regarding the transformation of an agricultural vehicle into an autonomous weeding robot. How can we transform an agricultural vehicle into an autonomous weeding robot? A robot that can run autonomously through a vegetable field, classify multiple types of weeds from real-time video feed and then spray specific herbicides based of previously classified weeds. In this research, we answer some of the theoretical and practical challenges regarding the transformation of an agricultural vehicle into an autonomous weeding robot. <br></div></div></div><div><br></div><div> First, we propose a solution for real-time crop row detection from autonomous navigation of agricultural vehicle using domain knowledge and unsupervised machine learning based approach. We implement projective transformation to transform camera image plane to an image plane exactly at the top of the crop rows, so that parallel crop rows remain parallel. Then we use color based segmentation to differentiate crop and weed pixels from background. We implement hierarchical density-based spatial clustering of applications with noise (HDBSCAN) clustering algorithm to differentiate between the crop row clusters and weed clusters. <br></div><div><br></div><div> Finally we use Random sample consensus (RANSAC) for robust line fitting through the detected crop row clusters. We test our algorithm against four different well established methods for crop row detection in-terms of processing time and accuracy. Our proposed method, Clustering Algorithm based RObust LIne Fitting (CAROLIF), shows significantly better accuracy compared to three other methods with average intersect over union (IoU) value of 73%. We also test our algorithm on a video taken from an agricultural vehicle at a corn field in Indiana. CAROLIF shows promising results under lighting variation, vibration and unusual crop-weed growth. <br></div><div><br></div><div><div> Then we propose a robust weed classification system based on convolutional neural network (CNN) and novel decision-level evidence-based multi-sensor fusion algorithm. We create a small dataset of three different weeds (Giant ragweed, Pigweed and Cocklebur) commonly available in corn fields. We train three different CNN architectures on our dataset. Based on classification accuracy and inference time, we choose VGG16 with transfer learning architecture for real-time weed classification.</div><div> </div><div> To create a robust and stable weed classification pipeline, a multi-sensor fusion algorithm based on Dempster-Shafer (DS) evidence theory with a novel entropy function is proposed. The proposed novel entropy function is inspired from Shannon and Deng entropy but it shows better results at understanding uncertainties in certain scenarios, compared to Shannon and Deng entropy, under DS framework. Our proposed algorithm has two advantages compared to other sensor fusion algorithms. First, it can be applied to both space and time domain to fuse results from multiple sensors and create more robust results. Secondly, it can detect which sensor is faulty in the sensors array and compensate for the faulty sensor by giving it lower weight at real-time. Our proposed algorithm calculates the evidence distance from each sensor and determines if one sensor agrees or disagrees with another. Then it rewards the sensors which agrees with another according to their information quality which is calculated using our novel entropy function. The proposed algorithm can combine highly conflicting evidences from multiple sensors and overcomes the limitation of original DS combination rule. After testing our algorithm with real and simulation data, it shows better convergence rate, anti-disturbing ability and transition property compared to other methods available from open literature.</div></div><div><br></div><div><div> Finally, we present a fuzzy-logic based approach to measure the confidence</div><div> of the detected object's bounding-box (BB) position from a CNN detector. The CNN detector gives us the position of BB with percentage accuracy of the object inside the BB on each image plane. But how do we know for sure that the position of the BB is correct? When we are detecting an object using multiple cameras, the position of the BB on the camera image plane may appear in different places based on the detection accuracy and the position of the cameras. But in 3D space, the object is at the exact same position for both cameras. We use this relation between the camera image planes to create a fuzzy-fusion system which will calculate the confidence value of detection. Based on the fuzzy-rules and accuracy of BB position, this system gives us confidence values at three different stages (`Low', `OK' and `High'). This proposed system is successful at giving correct confidence score for scenarios where objects are correctly detected, objects are partially detected and objects are incorrectly detected. </div></div>