Probabilistic behavior in perceiving the environment and take critical decisions have an inevitable role in human life. A decision is concerned with a choice among the available alternatives and is always subject to unknown elements concerning the future. The lack of complete data, insufficient scientific, behavioral, and industry development and of course defects in measurement methods, affect the reliability of an action’s outcome. Thus, having a proper estimation of this reliability or uncertainty could be very advantageous particularly when an individual or generally a subject is faced with a high risk. With the fact that there are always uncertainty elements whose values are unknown and these enter into a processes through multiple sources, it has been a primary challenge to design an efficient representation of confidence objectively. With the aim of addressing this problem, a variety of researches have been conducted to introduce frameworks in metrology of uncertainty quantification that are comprehensive enough and have transferability into different areas. Moreover, it’s also a challenging task to define a proper index that reflects more aspects of the problem and measurement process. With significant advances in Artificial Intelligence in the past decade, one of the key elements, in order to ease human life by giving more control to machines, is to heed the uncertainty estimation for a prediction. With a focus on measurement aspects, this thesis attends to demonstrate how a different measurement index affects the quality of evaluated predictive uncertainty of neural networks. Finally, we propose a novel index that shows uncertainty values with the same or higher quality than existing methods which emphasizes the benefits of having a proper measurement index in managing the risk of the outcome from a classification model.

Nowadays, researchers have shown convolutional neural network (CNN) can achieve an improved performance in multi-object tracking (MOT) by performing detection and re-identification (ReID) simultaneously. Many models have been created to overcome challenges and bring the state-of-the-art performance to a new level. However, due to the fact the CNN models only utilize feature from a local region, the potential of the model has not been fully utilized. The long range dependencies in spatial domain are usually difficult for a network to capture. Hence, how to obtain such dependencies has become the new focus in MOT field. One approach is to adopt the self-attention mechanism named transformer. Since it was successfully transferred from natural language processing to computer vision, many recent works have implemented it to their trackers. With the introduce of global information, the trackers become more robust and stable. There are also traditional methods which are re-designed in the manner of CNN and achieve satisfying performance such as optical flow. It can generate a correlated relation between feature maps and also obtain non-local information. However, the introduces of these mechanism usually causes a significant surge in computational power and memory. They also requires huge amount of epochs to train thus the training time is largely increased. To solve this issue, we propose a new method to gather non-local information based on the existing self-attention methods, we named it dual attention with regional-representation, which significantly reduces the training time as well as the inference time, but only causes a small increase in computational memory and are able to run with a reasonable speed. Our experiments shows this module can help the ReID be more stable to improve the performance in different tasks. / Thesis / Master of Applied Science (MASc)

Software bugs prediction is one of the most active research areas in the software engineering community. The process of testing and debugging code proves to be costly during the software development life cycle. Software metrics measure the quality of source code to identify software bugs and vulnerabilities. Traceable code patterns are able to de- scribe code at a finer granularity level to measure quality. Micro patterns will be used in this research to mechanically describe java code at the class level. Machine learning has also been introduced for bug prediction to localize source code for testing and debugging. Deep Learning is a branch of Machine Learning that is relatively new. This research looks to improve the prediction of software bugs by utilizing micro patterns with deep learning techniques. Software bug prediction at a finer granularity level will enable developers to localize code to test and debug during the development process.

Learning-based multi-view stereo (MVS) has shown promising results in the domain of general 3D reconstruction. However, no work before this thesis has applied learning-based MVS to urban 3D reconstruction from satellite images. In this thesis, learning-based MVS is used to infer depth maps from satellite images. Models are trained on both synthetic and real satellite images from Las Vegas with ground truth data from a high-resolution aerial-based 3D model. This thesis also evaluates different methods for reconstructing digital surface models (DSM) and compares them to existing satellite-based 3D models at Maxar Technologies. The DSMs are created by either post-processing point clouds obtained from predicted depth maps or by an end-to-end approach where the depth map for an orthographic satellite image is predicted.  This thesis concludes that learning-based MVS can be used to predict accurate depth maps. Models trained on synthetic data yielded relatively good results, but not nearly as good as for models trained on real satellite images. The trained models also generalize relatively well to cities not present in training. This thesis also concludes that the reconstructed DSMs achieve better quantitative results than the existing 3D model in Las Vegas and similar results for the test sets from other cities. Compared to ground truth, the best-performing method achieved an L1 and L2 error of 14 % and 29 % lower than Maxar's current 3D model, respectively. The method that uses a point cloud as an intermediate step achieves better quantitative results compared to the end-to-end system. Very promising qualitative results are achieved with the proposed methods, especially when utilizing an end-to-end approach.

No description available.

Single Particle Tracking (SPT) plays an important role in the study of physical and dynamic properties of biomolecules moving in their native environment. To date, many algorithms have been developed for localization and parameter estimation in SPT. Though the performance of these methods is good when the signal level is high and the motion model simple, they begin to fail as the signal level decreases or model complexity increases. In addition, the inputs to the SPT algorithms are sequences of images that are cropped from a large data set and that focus on a single particle. This motivates us to seek machine learning tools to deal with that initial step of extracting data from larger images containing multiple particles. This thesis makes contributions to both data extraction question and to the problem of state and parameter estimation. First, we build upon the Expectation Maximization (EM) algorithm to create a generic framework for joint localization refinement and parameter estimation in SPT. Under the EM-based scheme, two representative methods are considered for generating the filtered and smoothed distributions needed by EM: Sequential Monte Carlo - Expectation Maximization (SMC-EM), and Unscented - Expectation Maximization (U-EM). The selection of filtering and smoothing algorithms is very flexible so long as they provide the necessary distributions for EM. The versatility and reliability of EM based framework have been validated via data-intensive modeling and simulation where we considered a variety of influential factors, such as a wide range of {\color{red}Signal-to-background ratios (SBRs)}, diffusion speeds, motion blur, camera types, image length, etc. Meanwhile, under the EM-based scheme, we make an effort to improve the overall computational efficiency by simplifying the mathematical expression of models, replacing filtering/smoothing algorithms with more efficient ones {\color{purple} (trading some accuracy for reduced computation time)}, and using parallel computation and other computing techniques. In terms of localization refinement and parameter estimation in SPT, we also conduct an overall quantitative comparison among EM based methods and standard two-step methods. Regarding the U-EM, we conduct transformation methods to make it adapted to the nonlinearities and complexities of measurement model. We also extended the application of U-EM to more complicated SPT scenarios, including time-varying parameters and additional observation models that are relevant to the biophysical setting. The second area of contribution is in the particle detection and extraction problem to create data to feed into the EM-based approaches. Here we build Particle Identification Networks (PINs) covering three different network architectures. The first, \PINCNN{}, is based on a standard Convolutional Neural Network (CNN) structure that has previously been successfully applied in particle detection and localization. The second, \PINRES, uses a Residual Neural Network (ResNet) architecture that is significantly deeper than the CNN while the third, \PINFPN{}, is based on a more advanced Feature Pyramid Network (FPN) that can take advantage of multi-scale information in an image. All networks are trained using the same collection of simulated data created with a range of SBRs and fluorescence emitter densities, as well as with three different Point Spread Functions (PSFs): a standard Born-Wolf model, a model for astigmatic imaging to allow localization in three dimensions, and a model of the Double-Helix engineered PSF. All PINs are evaluated and compared through data-intensive simulation and experiments under a variety of settings. In the final contribution, we link all above together to create an algorithm that takes in raw camera data and produces trajectories and parameter estimates for multiple particles in an image sequence.

Matting, which aims to separate the foreground object from the background of an image, is an important problem in computer vision. Most existing methods rely on auxiliary information such as trimaps or scibbles to alleviate the difficulty arising from the underdetermined nature of the matting problem. However, such methods tend to be sensitive to the quality of auxiliary information, and are unsuitable for real-time deployment. In this paper, we propose a novel Attention-based Multi-scale Matting Network (AMMNet), which can estimate the alpha matte from a given RGB image without resorting to any auxiliary information. The proposed AMMNet consists of three (sub-)networks: 1) a multi-scale neural network designed to provide the semantic information of the foreground object, 2) a Unet-like network for attention mask generation, and 3) a Convolutional Neural Network (CNN) customized to integrate high- and low-level features extracted by the first two (sub-)networks. The AMMNet is generic in nature and can be trained end-to-end in a straightforward manner. The experimental results indicate that the performance of AMMNet is competitive against the state-of-the-art matting methods, which either require additional side information or are tailored to images with a specific type of content (e.g., portrait). / Thesis / Master of Applied Science (MASc)

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging modality that measures the hemodynamic responses to brain activation. With its cost-effectiveness and portability, fNIRS can be utilized to measure brain signals in the everyday world. However, factors such as blood pressure, cardiac rhythms, and motion can obscure the hemodynamic response function (HRF) obtained in fNIRS data. Motion, in particular, poses a significant challenge in obtaining the HRF for measurements conducted in everyday world activities when the subject is free to move. To address this, the General Linear Model (GLM) with temporally embedded Canonical Correlation Analysis (tCCA) has been shown to be effective in extracting the HRF by reducing motion and other systemic interferences. Recently, deep learning methods have also demonstrated its potential for time-series data analysis. The objective of this project is to evaluate the effectiveness of a novel transformer-based deep learning approach in comparison to the tradition method of GLM with tCCA

Interactive deep learning frameworks are crucial for effectively exploring and analyzing complex image datasets in visual analytics. However, existing approaches often face challenges related to inference accuracy and adaptability. To address these issues, we propose ImageSI, a framework integrating deep learning models with semantic interaction techniques for interactive image data analysis. Unlike traditional methods, ImageSI directly incorporates user feedback into the image model, updating underlying embeddings through customized loss functions, thereby enhancing the performance of dimension reduction tasks. We introduce three variations of ImageSI, ImageSI$_{text{MDS}^{-1}}$, prioritizing explicit pairwise relationships from user interaction, and ImageSI$_{text{DRTriplet}}$ and ImageSI$_{text{PHTriplet}}$, emphasizing clustering by defining groups of images based on user input. Through usage scenarios and quantitative analyses centered on algorithms, we demonstrate the superior performance of ImageSI$_{text{DRTriplet}}$ and ImageSI$_{text{MDS}^{-1}}$ in terms of inference accuracy and interaction efficiency. Moreover, ImageSI$_{text{PHTriplet}}$ shows competitive results. The baseline model, WMDS$^{-1}$, generally exhibits lower performance metrics. / Master of Science / Interactive deep learning frameworks are crucial for effectively exploring and analyzing complex image datasets in visual analytics. However, existing approaches often face challenges related to inference accuracy and adaptability. To address these issues, we propose ImageSI, a framework integrating deep learning models with semantic interaction techniques for interactive image data analysis. Unlike traditional methods, ImageSI directly incorporates user feedback into the image model, updating underlying embeddings through customized loss functions, thereby enhancing the performance of dimension reduction tasks. We introduce three variations of ImageSI, ImageSI$_{text{MDS}^{-1}}$, prioritizing explicit pairwise relationships from user interaction, and ImageSI$_{text{DRTriplet}}$ and ImageSI$_{text{PHTriplet}}$, emphasizing clustering by defining groups of images based on user input. Through usage scenarios and quantitative analyses centered on algorithms, we demonstrate the superior performance of ImageSI$_{text{DRTriplet}}$ and ImageSI$_{text{MDS}^{-1}}$ in terms of inference accuracy and interaction efficiency. Moreover, ImageSI$_{text{PHTriplet}}$ shows competitive results. The baseline model, WMDS$^{-1}$, generally exhibits lower performance metrics.

The abilities of powerful regression and multi-type data processing allow deep learning to effectively and accurately complete multi-tasks, which is the need of civil engineering. More cases showed that deep learning has become a greatly powerful and increasingly popular tool for civil engineering. Based on these, this dissertation developed deep learning studies for the condition assessment of civil infrastructure and materials. This dissertation included five main works: (1) Deep learning and infrared thermography for asphalt pavement crack severity classification. This work focused on longitudinal or transverse cracking. This work first built a dataset with four severity levels (no, low-severity, medium-severity, and high-severity) and three image types (visible, infrared, and fusion). Then this work applied the convolutional neural network (CNN) to classify the crack severity based on two strategies deep learning from scratch and transfer learning). This work also investigated the effect of image types on the accuracy of these two strategies and on the classification of different severity levels. (2) Asphalt pavement crack detection based on convolutional neural network and infrared thermography. This work first built an open dataset with three image types (visible, infrared, and fusion) and different conditions (single, multi, thin, and thick cracks; clean, rough, light, and dark backgrounds) and periods (morning, noon, and dusk). Then this work evaluated the performance of the CNN model based on the accuracy and complexity (computational and model). (3) An artificial neural network model on tensile behavior of hybrid steel-PVA fiber reinforced concrete containing fly ash and slag powder. This work considered a total of 23 factors for predicting the tensile behavior of hybrid fiber reinforced concrete (HFRC), including fibers' characteristics, mechanical properties of plain concrete, and concrete composition. Then this work compared the performance of the artificial neural network (ANN) method and the traditional equation-based method in terms of predicting the tensile stress, tensile strength, and strain corresponding to tensile strength. (4) Deep transfer learning-based vehicle classification by asphalt pavement vibration. This work first applied the pavement vibration IoT monitoring system to collect raw vibration signals and performed the wavelet transform to obtain denoised vibration signals. Then this work represented the vibration signals in two different ways, including the time-domain graph and the time-frequency graph. Finally, this work proposed two deep transfer learning-based vehicle classification methods according to these two representations of vibration signals. (5) Physical-informed long short-term memory (PI-LSTM) network for data-driven structural response modeling. This work first applied the single-degree-of-freedom (SDOF) system to investigate the performance of the proposed PI-LSTM network compared with the existing methods. Then this work further investigated and validated the proposed PI-LSTM network in terms of the experimental results of one six-story building and the numerical simulation results of another six-story building. / Doctor of Philosophy / With the development of technologies, deep learning has been applied to numerous fields to improve accuracy and efficiency. More work shows that deep learning has become a greatly powerful and increasingly popular tool for civil engineering. Since civil infrastructure and materials play a dominant role in civil engineering, this dissertation applied deep learning to the condition assessment of civil infrastructure and materials. Deep learning methods were applied to detect cracks in asphalt pavements. The mechanical properties of fiber reinforced concrete were investigated by deep learning methods. Based on the asphalt pavement vibration, the type of vehicles was classified by deep learning methods. Deep learning methods were also used to investigate the structural response.