Classification is an important task in the field of machine learning and when classifiers are trained on images, a variety of problems can surface during inference. 1) Recent trends of using convolutional neural networks (CNNs) for various machine learning tasks has borne many successes and CNNs are surprisingly expressive in their learning ability due to a large number of parameters and numerous stacked layers in the CNNs. This increased model complexity also increases the risk of overfitting to the training data. Increasing the size of the training data using synthetic or artificial means (data augmentation) helps CNNs learn better by reducing the amount of over-fitting and producing a regularization effect to improve generalization of the learned model. 2) CNNs have proven to be very good classifiers and generally localize objects well; however, the loss functions typically used to train classification CNNs do not penalize inability to localize an object, nor do they take into account an object's relative size in the given image when producing confidence measures. 3) Convolutional neural networks always output in the space of the learnt classes with high confidence while predicting the class of a given image regardless of what the image consists of. For example an ImageNet-1K trained CNN can not say if the given image has no objects that it was trained on if it is provided with an image of a dinosaur (not an ImageNet category) or if the image has the main object cut out of it (context only). We approach these three different problems using bounding box information and learning to produce high entropy predictions on out of distribution classes. To address the first problem, we propose a novel regularization method called CopyPaste. The idea behind our approach is that images from the same class share similar context and can be 'mixed' together without affecting the labels. We use bounding box annotations that are available for a subset of ImageNet images. We consistently outperform the standard baseline and explore the idea of combining our approach with other recent regularization methods as well. We show consistent performance gains on PASCAL VOC07, MS-COCO and ImageNet datasets. For the second problem we employ objectness measures to learn meaningful CNN predictions. Objectness is a measure of likelihood of an object from any class being present in a given image. We present a novel approach to object localization that combines the ideas of objectness and label smoothing during training. Unlike previous methods, we compute a smoothing factor that is adaptive based on relative object size within an image. We present extensive results using ImageNet and OpenImages to demonstrate that CNNs trained using adaptive label smoothing are much less likely to be overconfident in their predictions, as compared to CNNs trained using hard targets. We train CNNs using objectness computed from bounding box annotations that are available for the ImageNet dataset and the OpenImages dataset. We perform extensive experiments with the aim of improving the ability of a classification CNN to learn better localizable features and show object detection performance improvements, calibration and classification performance on standard datasets. We also show qualitative results using class activation maps to illustrate the improvements. Lastly, we extend the second approach to train CNNs with images belonging to out of distribution and context using a uniform distribution of probability over the set of target classes for such images. This is a novel way to use uniform smooth labels as it allows the model to learn better confidence bounds. We sample 1000 classes (mutually exclusive to the 1000 classes in ImageNet-1K) from the larger ImageNet dataset comprising about 22K classes. We compare our approach with standard baselines and provide entropy and confidence plots for in distribution and out of distribution validation sets. / Doctor of Philosophy / Categorization is an important task in everyday life. Humans can perform the task of classifying objects effortlessly in pictures. Machines can also be trained to classify objects in images. With the tremendous growth in the area of artificial intelligence, machines have surpassed human performance for some tasks. However, there are plenty of challenges for artificial neural networks. Convolutional Neural Networks (CNNs) are a type of artificial neural networks. 1) Sometimes, CNNs simply memorize the samples provided during training and fail to work well with images that are slightly different from the training samples. 2) CNNs have proven to be very good classifiers and generally localize objects well; however, the objective functions typically used to train classification CNNs do not penalize inability to localize an object, nor do they take into account an object's relative size in the given image. 3) Convolutional neural networks always produce an output in the space of the learnt classes with high confidence while predicting the class of a given image regardless of what the image consists of. For example, an ImageNet-1K (a popular dataset) trained CNN can not say if the given image has no objects that it was trained on if it is provided with an image of a dinosaur (not an ImageNet category) or if the image has the main object cut out of it (images with background only). We approach these three different problems using object position information and learning to produce low confidence predictions on out of distribution classes. To address the first problem, we propose a novel regularization method called CopyPaste. The idea behind our approach is that images from the same class share similar context and can be 'mixed' together without affecting the labels. We use bounding box annotations that are available for a subset of ImageNet images. We consistently outperform the standard baseline and explore the idea of combining our approach with other recent regularization methods as well. We show consistent performance gains on PASCAL VOC07, MS-COCO and ImageNet datasets. For the second problem we employ objectness measures to learn meaningful CNN predictions. Objectness is a measure of likelihood of an object from any class being present in a given image. We present a novel approach to object localization that combines the ideas of objectness and label smoothing during training. Unlike previous methods, we compute a smoothing factor that is adaptive based on relative object size within an image. We present extensive results using ImageNet and OpenImages to demonstrate that CNNs trained using adaptive label smoothing are much less likely to be overconfident in their predictions, as compared to CNNs trained using hard targets. We train CNNs using objectness computed from bounding box annotations that are available for the ImageNet dataset and the OpenImages dataset. We perform extensive experiments with the aim of improving the ability of a classification CNN to learn better localizable features and show object detection performance improvements, calibration and classification performance on standard datasets. We also show qualitative results to illustrate the improvements. Lastly, we extend the second approach to train CNNs with images belonging to out of distribution and context using a uniform distribution of probability over the set of target classes for such images. This is a novel way to use uniform smooth labels as it allows the model to learn better confidence bounds. We sample 1000 classes (mutually exclusive to the 1000 classes in ImageNet-1K) from the larger ImageNet dataset comprising about 22K classes. We compare our approach with standard baselines on `in distribution' and `out of distribution' validation sets.

Decisions for asset allocation and protection are predicated upon accurate knowledge of the current operating environment as well as correctly characterizing the evolution of the environment over time. The desired kinematic and kinetic states of objects in question cannot be measured directly in most cases and instead are inferred or estimated from available measurements using a filtering process. Often, nonlinear transformations between the measurement domain and desired state domain distort the state domain probability density function yielding a form which does not necessarily resemble the form assumed in the filtering algorithm. The distortion effect must be understood in greater detail and appropriately accounted for so that even if sensors, state estimation algorithms, and state propagation algorithms operate in different domains, they can all be effectively utilized without any information loss due to domain transformations. This research presents an analytical investigation into understanding how non-linear transformations of stochastic, but characterizable, processes affect state and uncertainty estimation with direct application to space object surveillance and space- craft attitude determination. Analysis is performed with attention to construction of the state domain probability density function since state uncertainty and correlation are derived from the statistical moments of the probability density function. Analytical characterization of the effect nonlinear transformations impart on the structure of state probability density functions has direct application to conventional non- linear filtering and propagation algorithms in three areas: (1) understanding how smoothing algorithms used to estimate indirectly observed states impact state uncertainty, (2) justification or refutation of assumed state uncertainty distribution for more realistic uncertainty quantification, and (3) analytic automation of initial state estimate and covariance in lieu of user tuning. A nonlinear filtering algorithm based upon Bayes’ Theorem is presented to ac- count for the impact nonlinear domain transformations impart on probability density functions during the measurement update and propagation phases. The algorithm is able to accommodate different combinations of sensors for state estimation which can also be used to hypothesize system parameters or unknown states from available measurements because information is able to appropriately accounted for.

During the operation of a geological carbon storage project, verifying that the CO₂ plume remains within the permitted zone is of particular interest both to regulators and to operators. However, the cost of many monitoring technologies, such as time-lapse seismic, limits their application. For adequate predictions of plume migration, proper representation of heterogeneous permeability fields is imperative. Previous work has shown that injection data (pressures, rates) from wells might provide a means of characterizing complex permeability fields in saline aquifers. Thus, given that injection data are readily available and inexpensive, they might provide an inexpensive alternative for monitoring; combined with a flow model like the one developed in this work, these data could even be used for predicting plume migration. These predictions of plume migration pathways can then be compared to field observations like time-lapse seismic or satellite measurements of surface-deformation, to ensure the containment of the injected CO₂ within the storage area. In this work, two novel methods for creating heterogeneous permeability fields constrained by injection data are demonstrated. The first method is an implementation of a probabilistic history matching algorithm to create models of the aquifer for predicting the movement of the CO₂ plume. The geologic property of interest, for example hydraulic conductivity, is updated conditioned to geological information and injection pressures. The resultant aquifer model which is geologically consistent can be used to reliably predict the movement of the CO₂ plume in the subsurface. The second method is a model selection algorithm that refines an initial suite of subsurface models representing the prior uncertainty to create a posterior set of subsurface models that reflect injection performance consistent with that observed. Such posterior models can be used to represent uncertainty in the future migration of the CO₂ plume. The applicability of both methods is demonstrated using a field data set from central Algeria. / text

This thesis provides a basic framework for probabilistic real-time urban flood forecasting based on data of varying degree of quality and quantity. The framework was developed based on precipitation data from two case study areas:Aarhus Denmark and Castries St. Lucia. Many practitioners have acknowledged that a combination of structural and non-structural measures are required to reduce the effects of flooding on urban environments, but the general dearth of the desired data and models makes the development of a flood forecasting system seem unattainable. Needless to say, high resolution data and models are not always achievable and it may be necessary to override accuracy in order to reduce flood risk in urban areas and focus on estimating and communicating the uncertainty in the available resource. Thus, in order to develop a pertinent framework, both primary and secondary data sources were used to discover the current practices and to identify relevant data sources. Results from an online survey revealed that we currently have the resources to make a flood forecast and also pointed to potential open source quantitative precipitation forecast (QPF) which is the single most important component in order to make a flood forecast. The design of a flood forecasting system entails the consideration of several factors, thus the framework provides an overview of the considerations and provides a description of the proposed methods that apply specifically to each component. In particular, this thesis focuses extensively on the verification of QPF and QPE from NWP weather radar and highlights a method for estimating the uncertainty in the QPF from NWP models based on a retrospective comparison of observed and forecasted rainfall in the form of probability distributions. The results from the application of the uncertainty model suggest that the rainfall forecasts has a large contribution to the uncertainty in the flood forecast and applying a method which bias corrects and estimates confidence levels in the forecast looks promising for real-time flood forecasting. This work also describes a method used to generate rainfall ensembles based on a catalogue of observed rain events at suitable temporal scales. Results from model calibration and validation highlights the invaluable potential in using images extracted from social network sites for model calibration and validation. This framework provides innovative possibilities for real-time urban flood forecasting.

Precipitation phase is fundamental to a catchment’s hydrological response to precipitation events in cold regions and is especially variable over time and space in complex topography. Phase is controlled by the microphysics of the falling hydrometeor, but microphysical calculations require detailed atmospheric information that is often unavailable and lacking from hydrological analyses. In hydrology, there have been many methods developed to estimate phase, but most are regionally calibrated and many depend on air temperature (Ta) and use daily time steps. Phase is not only related to Ta, but to other meteorological variables such as humidity. In addition, precipitation events are dynamic, adding uncertainties to the use of daily indices to estimate phase. To better predict precipitation phase with respect to meteorological conditions, the combined mass and energy balance of a falling hydrometeor was calculated and used to develop a model to estimate precipitation phase. Precipitation phase and meteorological data were observed at multiple elevations in a small Canadian Rockies catchment, Marmot Creek Research Basin, at 15-minute intervals over several years to develop and test the model. The mass and energy balance model was compared to other methods over varying time scales, seasons, elevations and topographic exposures. The results indicate that the psychrometric energy balance model performs much better than Ta methods and that this improvement increases as the calculation time interval decreases. The uncertainty that differing phase methods introduce to hydrological process estimation was assessed with the Cold Regions Hydrological Model (CRHM). The rainfall/total precipitation ratio, runoff, discharge and snowpack accumulation were calculated using a single and a double Ta threshold method and the proposed physically based mass and energy balance model. Intercomparison of the hydrological responses of the methods highlighted differences between Ta based and psychrometric approaches. Uncertainty of hydrological processes, as established by simulating a wide range of Ta methods, reached up to 20% for rain ratio, 1.5 mm for mean daily runoff, 0.4 mm for mean daily discharge and 160 mm of peak snow water equivalent. The range of Ta methods showed that snowcover duration, snow free date and peak discharge date could vary by up to 36, 26 and 10 days respectively. The greatest hydrological uncertainty due to precipitation phase methods was found at sub-alpine and sub-arctic headwater basins and the least uncertainty was found at a small prairie basin.

Background: Dynamic contrast-enhanced MRI (DCE-MRI) has the potential to produce images of physiological quantities such as blood flow, blood vessel volume fraction, and blood vessel permeability. Such information is highly valuable, e.g., in oncology. The focus of this work was to improve the quantitative aspects of DCE-MRI in terms of better understanding of error sources and their effect on estimated physiological quantities. Methods: Firstly, a novel parameter estimation algorithm was developed to overcome a problem with sensitivity to the initial guess in parameter estimation with a specific pharmacokinetic model. Secondly, the accuracy of the arterial input function (AIF), i.e., the estimated arterial blood contrast agent concentration, was evaluated in a phantom environment for a standard magnitude-based AIF method commonly used in vivo. The accuracy was also evaluated in vivo for a phase-based method that has previously shown very promising results in phantoms and in animal studies. Finally, a method was developed for estimation of uncertainties in the estimated physiological quantities. Results: The new parameter estimation algorithm enabled significantly faster parameter estimation, thus making it more feasible to obtain blood flow and permeability maps from a DCE-MRI study. The evaluation of the AIF measurements revealed that inflow effects and non-ideal radiofrequency spoiling seriously degrade magnitude-based AIFs and that proper slice placement and improved signal models can reduce this effect. It was also shown that phase-based AIFs can be a feasible alternative provided that the observed difficulties in quantifying low concentrations can be resolved. The uncertainty estimation method was able to accurately quantify how a variety of different errors propagate to uncertainty in the estimated physiological quantities. Conclusion: This work contributes to a better understanding of parameter estimation and AIF quantification in DCE-MRI. The proposed uncertainty estimation method can be used to efficiently calculate uncertainties in the parametric maps obtained in DCE-MRI.

The evolving complexity of electric power systems with higher levels of uncertainties is a new challenge faced by system operators. Therefore, new methods for power system prediction, monitoring and state estimation are relevant for the efficient exploitation of renewable energy sources and the secure operation of network assets. In order to estimate all possible operating conditions of power systems, this Thesis proposes the use of Gaussian mixture models to represent non-Gaussian correlated input variables, such as wind power output or aggregated load demands in the probabilistic load flow problem. The formulation, based on multiple Weighted Least Square runs, is also extended to monitor distribution radial networks where the uncertainty of these networks is aggravated by the lack of sufficient real-time measurements. This research also explores reduction techniques to limit the computational demands of the probabilistic load flow and it assesses the impact of the reductions on the resulting probability density functions of power flows and bus voltages. The development of synchronised measurement technology to support monitoring of electric power systems in real-time is also studied in this work. The Thesis presents and compares different formulations for incorporating conventional and synchronised measurements in the state estimation problem. As a result of the study, a new hybrid constrained state estimator is proposed. This constrained formulation makes it possible to take advantage of the information from synchronised phasor measurements of branch currents and bus voltages in polar form. Additionally, the study is extended to assess the advantages of PMU measurements in multi-area state estimators and it explores a new algorithm that minimises the data exchange between local area state estimators. Finally, this research work also presents the advantages of dynamic state estimators supported by Synchronised Measurement Technology. The dynamic state estimator is compared with the static approach in terms of accuracy and performance during sudden changes of states and the presence of bad data. All formulations presented in this Thesis were validated in different IEEE test systems.

Deep learning (DL) has demonstrated outstanding performance in a variety of applications. With the assistance of DL, healthcare seeks to reduce labor costs and increase access to high-quality medical resources. To ensure the stability and robustness of DL applications in medicine, it is essential to estimate the uncertainty. In this thesis, the research focuses on generating an uncertainty-aware nuclei detection framework for digital pathology. A neural network (NN) with uncertainty estimation is implemented using a Bayesian deep learning method based on MC Dropout to evaluate and study the method's reliability. By evaluating and discussing the uncertainty in DL, it is possible to comprehend why it is essential to include a mechanism for measuring uncertainty. With the implementation of the framework, the results demonstrate that uncertainty-aware DL approaches enable doctors to minimize manual labeling tasks and make better decisions based on uncertainty in diagnosis and treatment. We evaluate the models in terms of both model performance and model calibration. The results demonstrate that our solution increases precision and f1 score by 15% and 11%, respectively. Using our method, the negative log likelihood (NLL) was reduced by 12% as well.

Autonomous vehicles are equipped with systems that can detect and track the objects in a vehicle’s vicinity and make appropriate driving decisions accordingly. Infrared (IR) cameras are not typically employed on these systems, but the new information that can be supplied by IR cameras can help improve the probability of detecting all objects in a vehicle’s surroundings. The purpose of this research is to investigate how IR imaging can be leveraged to improve existing autonomous driving detection systems. This research serves as a proof-of-concept demonstration. In order to achieve detection using thermal images, raw data from seven different driving scenarios was captured and labelled using a calibrated camera. Calibrating the camera made it possible to estimate the distance to objects within the image frame. The labelled images (ground truth data) were then used to train several YOLOv2 neural networks to detect similar objects in other image frames. Deeper YOLOv2 networks trained on larger amounts of data were shown to perform better on both precision and recall metrics. A novel method of estimating pixel error in detected object locations has also been proposed which can be applied to any detection algorithm that has corresponding ground truth data. The pixel errors were shown to be normally distributed with unique spreads about different ranges of y-pixels. Low correlations were seen in detection errors in the x-pixel direction. This methodology can be used to create a gate estimation for the detected pixel location of an object. Detection using IR imaging has been shown to have promising results for applications where typical autonomous sensors can have difficulties. The work done in this thesis has shown that the additional information supplied by IR cameras has potential to improve existing autonomous sensory systems. / Thesis / Master of Applied Science (MASc)

<p dir="ltr">In the context of reinforcement learning, extensive research has shown how reinforcement learning was facilitated by the estimation of uncertainty to improve the ability to make decisions. However, the constraints imposed by the limited observation of the process of forming environment representation have seldom been a subject of discussion. Thus, the study intended to demonstrate that when incorporating a limited memory into uncertainty estimation, individuals potentially misestimate outcomes and environmental statistics. The study included a computational model that included the process of active working memory and lateral inhibition in working memory (WM) to describe how the relevant information was chosen and stored to form estimations of uncertainty in forming outcome expectations. The active working memory maintained relevant information not just by the recent memory, but also with utility. With the relevant information stored in WM, the model was able to estimate expected uncertainty and perceived volatility and detect contextual changes or dynamics in the outcome structure. Two experiments to investigate limitations in information availability and uncertainty estimation were carried out. The first experiment investigated the impact of cognitive loading on the reliance on memories to form outcome estimation. The findings revealed that introducing cognitive loading diminished the reliance on memory for uncertainty estimations and lowered the expected uncertainty, leading to an increased perception of environmental volatility. The second experiment investigated the ability to detect changes in outcome noise under different conditions of outcome exposure. The study found differences in the mechanisms used for detecting environmental changes in various conditions. Through the experiments and model fitting, the study showed that the misestimation of uncertainties was reliant on individual experiences and relevant information stored in WM under a limited capacity.</p>