Developing advanced semantic models is important in building computational systems that can not only understand language but also convey ideas and concepts to others. Semantic models can allow a creative image-producing-agent to autonomously produce artifacts that communicate an intended meaning. This notion of communicating meaning through art is often considered a necessary part of eliciting an aesthetic experience in the viewer and can thus enhance the (perceived) creativity of the agent. Computational creativity, a subfield of artificial intelligence, deals with designing computational systems and algorithms that either automatically create original and functional products, or that augment the ability of humans to do so. We present work on DARCI (Digital ARtist Communicating Intention), a system designed to autonomously produce original images that convey meaning. In order for DARCI to automatically express meaning through the art it creates, it must have its own semantic model that is perceptually grounded with visual capabilities.The work presented here focuses on designing, building, and incorporating advanced semantic and perceptual models into the DARCI system. These semantic models give DARCI a better understanding of the world and enable it to be more autonomous, to better evaluate its own artifacts, and to create artifacts with intention. Through designing, implementing, and studying DARCI, we have developed evaluation methods, models, frameworks, and theories related to the creative process that can be generalized to other domains outside of visual art. Our work on DARCI has even influenced the visual art community through several collaborative efforts, art galleries, and exhibits. We show that the DARCI system is successful at autonomously producing original art that is meaningful to human viewers. We also discuss insights that our efforts have contributed to the field of computational creativity.

In automotive industry there is a growing need for predicting the failure of a component, to achieve the cost saving and customer satisfaction. As failure in a component leads to the work breakdown for the customer. This paper describes an effort in making a prediction failure monitoring model for NOx sensor in trucks. It is a component that used to measure the level of nitrogen oxide emission from the truck. The NOx sensor has chosen because its failure leads to the slowdown of engine efficiency and it is fragile and costly to replace. The data from a good and contaminated NOx sensor which is collated from the test rigs is used the input to the model. This work in this paper shows approach of complementing the Deep Learning models with Machine Learning algorithm to achieve the results. In this work LSTMs are used to detect the gain in NOx sensor and Encoder-Decoder LSTM is used to predict the variables. On top of it Multiple Linear Regression model is used to achieve the end results. The performance of the monitoring model is promising. The approach described in this paper is a general model and not specific to this component, but also can be used for other sensors too as it has a universal kind of approach.

Deep Learning models rely on a huge amount of labeled data to be created. However, there are a number of areas where labeling data is a costly process, making Deep Learning approaches unfeasible. One way to handle that situation is by using the Active Learning technique. Initially, it creates a model with the available labeled data. After that, it incrementally chooses new unlabeled data that will potentially increase the model accuracy, if added to the training data. To select which data will be labeled next, this technique requires a measurement of uncertainty from the model prediction, which is usually not computed for Deep Learning methods. A new approach has been proposed to measure uncertainty in those models, called Monte Carlo Dropout . This technique allowed Active Learning to be used together with Deep Learning for image classification. This research will evaluate if modeling uncertainty on Deep Learning models with Monte Carlo Dropout will make the use of Active Learning feasible for the task of sentiment analysis, an area with huge amount of data, but few of them labeled. / Modelos de Aprendizado Profundo necessitam de uma vasta quantidade de dados anotados para serem criados. Entretanto, existem muitas áreas onde obter dados anotados é uma tarefa custosa. Neste cenário, o uso de Aprendizado Profundo se torna bastante difícil. Uma maneira de lidar com essa situação é usando a técnica de Aprendizado Ativo. Inicialmente, essa técnica cria um modelo com os dados anotados disponíveis. Depois disso, ela incrementalmente escolhe dados não anotados que irão, potencialmente, melhorar à acurácia do modelo, se adicionados aos dados de treinamento. Para selecionar quais dados serão anotados, essa técnica necessita de uma medida de incerteza sobre as predições geradas pelo modelo. Entretanto, tal medida não é usualmente realizada em modelos de Aprendizado Profundo. Uma nova técnica foi proposta para lidar com a problemática de medir a incerteza desses modelos, chamada de Monte Carlo Dropout . Essa técnica permitiu o uso de Aprendizado Ativo junto com Aprendizado Profundo para tarefa de classificação de imagens. Essa pesquisa visa averiguar se ao modelarmos a incerteza em modelos de Aprendizado Profundo com a técnica de Monte Carlo Dropout , será possível usar a técnica de Aprendizado Ativo para tarefa de análise de sentimento, uma área com uma vasta quantidade de dados, mas poucos deles anotados.

The task of automatically segmenting 3-D surfaces representing object boundaries is important in quantitative analysis of volumetric images, which plays a vital role in numerous biomedical applications. For the diagnosis and management of disease, segmentation of images of organs and tissues is a crucial step for the quantification of medical images. Segmentation finds the boundaries or, limited to the 3-D case, the surfaces, that separate regions, tissues or areas of an image, and it is essential that these boundaries approximate the true boundary, typically by human experts, as closely as possible. Recently, graph-based methods with a global optimization property have been studied and used for various applications. Sepecifically, the state-of-the-art graph search (optimal surface segmentation) method has been successfully used for various such biomedical applications. Despite their widespread use for image segmentation, real world medical image segmentation problems often pose difficult challenges, wherein graph based segmentation methods in its purest form may not be able to perform the segmentation task successfully. This doctoral work has a twofold objective. 1)To identify medical image segmentation problems which are difficult to solve using existing graph based method and develop novel methods by employing graph search as a building block to improve segmentation accuracy and efficiency. 2) To develop a novel multiple surface segmentation strategy using deep learning which is more computationally efficient and generic than the exisiting graph based methods, while eliminating the need for human expert intervention as required in the current surface segmentation methods. This developed method is possibly the first of its kind where the method does not require and human expert designed operations. To accomplish the objectives of this thesis work, a comprehensive framework of graph based and deep learning methods is proposed to achieve the goal by successfully fulfilling the follwoing three aims. First, an efficient, automated and accurate graph based method is developed to segment surfaces which have steep change in surface profiles and abrupt distance changes between two adjacent surfaces. The developed method is applied and validated on intra-retinal layer segmentation of Spectral Domain Optical Coherence Tomograph (SD-OCT) images of eye with Glaucoma, Age Related Macular Degneration and Pigment Epithelium Detachment. Second, a globally optimal graph based method is developed to attain subvoxel and super resolution accuracy for multiple surface segmentation problem while imposing convex constraints. The developed method was applied to layer segmentation of SD-OCT images of normal eye and vessel walls in Intravascular Ultrasound (IVUS) images. Third, a deep learning based multiple surface segmentation is developed which is more generic, computaionally effieient and eliminates the requirement of human expert interventions (like transformation designs, feature extrraction, parameter tuning, constraint modelling etc.) required by existing surface segmentation methods in varying capacities. The developed method was applied to SD-OCT images of normal and diseased eyes, to validate the superior segmentaion performance, computation efficieny and the generic nature of the framework, compared to the state-of-the-art graph search method.

Technological advances in next-generation sequencing and biomedical imaging have led to a rapid increase in biomedical data dimension and acquisition rate, which is challenging the conventional data analysis strategies. Modern machine learning techniques promise to leverage large data sets for finding hidden patterns within them, and for making accurate predictions. This dissertation aims to design novel machine learning-based models to transform biomedical big data into valuable biological insights. The research presented in this dissertation focuses on three bioinformatics domains: splice junction classification, gene regulatory network reconstruction, and lesion detection in mammograms. A critical step in defining gene structures and mRNA transcript variants is to accurately identify splice junctions. In the first work, we built the first deep learning-based splice junction classifier, DeepSplice. It outperforms the state-of-the-art classification tools in terms of both classification accuracy and computational efficiency. To uncover transcription factors governing metabolic reprogramming in non-small-cell lung cancer patients, we developed TFmeta, a machine learning approach to reconstruct relationships between transcription factors and their target genes in the second work. Our approach achieves the best performance on benchmark data sets. In the third work, we designed deep learning-based architectures to perform lesion detection in both 2D and 3D whole mammogram images.

Road crashes are one of the main causes of death in the United States. To reduce the number of accidents, roadway assessment programs take a proactive approach, collecting data and identifying high-risk roads before crashes occur. However, the cost of data acquisition and manual annotation has restricted the effect of these programs. In this thesis, we propose methods to automate the task of roadway safety assessment using deep learning. Specifically, we trained convolutional neural networks on publicly available roadway images to predict safety-related metrics: the star rating score and free-flow speed. Inference speeds for our methods are mere milliseconds, enabling large-scale roadway study at a fraction of the cost of manual approaches.

<div> The structural defects can lead to serious safety issues and the corrosponding economic losses. In 2013, it was estimated that 2.5 trillion US dollars were spent on corrosion around the world, which was 3.4\% of the global Gross Domestic Product (GDP) (Koch, 2016). Periodical inspection of corrosion and maintenance of steel structures are essential to minimize these losses. Current corrosion inspection guidelines require inspectors to visually assess every critical member within arm's reach. This process is time-consuming, subjective and labor-intensive, and therefore is done only once every two years. </div><div><br></div><div>A promising solution is to use a robotic system, such as an Unmanned Aerial Vehicle (UAV), with computer vision techniques to assess corrosion on metallic surfaces. Several studies have been conducted in this area, but the shortcoming is that they cannot quantify the corroded region reliably: some studies only classify whether corrosion exists in the image or not; some only draw a box around corroded region; and some need human-engineered features to identify corrosion. This study aims to address this problem by using deep learning-based semantic segmentation to let the computer capture useful features and find the bounding of corroded regions accurately.</div><div><br></div><div>In this study, the performance of four state-of-the-art deep learning techniques for semantic segmentation was investigated for corrosion assessment task，including U-Net, DeepLab, PSPNet, and RefineNet. Six hundred high-resolution images of corroded regions were used to train and test the networks. Ten sets of experiments were performed on each architecture for cross-validation. Since the images were large, two approaches were used to analyze images: 1) subdividing images, 2) down-sampling images. A parametric analysis on these two prepossessing methods was also considered.</div><div><br></div><div>Prediction results were evaluated based on intersection over union (IoU), recall and precision scores. Statistical analysis using box chart and Wilcoxon singled ranked test showed that subdivided image dataset gave a better result, while resized images required less time for prediction. Performance of PSPNet outperformed the other three architectures on the subdivided dataset. DeepLab showed the best performance on the resized dataset. It was found Refinenet was not appropriate for corrosion detection task. U-Net was found to be ideal for real-time processing of image while RefineNet did not perform well for corrosion assessment.</div><div> </div>

Developing unsupervised vision systems in Dynamic Environments is one of the next challenges in Computer Vision. In Dynamic Environments, we usually lack the complete domain knowledge of the applied environments before deployment, and computation is also limited due to the need for prompt reaction and on-board computational capacity. This thesis studies a series of key Computer Vision problems in Dynamic Environments. <div><br></div><div>First, we propose a stream clustering algorithm and a number of variants for unsupervised feature learning and object discovery, which possess several crucial characteristics required by applications in Dynamic Environments, e.g. fully progressive, arbitrary similarity measure, matching object while the feature space is increasing, etc. We give strong provable guarantees of the clustering accuracy in statistic view. Based on the above the approaches, we tackle the problem of discovering aerial objects on-the-fly, where we assume all of the objects are unknown at the beginning of the deployment. The vision system is required to discover from the low-level features to salient objects on-the-fly without any supervision. We propose a number of approaches with respect to object proposal, tracking, recognition, and localization to achieve real-time performance. Extensive experiments on prevalent aerial video datasets showed that the approaches efficiently and accurately discover salient ground objects. </div><div><br></div><div>To explore complex and deep architectures in Dynamic Environments, we propose Unsupervised Deep Encoding which unifies traditional Visual Encoding and Convolutional Neural Networks. We found strong relationships between single-layer Neural Networks and Clustering and thus performed unsupervised feature learning at each layer from the feature maps of the previous layer. We replaced the dot product inside each neuron with a similarity measure, which is also used in unsupervised feature learning. The weight vectors of our network are initialized by cluster centers. Therefore, one feature map is a visual encoding of its previous feature map. We applied this mechanism to pre-training Convolutional Neural Networks for image classification. It has been found by extensive experiments that pre-training benefits the network more reliable learning dynamics (e.g.fast convergence without Batch Normalization) and better classification accuracy.</div>

Environment maps are widely used in several computer graphics fields, such as realistic architectural rendering or computer games as sources of the light in the scene. Obtaining these maps is not easy, since they have to have both a high- dynamic range as well as a high resolution. As a result, they are expensive to make and the supply is limited. Deep neural networks are a widely unexplored research area and have been successfully used for generating complex and realistic images like human portraits. Neural networks perform well at predicting data from complex models, which are easily observable, such as photos of the real world. This thesis explores the idea of generating physically plausible environment maps by utilizing deep neural networks known as generative adversarial networks. Since a skydome dataset is not publicly available, we develop a scalable capture process with both low-end and high-end hardware. We implement a pipeline to process the captured data before feeding it to a network and extend an already existing network architecture to generate HDR environment maps. We then run a series of experiments to determine the quality of the results and uncover the directions of possible further research.

Convolutional neural networks (ConvNets) have quickly become the most widely used tool for image perception and interpretation tasks over the past several years. The single most important resource needed for training a ConvNet that will successfully generalize to unseen examples is an adequately sized labeled dataset. In many interesting medical imaging cases, the necessary size or quality of training data is not suitable for directly training a ConvNet. Furthermore, access to the expertise to manually label such datasets is often infeasible. To address these barriers, we investigate a method for iterative refinement of the ConvNet training. Initially, unlabeled images are attained, minimal labeling is performed, and a model is trained on the sparse manual labels. At the end of each training iteration, full images are predicted, and additional manual labels are identified to improve the training dataset. In this work, we show how to utilize patch-based ConvNets to iteratively build a training dataset for automatically segmenting MRI images of the human cerebellum. We construct this training dataset using a small collection of high-resolution 3D images and transfer the resulting model to a much larger, much lower resolution, collection of images. Both T1-weighted and T2-weighted MRI modalities are utilized to capture the additional features that arise from the differences in contrast between modalities. The objective is to perform tissue-level segmentation, classifying each volumetric pixel (voxel) in an image as white matter, gray matter, or cerebrospinal fluid (CSF). We will present performance results on the lower resolution dataset, and report achieving a 12.7% improvement in accuracy over the existing segmentation method, expectation maximization. Further, we will present example segmentations from our iterative approach that demonstrate it’s ability to detect white matter branching near the outer regions of the anatomy, which agrees with the known biological structure of the cerebellum and has typically eluded traditional segmentation algorithms.