Deep Neural Networks have long been considered black box systems, where their interpretability is a concern when applied in safety critical systems. In this work, a novel approach of interpreting the decisions of DNNs is proposed. The approach depends on exploiting generative models and the interpretability of their latent space. Three methods for ranking features are explored, two of which depend on sensitivity analysis, and the third one depends on Random Forest model. The Random Forest model was the most successful to rank the features, given its accuracy and inherent interpretability.

Deep nueral networks (DNNs) have become popular due to their predictive power and flexibility in model fitting. In unsupervised learning, variational autoencoders (VAEs) and generative adverarial networks (GANs) are two most popular and successful generative models. How to provide a unifying framework combining the best of VAEs and GANs in a principled way is a challenging task. In supervised learning, the demand for high-dimensional data analysis has grown significantly, especially in the applications of social networking, bioinformatics, and neuroscience. How to simultaneously approximate the true underlying nonlinear system and identify relevant features based on high-dimensional data (typically with the sample size smaller than the dimension, a.k.a. small-n-large-p) is another challenging task.<div><br></div><div>In this dissertation, we have provided satisfactory answers for these two challenges. In addition, we have illustrated some promising applications using modern machine learning methods.<br></div><div><br></div><div>In the first chapter, we introduce a novel inferential Wasserstein GAN (iWGAN) model, which is a principled framework to fuse auto-encoders and WGANs. GANs have been impactful on many problems and applications but suffer from unstable training. The Wasserstein GAN (WGAN) leverages the Wasserstein distance to avoid the caveats in the minmax two-player training of GANs but has other defects such as mode collapse and lack of metric to detect the convergence. The iWGAN model jointly learns an encoder network and a generator network motivated by the iterative primal dual optimization process. The encoder network maps the observed samples to the latent space and the generator network maps the samples from the latent space to the data space. We establish the generalization error bound of iWGANs to theoretically justify the performance of iWGANs. We further provide a rigorous probabilistic interpretation of our model under the framework of maximum likelihood estimation. The iWGAN, with a clear stopping criteria, has many advantages over other autoencoder GANs. The empirical experiments show that the iWGAN greatly mitigates the symptom of mode collapse, speeds up the convergence, and is able to provide a measurement of quality check for each individual sample. We illustrate the ability of iWGANs by obtaining a competitive and stable performance with state-of-the-art for benchmark datasets. <br></div><div><br></div><div>In the second chapter, we present a general framework for high-dimensional nonlinear variable selection using deep neural networks under the framework of supervised learning. The network architecture includes both a selection layer and approximation layers. The problem can be cast as a sparsity-constrained optimization with a sparse parameter in the selection layer and other parameters in the approximation layers. This problem is challenging due to the sparse constraint and the nonconvex optimization. We propose a novel algorithm, called Deep Feature Selection, to estimate both the sparse parameter and the other parameters. Theoretically, we establish the algorithm convergence and the selection consistency when the objective function has a Generalized Stable Restricted Hessian. This result provides theoretical justifications of our method and generalizes known results for high-dimensional linear variable selection. Simulations and real data analysis are conducted to demonstrate the superior performance of our method.<br></div><div><br></div><div><div>In the third chapter, we develop a novel methodology to classify the electrocardiograms (ECGs) to normal, atrial fibrillation and other cardiac dysrhythmias as defined by the Physionet Challenge 2017. More specifically, we use piecewise linear splines for the feature selection and a gradient boosting algorithm for the classifier. In the algorithm, the ECG waveform is fitted by a piecewise linear spline, and morphological features related to the piecewise linear spline coefficients are extracted. XGBoost is used to classify the morphological coefficients and heart rate variability features. The performance of the algorithm was evaluated by the PhysioNet Challenge database (3658 ECGs classified by experts). Our algorithm achieves an average F1 score of 81% for a 10-fold cross validation and also achieved 81% for F1 score on the independent testing set. This score is similar to the top 9th score (81%) in the official phase of the Physionet Challenge 2017.</div></div><div><br></div><div>In the fourth chapter, we introduce a novel region-selection penalty in the framework of image-on-scalar regression to impose sparsity of pixel values and extract active regions simultaneously. This method helps identify regions of interest (ROI) associated with certain disease, which has a great impact on public health. Our penalty combines the Smoothly Clipped Absolute Deviation (SCAD) regularization, enforcing sparsity, and the SCAD of total variation (TV) regularization, enforcing spatial contiguity, into one group, which segments contiguous spatial regions against zero-valued background. Efficient algorithm is based on the alternative direction method of multipliers (ADMM) which decomposes the non-convex problem into two iterative optimization problems with explicit solutions. Another virtue of the proposed method is that a divide and conquer learning algorithm is developed, thereby allowing scaling to large images. Several examples are presented and the experimental results are compared with other state-of-the-art approaches. <br></div>

The main objective of this thesis is to propose a localization method for a UAV using various computer vision and machine learning techniques. It plays a major role in planning the strategy for the flight, and acts as a navigational contingency method, in event of a GPS failure. The implementation of the algorithms employs high processing capabilities of the graphics processing unit, making it more efficient. The method involves the working of various neural networks, working in synergy to perform the localization. This thesis is a part of a collaborative project between The University of North Texas, Denton, USA, and the University of Windsor, Ontario, Canada. The localization has been divided into three phases namely object detection, recognition, and location estimation. Object detection and position estimation were discussed in this thesis while giving a brief understanding of the recognition. Further, future strategies to aid the UAV to complete the mission, in case of an eventuality, like the introduction of an EDGE server and wireless charging methods, was also given a brief introduction.

Indiana University-Purdue University Indianapolis (IUPUI) / Humans can visually see things and can differentiate objects easily but for computers, it is not that easy. Computer Vision is an interdisciplinary field that allows computers to comprehend, from digital videos and images, and differentiate objects. With the Introduction to CNNs/DNNs, computer vision is tremendously used in applications like ADAS, robotics and autonomous systems, etc. This thesis aims to propose an architecture, RMNv2, that is well suited for computer vision applications such as ADAS, etc. RMNv2 is inspired by its original architecture Mobilenet V2. It is a modified version of Mobilenet V2. It includes changes like disabling downsample layers, Heterogeneous kernel-based convolutions, mish activation, and auto augmentation. The proposed model is trained from scratch in the CIFAR10 dataset and produced an accuracy of 92.4% with a total number of parameters of 1.06M. The results indicate that the proposed model has a model size of 4.3MB which is like a 52.2% decrease from its original implementation. Due to its less size and competitive accuracy the proposed model can be easily deployed in resource-constrained devices like mobile and embedded devices for applications like ADAS etc. Further, the proposed model is also implemented in real-time embedded devices like NXP Bluebox 2.0 and NXP i.MX RT1060 for image classification tasks.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第21909号 / 情博第692号 / 新制||情||119(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)教授 神谷 之康, 教授 熊田 孝恒, 教授 西野 恒 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DGAM

No description available.

With the development of small satellites it has become easier and cheaper to deploy satellites for earth observation from space. While optical sensors capture high-resolution data, this data is traditionally sent to earth for analysis which puts a high constraint on the data link and increases the time for making data based decisions. This thesis explores the possibilities of deploying an AI model in small satellites for detecting objects in satellite imagery and therefore reduce the amount of data that needs to be transmitted. The neural network model YOLOv8 was trained on the xView and DIOR dataset and evaluated in a hardware restricted execution environment. The model achieved a mAP50 of 0.66 and could process satellite images at a speed of 309m2/s.

Battery Electric Vehicles (BEV) are gaining market share but still must overcome several engineering challenges related to the lithium-ion battery packs powering them. The batteries must be carefully managed to optimize safety and performance. The estimation of battery states, which cannot be measured directly, is an important part of battery management and remains an active area of research since small gains in estimation accuracy can help reduce cost and increase BEV range. This thesis presents several improvements to battery state estimation using different methods. Electrochemical Impedance Spectroscopy (EIS) is receiving increased attention from researchers as a method for state estimation and diagnostics for real-time applications. Due to battery relaxation behaviour, long rest times are commonly used before performing the EIS measurement. In this work, methods were developed to significantly shorten the required rest times, and a State of Health (SoH) estimation strategy was proposed by taking advantage of the relaxation effect as measured by EIS. This method was demonstrated to have an estimation error of below 1%. At low temperatures, the accuracy of the battery model becomes poor due to the non-linear battery response to current. By using an adaptive filter called the Interacting Multiple Model (IMM) filter, the next part of this work showed how to significantly improve low temperature State of Charge (SoC) estimation. Further reduction in estimation errors was achieved by pairing the IMM with the Smooth Variable Structure Filter (SVSF), for SoC estimation errors below 2%. The work presented in this thesis also includes the application of Deep Neural Networks (DNN) for SoC estimation from EIS data. Finally, an extensive aging study was conducted and an accelerated protocol was compared to a realistic drive cycle based protocol using EIS as a characterization tool. / Thesis / Doctor of Philosophy (PhD) / Replacing conventional gasoline/diesel powered cars with battery powered vehicles is part of a solution to the climate crisis. However, the initial costs paired with range anxiety stops many from switching to electric cars. Both cost and range are related to the battery pack. To achieve the best possible range for the lowest possible cost, battery packs must be carefully controlled by sophisticated algorithms. Unfortunately, battery range or health cannot be measured directly, but must be inferred through measurable indicators. This thesis explores battery behavior under different operating conditions and develops improved methods which can be used to determine battery health and/or range. A powerful method usually used only in laboratory settings is studied and improved to make it more suitable for implementation in electric cars. In this work it is used for accurate battery health determination. Furthermore, a strategy for improving battery range determination at low temperatures is also proposed.

The ability to extract useful information from texts and present them in the form of structured attributes is an important step towards making product comparison algorithm in fashion smarter and better. Some previous work exploits statistical features like the word frequency and graph models to predict keyphrases. In recent years, deep neural networks have proved to be the state-of-the-art methods to handle language modeling. Successful examples include Long Short Term Memory (LSTM), Gated Recurrent Units (GRU), Bidirectional Encoder Representations from Transformers(BERT) and their variations. In addition, some word embedding techniques like word2vec[1] are also helpful to improve the performance. Besides these techniques, a high-quality dataset is also important to the effectiveness of models. In this project, we aim to develop reliable and efficient machine learning models for keyphrase extraction. At Norna AB, we have a collection of product descriptions from different vendors without keyphrase annotations, which motivates the use of unsupervised methods. They should be capable of extracting useful keyphrases that capture the features of a product. To further explore the power of deep neural networks, we also implement several deep learning models. The dataset has two parts, the first part is called the fashion dataset where keyphrases are extracted by our unsupervised method. The second part is a public dataset in the domain of news. We find that deep learning models are also capable of extracting meaningful keyphrases and outperform the unsupervised model. Precision, recall and F1 score are used as evaluation metrics. The result shows that the model that uses LSTM and CRF achieves the optimal performance. We also compare the performance of different models with respect to keyphrase lengths and keyphrase numbers. The result indicates that all models perform better on predicting short keyphrases. We also show that our refined model has the advantage of predicting long keyphrases, which is challenging in this field. / Förmågan att extrahera användbar information från texter och presentera den i form av strukturerade attribut är ett viktigt steg mot att göra produktjämförelsesalgoritmen på ett smartare och bättre sätt. Vissa tidigare arbeten utnyttjar statistiska funktioner som ordfrekvens och grafmodeller för att förutsäga nyckelfraser. Under de senaste åren har djupa neurala nätverk visat sig vara de senaste metoderna för att hantera språkmodellering. Framgångsrika exempel inkluderar Long Short Term Memory (LSTM), Gated Recurrent Units (GRU), Bidirectional Encoder Representations from Transformers (BERT) och deras variationer. Dessutom kan vissa ordinbäddningstekniker som word2vec[1] också vara till hjälp för att förbättra prestandan. Förutom dessa tekniker är en datauppsättning av hög kvalitet också viktig för modellernas effektivitet. I detta projekt strävar vi efter att utveckla pålitliga och effektiva maskininlärningsmodeller för utvinning av nyckelfraser. På Norna AB har vi en samling produktbeskrivningar från olika leverantörer utan nyckelfrasnoteringar, vilket motiverar användningen av metoder utan tillsyn. De bör kunna extrahera användbara nyckelfraser som fångar funktionerna i en produkt. För att ytterligare utforska kraften i djupa neurala nätverk implementerar vi också flera modeller för djupinlärning. Datasetet har två delar, den första delen kallas modedataset där nyckelfraser extraheras med vår metod utan tillsyn. Den andra delen är en offentlig dataset i nyhetsdomänen. Vi finner att deep learning-modeller också kan extrahera meningsfulla nyckelfraser och överträffa den oövervakade modellen. Precision, återkallning och F1-poäng används som utvärderingsmått. Resultatet visar att modellen som använder LSTM och CRF uppnår optimal prestanda. Vi jämför också prestanda för olika modeller med avseende på keyphrase längder och nyckelfras nummer. Resultatet indikerar att alla modeller presterar bättre på att förutsäga korta tangentfraser. Vi visar också att vår raffinerade modell har fördelen att förutsäga långa tangentfraser, vilket är utmanande inom detta område.

No description available.