Self-driving cars rely on their sense of sight to function effectively in chaotic and uncontrolled environments. Thanks to recent developments in computer vision, specifically convolutional neural networks, autonomous vehicles have developed the ability to see at or above human-level capabilities, which in turn has allowed for rapid advances in self-driving cars. Unfortunately, much like humans being confused by simple optical illusions, convolutional neural networks are susceptible to simple adversarial inputs. As there is no overlap between the optical illusions that fool humans and the adversarial examples that threaten convolutional neural networks, little is understood as to why these adversarial examples dupe such advanced models and what effective mitigation techniques might exist to resolve these issues. This thesis focuses on these adversarial images. By extending existing work, this thesis is able to offer a unique perspective on adversarial examples. Furthermore, these extensions are used to develop a novel attack that can generate physically robust adversarial examples. These physically robust instances provide a unique challenge as they transcend both individual models and the digital domain, thereby posing a significant threat to the efficacy of convolutional neural networks and their dependent applications.

Training a neural network requires a large amount of labeled data that has to be created by either human annotation or by very specifically created methods. Currently, there is a vast abundance of unlabeled data that is neglected sitting on servers, hard drives, websites, etc. These untapped data sources serve as the inspiration for this paper. The goal of this thesis is to explore and test various methods of semi-supervised learning (SSL) for convolutional neural networks (CNN). These methods will be analyzed and evaluated based on their accuracy on a test set of data. Since this particular neural network will be used to offer paths for an autonomous robot, it is important for the networks to be lightweight in scale. This paper will then take this assortment of smaller neural networks and run them through a variety of semi-supervised training methods. The first method is to have a teacher model that is trained on properly labeled data create labels for unlabeled data and add this to the training set for the next student model. From this base method, a few variations were tried in the hopes of getting a significant improvement. The first variation tested by this thesis is the effects of having this teacher and student cycle run more than one iteration. After that, the effects of using the confidence values that the models produced were explored by both including only data with confidence above a certain value and in a different test, relabeling data below a confidence threshold. The last variation this thesis explored was to have two teacher models concurrently and have the combination of those two models decide on the proper label for the unlabeled data. Through exploration and testing, these methods are evaluated in the results section as to which one produces the best results for SSL.

Computer vision research has been growing rapidly over the last decade. Recent advancements in the field have been widely used in staple products across various industries. The automotive and medical industries have even pushed cars and equipment into production that use computer vision. However, there seems to be a lack of computer vision research in the game industry. With the advent of e-sports, competitive and casual gaming have reached new heights with regard to players, viewers, and content creators. This has allowed for avenues of research that did not exist prior. In this thesis, we explore the practicality of object detection as applied in games. We designed a custom convolutional neural network detection model, SmashNet. The model was improved through classification weights generated from pre-training on the Caltech101 dataset with an accuracy of 62.29%. It was then trained on 2296 annotated frames from the competitive 2.5-dimensional fighting game Super Smash Brothers Melee to track coordinate locations of 4 specific characters in real-time. The detection model performs at a 68.25% accuracy across all 4 characters. In addition, as a demonstration of a practical application, we designed KirbyBot, a black-box adaptive bot which performs basic commands reactively based only on the tracked locations of two characters. It also collects very simple data on player habits. KirbyBot runs at a rate of 6-10 fps. Object detection has several practical applications with regard to games, ranging from better AI design, to collecting data on player habits or game characters for competitive purposes or improvement updates.

京都大学 / 新制・論文博士 / 博士(医学) / 乙第13501号 / 論医博第2260号 / 新制||医||1061(附属図書館) / 京都大学大学院医学研究科医学専攻 / (主査)教授 小林 恭, 教授 中本 裕士, 教授 黒田 知宏 / 学位規則第4条第2項該当 / Doctor of Medical Science / Kyoto University / DFAM

In shotgun proteomics, the liquid chromatography step is used to separate peptides in order to analyze as few as possible at the same time in the mass spectrometry step. Each peptide has a retention time, that is how long it takes to pass through the chromatography column. Prediction of the retention time can be used to gain increased identification of peptides or in order to create targeted proteomics experiments. Using machine learning methods such as support vector machines has given a high prediction accuracy, but such methods require known features that the retention time depends on. In this thesis we let a convolutional network, learn to rank the retention times instead of predicting the retention times themselves. We also tested how the prediction accuracy depends on the size of the training set. We found that pairwise ranking of peptides outperforms pointwise ranking and that adding more training data increased accuracy until the end without an increase in training time. This implies that accuracy can be further increased by training on even greater training sets.

Indiana University-Purdue University Indianapolis (IUPUI) / Smart wearables, known as emerging paradigms for vital big data capturing, have been attracting intensive attentions. However, one crucial problem is their power-hungriness, i.e., the continuous data streaming consumes energy dramatically and requires devices to be frequently charged. Targeting this obstacle, we propose to investigate the biodynamic patterns in the data and design a data-driven approach for intelligent data compression. We leverage Deep Learning (DL), more specifically, Convolutional Autoencoder (CAE), to learn a sparse representation of the vital big data. The minimized energy need, even taking into consideration the CAE-induced overhead, is tremendously lower than the original energy need. Further, compared with state-of-the-art wavelet compression-based method, our method can compress the data with a dramatically lower error for a similar energy budget. Our experiments and the validated approach are expected to boost the energy efficiency of wearables, and thus greatly advance ubiquitous big data applications in era of smart health. In recent years, there has also been a growing interest in edge intelligence for emerging instantaneous big data inference. However, the inference algorithms, especially deep learning, usually require heavy computation requirements, thereby greatly limiting their deployment on the edge. We take special interest in the smart health wearable big data mining and inference. Targeting the deep learning’s high computational complexity and large memory and energy requirements, new approaches are urged to make the deep learning algorithms ultra-efficient for wearable big data analysis. We propose to leverage knowledge distillation to achieve an ultra-efficient edge-deployable deep learning model. More specifically, through transferring the knowledge from a teacher model to the on-edge student model, the soft target distribution of the teacher model can be effectively learned by the student model. Besides, we propose to further introduce adversarial robustness to the student model, by stimulating the student model to correctly identify inputs that have adversarial perturbation. Experiments demonstrate that the knowledge distillation student model has comparable performance to the heavy teacher model but owns a substantially smaller model size. With adversarial learning, the student model has effectively preserved its robustness. In such a way, we have demonstrated the framework with knowledge distillation and adversarial learning can, not only advance ultra-efficient edge inference, but also preserve the robustness facing the perturbed input. / 2023-06-01

Indiana University-Purdue University Indianapolis (IUPUI) / Semantic segmentation is a machine learning task that is seeing increased utilization in multiples fields, from medical imagery, to land demarcation, and autonomous vehicles. Semantic segmentation performs the pixel-wise classification of images, creating a new, seg- mented representation of the input that can be useful for detected various terrain and objects within and image. Recently, convolutional neural networks have been heavily utilized when creating neural networks tackling the semantic segmentation task. This is particularly true in the field of autonomous driving systems. The requirements of automated driver assistance systems (ADAS) drive semantic seg- mentation models targeted for deployment on ADAS to be lightweight while maintaining accuracy. A commonly used method to increase accuracy in the autonomous vehicle field is to fuse multiple sensory modalities. This research focuses on leveraging the fusion of long wave infrared (LWIR) imagery with visual spectrum imagery to fill in the inherent perfor- mance gaps when using visual imagery alone. This comes with a host of benefits, such as increase performance in various lighting conditions and adverse environmental conditions. Utilizing this fusion technique is an effective method of increasing the accuracy of a semantic segmentation model. Being a lightweight architecture is key for successful deployment on ADAS, as these systems often have resource constraints and need to operate in real-time. Multi-Spectral Fusion Network (MFNet) [1] accomplishes these parameters by leveraging a sensory fusion approach, and as such was selected as the baseline architecture for this research. Many improvements were made upon the baseline architecture by leveraging a variety of techniques. Such improvements include the proposal of a novel loss function categori- cal cross-entropy dice loss, introduction of squeeze and excitation (SE) blocks, addition of pyramid pooling, a new fusion technique, and drop input data augmentation. These improve- ments culminated in the creation of the Fast Thermal Fusion Network (FTFNet). Further improvements were made by introducing depthwise separable convolutional layers leading to lightweight FTFNet variants, FTFNet Lite 1 & 2. 13 The FTFNet family was trained on the Multi-Spectral Road Scenarios (MSRS) and MIL- Coaxials visual/LWIR datasets. The proposed modifications lead to an improvement over the baseline in mean intersection over union (mIoU) of 2.92% and 2.03% for FTFNet and FTFNet Lite 2 respectively when trained on the MSRS dataset. Additionally, when trained on the MIL-Coaxials dataset, the FTFNet family showed improvements in mIoU of 8.69%, 4.4%, and 5.0% for FTFNet, FTFNet Lite 1, and FTFNet Lite 2.

Bearings are the essential components of modern rotating machines. Bearing faults can cause severe machine damages or even breakdowns. In recent years, artificial intelligence and deep learning have been successfully applied to fault detection. In this thesis, convolutional neural networks (CNN) are employed for bearing fault detection and classification. Computer simulations results demonstrate that the CNN based approach is advantageous over the conventional regression model, with an overall accuracy of 99.5%.

Nowadays, space weather has become an international issue to the world's countries because of its catastrophic effect on space-borne and ground-based systems, and industries, impacting our lives. One of the main solar activities that is considered as a major driver of space weather is solar flares. Solar flares can be defined as an enormous eruption in the sun's atmosphere. This phenomenon happens when magnetic energy stored in twisted magnetic fields, usually near sunspots, is suddenly released. Yet, their occurrence is not fully understood. These flares can affect the Earth by the release of massive quantities of charged particles and electromagnetic radiation. Investigating the associations between solar flares and sunspot groups is helpful in comprehending the possible cause and effect relationships among solar flares and sunspot features. 01 This thesis proposes a new approach developed by integrating advances in image processing, machine learning, and deep learning with advances in solar physics to extract valuable knowledge from historical solar data related to sunspot regions and flares. This dissertation aims to achieve the following: 1) We developed a new prediction algorithm based on the Automated Solar Activity Prediction system (ASAP) system. The proposed algorithm updates the ASAP system by extending the training process and optimizing the learning rules to the optimize performance better. Two neural networks are used in the proposed approach. The first neural network is used to predict whether a specific sunspot class at a particular time is likely to produce a significant flare or not. The second neural network is used to predict the type of this flare, X or M-class. 2) We proposed a new system called the ASAP_Deep system built on top of the ASAP system introduced in [6] but improves the system with an updated deep learning-based prediction capability. In addition, we successfully apply Convolutional Neural Network (CNN) to the sunspot group image without any pr-eprocessing or feature extraction. Moreover, our system results are considerably better, especially for the false alarm ratio (FAR); this reduces the losses resulting from the protection measures applied by companies. In addition, the proposed system achieves a relatively high score of True Skill Statistic (TSS) and Heidke Skill Score (HSS). 3) We presented a novel system that used the Deep Belief Networks (DBNs) to predict the solar flares occurrence. The input data are SDO/HMI Intensitygram and Magnetogram images. The model outputs are "Flare or No-Flare" of significant flare occurrence (M and X-class flares). In addition, we created a dataset from the sunspots groups extracted from SDO HMI Intensitygram images. We compared the results obtained from the complete suggested system with those of three previous flare forecast models using several statistical metrics. In our view, these developed methods and results represent an excellent initial step toward enhancing the accuracy of flare forecasting, enhance our understanding of flare occurrence, and develop efficient flare prediction systems. The systems, implementation, results, and future work are explained in this dissertation.

This thesis leverages vibration-based unsupervised learning and deep transfer learning to reduce the manual labour involved in building algorithms that perform intelligent fault detection (IFD) on roller element bearings. A review of theory and literature in the field of IFD is presented, and challenges are discussed. An issue is then introduced; current machine learning models built for IFD show strong performance on a small subset of specific data, but do not generalize to a broader range of applications. Signal processing, machine learning, and transfer learning concepts are then explained and discussed. Time-frequency fingerprinting, as well as feature engineering, is used in conjunction with principal component analysis (PCA) to prepare vibration signals to be clustered by a gaussian mixture model (GMM). This process allows for the intelligent referral of data towards algorithms that have performed well on similar datasets and favours the re-use of domain-specific tasks. An algorithm is then proposed that promotes generalization in convolutional neural networks (CNNs) and simplifies the hyperparameter tuning process to allow machine learning models to be applied to a broader set of problems. The machine learning process is then automated as much as possible through meta learning and ensemble models: data similarity measurements are used to evaluate the data fit for transfer and propose training guidelines. Throughout the thesis, three open-source bearing fault datasets are used to test and validate the hypotheses. This thesis focuses on developing and adapting current deep learning models to succeed in challenging domains and real-world scenarios, while improving performance with unsupervised learning and transfer learning.