No description available.

No description available.

We introduce Domain Aware Continual Zero-Shot Learning (DACZSL), the task of visually recognizing images of unseen categories in unseen domains sequentially. We created DACZSL on top of the DomainNet dataset by dividing it into a sequence of tasks, where classes are incrementally provided on seen domains during training and evaluation is conducted on unseen domains for both seen and unseen classes. We also proposed a novel Domain-Invariant CZSL Network (DIN), which outperforms state-of-the-art baseline models that we adapted to DACZSL setting. We adopt a structure-based approach to alleviate forgetting knowledge from previous tasks with a small per-task private network in addition to a global shared network. To encourage the private network to capture the domain and task-specific representation, we train our model with a novel adversarial knowledge disentanglement setting to make our global network task-invariant and domain-invariant over all the tasks. Our method also learns a class-wise learnable prompt to obtain better class-level text representation, which is used to represent side information to enable zero-shot prediction of future unseen classes. Our code and benchmarks are made available at https://zero-shot-learning.github.io/daczsl.

Indiana University-Purdue University Indianapolis (IUPUI) / Deep learning solutions to computer vision tasks have revolutionized many industries in recent years, but embedded systems have too many restrictions to take advantage of current state-of-the-art configurations. Typical embedded processor hardware configurations must meet very low power and memory constraints to maintain small and lightweight packaging, and the architectures of the current best deep learning models are too computationally-intensive for these hardware configurations. Current research shows that convolutional neural networks (CNNs) can be deployed with a few architectural modifications on Field-Programmable Gate Arrays (FPGAs) resulting in minimal loss of accuracy, similar or decreased processing speeds, and lower power consumption when compared to general-purpose Central Processing Units (CPUs) and Graphics Processing Units (GPUs). This research contributes further to these findings with the FPGA implementation of a YOLOv4 object detection model that was developed with the use of transfer learning. The transfer-learned model uses the weights of a model pre-trained on the MS-COCO dataset as a starting point then fine-tunes only the output layers for detection on more specific objects of five classes. The model architecture was then modified slightly for compatibility with the FPGA hardware using techniques such as weight quantization and replacing unsupported activation layer types. The model was deployed on three different hardware setups (CPU, GPU, FPGA) for inference on a test set of 100 images. It was found that the FPGA was able to achieve real-time inference speeds of 33.77 frames-per-second, a speedup of 7.74 frames-per-second when compared to GPU deployment. The model also consumed 96% less power than a GPU configuration with only approximately 4% average loss in accuracy across all 5 classes. The results are even more striking when compared to CPU deployment, with 131.7-times speedup in inference throughput. CPUs have long since been outperformed by GPUs for deep learning applications but are used in most embedded systems. These results further illustrate the advantages of FPGAs for deep learning inference on embedded systems even when transfer learning is used for an efficient end-to-end deployment process. This work advances current state-of-the-art with the implementation of a YOLOv4 object detection model developed with transfer learning for FPGA deployment.

This thesis studies to automatically solve Arabic Math Word Problems (MWPs) by deep learning models. MWP is a text description of a mathematical problem, which should be solved by deriving a math equation and reach the answer. Due to their strong learning capacity, deep learning based models can learn from the given problem description and generate the correct math equation for solving the problem. Effective models have been developed for solving MWPs in English and Chinese. However, Arabic MWPs are rarely studied. To initiate the study in Arabic MWPs, this thesis contributes the first large-scale dataset for Arabic MWPs, which contain 6,000 samples. Each sample is composed of an Arabic MWP description and the corresponding equation to solve this MWP. Arabic MWP solvers are then built with deep learning models, and verified on this dataset for their effectiveness. In addition, a transfer learning model is built to let the high-resource Chinese MWP solver to promote the performance of the low-resource Arabic MWP solver. This work is the first to use deep learning methods to solve Arabic MWP and the first to use transfer learning to solve MWP across different languages. The solver enhanced by transfer learning has accuracy 74.15%, which is 3% higher than the baseline that does not use transfer learning. In addition, the accuracy is more than 7% higher than the baseline for templates with few samples representing them. Furthermore, The model can generate new sequences that were not seen before during the training with an accuracy of 27% (11% higher than the baseline).

Handwriting recognition systems have achieved remarkable performance over the past several years with the advent of deep neural networks. For high-quality recognition, these models require large amounts of labeled training data, which can be difficult to obtain. Various methods to reduce this effort have been proposed in the realms of active and transfer learning, but not in combination. We propose a framework for fitting new handwriting recognition models that joins active and transfer learning into a unified framework. Empirical results show the superiority of our method compared to traditional active learning, transfer learning, or standard supervised training schemes.

abstract: Deep learning is a sub-field of machine learning in which models are developed to imitate the workings of the human brain in processing data and creating patterns for decision making. This dissertation is focused on developing deep learning models for medical imaging analysis of different modalities for different tasks including detection, segmentation and classification. Imaging modalities including digital mammography (DM), magnetic resonance imaging (MRI), positron emission tomography (PET) and computed tomography (CT) are studied in the dissertation for various medical applications. The first phase of the research is to develop a novel shallow-deep convolutional neural network (SD-CNN) model for improved breast cancer diagnosis. This model takes one type of medical image as input and synthesizes different modalities for additional feature sources; both original image and synthetic image are used for feature generation. This proposed architecture is validated in the application of breast cancer diagnosis and proved to be outperforming the competing models. Motivated by the success from the first phase, the second phase focuses on improving medical imaging synthesis performance with advanced deep learning architecture. A new architecture named deep residual inception encoder-decoder network (RIED-Net) is proposed. RIED-Net has the advantages of preserving pixel-level information and cross-modality feature transferring. The applicability of RIED-Net is validated in breast cancer diagnosis and Alzheimer’s disease (AD) staging. Recognizing medical imaging research often has multiples inter-related tasks, namely, detection, segmentation and classification, my third phase of the research is to develop a multi-task deep learning model. Specifically, a feature transfer enabled multi-task deep learning model (FT-MTL-Net) is proposed to transfer high-resolution features from segmentation task to low-resolution feature-based classification task. The application of FT-MTL-Net on breast cancer detection, segmentation and classification using DM images is studied. As a continuing effort on exploring the transfer learning in deep models for medical application, the last phase is to develop a deep learning model for both feature transfer and knowledge from pre-training age prediction task to new domain of Mild cognitive impairment (MCI) to AD conversion prediction task. It is validated in the application of predicting MCI patients’ conversion to AD with 3D MRI images. / Dissertation/Thesis / Doctoral Dissertation Industrial Engineering 2019

The Contracts Register is a public database containing contracts concluded by public institutions. Due to the number of documents in the database, data analysis is proble- matic. The objective of this thesis is to find a machine learning approach for sorting the contracts into categories by their area of interest (real estate services, construction, etc.) and implement the approach for usage on the web portal Hlídač státu. A large number of categories and a lack of a tagged dataset of contracts complicate the solution. 1

We refactor the traditional two-step approach of automatic speech recognition for spoken language translation. Instead of conventional graphemes, we use phonemes as an intermediate speech representation. Starting with the acoustic model, we revise the cross-lingual transfer and propose a coarse-to-fine method providing further speed-up and performance boost. Further, we review the translation model. We experiment with source and target encoding, boosting the robustness by utilizing the fine-tuning and transfer across ASR and SLT. We empirically document that this conventional setup with an alternative representation not only performs well on standard test sets but also provides robust transcripts and translations on challenging (e.g., non-native) test sets. Notably, our ASR system outperforms commercial ASR systems. 1

Computational ECG (electrocardiogram) analysis enables accurate and faster diagnosis and early prediction of heart failure related symptoms (heart failure worsening). Machine learning, particularly deep learning, has been applied for ECG data successfully. The previous applications, however, either mainly focused on classifying occurrent, known patterns of on-going heart failure or heart failure related diseases such arrhythmia, which have undesirable predictability beforehand, or emphasizing on data from pre-processed public database data. In this dissertation, we developed an approach, however, does not fully capitalize on the potential of deep learning, which directly learns important features from raw input data without relying on a priori knowledge. Here, we present a deep transfer learning pipeline which combines an image-based pretrained deep neural network model with manifold learning to predict the precursors of heart failure (heart failure-worsening and recurrent heart failure related re-hospitalization) using raw ECG time series from wearable devices. In this dissertation, we used the unprocessed real-life ECG data from the SENTINEL-HF study by Dovancescu, et al. to predict the precursors of heart failure worsening. To extract rich features from ECG time series, we took a deep transfer learning approach where 1D time-series of five heartbeats were transformed to 2D images by Gramian Angular Summation Field (GASF) and then the pretrained models, VGG19 were used for feature extraction. Then, we applied UMAP (Uniform Manifold Approximation and Projection) to capture the manifold of the standardized feature space and reduce the dimension, followed by SVM (Support Vector Machine) training. Using our pipeline, we demonstrated that our classifier was able to predict heart failure worsening with 92.1% accuracy, 92.9% precision, 92.6% recall and F1 score of 0.93 bypassing the detection of known abnormal ECG patterns. In conclusion, we demonstrate the feasibility of early alerts of heart failure by predicting the precursor of heart failure worsening based on raw ECG signals. We expected that our approached provided an innovative method to assess the recovery and successfulness for the treatment patient received during the first hospitalization, to predict whether recurrent heart failure is likely to occur, and to evaluate whether the patient should be discharged.