Indiana University-Purdue University Indianapolis (IUPUI) / Medication non-adherence and non-compliance left unaddressed can compound into severe medical problems for patients. Identifying patients that are likely to become non-adherent can help reduce these problems. Despite these benefits, monitoring adherence at scale is cost-prohibitive. Social forums offer an easily accessible, affordable, and timely alternative to the traditional methods based on claims data. This study investigates the potential of medication adherence prediction based on social forum data for diabetes and fibromyalgia therapies by using transfer learning from the Medical Expenditure Panel Survey (MEPS). Predictive adherence models are developed by using both survey and social forums data and different random forest (RF) techniques. The first of these implementations uses binned inputs from k-means clustering. The second technique is based on ternary trees instead of the widely used binary decision trees. These techniques are able to handle missing data, a prevalent characteristic of social forums data. The results of this study show that transfer learning between survey models and social forum models is possible. Using MEPS survey data and the techniques listed above to derive RF models, less than 5% difference in accuracy was observed between the MEPS test dataset and the social forum test dataset. Along with these RF techniques, another RF implementation with imputed means for the missing values was developed and shown to predict adherence for social forum patients with an accuracy >70%. This thesis shows that a model trained with verified survey data can be used to complement traditional medical adherence models by predicting adherence from unverified, self-reported data in a dynamic and timely manner. Furthermore, this model provides a method for discovering objective insights from subjective social reports. Additional investigation is needed to improve the prediction accuracy of the proposed model and to assess biases that may be inherent to self-reported adherence measures in social health networks.

京都大学 / 新制・課程博士 / 博士(工学) / 甲第23615号 / 工博第4936号 / 新制||工||1771(附属図書館) / 京都大学大学院工学研究科機械理工学専攻 / (主査)教授 椹木 哲夫, 教授 松野 文俊, 教授 藤本 健治 / 学位規則第4条第1項該当 / Doctor of Philosophy (Engineering) / Kyoto University / DFAM

The introduction of Machine Learning (ML) and Deep Learning (DL) techniques into modern radio communications system, a field known as Radio Frequency Machine Learning (RFML), has the potential to provide increased performance and flexibility when compared to traditional signal processing techniques and has broad utility in both the commercial and defense sectors. Existing RFML systems predominately utilize supervised learning solutions in which the training process is performed offline, before deployment, and the learned model remains fixed once deployed. The inflexibility of these systems means that, while they are appropriate for the conditions assumed during offline training, they show limited adaptability to changes in the propagation environment and transmitter/receiver hardware, leading to significant performance degradation. Given the fluidity of modern communication environments, this rigidness has limited the widespread adoption of RFML solutions to date. Transfer Learning (TL) is a means to mitigate such performance degradations by re-using prior knowledge learned from a source domain and task to improve performance on a "similar" target domain and task. However, the benefits of TL have yet to be fully demonstrated and integrated into RFML systems. This dissertation begins by clearly defining the problem space of RF TL through a domain-specific TL taxonomy for RFML that provides common language and terminology with concrete and Radio Frequency (RF)-specific example use- cases. Then, the impacts of the RF domain, characterized by the hardware and channel environment(s), and task, characterized by the application(s) being addressed, on performance are studied, and methods and metrics for predicting and quantifying RF TL performance are examined. In total, this work provides the foundational knowledge to more reliably use TL approaches in RF contexts and opens directions for future work that will improve the robustness and increase the deployability of RFML. / Doctor of Philosophy / The field of Radio Frequency Machine Learning (RFML) introduces Machine Learning (ML) and Deep Learning (DL) techniques into modern radio communications systems, and is expected to be a core component of 6G technologies and beyond. While RFML provides a myriad of benefits over traditional radio communications systems, existing approaches are generally incapable of adapting to changes that will inevitably occur over time, which causes severe performance degradation. Transfer Learning (TL) offers a solution to the inflexibility of current RFML systems, through techniques for re-using and adapting existing models for new, but similar, problems. TL is an approach often used in image and language-based ML/DL systems, but has yet to be commonly used by RFML researchers. This dissertation aims to provide the foundational knowledge necessary to reliably use TL in RFML systems, from the definition and categorization of RF TL techniques to practical guidelines for when to use RF TL in real-world systems. The unique elements of RF TL not present in other modalities are exhaustively studied, and methods and metrics for measuring and predicting RF TL performance are examined.

Retrogressive thaw slumps (RTSs) are thermokarst landforms that develop on slopes in permafrost regions when thawing permafrost causes the land surface to collapse. RTSs are an indicator of climate change and pose a threat to human infrastructure and ecosystems in the affected areas. As the availability of ready-to-use high-resolution satellite imagery increases, automatic RTS mapping is being explored with deep learning methods. We employed a pre-trained Mask-RCNN model to automatically map RTSs on Banks Island and Victoria Island in the western Canadian Arctic, where there is extensive RTS activity. We tested the model with different settings, including image band combinations, backbones, and backbone trainable layers, and performed hyper-parameter tuning and determined the optimal learning rate, momentum, and decay rate for each of the model settings. Our final model successfully mapped most of the RTSs in our test sites, with F1 scores ranging from 0.61 to 0.79. Our study demonstrates that transfer learning from a pre-trained Mask-RCNN model is an effective approach that has the potential to be applied for RTS mapping across the Canadian Arctic.

Deep learning is increasingly becoming a viable way of classifying all types of data. Modern deep learning algorithms, such as one dimensional convolutional neural networks, have demonstrated excellent performance in classifying time series data because of the ability to identify time invariant features. A primary challenge of deep learning for time series classification is the large amount of data required for training and many application domains, such as in medicine, have challenges obtaining sufficient data. Transfer learning is a deep learning method used to apply feature knowledge from one deep learning model to another; this is a powerful tool when both training datasets are similar and offers smaller datasets the power of more robust larger datasets. This makes it vital that the best source dataset is selected when performing transfer learning and presently there is no metric for this purpose. In this thesis a metric of predicting the performance of transfer learning is proposed. To develop this metric this research will focus on classification and transfer learning for human-activity-recognition time series data. For general time series data, finding temporal relations between signals is computationally intensive using non-deep learning techniques. Rather than time-series signal processing, a neural network autoencoder was used to first transform the source and target datasets into a time independent feature space. To compare and quantify the suitability of transfer learning datasets, two metrics were examined: i) average embedded signal from each dataset was used to calculate the distance between each datasets centroid, and ii) a Generative Adversarial Network (GAN) model was trained and the discriminator portion of the GAN is then used to assess the dissimilarity between source and target. This thesis measures a correlation between the distance between two dataset and their similarity, as well as the ability for a GAN to discriminate between two datasets and their similarity. The discriminator metric, however, does suffer from an upper limit of dissimilarity. These metrics were then used to predict the success of transfer learning from one dataset to another for the purpose of general time series classification. / Thesis / Master of Applied Science (MASc) / Over the past decade, advances in computational power and increases in data quantity have made deep learning a useful method of complex pattern recognition and classification in data. There is a growing desire to be able to use these complex algorithms on smaller quantities of data. To achieve this, a deep learning model is first trained on a larger dataset and then retrained on the smaller dataset; this is called transfer learning. For transfer learning to be effective, there needs to be a level of similarity between the two datasets so that properties from larger dataset can be learned and then refined using the smaller dataset. Therefore, it is of great interest to understand what level of similarity exists between the two datasets. The goal of this research is to provide a similarity metric between two time series classification datasets so that potential performance gains from transfer learning can be better understood. The measure of similarity between two time series datasets presents a unique challenge due to the nature of this data. To address this challenge an encoder approach was implemented to transform the time series data into a form where each signal example can be compared against one another. In this thesis, different similarity metrics were evaluated and correlated to the performance of a deep learning model allowing the prediction of how effective transfer learning may be when applied.

The objective of this dissertation is to provide key methodological advancements towards the use of transfer learning in Laser-Based Additive Manufacturing (LBAM), to assist practitioners in producing high-quality repeatable parts. Currently, in LBAM processes, there is an urgent need to improve the quality and repeatability of the manufacturing process. Fabricating parts using LBAM is often expensive, due to the high cost of materials, the skilled machine operators needed for operation, and the long build times needed to fabricate parts. Additionally, monitoring the LBAM process is expensive, due to the highly specialized infrared sensors needed to monitor the thermal evolution of the part. These factors lead to a key challenge of improving the quality of additively manufactured parts, because additional experiments and/or sensors is expensive. We propose to use transfer learning, which is a statistical technique for transferring knowledge from one domain to a similar, yet distinct, domain, to leverage previous non-identical experiments to assist practitioners in expediting part certification. By using transfer learning, previous experiments completed in similar, but non-identical, domains can be used to provide insight towards the fabrication of high-quality parts. In this dissertation, transfer learning is applied to four key domains within LBAM. First, transfer learning is used for sensor fusion, specifically to calibrate the infrared camera with true temperature measurements from the pyrometer. Second, a Bayesian transfer learning approach is developed to transfer knowledge across different material systems, by modelling material differences as a lurking variable. Third, a Bayesian transfer learning approach for predicting distortion is developed to transfer knowledge from a baseline machine system to a new machine system, by modelling machine differences as a lurking variable. Finally, compensation plans are developed from the transfer learning models to assist practitioners in improving the quality of parts using previous experiments. The work of this dissertation provides current practitioners with methods for sensor fusion, material/machine calibration, and efficient learning of compensation plans with few samples.

<p dir="ltr">Despite the remarkable advancements of machine learning (ML) technologies in biomedical research, especially in tackling complex human diseases such as cancer and Alzheimer's disease, a considerable gap persists between promising theoretical results and dependable clinical applications in diagnosis, prognosis, and therapeutic decision-making. One of the primary challenges stems from the absence of large high-quality patient datasets, which arises from the cost and human labor required for collecting such datasets and the scarcity of patient samples. Moreover, the inherent complexity of the data often leads to a feature space dimension that is large compared with the sample size, potentially causing instability during training and unreliability in inference. To address these challenges, the transfer learning (TL) approach has been embraced in biomedical ML applications to facilitate knowledge transfer across diverse and related biological contexts. Leveraging this principle, we introduce an unsupervised multi-view TL algorithm, named MVTOT [1], which enables the analysis of various biomarkers across different cancer types. Specifically, we compress high-dimensional biomarkers from different cancer types into a low-dimensional feature space via nonnegative matrix factorization and distill common information shared by various cancer types using the Wasserstein distance defined by Optimal Transport theory. We evaluate the stratification performance on three early-stage cancers from the Cancer Genome Atlas (TCGA) project. Our framework, compared with other benchmark methods, demonstrates superior accuracy in patient survival outcome stratification.</p><p dir="ltr">Additionally, while patient-level stratification has enhanced clinical decision-making, our understanding of diseases at the single-cell (SC) level remains limited, which is crucial for deciphering disease progression mechanisms, monitoring drug responses, and prioritizing drug targets. It is essential to associate each SC with patient-level clinical traits such as survival hazard, drug response, and disease subtypes. However, SC samples often lack direct labeling with these traits, and the significant statistical gap between patient and SC-level gene expressions impedes the transfer of well-annotated patient-level disease attributes to SCs. Domain adaptation (DA), a TL subfield, addresses this challenge by training a domain-invariant feature extractor for both patient and SC gene expression matrices, facilitating the successful application of ML models trained on patient-level data to SC samples. Expanding upon an established deep-learning-based DA model, DEGAS [2], we substitute their computationally ineffective maximum mean discrepancy loss with the Wasserstein distance as the metric for domain discrepancy. This substitution facilitates the embedding of both SC and patient inputs into a common latent feature space. Subsequently, employing the model trained on patient-level disease attributes, we predict SC-level survival hazard, disease status, and drug response for prostate cancer, Alzheimer's SC data, and multiple myeloma data, respectively. Our approach outperforms benchmark studies, uncovering clinically significant cell subgroups and revealing the correlation between survival hazard and drug response at the SC level.</p><p dir="ltr">Furthermore, in addition to these approaches, we acknowledge the effectiveness of TL and image analysis in stratifying patients with early and late-stage Mild Cognitive Impairment based on neuroimaging, as well as predicting survival and metastasis in melanoma based on histological images. These applications underscore the potential of employing ML methods, especially TL algorithms, in addressing biomedical issues from various angles, thereby enhancing our understanding of disease mechanisms and developing new biomarkers predicting patient outcomes.</p>

Radio frequency fingerprinting, a type of physical layer identification, allows identifying wireless transmitters based on their unique hardware. Every wireless transmitter has slight manufacturing variations and differences due to the layout of components. These are manifested as differences in the signal emitted by the device. A variety of techniques have been proposed for identifying transmitters, at the physical layer, based on these differences. This has been successfully demonstrated on a large variety of transmitters and other devices. However, some situations still pose challenges: Some types of fingerprinting feature are very dependent on the modulated signal, especially features based on the frequency content of a signal. This means that changes in transmitter configuration such as bandwidth or modulation will prevent wireless fingerprinting. Such changes may occur frequently with cognitive radios, and in dynamic spectrum access networks. A method is proposed to transform features to be invariant with respect to changes in transmitter configuration. With the transformed features it is possible to re-identify devices with a high degree of certainty. Next, improving performance with limited data by identifying devices using observations crowdsourced from multiple receivers is examined. Combinations of three types of observations are defined. These are combinations of fingerprinter output, features extracted from multiple signals, and raw observations of multiple signals. Performance is demonstrated, although the best method is dependent on the feature set. Other considerations are considered, including processing power and the amount of data needed. Finally, drift in fingerprinting features caused by changes in temperature is examined. Drift results from gradual changes in the physical layer behavior of transmitters, and can have a substantial negative impact on fingerprinting. Even small changes in temperature are found to cause drift, with the oscillator as the primary source of this drift (and other variation) in the fingerprints used. Various methods are tested to compensate for these changes. It is shown that frequency based features not dependent on the carrier are unaffected by drift, but are not able to distinguish between devices. Several models are examined which can improve performance when drift is present. / Doctor of Philosophy / Radio frequency fingerprinting allows uniquely identifying a transmitter based on characteristics of the signal it emits. In this dissertation several extensions to current fingerprinting techniques are given. Together, these allow identification of transmitters which have changed the signal sent, identifying using different measurement types, and compensating for variation in a transmitter's behavior due to changes in temperature.

Predicting formation energies of crystals is a common but computationally expensive task. In this work, it is therefore investigated how a neural network can be used as a tool for predicting formation energies with less computational cost compared to conventional methods. The investigated model shows promising results in predicting formation energies, reaching below a mean absolute error of 0.05 eV/atom with less than 4000 training datapoints. The model also shows great transferability, being able to reach below an MAE of 0.1 eV/atom with less than 100 training points when transferring from a pre-trained model. A drawback of the model is however that it is relying on descriptions of the crystal structures that include interatomic distances. Since these are not always accurately known, it is investigated how inaccurate structure descriptions affect the performance of the model. The results show that the quality of the descriptions definitely worsen the accuracy. The less accurate descriptions can however be used to reduce the search space in the creation of phase diagrams, and the proposed workflow which combines conventional density functional theory and machine learning shows a reduction in time consumption of more than 50 \% compared to only using density functional theory for creating a ternary phase diagram.

The automatic interpretation of signs of a sign language involves image recognition. An appropriate approach for this task is to use Deep Learning, and in particular, Convolutional Neural Networks. This method typically needs large amounts of data to be able to perform well. Transfer learning could be a feasible approach to achieve high accuracy despite using a small data set. The hypothesis of this thesis is to test if transfer learning works well to interpret the hand alphabet of the Swedish Sign Language. The goal of the project is to implement a model that can interpret signs, as well as to build a user-friendly web application for this purpose. The final testing accuracy of the model is 85%. Since this accuracy is comparable to those received in other studies, the project’s hypothesis is shown to be supported. The final network is based on the pre-trained model InceptionV3 with five frozen layers, and the optimization algorithm mini-batch gradient descent with a batch size of 32, and a step-size factor of 1.2. Transfer learning is used, however, not to the extent that the network became too specialized in the pre-trained model and its data. The network has shown to be unbiased for diverse testing data sets. Suggestions for future work include integrating dynamic signing data to interpret words and sentences, evaluating the method on another sign language’s hand alphabet, and integrate dynamic interpretation in the web application for several letters or words to be interpreted after each other. In the long run, this research could benefit deaf people who have access to technology and enhance good health, quality education, decent work, and reduced inequalities. / Automatisk tolkning av tecken i ett teckenspråk involverar bildigenkänning. Ett ändamålsenligt tillvägagångsätt för denna uppgift är att använda djupinlärning, och mer specifikt, Convolutional Neural Networks. Denna metod behöver generellt stora mängder data för att prestera väl. Därför kan transfer learning vara en rimlig metod för att nå en hög precision trots liten mängd data. Avhandlingens hypotes är att utvärdera om transfer learning fungerar för att tolka det svenska teckenspråkets handalfabet. Målet med projektet är att implementera en modell som kan tolka tecken, samt att bygga en användarvänlig webapplikation för detta syfte. Modellen lyckas klassificera 85% av testinstanserna korrekt. Då denna precision är jämförbar med de från andra studier, tyder det på att projektets hypotes är korrekt. Det slutgiltiga nätverket baseras på den förtränade modellen InceptionV3 med fem frysta lager, samt optimiseringsalgoritmen mini-batch gradient descent med en batchstorlek på 32 och en stegfaktor på 1,2. Transfer learning användes, men däremot inte till den nivå så att nätverket blev för specialiserat på den förtränade modellen och dess data. Nätverket har visat sig vara ickepartiskt för det mångfaldiga testningsdatasetet. Förslag på framtida arbeten inkluderar att integrera dynamisk teckendata för att kunna tolka ord och meningar, evaluera metoden på andra teckenspråkshandalfabet, samt att integrera dynamisk tolkning i webapplikationen så flera bokstäver eller ord kan tolkas efter varandra. I det långa loppet kan denna studie gagna döva personer som har tillgång till teknik, och därmed öka chanserna för god hälsa, kvalitetsundervisning, anständigt arbete och minskade ojämlikheter.