Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 246 for “"Knowledge Distillation"”.

  1. Continual Domain Adaptation through Knowledge Distillation

    … called Continual Domain Adaptation through Knowledge Distillation (CAKE), that uses knowledge distillation (KD) to transfer to a CNN the more complex Vision Transformer's knowledge. By doing so, CNN-based adaptation obtains a similar performance to transformer-based adaptation while reducing …

    rit Repository record for Continual Domain Adaptation through Knowledge Distillation (opens in a new tab)

  2. Enhanced knowledge distillation by auxiliary classifiers

    … proposed approaches to mitigate this limitation, knowledge distillation has gained much attention due to its generalizability and simplicity in implementation. This thesis introduces the enhanced knowledge distillation (EKD), a simple yet effective approach to outperform the canonical knowledge …

    uoit Repository record for Enhanced knowledge distillation by auxiliary classifiers (opens in a new tab)

  3. DEEPFAKE DETECTION GENERALIZATION VIA KNOWLEDGE DISTILLATION

    … to address these problems, through the use of knowledge distillation (KD) using heterogeneous teachers. In a typical class setting, a student learns from several teachers, each an expert in their domain. This process is analogous to the use of KD with heterogeneous teachers. A contribution of …

    hawaii Repository record for DEEPFAKE DETECTION GENERALIZATION VIA KNOWLEDGE DISTILLATION (opens in a new tab)

  4. KNOWLEDGE DISTILLATION FOR MOLECULAR PROPERTY PREDICTION

    Knowledge distillation (KD) is a powerful model compression technique that transfers knowledge from complex teacher models to compact student models, reducing computational costs while preserving predictive accuracy. This study investigated KD's efficacy in molecular property prediction across …

    cornell Repository record for KNOWLEDGE DISTILLATION FOR MOLECULAR PROPERTY PREDICTION (opens in a new tab)

  5. StarGAN-v2 compression using knowledge distillation

    … on the task of compressing StarGAN-v2 using knowledge distillation. Using depthwise separable convolutional layers and reduced channels for intermediate layers, we develop efficient architectures for different StarGAN-v2 modules. In a GAN mini-max optimization environment, the efficient …

    concordia Download PDF for StarGAN-v2 compression using knowledge distillation (opens in a new tab)

  6. Compressing Deep Neural Networks via Knowledge Distillation

    There has been a continuous evolution in deep neural network architectures since Alex Krizhevsky proposed AlexNet in 2012. Part of this has been due to increased complexity of the data and easier availability of datasets and part of it has been due to increased complexity of applications. These two …

    clemson Repository record for Compressing Deep Neural Networks via Knowledge Distillation (opens in a new tab)

  7. Minifying Deep Denoising Networks with Knowledge Distillation

    … investigating and measuring the effectiveness of knowledge distillation in creating a smaller and faster model. In using knowledge distillation, we extract the capabilities of a larger, more powerful teacher model into a smaller, more efficient student model, optimizing its suitability for …

    calpoly Repository record for Minifying Deep Denoising Networks with Knowledge Distillation (opens in a new tab)

  8. Multi-Teacher Knowledge Distillation Using Teacher's Domain Expertise

    … with low computing power and storage capacity. Knowledge Distillation solves this problem by distilling knowledge into a smaller BERT model while retaining much of the teacher’s accuracy in student. A teacher expert in predicting one class should be chosen, by student, over others for that class …

    ndsu Repository record for Multi-Teacher Knowledge Distillation Using Teacher's Domain Expertise (opens in a new tab)

  9. Improving Deep Neural Network Training with Knowledge Distillation

    Knowledge distillation, as a popular compression technique, has been widely used to reduce deep neural network (DNN) size for a variety of applications. However, in recent years, some research had found its potential for improving deep neural network performance. This dissertation focuses on …

    ucf Repository record for Improving Deep Neural Network Training with Knowledge Distillation (opens in a new tab)

  10. Training Binary Deep Neural Networks Using Knowledge Distillation

    … of a binary network can be improved by using knowledge distillation. Three different knowledge distillation methods were tested for various network types. Additionally, different architectures of a residual block in ResNet were suggested and tested. Test on CIFAR10 showed an 1.5% increase in …

    chalmers Repository record for Training Binary Deep Neural Networks Using Knowledge Distillation (opens in a new tab)

  11. Knowledge Distillation as a Solution for Domain Transfer

    … in fact verification tasks. We present a data distillation technique for delexicalization, which we then combine with a model distillation method to prevent aggressive data distillation. We show that by using our solution, not only does the performance of an ex- isting state-of-the-art model …

    arizona-thes Repository record for Knowledge Distillation as a Solution for Domain Transfer (opens in a new tab)

  12. Efficient transformer-based panoptic segmentation via knowledge distillation

    Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms

    uiuc Repository record for Efficient transformer-based panoptic segmentation via knowledge distillation (opens in a new tab)

  13. Knowledge Distillation Circumvents Nonlinearity for Optical Convolutional Neural Networks

    … (SCLC) network architecture and develop a Knowledge Distillation (KD) approach to circumvent the need for nonlinearity and successfully train such networks. While the KD approach is known in machine learning as an effective process for network pruning, we adapt the approach to transfer the …

    washington Repository record for Knowledge Distillation Circumvents Nonlinearity for Optical Convolutional Neural Networks (opens in a new tab)

  14. Knowledge Distillation and Its Application to Network Traffic Classification

    Knowledge distillation improves the performance of a low-complexity student model with the help of a more powerful teacher. In this dissertation, we study knowledge distillation and its application to network traffic. We posit that knowledge distillation is an essential ingredient of traffic …

    toronto-retro Repository record for Knowledge Distillation and Its Application to Network Traffic Classification (opens in a new tab)

  15. Unsupervised Contrastive Representation Learning for Knowledge Distillation and Clustering

    … learning methods and apply them to knowledge distillation and clustering. The knowledge distillation transfers knowledge from high-capacity teachers to small student models and then improves the performance of students. And the representational knowledge distillation methods try to …

    clemson Repository record for Unsupervised Contrastive Representation Learning for Knowledge Distillation and Clustering (opens in a new tab)

  16. Knowledge Distillation in Federated Magnetic Resonance Image (MRI) Reconstruction

    … and data privacy issues. On the other hand, knowledge distillation is a student-teacher-based model compression technique that can address communication and data privacy issues. In this thesis, we explore the inclusion of knowledge distillation in a Federated MRI reconstruction framework to …

    buffalo Repository record for Knowledge Distillation in Federated Magnetic Resonance Image (MRI) Reconstruction (opens in a new tab)

  17. DistilBERT-EL-KLD: Knowledge Distillation and Greek Language Modeling

    Σκοπός της εν λόγω διπλωματικής εργασίας είναι η ανάπτυξη, εκπαίδευση και αξιολόγηση ενός μοντέλου DistilBERT, εξειδικευμένου αποκλειστικά στην Ελληνική γλώσσα. Μετά από εμβριθή ανασκόπηση του θεωρητικού υποβάθρου (κλασική μηχανική μάθηση, βαθιά μάθηση και νευρωνικά δίκτυα) με σκοπό να καταστεί …

    athens Repository record for DistilBERT-EL-KLD: Knowledge Distillation and Greek Language Modeling (opens in a new tab)

  18. Domain-Aware Reasoning with Lightweight Models via Knowledge Distillation

    … This thesis investigates the efficacy of knowledge distillation for transferring advanced reasoning from a large teacher model (DeepSeek-R1) to a lightweight student model (Meta-Llama-3.1-8B-Instruct) within a Retrieval-Augmented Generation (RAG) framework for cybersecurity intelligence. …

    chalmers Repository record for Domain-Aware Reasoning with Lightweight Models via Knowledge Distillation (opens in a new tab)

  19. Temperature-centric investigation of speculative decoding with knowledge distillation

    … speculative decoding’s efficacy. Beginning with knowledge distillation (KD), we first highlight the challenge of decoding at higher temperatures, and demonstrate KD in a consistent temperature setting could be a remedy. We also investigate the effects of out-of-domain testing sets with …

    uiuc Repository record for Temperature-centric investigation of speculative decoding with knowledge distillation (opens in a new tab)

Page 1 of 13