Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 21 for “"QLoRA"”.
-
Chatbot založený na hlubokých neuronových sítích
… jako jsou LoRA (Low-Rank Adaptation) a QLoRA (Quantized Low-Rank Adaptation), si kladou za cíl tyto nároky snížit při zachování konkurenceschopného výkonu. Tato práce experimentálně porovnává tři přístupy k adaptaci velkých jazykových modelů: plné dolaďování, LoRA a QLoRA. Metody jsou …
-
Minimal-Data Peptide Design and Accessible Protein Language Models for Protein Engineering
… I present Quantized Low-Rank Adaptation (QLoRA) for protein language models (pLMs), combining 4-bit weight quantization with low-rank adapters to enable efficient fine-tuning on lower cost and accessible GPUs. Across several pLMs (8 million–3 billion parameters) QLoRA cuts required …
-
Advanced Quantization Techniques for Communication Efficiency and Privacy in Federated Learning, and Memory-efficient Fine-tuning of LLMs
… of LLMs, conventional quantization methods like QLoRA fall short for extremely low-bit fine-tuning tasks. This dissertation addresses these three challenges of quantization from an optimization perspective. Specifically, it presents: 1. Federated Optimization Algorithm with Acceleration and …
-
A First Study of Generative CommonSense Reasoning in Greek
… υποστεί προσαρμογή παραμέτρων (finetuning) μέσω QLoRA. Τα πειράματα περιλάμβαναν διαφορετικές στρατηγικές prompting και αποκωδικοποίησης. Η αξιολόγηση συνδυάζει αυτόματες μετρικές, όπως BLEU-4 και CIDEr, με ανθρώπινη αξιολόγηση που μετρά τη γραμματική ορθότητα, τη λογική συνοχή και την κάλυψη των …
-
Autonomous Deficiency Detection and Vision-Language Summarization for Underground Infrastructure on Embedded Edge Systems
… Model pipeline employing LoRA and 4-bit QLoRA quantization compresses Phi-3.5 for local deployment, en- abling autonomous technical report generation without cloud connectivity. Third, multi-modal sensor fusion integrating sparse LiDAR with RGB imagery via EfficientPENet and DMD3C …
-
Velké jazykové modely pro generování kódu se zaměřením na vestavěné systémy
… 7B a pri jeho ladení bola využitá technika QLoRA pre minimalizáciu výpočtových nárokov ladenia. Druhý model, MicroCoderFIM, je založený na modeli StarCoderBase 1B a podporuje vyplňovanie kódu na základe okolia (fill-in-the-middle). Jednotlivé modely boli porovnávané na základe metrík BLEU, …
-
Generování popisů obrázků pomocí neuronových sítí
… historických dokumentov pomocou techník LoRA a QLoRA. Pre potreby trénovania a evaluácie bol vytvorený vlastný anotovaný dataset obsahujúci štruktúrované popisy vo forme detailného opisu, stručného titulku a kľúčových slov. Hyperparametre modelov boli optimalizované sekvenčným prístupom. Kvalita …
-
Systém pro vytěžování zpravodajských informací založený na velkých jazykových modelech
… –- zero-shot prompting, few-shot prompting a QLoRa fine-tuning – ve srovnání se dvěma baseline modely: OpenAI GPT-4o-mini a XLM-RoBERTa (fine-tuned, s zploštělými zjednodušenými entitami). Vyhodnocení proběhlo na sto oanotovaných dokumentech v českém jazyce z blogu “valka.online” a metriky …
-
Adapting Without Forgetting
… Cellpose-SAM was subsequently fine-tuned using QLoRA, a parameter-efficient method that adapts only a small fraction of the model’s weights. Fine-tuning on a small curated dataset yielded substantial improvement on challenging microscopy images while retaining near-baseline performance on …
-
"TheraBot" : Ανάπτυξη και Αξιολόγηση Ελληνόγλωσσου Chatbot Ψυχικής Υγείας
… Στη συνέχεια, το Krikri προσαρμόστηκε με LoRA/QLoRA για το αντικείμενο της εργασίας, με τεκμηρίωση υπερπαραμέτρων, υποδομής και σημείων ελέγχου. Παρουσιάζονται επίσης οι καμπύλες εκπαίδευσης και ο σχεδιασμός prompt/system-role, που επιβάλλει την αποφυγή «θεραπευτικών» ισχυρισμών και τη χρήση …
-
Efficient Learning for Large Language Models
… pass, achieving competitive performance to QLoRA while reducing GPU memory usage by up to 65% and supporting flexible rank selection at inference time. Second, Sorted-LoRA introduces a stochastic depth–aware fine-tuning framework that co-trains multiple sub-models of varying depths within a …
-
Radiology Report Summarization With Large Language Models
… count, the LLaMa model was fine-tuned using QLoRA—a quantized adaptation approach that enables memory-efficient training—while T5-Base was fine-tuned using standard LoRA. Our results demonstrate substantial performance gains over the respective base models. For LLaMa, improvements of 174.18%, …
-
Efficient Scaling of Language Models
… with minimal computational overhead. QLoRA combines novel 4-bit quantization with parameter efficient finetuning, reducing memory requirements of supervised learning by 15× without performance degradation. This efficiency enabled comprehensive instruction-tuning studies revealing that …
-
Accessible Foundation Models: Systems, Algorithms, and Science
… parameter. For finetuning, I will introduce the QLoRA algorithm, which pushes such quantization much further to unlock the finetuning of very large models on a single GPU by only updating a small set of the parameters while keeping most of the network in a new information-theoretically optimal …
-
Modular LLM Pipeline for Task-Oriented Customer Service Dialogue
… μοντέλα για τον ρόλο που τους έχει ανατεθεί μέσω QLoRA. Και οι τρεις αρχιτεκτονικές αξιολογούνται με βάση το MultiWOZ, αναφέροντας τόσο μετρικές κατάστασης διαλόγου (Joint Goal Accuracy, Slot F1) όσο και μετρικές end-to-end (Inform, Success, BLEU, Combined), και επιπλέον μετρικές όπως …
-
Enhancing language models for telecom networks using retrieval-augmented generation: A pipeline for telecom-specific QA system
… vector, and parameter-efficient tuning with QLoRA (Quantized Low-Rank Adaptation) for computational scalability. We incorporate an incremental fine-tuning method of the Small Language Models (SLMs) to reduce hardware and resource overhead while ensuring superior performance in answering the …
-
Advanced machine learning techniques for data analysis and processing
… descriptors and fine-tunes a causal LLM via QLoRA on structured prompts derived from inertial sensor data. The framework is evaluated on two human activity recognition benchmarks and includes an attention-based interpretability analysis that links predictions to specific symbolic motifs on …
-
Intent-driven Radio Access Network Management Using Advanced Machine Learning Techniques
… enhanced through Quantized Low-Rank Adapter (QLORA)-based fine-tuning to improve domain adaptation and intent interpretation accuracy. A key limitation of the hierarchical DT-based RAN controller is its reliance on high- quality offline Reinforcement Learning (RL) trajectories for training. As …
-
Sensi-Train: Context-aware Sensitive Content Detection using Transformers and Large Language Models
The detection of sensitive content has become a critical task in natural language processing due to the widespread use of online platforms and the social harm caused by it. Even though current methods have advanced significantly, many of them mainly rely on surface-level text and neglect to take …
-
Adapting Large Language Models for Effective and Scalable Information Extraction from Oncologic FDG PET-CT Imaging Reports
… and was adapted to the task using QLoRA and chain-of-thought prompting. This achieved document-level F1 scores of 0.92 and 0.79, and sentence-level F1 scores of 0.75 and 0.52, on the internal and external datasets respectively. Qualitative analysis demonstrated how sentence-level …
Page 1 of 2