Spelling suggestions: "subject:"[een] TRANSFORMERS"" "subject:"[enn] TRANSFORMERS""
411 |
Active Learning for Extractive Question AnsweringMarti Roman, Salvador January 2022 (has links)
Data labelling for question answering tasks (QA) is a costly procedure that requires oracles to read lengthy excerpts of texts and reason to extract an answer for a given question from within the text. QA is a task in natural language processing (NLP), where a majority of recent advancements have come from leveraging the vast corpora of unlabelled and unstructured text available online. This work aims to extend this trend in the efficient use of unlabelled text data to the problem of selecting which subset of samples to label in order to maximize performance. This practice of selective labelling is called active learning (AL). Recent developments in AL for NLP have introduced the use of self-supervised learning on large corpora of text in the labelling process of samples for classification problems. This work adapts this research to the task of question answering and performs an initial exploration of expected performance. The methods covered in this work use uncertainty estimates obtained from neural networks to guide an incremental labelling process. These estimates are obtained from transformer-based models, previously trained in a self-supervised manner, by calculating the entropy of the confidence scores or with an approximation of Bayesian uncertainty obtained through Monte Carlo dropout. These methods are evaluated on two different benchmarking QA datasets: SQuAD v1 and TriviaQA. Several factors are observed to influence the behaviour of these uncertainty-based acquisition functions, including the choice of language model used, the presence of unanswered questions and the acquisition size used in the incremental process. The study produces no evidence to support that averaging or selecting maximal uncertainty values between the classification of an answer’s starting and ending positions affects sample acquisition quality. However, language model choice, the presence of unanswerable questions and acquisition size are all identified as key factors affecting consistency between runs and degree of success.
|
412 |
Optimering av en chattbot för det svenska språket / Optimization of a Chatbot for the Swedish LanguageMutaliev, Mohammed, Almimar, Ibrahim January 2021 (has links)
Chattbotutvecklare på Softronic använder i dagsläget Rasa-ramverket och dess standardkomponenter för bearbetning av användarinmatning. Det här är problematiskt då standardkomponenterna inte är optimerade för det svenska språket. Till följd av detta efterfrågades en utvärdering av samtliga Rasa-komponenter med syfte att identifiera de mest gynnsamma komponenterna för att maximera klassificeringsträffsäkerhet. I detta examensarbete framtogs och jämfördes flera Rasa-pipelines med olika komponenter för tokenisering, känneteckensextrahering och klassificering. Resultaten av komponenterna för tokenisering visade att Rasas WhitespaceTokenizer överträffade både SpacyTokenizer och StanzaTokenizer. För känneteckensextrahering var CountVectorsFeaturizer, LanguageModelFeaturizer (med LaBSE-modellen) och FastTextFeaturizer (med den officiella fastText-modellen tränad på svenska Wikipedia) de mest optimala komponenterna. Den klassificerare som i allmänhet presterade bäst var DIETClassifier, men det fanns flera tillfällen där SklearnIntentClassifier överträffade den. Detta arbete resulterade i flera pipelines som överträffade Rasas standard-pipeline. Av dessa pipelines var det två som presterade bäst. Den första pipeline implementerade komponenterna WhitespaceTokenizer, CountVectorsFeaturizer, FastTextFeaturizer (med den officiella fastText-modellen tränad på svenska Wikipedia) och DIETClassifier med en klassificeringsträffsäkerhet på 91% (F1-score). Den andra pipeline implementerade komponenterna WhitespaceTokenizer, LanguageModelFeaturizer (med LaBSE-modellen) och SklearnIntentClassifier med en klassificeringsträffsäkerhet på 91,5% (F1-score). / Chatbot developers at Softronic currently use the Rasa framework and its default components for processing user input. This is problematic as the default components are not optimized for the Swedish language. Following this an evaluation of all Rasa components was requested with the purpose of identifying the most favorable components to maximize classification accuracy. In this thesis, several Rasa pipelines were developed and compared with different components for tokenization, feature extraction and classification. The results of the tokenization components showed that Rasa's WhitespaceTokenizer surpassed both SpacyTokenizer and StanzaTokenizer. For feature extraction, CountVectorsFeaturizer, LanguageModelFeaturizer (with the LaBSE model) and FastTextFeaturizer (with the official fastText model trained on Swedish Wikipedia) were the most optimal components. The classifier that generally performed best was DIETClassifier, but there were several occasions where SklearnIntentClassifier surpassed it. This work resulted in several pipelines that exceeded Rasa’s standard pipeline. Of these pipelines, two performed best. The first pipeline implemented the components WhitespaceTokenizer, CountVectorsFeaturizer, FastTextFeaturizer (with the official fastText model trained on Swedish Wikipedia) and DIETClassifier with a classification accuracy of 91% (F1 score). The other pipeline implemented the components WhitespaceTokenizer, LanguageModelFeaturizer (with the LaBSE model) and SklearnIntentClassifier with a classification accuracy of 91.5% (F1 score).
|
413 |
Task-agnostic knowledge distillation of mBERT to Swedish / Uppgiftsagnostisk kunskapsdestillation av mBERT till svenskaKina, Added January 2022 (has links)
Large transformer models have shown great performance in multiple natural language processing tasks. However, slow inference, strong dependency on powerful hardware, and large energy consumption limit their availability. Furthermore, the best-performing models use high-resource languages such as English, which increases the difficulty of using these models for low-resource languages. Research into compressing large transformer models has been successful, using methods such as knowledge distillation. In this thesis, an existing task-agnostic knowledge distillation method is employed by using Swedish data for distillation of mBERT models further pre-trained on different amounts of Swedish data, in order to obtain a smaller multilingual model with performance in Swedish competitive with a monolingual student model baseline. It is shown that none of the models distilled from a multilingual model outperform the distilled Swedish monolingual model on Swedish named entity recognition and Swedish translated natural language understanding benchmark tasks. It is also shown that further pre-training mBERT does not significantly affect the performance of the multilingual teacher or student models on downstream tasks. The results corroborate previously published results showing that no student model outperforms its teacher. / Stora transformator-modeller har uppvisat bra prestanda i flera olika uppgifter inom naturlig bearbetning av språk. Men långsam inferensförmåga, starkt beroende av kraftfull hårdvara och stor energiförbrukning begränsar deras tillgänglighet. Dessutom använder de bäst presterande modellerna högresursspråk som engelska, vilket ökar svårigheten att använda dessa modeller för lågresursspråk. Forskning om att komprimera dessa stora transformatormodeller har varit framgångsrik, med metoder som kunskapsdestillation. I denna avhandling används en existerande uppgiftsagnostisk kunskapsdestillationsmetod genom att använda svensk data för destillation av mBERT modeller vidare förtränade på olika mängder svensk data för att få fram en mindre flerspråkig modell med prestanda på svenska konkurrerande med en enspråkig elevmodell baslinje. Det visas att ingen av modellerna destillerade från en flerspråkig modell överträffar den destillerade svenska enspråkiga modellen på svensk namngiven enhetserkännande och svensk översatta naturlig språkförståelse benchmark uppgifter. Det visas också att ytterligare förträning av mBERTpåverkar inte väsentligt prestandan av de flerspråkiga lärar- eller elevmodeller för nedströmsuppgifter. Resultaten bekräftar tidigare publicerade resultat som visar att ingen elevmodell överträffar sin lärare.
|
414 |
The Impact of the Retrieval Text Set for Text Sentiment Classification With the Retrieval-Augmented Language Model REALM / Effekten av hämtningstextsetet för sentimenttextklassificering med den hämtningsförstärkta språkmodellen REALMBlommegård, Oscar January 2023 (has links)
Large Language Models (LLMs) have demonstrated impressive results across various language technology tasks. By training on large corpora of diverse text collections from the internet, these models learn to process text effectively, allowing them to acquire comprehensive world knowledge. However, this knowledge is stored implicitly in the parameters of the model, and it is necessary to train ever-larger networks to capture more information. Retrieval-augmented language models have been proposed as a way of improving the interpretability and adaptability of normal language models by utilizing a separate retrieval text set during application time. These models have demonstrated state-of-the-art results on knowledge-intensive tasks such as question-answering and fact-checking. However, their effectiveness in text classification remains unexplored. This study investigates the impact of the retrieval text set on the performance of the retrieval-augmented language model REALM model for sentiment text classification tasks. The results indicate that the addition of retrieval text data fails to improve the prediction capabilities of REALM for sentiment text classification tasks. This outcome is mainly due to the difference in functionality of the retrieval mechanisms during pre-training and fine-tuning. During pre-training, the neural knowledge retriever focuses on retrieving factual knowledge such as dates, cities and names to enhance the prediction of the model. During fine-tuning, the retriever aims to retrieve texts that can strengthen the prediction of the text sentiment classification task. The findings suggest that retrieval models may hold limited potential to enhance performance for text sentiment classification tasks. / Stora språkmodeller har visat imponerande resultat inom många olika språkteknologiska uppgifter. Genom att träna på stora textmängder från internet lär sig dessa modeller att effektivt processa text, vilket gör att de kan förvärva omfattande världskunskap. Denna kunskap lagras emellertid implicit i modellernas parametrar, och det är nödvändigt att träna allt större nätverk för att fånga mer information. Hämtningsförstärkta språkmodeller (retrieval-augmented language models) har föreslagits som ett sätt att förbättra tolknings- och anpassningsförmågan hos språkmodeller genom att använda en separat hämtningstextmängd (retrieval text set) vid prediktion. Dessa modeller har visat imponerande resultat på kunskapsintensiva uppgifter som frågebesvarande (question-answering) och faktakontroll. Deras effektivitet för textklassificering är dock outforskad. Denna studie undersöker effekten av hämtningstextmängden på prestandan för den hämtningsförstärkta språkmodellen REALM för sentimenttextklassificeringsuppgifter. Resultaten indikerar att användning av hämtningstextmängd vid predicering inte lyckas förbättra REALM prediktionsförmåga för sentimenttextklassificeringsuppgifter. Detta beror främst på skillnaden i funktionalitet hos hämtningsmekanismen under förträning och finjustering. Under förträningen fokuserar hämtningsmekanismen på att hämta fakta som datum, städer och namn för att förbättra modellens predicering. Under finjusteringen syftar hätmningsmekanismen till att hämta texter som kan stärka förutsägelsen av sentimenttextklassificeringsuppgiften. Resultaten tyder på att hämtningsförstärkta modeller kan ha begränsad potential att förbättra prestandan för sentimenttextklassificeringsuppgifter.
|
415 |
<b>Sparse Ensemble Networks for Hyperspectral Image Classification</b>Rakesh Kumar Iyer (18424698) 23 April 2024 (has links)
<p dir="ltr">We explore the efficacy of sparsity and ensemble model in the classification of hyperspectral images, a pivotal task in remote sensing applications. While Convolutional Neural Networks (CNNs) and Transformer models have shown promise in this domain, each exhibits distinct limitations; CNNs excel in capturing the spatial/local features but falter to capture spectral features, whereas Transformers captures the spectral features at the expense of spatial features. Furthermore, the computational cost associated with training several independent CNN and Transformer networks becomes expensive. To address these limitations, we propose a novel ensemble framework comprising pruned CNNs and Transformers, optimizing both spatial and spectral feature utilization while curbing computational costs. By integrating sparsity through model pruning, our approach effectively reduces redundancy and computational complexity without compromising accuracy. Through extensive experimentation, we find that our method achieves comparable accuracy to its non-sparse counterparts while decreasing the computational cost. Our contribution enhances remote sensing analytics by demonstrating the potential of sparse and ensemble models in improving the precision and computational efficiency of hyperspectral image classification.</p>
|
416 |
Transformer-Based Networks for Fault Detection and Diagnostics of Rotating MachineryWong, Jonathan January 2024 (has links)
Machine health and condition monitoring are billion-dollar concerns for industry. Quality control and continuous improvement are some of the most important factors for manufacturers to consider in order to maintain a successful business. When work floor interruptions occur, engineers frequently employ “Band-Aid” fixes due to resource, timing, or technical constraints without solving for the root cause. Thus, a need for quick, reliable, and accurate fault detection and diagnosis methods are required.
Within complex rotating machinery, a fundamental component that accounts for large amounts of downtime and failure involves a very basic yet crucial element, the rolling-element bearing. A worn-out bearing constitutes to some of the most drastic failures in any mechanical system next to electrical failures associated with stator windings. The cyclical motion provides a way for measurements to be taken via vibration sensors and analyzed through signal processing techniques. Methods will be discussed to transform these acquired signals into usable input data for neural network training in order to classify the type of fault that is present within the system.
With the wide-spread utilization and adoption of neural networks, we turn our attention to the growing field of sequence-to-sequence deep learning architectures. Language based models have since been adapted to a multitude of tasks outside of text translation and word prediction. We now see powerful Transformers being used to accomplish generative modeling, computer vision, and anomaly detection -- spanning across all industries.
This research aims to determine the efficacy of the Transformer neural network for use in the detection and classification of faults within 3-phase induction motors for the automotive industry. We require a quick turnaround, often leading to small datasets in which methods such as data augmentation will be employed to improve the training process of our time-series signals. / Thesis / Master of Applied Science (MASc)
|
417 |
Enhancing Industrial Process Interaction Using Deep Learning, Semantic Layers, and Augmented RealityIzquierdo Doménech, Juan Jesús 24 June 2024 (has links)
Tesis por compendio / [ES] La Realidad Aumentada (Augmented Reality, AR) y su capacidad para integrar contenido sintético sobre una imagen real proporciona un valor incalculable en diversos campos; no obstante, la industria es uno de estos campos que más se puede aprovechar de ello. Como tecnología clave en la evolución hacia la Industria 4.0 y 5.0, la AR no solo complementa sino que también potencia la interacción humana con los procesos industriales. En este contexto, la AR se convierte en una herramienta esencial que no sustituye al factor humano, sino que lo enriquece, ampliando sus capacidades y facilitando una colaboración más efectiva entre humanos y tecnología. Esta integración de la AR en entornos industriales no solo mejora la eficiencia y precisión de las tareas, sino que también abre nuevas posibilidades para la expansión del potencial humano.
Existen numerosas formas en las que el ser humano interactúa con la tecnología, siendo la AR uno de los paradigmas más innovadores respecto a cómo los usuarios acceden a la información; sin embargo, es crucial reconocer que la AR, por sí misma, tiene limitaciones en cuanto a la interpretación del contenido que visualiza. Aunque en la actualidad podemos acceder a diferentes librerías que utilizan algoritmos para realizar una detección de imágenes, objetos, o incluso entornos, surge una pregunta fundamental: ¿hasta qué punto puede la AR comprender el contexto de lo que ve? Esta cuestión se vuelve especialmente relevante en entornos industriales. ¿Puede la AR discernir si una máquina está funcionando correctamente, o su rol se limita a la presentación de indicadores digitales superpuestos? La respuesta a estas cuestiones subrayan tanto el potencial como los límites de la AR, impulsando la búsqueda de innovaciones que permitan una mayor comprensión contextual y adaptabilidad a situaciones específicas dentro de la industria.
En el núcleo de esta tesis yace el objetivo de no solo dotar a la AR de una "inteligencia semántica" capaz de interpretar y adaptarse al contexto, sino también de ampliar y enriquecer las formas en que los usuarios interactúan con esta tecnología. Este enfoque se orienta particularmente a mejorar la accesibilidad y la eficiencia de las aplicaciones de AR en entornos industriales, que son por naturaleza restringidos y complejos. La intención es ir un paso más allá de los límites tradicionales de la AR, proporcionando herramientas más intuitivas y adaptativas para los operadores en dichos entornos.
La investigación se despliega a través de tres artículos de investigación, donde se ha desarrollado y evaluado una arquitectura multimodal progresiva. Esta arquitectura integra diversas modalidades de interacción usuario-tecnología, como el control por voz, la manipulación directa y el feedback visual en AR. Además, se incorporan tecnologías avanzadas basadas en modelos de aprendizaje automática (Machine Learning, ML) y aprendizaje profundo (Deep Learning, DL) para extraer y procesar información semántica del entorno. Cada artículo construye sobre el anterior, demostrando una evolución en la capacidad de la AR para interactuar de manera más inteligente y contextual con su entorno, y resaltando la aplicación práctica y los beneficios de estas innovaciones en la industria. / [CA] La Realitat Augmentada (Augmented Reality, AR) i la seua capacitat per integrar contingut sintètic sobre una imatge real ofereix un valor incalculable en diversos camps; no obstant això, la indústria és un d'aquests camps que més pot aprofitar-se'n. Com a tecnologia clau en l'evolució cap a la Indústria 4.0 i 5.0, l'AR no només complementa sinó que també potencia la interacció humana amb els processos industrials. En aquest context, l'AR es converteix en una eina essencial que no substitueix al factor humà, sinó que l'enriqueix, ampliant les seues capacitats i facilitant una col·laboració més efectiva entre humans i tecnologia. Esta integració de l'AR en entorns industrials no solament millora l'eficiència i precisió de les tasques, sinó que també obri noves possibilitats per a l'expansió del potencial humà.
Existeixen nombroses formes en què l'ésser humà interactua amb la tecnologia, sent l'AR un dels paradigmes més innovadors respecte a com els usuaris accedeixen a la informació; no obstant això, és crucial reconéixer que l'AR, per si mateixa, té limitacions quant a la interpretació del contingut que visualitza. Encara que en l'actualitat podem accedir a diferents llibreries que utilitzen algoritmes per a realitzar una detecció d'imatges, objectes, o fins i tot entorns, sorgeix una pregunta fonamental: fins a quin punt pot l'AR comprendre el context d'allò veu? Esta qüestió esdevé especialment rellevant en entorns industrials. Pot l'AR discernir si una màquina està funcionant correctament, o el seu rol es limita a la presentació d'indicadors digitals superposats? La resposta a estes qüestions subratllen tant el potencial com els límits de l'AR, impulsant la recerca d'innovacions que permeten una major comprensió contextual i adaptabilitat a situacions específiques dins de la indústria.
En el nucli d'esta tesi jau l'objectiu de no solament dotar a l'AR d'una "intel·ligència semàntica" capaç d'interpretar i adaptar-se al context, sinó també d'ampliar i enriquir les formes en què els usuaris interactuen amb esta tecnologia. Aquest enfocament s'orienta particularment a millorar l'accessibilitat i l'eficiència de les aplicacions d'AR en entorns industrials, que són de naturalesa restringida i complexos. La intenció és anar un pas més enllà dels límits tradicionals de l'AR, proporcionant eines més intuïtives i adaptatives per als operaris en els entorns esmentats.
La recerca es desplega a través de tres articles d'investigació, on s'ha desenvolupat i avaluat una arquitectura multimodal progressiva. Esta arquitectura integra diverses modalitats d'interacció usuari-tecnologia, com el control per veu, la manipulació directa i el feedback visual en AR. A més, s'incorporen tecnologies avançades basades en models d'aprenentatge automàtic (ML) i aprenentatge profund (DL) per a extreure i processar informació semàntica de l'entorn. Cada article construeix sobre l'anterior, demostrant una evolució en la capacitat de l'AR per a interactuar de manera més intel·ligent i contextual amb el seu entorn, i ressaltant l'aplicació pràctica i els beneficis d'estes innovacions en la indústria. / [EN] Augmented Reality (AR) and its ability to integrate synthetic content over a real image provides invaluable value in various fields; however, the industry is one of these fields that can benefit most from it. As a key technology in the evolution towards Industry 4.0 and 5.0, AR not only complements but also enhances human interaction with industrial processes. In this context, AR becomes an essential tool that does not replace the human factor but enriches it, expanding its capabilities and facilitating more effective collaboration between humans and technology. This integration of AR in industrial environments not only improves the efficiency and precision of tasks but also opens new possibilities for expanding human potential.
There are numerous ways in which humans interact with technology, with AR being one of the most innovative paradigms in how users access information; however, it is crucial to recognize that AR, by itself, has limitations in terms of interpreting the content it visualizes. Although today we can access different libraries that use algorithms for image, object, or even environment detection, a fundamental question arises: To what extent can AR understand the context of what it sees? This question becomes especially relevant in industrial environments. Can AR discern if a machine functions correctly, or is its role limited to presenting superimposed digital indicators? The answer to these questions underscores both the potential and the limits of AR, driving the search for innovations that allow for greater contextual understanding and adaptability to specific situations within the industry.
At the core of this thesis lies the objective of not only endowing AR with "semantic intelligence" capable of interpreting and adapting to context, but also of expanding and enriching the ways users interact with this technology. This approach mainly aims to improve the accessibility and efficiency of AR applications in industrial environments, which are by nature restricted and complex. The intention is to go beyond the traditional limits of AR, providing more intuitive and adaptive tools for operators in these environments.
The research unfolds through three articles, where a progressive multimodal architecture has been developed and evaluated. This architecture integrates various user-technology interaction modalities, such as voice control, direct manipulation, and visual feedback in AR. In addition, advanced technologies based on Machine Learning (ML) and Deep Learning (DL) models are incorporated to extract and process semantic information from the environment. Each article builds upon the previous one, demonstrating an evolution in AR's ability to interact more intelligently and contextually with its environment, and highlighting the practical application and benefits of these innovations in the industry. / Izquierdo Doménech, JJ. (2024). Enhancing Industrial Process Interaction Using Deep Learning, Semantic Layers, and Augmented Reality [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/205523 / Compendio
|
418 |
Klasická i neklasická řešení venkovních rozvoden 123 kV / Conventional and Unconventional Solving of 123 kV Outdoor SwitchgearsPetrucha, Lukáš January 2008 (has links)
This graduation theses shows some of possible versions of outdoor switchgears with very high voltage, especially on the level 123 kV both concerning own complement of classic outdoor switchgears with devices as are overvoltages limiter, disconnecting switchgears, circuit breakers, etc., and compact connections. In the introduction of my theses there are explained basic ideas and theories of switchgears and described main devices and equipments which create classic (from equipment setting point of view) outdoor switchgears of very high voltage. Subsequently it describes possible ways of linking-up of these devices in the complex of switchgears themselves according to possible dispositions arranged and busbars systems. The same focus is dedicated to new, non-classic (non-standard solution in terms of devices solution), compact solving of outdoor very high voltage modules either by air isolated or by means of enclosed technology with gas SF6 which represent innovative solving first of all from reduction of built up area point of view which is very important from economic point of view especially during constructions of new switchgears. In the end of my theses there are mentioned also brief evaluations of producers and economics for development and operation of individual technologies. I used for my work materials of companies CEZ, a. s., Siemens, a. s. and Abb, a.s.
|
419 |
3D Gaze Estimation on Near Infrared Images Using Vision Transformers / 3D Ögonblicksuppskattning på Nära Infraröda Bilder med Vision TransformersVardar, Emil Emir January 2023 (has links)
Gaze estimation is the process of determining where a person is looking, which has recently become a popular research area due to its broad range of applications. For example, tools that estimate gaze are used for research, medical diagnosis, virtual and augmented reality, driver assistance system, and many more. Therefore, better products are sought by many. Gaze estimation methods typically use images of only the eyes or the whole face to estimate the gaze since these methods are the most practical and convenient options. Recently, Convolutional Neural Networks (CNNs) have been appealing candidates for estimating the gaze. Nevertheless, the recent success of Vision Transformers (ViTs) in image classification tasks has introduced a new potential alternative. Hence, this work investigates the potential of using ViTs to estimate the gaze on Near-Infrared (NIR) images. This is done in terms of average error and computational complexity. Furthermore, this work examines not only pure ViTs but other models, such as hybrid ViTs and CNN-Formers, which combine CNNs and ViTs. The empirical results showed that hybrid ViTs are the only models that can outperform state-of-the-art CNNs such as MobileNetV2 and ResNet-18 while maintaining similar computational complexity to ResNet-18. The results on hybrid ViTs indicate that the convolutional stem is the most crucial part of them. Improved convolutional stems lead to better outcomes. Moreover, in this work, we defined a new training algorithm for hybrid ViTs, the hybrid Data-Efficient Image Transformer (DeiT) procedure, which has shown remarkable results. It is 3.5% better than the pretrained ResNet-18 while having the same time complexity. / Blickuppskattning är processen att uppskatta en persons blick, vilket nyligen har blivit ett populärt forskningsområde på grund av dess breda användningsområde. Till exempel, verktyg för blickuppskattning används inom forskning, medicinsk diagnos, virtuell och förstärkt verklighet, förarassistanssystem och för mycket mer. Därför, bättre produkter för blickuppskattning eftersträvas av många. Blickuppskattnings metoder vanligtvis använder bilder av endast ögonen eller hela ansiktet för att uppskatta blicken eftersom denna typen av metoder är de mest praktiska och lämliga alternativ. På sistånde har Convolutional Neural Networks (CNNs) varit tilltalande kandidater för att uppskatta blicken. Dock, har den senaste framgången med Vision Transformers (ViTs) i bildklassificeringsuppgifter introducerat ett nytt potentiellt alternativ. Därför undersöker detta arbete potentialen av att använda ViTs för att uppskatta blicken på Nära-infraröda (NIR) bilder. Undersökningen görs både i termer av medelfel och beräkningskomplexitet. Hursomhelst, detta arbete undersöker inte enbart rena ViTs utan andra modeller, som hybrida ViTs och CNN-Formers, som kombinerar CNNs och ViTs. De empiriska resultaten visade att hybrida ViTs är de enda modellerna som kan överträffa toppmoderna CNNs som MobileNetV2 och ResNet-18 samtidigt som de bibehåller liknande beräkningskomplexitet som ResNet-18. Resultaten på hybrida ViTs indikerar att faltningsstammen är den mest avgörande delen av dem. Det vill säga, desto bättre faltningsstamm en har desto bättre resultat kan man erhålla. Dessutom definierade vi i detta arbete en ny träningsalgoritm för hybrida ViTs, vilket vi kallar hybrida Data-Efficient Image Transformer (DeiT) procedur som har visat anmärkningsvärda resultat. Den är 3,5% bättre än den förtränade ResNet-18 samtidigt som den har samma tid komplexitet.
|
420 |
Classification of Radar Emitters Based on Pulse Repetition Interval using Machine LearningSvensson, André January 2022 (has links)
In electronic warfare, one of the key technologies is radar. Radar is used to detect and identify unknown aerial, nautical or land-based objects. An attribute of of a pulsed radar signal is the Pulse Repetition Interval (PRI) which is the time interval between pulses in a pulse train. In a passive radar receiver system, the PRI can be used to recognize the emitter system. Correct classification of emitter systems is a crucial part of Electronic Support Measures (ESM) and Radar Warning Receivers (RWR) in order to deploy appropriate measures depending on the emitter system. Inaccurate predictions of emitter systems can have lethal consequences and variables such as time and confidence in the predictions are essential for an effective predictive method. Due to the classified nature of military systems and techniques, there are no industry standard systems or techniques that perform quick and accurate classifications of emitter systems based on PRI. Therefore, methods that allows for fast and accurate predictions based on PRI is highly desirable and worthy of research. This thesis explores and compares the capabilities of two machine learning methods for the task of classifying emitters based on received PRI. The first method is an attention based model which performs well throughout all levels of realistic noise and is quick to learn and even quicker to give accurate predictions. The second method is a K-Nearest Neighbor (KNN) implementation that, while performing well for noise-free PRI, finds its performance degrading as the amount of noise increases. An additional outcome of this thesis is the development of a system to generate samples in an automated fashion. The attention based model performs well, achieving a macro avarage F1-score of 63% in the 59-class recognition task whereas the performance of the KNN is lower, achieving a macro avarage F1-score of 43%. Future research could be conducted with the purpose of designing a better attention based model for producing higher and more confident predictions and designing algorithms to reduce the time complexity of the KNN implementation. / En av de viktigaste teknikerna inom telektrig är radarn. Radar används för att upptäcka och identifiera okända, luftburna, sjögående eller landbaserade förmål. En komponent av radar är Pulsrepetitionsinterval (Pulse Repetition Intervall, PRI) som beskrivs som tidsintervallet mellan två inkommande pulser. I ett radarvarnar system (Radar Warning Receiver, RWR) kan PRI användas för att identifiera radarsystem. Korrekt identifiering av radarsystem är en viktig uppgift för elektroniska understödsmedel (Electronic Support Measures, ESM) med syfte att tillsätta lämpliga medel beroende på radarsystemet i fråga. Icke tillförlitlig identifiering av radarsystem kan ha dödliga konsekvenser och variabler som tid och säkerhet i identifieringen är avgörande för ett effektivt system. Då dokumentation och specifikationer för militära system i regel är hemligstämplade är det svårt att utröna någon typ av industristandard för att utföra snabb och säker klassificering av radarsystem baserat på PRI. Därför är det av stort intresse detta område och möjligheterna för sådana lösningar utforskas. Detta examensarbete utforskar och jämför förmågorna hos två maskininlärningsmetoder i avseende att korrekt identifiera radarsändare baserat på genererat PRI. Den första metoden är ett djupt neuralt nätverk som använder sig av tekniken ”attention”. Det djupa nätverket presterar bra för alla brusnivåer och lär sig snabbt att känna igen attributen hos PRI som kännetecknar vilken radarsändare och som efter träning dessutom är snabb på att korrekt identifiera PRI. Den andra metoden är en K-Nearest Neighbor implementation som förvisso presterar bra på icke brusig data men vars förmåga försämras allt eftersom brusnivåerna ökar. Ett ytterligare resultat av arbetet är utvecklingen och implementationen av en metod för att specificera PRI och sedan generera PRI efter specifikation. Attention modellen genererar bra prediktioner för data bestående av 59 klasser, med ett F1-score snitt om 63% medan KNN-implementationen för samma uppgift har en lägre träffsäkerhet med ett F1-score snitt om 43%. Vidare forskning kan innefatta utökad utveckling av det djupa, neurala nätverket i syfte att förbättra dess förmåga för identifiering och metoder för att minimera tidsåtgången för KNN implementationen.
|
Page generated in 0.0468 seconds