[pt] Transformadores de guia de onda são amplamente empregados no projeto de componentes em onda guiada e são encontrados em praticamente todas as cadeias alimentadoras de antenas e demais estruturas de onda guiada na faixa de microondas. Embora a teoria de transformadores seja conhecida, os requisitos de ordem sistêmica têm levado os projetos de transformadores de guia de onda ao seu limite. Para tal nível de exigência, e considerando o número de variáveis no projeto de transformadores, técnicas numéricas de análise (tais como FDTD e expansão modal dentre outros), e otimização têm sido obrigatoriamente empregadas. Por outro lado, o número de variáveis de um transformador, acaba sendo um processo de alto consumo de tempo computacional, incoerente com o porte e objetivo de custo desses transformadores. Este trabalho propõe uma possibilidade alternativa para a análise mais rápida para essas estruturas, através do emprego de formulações fechadas derivadas de métodos varacionais. Um modelo heurístico é proposto para o caso de descontinuidades em dois planos, sejam para o caso de descontinuidades homogêneas ou para não-homogêneas. / [en] Waveguide transformers are widely used on antenna’s feeder chains and other microwave devices. Although the theory of quarter wavelength transformers is well known, the current electrical performance of such microwave devices has been pushing the waveguide transformers design to its limit. For attending such level of requirements, and considering the number of existing variables on a waveguide transformer design, very accurate numerical techniques has been applied on its analyses, (such as FDTD, mode matching, etc), and optimization techniques as well. On the other hand, such numerical techniques are very memory and/or CPU/time consuming, which do not match with the cost objective of those simple concept transformers. This work proposes an alternative technique, based on close-form models derived from varational theory. A heuristic model is also proposed for attending the two plane transformer case, which can be easily applied for both homogeneous and inhomogeneous structures. Keywords Waveguide;

<div>Textual emotion classification is a task in affective AI that branches from sentiment analysis and focuses on identifying emotions expressed in a given text excerpt. </div><div>It has a wide variety of applications that improve human-computer interactions, particularly to empower computers to understand subjective human language better. </div><div>Significant research has been done on this task, but very little of that research leverages one of the most emotion-bearing symbols we have used in modern communication: Emojis.</div><div>In this thesis, we propose several transformer-based models for emotion classification that processes emojis as input tokens and leverages pretrained models and uses them</div><div>, a model that processes Emojis as textual inputs and leverages DeepMoji to generate affective feature vectors used as reference when aggregating different modalities of text encoding. </div><div>To evaluate ReferEmo, we experimented on the SemEval 2018 and GoEmotions datasets, two benchmark datasets for emotion classification, and achieved competitive performance compared to state-of-the-art models tested on these datasets. Notably, our model performs better on the underrepresented classes of each dataset.</div>

In the digital age where video content is abundant, this thesis investigates the efficient adaptation of an existing video-language model (VLM) to new data. The research leverages CLIP, a robust language-vision model, for various video-related tasks including video retrieval. The study explores using pre-trained VLMs to extract video embeddings without the need for extensive retraining. The effectiveness of a smaller model using aggregation is compared with larger models and the application of logistic regression for few-shot learning on video embeddings is examined. The aggregation was done using both non-learning through mean-pooling and also by utilizing a transformer. The video-retrieval models were evaluated on the ActivityNet Captions dataset which contains long videos with dense descriptions while the linear probes were evaluated on ActivityNet200 a video classification dataset.  The study's findings suggest that most models improved when additional frames were employed through aggregation, leading to improved performance. A model trained with fewer frames was able to surpass those trained with two or four times more frames by instead using aggregation. The incorporation of patch dropout and the freezing of embeddings proved advantageous by enhancing performance and conserving training resources. Furthermore, using a linear probe showed that the extracted features were of high quality requiring only 2-4 samples per class to match the zero-shot performance.

This thesis investigates how various driver signals, signals that a truck driver can interact with, influences fuel consumption and what are the optimal values of these signals in various traffic conditions. More specifically, the objective is to estimate good driver behavior in various traffic conditions and compare bad driver behavior in similar situations to see how performing a specific driver action, changing a driver signal from the bad driver value to the corresponding good driver value impacts the fuel consumption. The result is an AI-based algorithm that utilizes the transformer model architecture to estimate good driver behavior, based on environmental describing signals, as well as fuel consumption. Utilizing these, causal inference is used to estimate how much fuel can be saved by switching a driver signal from a bad driver value to a good driver value.

When two human beings engage in a conversation, feedback is generally present since it helps in modulating and guiding the conversation for the involved parties. When a robotic agent engages in a conversation with a human, the robot is not capable of understanding the feedback given by the human as other humans would. In this thesis, we model human feedback as a Multivariate Time Series to be classified as positive, negative or neutral. We explore state-of-the-art Deep Learning architectures such as InceptionTime, a Convolutional Neural Network approach, and the Time Series Encoder, a Transformer approach. We demonstrate state-of-the art performance in accuracy, loss and f1-score of such models and improved performance in all metrics when compared to best performing approaches in previous studies such as the Random Forest Classifier. While InceptionTime and the Time Series Encoder reach an accuracy of 85.09% and 84.06% respectively, the Random Forest Classifier stays back with an accuracy of 81.99%. Moreover, InceptionTime reaches an f1-score of 85.07%, the Time Series Encoder of 83.27% and the Random Forest Classifier of 77.61%. In addition to this, we study the data classified by both Deep Learning approaches to outline relevant, redundant and trivial human feedback signals over the whole dataset as well as for the positive, negative and neutral cases. / När två människor konverserar, är feedback (återmatning) en del av samtalet eftersom det hjälper till att styra och leda samtalet för de samtalande parterna. När en robot-agent samtalar med en människa, kan den inte förstå denna feedback på samma sätt som en människa skulle kunna. I den här avhandlingen modelleras människans feedback som en flervariabeltidsserie (Multivariate Time Series) som klassificeras som positiv, negativ eller neutral. Vi utforskar toppmoderna djupinlärningsarkitekturer som InceptionTime, en CNN-metod och Time Series Encoder, som är en Transformer-metod. Vi uppnår hög noggrannhet, F1 och lägre värden på förlustfunktionen jämfört med tidigare högst presterande metoder, som Random Forest-metoder. InceptionTime och Time Series Encoder uppnår en noggrannhet på 85,09% respektive 84,06%, men Random Forest-klassificeraren uppnår endast 81,99%. Dessutom uppnår InceptionTime ett F1 på 85,07%, Time Series Encoder 83,27%, och Random Forest-klassificeraren 77,61. Utöver detta studerar vi data som har klassificerats av båda djupinlärningsmetoderna för att hitta relevanta, redundanta och enklare mänskliga feedback-signaler över hela datamängden, samt för positiva, negativa och neutrala datapunkter.

There is a growing use of the internet and alongside that, there has been an increase in the use of toxic language towards other people that can be harmful to those that it targets. The usefulness of artificial intelligence has exploded in recent years with the development of natural language processing, especially with the use of transformers. One of the first ones was BERT, and that has spawned many variations including ones that aim to be more lightweight than the original ones. The goal of this project was to train three different kinds of transformer models, RoBERTa, ALBERT, and DistilBERT, and find out which one was best at identifying toxic language online. The models were trained on a handful of existing datasets that had labelled data as abusive, hateful, harassing, and other kinds of toxic language. These datasets were combined to create a dataset that was used to train and test all of the models. When tested against data collected in the datasets, there was very little difference in the overall performance of the models. The biggest difference was how long it took to train them with ALBERT taking approximately 2 hours, RoBERTa, around 1 hour and DistilBERT just over half an hour. To understand how well the models worked in a real-world scenario, the models were evaluated by labelling text as toxic or non-toxic on three different subreddits. Here, a larger difference in performance showed up. DistilBERT labelled significantly fewer instances as toxic compared to the other models. A sample of the classified data was manually annotated, and it showed that the RoBERTa and DistilBERT models still performed similarly to each other. A second evaluation was done on the data from Reddit and a threshold of 80% certainty was required for the classification to be considered toxic. This led to an average of 28% of instances being classified as toxic by RoBERTa, whereas ALBERT and DistilBERT classified an average of 14% and 11% as toxic respectively. When the results from the RoBERTa and DistilBERT models were manually annotated, a significant improvement could be seen in the performance of the models. This led to the conclusion that the DistilBERT model was the most suitable model for training and classifying toxic language of the lightweight models tested in this work.

Training a deep neural network from scratch can be very expensive in terms of resources.In addition, training a neural network on a new task is usually done by training themodel form scratch. Recently there are new approaches in machine learning which usesthe knowledge from a pre-trained deep neural network on a new task. The technique ofreusing the knowledge from previously trained deep neural networks is called Transferlearning. In this paper we are going to evaluate transfer learning capabilities of deep neuralnetwork architectures for image classification. This research attempts to implementtransfer learning with different datasets and models in order to investigate transfer learningin different situations.

Bidirectional Encoder Representations from Transformers (BERT) is a recently proposed language representation model, designed to pre-train deep bidirectional representations, with the goal of extracting context-sensitive features from an input text [1]. One of the challenging problems in the field of Natural Language Processing is Conversational Machine Comprehension (CMC). Given a context passage, a conversational question and the conversational history, the system should predict the answer span of the question in the context passage. The main challenge in this task is how to effectively encode the conversational history into the prediction of the next answer. In this thesis work, we investigate the use of the BERT language model for the CMC task. We propose a new architecture, named BERT-CMC, using the BERT model as a base. This architecture includes a new module for encoding the conversational history, inspired by the Transformer-XL model [2]. This module serves the role of memory throughout the conversation. The proposed model is trained and evaluated on the Conversational Question Answering dataset (CoQA) [3]. Our hypothesis is that the BERT-CMC model will effectively learn the underlying context of the conversation, leading to better performance than the baseline model proposed for CoQA. Our results of evaluating the BERT-CMC on the CoQA dataset show that the model performs poorly (44.7% F1 score), comparing to the CoQA baseline model (66.2% F1 score). In the light of model explainability, we also perform a qualitative analysis of the model behavior in questions with various linguistic phenomena eg coreference, pragmatic reasoning. Additionally, we motivate the critical design choices made, by performing an ablation study of the effect of these choices on the model performance. The results suggest that fine tuning the BERT layers boost the model performance. Moreover, it is shown that increasing the number of extra layers on top of BERT leads to bigger capacity of the conversational memory. / Bidirectional Encoder Representations from Transformers (BERT) är en nyligen föreslagen språkrepresentationsmodell, utformad för att förträna djupa dubbelriktade representationer, med målet att extrahera kontextkänsliga särdrag från en inmatningstext [1]. Ett utmanande problem inom området naturligtspråkbehandling är konversationsförståelse (förkortat CMC). Givet en bakgrundstext, en fråga och konversationshistoriken ska systemet förutsäga vilken del av bakgrundstexten som utgör svaret på frågan. Den viktigaste utmaningen i denna uppgift är hur man effektivt kan kodifiera konversationshistoriken i förutsägelsen av nästa svar. I detta examensarbete undersöker vi användningen av BERT-språkmodellen för CMC-uppgiften. Vi föreslår en ny arkitektur med namnet BERT-CMC med BERT-modellen som bas. Denna arkitektur innehåller en ny modul för kodning av konversationshistoriken, inspirerad av Transformer-XL-modellen [2]. Den här modulen tjänar minnets roll under hela konversationen. Den föreslagna modellen tränas och utvärderas på en datamängd för samtalsfrågesvar (CoQA) [3]. Vår hypotes är att BERT-CMC-modellen effektivt kommer att lära sig det underliggande sammanhanget för konversationen, vilket leder till bättre resultat än basmodellen som har föreslagits för CoQA. Våra resultat av utvärdering av BERT-CMC på CoQA-datasetet visar att modellen fungerar dåligt (44.7% F1 resultat), jämfört med CoQAbasmodellen (66.2% F1 resultat). För att bättre kunna förklara modellen utför vi också en kvalitativ analys av modellbeteendet i frågor med olika språkliga fenomen, t.ex. koreferens, pragmatiska resonemang. Dessutom motiverar vi de kritiska designvalen som gjorts genom att utföra en ablationsstudie av effekten av dessa val på modellens prestanda. Resultaten tyder på att finjustering av BERT-lager ökar modellens prestanda. Dessutom visas att ökning av antalet extra lager ovanpå BERT leder till större konversationsminne.

Detta arbete har undersökt påverkan av en ökad solcellsinstallation på fördelningsstationen FS25 Änge med hjälp av historiska data och antaganden. Det har även undersökts hur ett batterilagringssystem skulle kunna hjälpa nätet vid den ökade mängden solcellsinstallationer. Arbetet har riktat sig mot den dagen då solproduktion förväntas vara som högst samtidigt som övrig tid på året beaktats. Den komponent som varit huvudfokus är transformatorerna i nätet. Tre scenarion har kollat på där solcellsimplementering skett i graderna 100%, 70% och 50% av alla kunder under respektive nätstation. Det visade sig att det största problemet är i fördelningsstationen som är dimensionerad utefter sammanlagring. När det kommer till solproduktion går det inte räkna med sammanlagring då den producerar som högst för alla anläggningar samtidigt. Därefter diskuteras de möjligheter som finns för att hjälpa FS25 Änge under överbelastning där BESS implementerades men även flexmarknaden kom upp som en möjlighet. Avslutningsvis diskuterades framtida arbete som skulle behöva göras på detta ämne, där det arbetet som är mest aktuellt är att se påverkan hos elsystemet om det installerades batterier hos alla kunder och inte bara de som ligger under en överbelastad transformator. / This work has examined the impact of increased PV-implementation on distribution station FS25 Änge using historical data and assumptions. It has also examined how a battery energy storage system could assist the grid with the increased amount of PV- installations. The focus of the work has been on the day when the solar production is expected to be at its highest while considering the rest of the year. The component that has been the focus is the power transformers in the grid. Three scenarios have been examined where solar cell implementation has occurred to the degrees of 100%, 70%, and 50% of all customers at each substation, respectively. It turned out that the biggest problem is in the distribution station, which is dimensioned based on aggregated storage. When it comes to solar production, aggregated storage cannot be considered as it produces at its highest for all consumers simultaneously. Furthermore, the possibilities to assist FS25 Änge during overload were discussed, where Battery Energy Storage Systems (BESS) were implemented, and the flexibility local market emerged as a potential solution. Finally, future work that needs to be done on this topic was discussed, with the most relevant being to examine the impact on the electrical system if batteries were installed for all customers, not just those under an overloaded power transformer.

La diffusion des Fake News sur les réseaux sociaux est devenue un problème central ces dernières années. Notamment, hoaxy rapporte que les efforts de fact checking prennent généralement 10 à 20 heures pour répondre à une fake news, et qu'il y a un ordre de magnitude en plus de fake news que de fact checking. Le fact checking automatique pourrait aider en accélérant le travail humain et en surveillant les tendances dans les fake news. Dans un effort contre la désinformation, nous résumons le domaine de Fact Checking Automatique basé sur le contenu en 3 approches: les modèles avec aucune connaissances externes, les modèles avec un Graphe de Connaissance et les modèles avec une Base de Connaissance. Afin de rendre le Fact Checking Automatique plus accessible, nous présentons pour chaque approche une architecture efficace avec le poids en mémoire comme préoccupation, nous discutons aussi de comment chaque approche peut être appliquée pour faire usage au mieux de leur charactéristiques. Nous nous appuyons notamment sur la version distillée du modèle de langue BERT tinyBert, combiné avec un partage fort des poids sur 2 approches pour baisser l'usage mémoire en préservant la précision. / The spreading of fake news on social media has become a concern in recent years. Notably, hoaxy found that fact checking generally takes 10 to 20 hours to respond to a fake news, and that there is one order of magnitude more fake news than fact checking. Automatic fact checking could help by accelerating human work and monitoring trends in fake news. In the effort against disinformation, we summarize content-based automatic fact-checking into 3 approaches: models with no external knowledge, models with a Knowledge Graph and models with a Knowledge Base. In order to make Automatic Fact Checking more accessible, we present for each approach an effective architecture with memory footprint in mind and also discuss how they can be applied to make use of their different characteristics. We notably rely on distilled version of the BERT language model tinyBert, combined with hard parameter sharing on two approaches to lower memory usage while preserving the accuracy.