Tracking algorithms provide the model to recognize objects’ motion in the past. Moreover, applied to an artificial intelligence algorithm, these algorithms allow, to some degree, the capacity to forecast the future position of an object. This thesis uses deep learning algorithms to predict the ball’s position in the three-dimensional (3D) Cartesian space given the players’ motion and referees on the 2D space. The algorithms implemented are the encoder-decoder attention-based Transformer and the Inception Time, which is comprised of an ensemble of Convolutional Neural Networks. They are compared to each other under different parametrizations to understand their ability to capture temporal and spatial aspects of the tracking data on the ball prediction. The Inception Time proved to be more inconsistent on different areas of the pitches, especially on the end-lines and corners, motivating the decision to choose the Transformer network as the optimal algorithm to predict the ball position since it achieved less volatile errors on the pitch. / Spårningsalgoritmer möjliggör för modellen att känna igen objekts tidigare rörelser. Dessutom om tillämpad till en Artificiell intelligensalgoritm, de tillåter till viss mån att prognostisera ett objekts framtida position. Detta examensarbete använder djupinlärningsalgoritmer för att förutsäga bollens position i det tredimensionella (3D) kartesiska utrymmet baserat på spelarnas och domarnas rörelse i 2D-rymden. De implementerade algoritmerna är den kodare-avkodare-uppmärksamhetsbaserade Transformer och Inception Time, som består av en sammansättning faltningsnätverk (CNN). De jämförs med varandra med olika parametriseringar för att se deras förmåga att fånga upp tidsmässiga och rumsliga aspekter av spårningsdata för att förutsäga bollens rörelse. Inception Time visade sig vara mer inkonsekvent på olika områden på planen. Det var extra tydligt på slutlinjerna och i hörnen. Det motiverade beslutet att välja Transformer-nätverket som den optimala algoritmen för att förutsäga bollpositionen, eftersom den resulterade i färre ojämna fel på planen.

Diagnostic trouble codes (DTC) have traditionally been used by mechanics to figure out what is wrong with a vehicle. A vehicle generates a DTC when a specific condition in the vehicle is met. This condition has been defined by an engineer and represents some fault that has happened. Therefore the intuition is that DTC’s contain useful information about the health of the vehicle. Due to the sequential ordering of DTC’s and the high count of unique values, this modality of data has characteristics that resemble those of natural language. This thesis investigates if an algorithm that has shown to be promising in the field of Natural Language Processing can be applied to sequences of DTC’s. More specifically, the deep learning model called the transformer decoder will be compared to a baseline model called n-gram in terms of how well they estimate a probability distribution of the next DTC condition on previously seen DTC’s. Estimating a probability distribution could then be useful for manufacturers of heavy commercial vehicles such as Scania when creating systems that help them in their mission of ensuring a high uptime of their vehicles. The algorithms were compared by firstly doing a hyperparameter search for both algorithms and then comparing the models using the 5x2 cross-validation paired t-test. Three metrics were evaluated, perplexity, Top- 1 accuracy, and Top-5 accuracy. It was concluded that there was a significant difference in the performance of the two models where the transformer decoder was the better method given the metrics that were used in the evaluation. The transformer decoder had a perplexity of 22.1, Top-1 accuracy of 37.5%, and a Top-5 accuracy of 59.1%. In contrast, the n-gram had a perplexity of 37.6, Top-1 accuracy of 7.5%, and a Top-5 accuracy of 30%. / Felkoder har traditionellt använts av mekaniker för att ta reda på vad som är fel med ett fordon. Ett fordon genererar en felkod när ett visst villkor i fordonet är uppfyllt, detta villkor har definierats av en ingenjör och representerar något fel som har skett. Därför är intuitionen att felkoder innehåller användbar information om fordonets hälsa. På grund av den sekventiella ordningen av felkoder och det höga antalet unika värden, har denna modalitet av data egenskaper som liknar de för naturligt språk. Detta arbete undersöker om en algoritm som har visat sig vara lovande inom språkteknologi kan tillämpas på sekvenser av felkoder. Mer specifikt kommer den djupainlärnings modellen som kallas Transformer Decoder att jämföras med en basmodell som kallas n- gram. Med avseende på hur väl de estimerar en sannolikhetsfördelning av nästa felkod givet tidigare felkoder som har setts. Att uppskatta en sannolikhetsfördelning kan vara användbart för tillverkare av tunga fordon så som Scania, när de skapar system som hjälper dem i deras uppdrag att säkerställa en hög upptid för sina fordon. Algoritmerna jämfördes genom att först göra en hyperparametersökning för båda modellerna och sedan jämföra modellerna med hjälp av 5x2 korsvalidering parat t-test. Tre mätvärden utvärderades, perplexity, Top-1 träffsäkerhet och Top-5 träffsäkerhet. Man drog slutsatsen att det fanns en signifikant skillnad i prestanda för de två modellerna där Transformer Decoder var den bättre metoden givet mätvärdena som användes vid utvärderingen. Transformer Decoder hade en perplexity på 22.1, Top-1 träffsäkerhet på 37,5% och en Top-5 träffsäkerhet på 59,1%. I kontrast, n-gram modellen hade en perplexity på 37.6, Top-1 träffsäkerhet på 7.5% och en Top-5 träffsäkerhet på 30%.

Transformers can be described as ’slumbering giants’ in the electric power system. This marks transformers to be big and expensive parts of equipment. Calling them slumbering refers to the unused capacity in many of them. Dynamic Transformer Rating (DTR) is a concept to utilize this potential and wind power connected transformers have been identified as a well-fitting application due to the naturally limited capacity factor and the correlation of low ambient temperature and high wind speeds. Previous scientific work and a small number of applied projects show the feasibility and benefits of combining DTR and wind power. Wind power forecasting is a standard procedure for dispatch planning and electricity trading. This thesis project aims at combining both subjects and focuses on providing and analyzing a forecasting tool. At various forecasting steps Machine-Learning (ML) approaches are tested and evaluated. The developed tool is designed for and tested on a case study comprising an existing wind farm and transformer. It is shown that in many, but not all cases an overheating (exceeding of the Hot Spot Temperature (HST) limit) can be predicted. Applying DTR adds a level of uncertainty to wind power forecasts since not only the wind power but also the transformer capacity must be predicted. In this project however the wind power forecast is identified as the main source of uncertainty. / Transformatorer kan beskrivas som ‚sovande jättar‘ i det elektriska systemet eftersom transformatorer karakteriseras som stor och kostsam utrustning. Att kalla dem sovande hänvisar till den oanvända kapaciteten som finns i många. Dynamic Transformer Rating (DTR) är ett koncept för att använda denna potential och transformatorer kopplade till vindkraftsanläggningar blev utnämnd som en passande tillämpning på grund av deras begränsade kapacitetsfaktor och korrelationen mellan låga temperaturer och höga vindhastigheter. Tidigare vetenskapligt arbete och ett fåtal realiserade projekt visar genomförbarhet och fördelarna med kombinationen av DTR och vindkraft. Vindkraftsprognoser är något man vanligen använder inom driftplanering av vindkraftverk och elhandel. Detta examensarbete har som mål att kombinera båda metoderna och fokuserar på att framställa och analysera ett prognosverktyg. Olika tillvägagångssätt testas och evalueras vid de olika stegen som tas. Verktyget är skapat och testat på en fallstudie som i sig är baserad på data från existerande vindkraftverk och transformatorer. Det visar sig att man kan förutsäga överskridandet av Hot spot temperature (HST) vid många tillfällen men inte alla. Tillämpning av DTR lägger till osäkerheter till vinkraftsprognoser för att både kapaciteten på vindkraftverk och på transformatorn måste förutsägas. I detta projekt visade sig vinkraftsprognosen vara den största källan till osäkerhet.

Presented here is a method for automatic punctuation restoration in Swedish using a BERT model. The method is based on KB-BERT, a publicly available, neural network language model pre-trained on a Swedish corpus by National Library of Sweden. This model has then been fine-tuned for this specific task using a corpus of government texts. With a lower-case and unpunctuated Swedish text as input, the model is supposed to return a grammatically correct punctuated copy of the text as output. A successful solution to this problem brings benefits for an array of NLP domains, such as speech-to-text and automated text. Only the punctuation marks period, comma and question marks were considered for the project, due to a lack of data for more rare marks such as semicolon. Additionally, some marks are somewhat interchangeable with the more common, such as exclamation points and periods. Thus, the data set had all exclamation points replaced with periods. The fine-tuned Swedish BERT model, dubbed prestoBERT, achieved an overall F1-score of 78.9. The proposed model scored similarly to international counterparts, with Hungarian and Chinese models obtaining F1-scores of 82.2 and 75.6 respectively. As further comparison, a human evaluation case study was carried out. The human test group achieved an overall F1-score of 81.7, but scored substantially worse than prestoBERT on both period and comma. Inspecting output sentences from the model and humans show satisfactory results, despite the difference in F1-score. The disconnect seems to stem from an unnecessary focus on replicating the exact same punctuation used in the test set, rather than providing any of the number of correct interpretations. If the loss function could be rewritten to reward all grammatically correct outputs, rather than only the one original example, the performance could improve significantly for both prestoBERT and the human group. / Här presenteras en metod för automatisk återinföring av skiljetecken på svenska med hjälp av ett neuralt nätverk i formen av en BERT-modell. Metoden bygger på KB-BERT, en allmänt tillgänglig språkmodell, tränad på ett svensk korpus, av Kungliga Biblioteket. Denna modell har sedan finjusterats för den här specifika uppgiften med hjälp av ett korpus av offentliga texter från landsting och dylikt. Med svensk text utan versaler och skiljetecken som inmatning, ska modellen returnera en kopia av texten där korrekta skiljetecken har placerats ut på rätta platser. En framgångsrik modell ger fördelar för en rad domäner inom neurolingvistisk programmering, såsom tal- till- texttranskription och automatiserad textgenerering. Endast skiljetecknen punkt, kommatecken och frågetecken tas i beaktande i projektet på grund av en brist på data för de mer sällsynta skiljetecknen såsom semikolon. Dessutom är vissa skiljetecken någorlunda utbytbara mot de vanligaste tre, såsom utropstecken mot punkt. Således har datasetets alla utropstecken ersatts med punkter. Den finjusterade svenska BERT-modellen, kallad prestoBERT, fick en övergripande F1-poäng på 78,9. De internationella motsvarande modellerna för ungerska och kinesiska fick en övergripande F1-poäng på 82,2 respektive 75,6. Det tyder på att prestoBERT är på en liknande nivå som toppmoderna motsvarigheter. Som ytterligare jämförelse genomfördes en fallstudie med mänsklig utvärdering. Testgruppen uppnådde en övergripande F1-poäng på 81,7, men presterade betydligt sämre än prestoBERT på både punkt och kommatecken. Inspektion av utdata från modellen och människorna visar tillfredsställande resultat från båda, trots skillnaden i F1-poäng. Skillnaden verkar härstamma från ett onödigt fokus på att replikera exakt samma skiljetecken som används i indatan, snarare än att återge någon av de många korrekta tolkningar som ofta finns. Om loss-funktionen kunde skrivas om för att belöna all grammatiskt korrekt utdata, snarare än bara originalexemplet, skulle prestandan kunna förbättras avsevärt för både prestoBERT såväl som den mänskliga gruppen.

The integration of Vision Transformers (ViTs) using Masked Autoencoder pre-training (ViTMAE) into real estate valuation is investigated in this Master’s thesis, addressing the challenge of effectively analyzing visual information from real estate images. This integration aims to enhance the accuracy and efficiency of valuation, a task traditionally dependent on realtor expertise. The research involved developing a model that combines ViTMAE-extracted visual features from real estate images with traditional property data. Focusing on residential properties in Sweden, the study utilized a dataset of images and metadata from online real estate listings. An adapted ViTMAE model, accessed via the Hugging Face library, was trained on the dataset for feature extraction, which was then integrated with metadata to create a comprehensive multimodal valuation model. Results indicate that including ViTMAE-extracted image features improves prediction accuracy in real estate valuation models. The multimodal approach, merging visual and traditional metadata, improved accuracy over metadata-only models. This thesis contributes to real estate valuation by showcasing the potential of advanced image processing techniques in enhancing valuation models. It lays the groundwork for future research in more refined holistic valuation models, incorporating a wider range of factors beyond visual data. / Detta examensarbete undersöker integrationen av Vision Transformers (ViTs) med Masked Autoencoder pre-training (ViTMAE) i bostadsvärdering, genom att addressera utmaningen att effektivt analysera visuell information från bostadsannonser. Denna integration syftar till att förbättra noggrannheten och effektiviteten i fastighetsvärdering, en uppgift som traditionellt är beroende av en fysisk besiktning av mäklare. Arbetet innefattade utvecklingen av en modell som kombinerar bildinformation extraherad med ViTMAE från fastighetsbilder med traditionella fastighetsdata. Med fokus på bostadsfastigheter i Sverige använde studien en databas med bilder och metadata från bostadsannonser. Den anpassade ViTMAE-modellen, tillgänglig via Hugging Face-biblioteket, tränades på denna databas för extraktion av bildinformation, som sedan integrerades med metadata för att skapa en omfattande värderingsmodell. Resultaten indikerar att inklusion av ViTMAE-extraherad bildinformation förbättrar noggranheten av bostadssvärderingsmodeller. Den multimodala metoden, som kombinerar visuell och traditionell metadata, visade en förbättring i noggrannhet jämfört med modeller som endast använder metadata. Denna uppsats bidrar till bostadsvärdering genom att visa på potentialen hos avancerade bildanalys för att förbättra värderingsmodeller. Den lägger grunden för framtida forskning i mer raffinerade holistiska värderingsmodeller som inkluderar ett bredare spektrum av faktorer utöver visuell data.

Financial trading data presents a unique challenge for anomaly detection due to its high dimensionality and often lack of labelled anomalous examples. Nevertheless, it is of great interest for financial institutions to gain insight into potential trading activities that might lead to financial losses and reputational damage. Given the complexity and unlabelled nature of this financial data, deep learning models such as the Transformer model are particularly suited for this task.   This work investigates the application of a Transformer-based autoencoder for anomaly detection in unlabelled financial transaction data with sequential characteristics. To assess the model's ability to detect anomalies and analyse the effects of class imbalance, synthetic anomalies are injected into the dataset. This creates a controlled environment where the model's performance can be evaluated but also the affects of imbalance can be investigated.    Two approaches are particularly explored for anomaly detection purposes: an unsupervised approach and a semi-supervised approach that explicitly leverages the presence of anomalies in the training data. Experiments suggest that while the unsupervised approach can detect anomalies with distinctive features, its performance suffers when anomalies are included in the training data since the model tends to reconstruct them. Conversely, the semi-supervised approach effectively addresses this limitation, demonstrating a clear advantage in the presence of class imbalance. While synthetic anomalies enable controlled evaluation and class imbalance analysis, generalizability to real-world financial data requires true anomalies.

Having the ability to perform large automatic software changes in a code base gives new possibilities for software restructuring and cost savings. The possibility of replacing software libraries in a semi-automatic way has been studied. String metrics are used to find equivalents between two libraries by looking at class- and method names. Rules based on the equivalents are then used to describe how to apply the transformation to the code base. Using the abstract syntax tree, locations for replacements are found and transformations are performed. After the transformations have been performed, an evaluation of the saved effort of doing the replacement automatically versus manually is made. It shows that a large part of the cost can be saved. An additional evaluation calculating the maintenance cost saved annually by changing libraries is also performed in order to prove the claim that an exchange can reduce the annual cost for the project.

Fuel cost represents a significant operating expense for owners and fleet managers of hydraulic off-highway vehicles. Further, the upcoming Tier IV compliance for off-highway applications will create further expense for after-treatment and cooling. Solutions that help address these factors motivate fleet operators to consider and pursue more fuelefficient hydraulic energy recovery systems. Electrical hybridization schemes are typically complex, expensive, and often do not satisfy customer payback expectations. This paper presents a hydraulic energy recovery architecture to realize energy recovery and utilization through a hydraulic hydro-mechanical transformer. The proposed system can significantly reduce hydraulic metering losses and recover energy from multiple services. The transformer enables recovered energy to be stored in a high-pressure accumulator, maximizing energy density. It can also provide system power management, potentially allowing for engine downsizing. A hydraulic test stand is used in the development of the transformer system. The test stand is easily adaptable to simulate transformer operations on an excavator by enabling selected mode valves. The transformer’s basic operations include shaft speed control, pressure transformation control, and output flow control. This paper presents the test results of the transformer’s basic operations on the test stand, which will enable a transformer’s full function on an excavator.

<p>Frequency response analysis (FRA) is a method which is used to detect mechanical faults in transformers. The FRA response of a transformer is determined by its geometry and material properties, and it can be considered as the transformer’s fingerprint. If there are any mechanical changes in the transformer, for example if the windings are moved or distorted, its fingerprint will also be changed so, theoretically, mechanical changes in the transformer can be detected with FRA.</p><p>The purpose of this thesis is to partly create a simple model for the ferromagnetic material in the transformer core, and partly to investigate the high frequency part of the FRA response of the transformer winding. To be able to realize these goals, two different models are developed separately from each other. The first one is a time- and frequency domain complex permeability model for the ferromagnetic core material, and the second one is a time- and frequency domain winding model based on lumped circuits, in which the discretization is made finer and finer in three steps. Capacitances and inductances in the circuit are calculated with use of analytical expressions derived from approximated geometrical parameters.</p><p>The developed core material model and winding model are then implemented in MATLAB separately, using state space analysis for the winding model, to simulate the time- and frequency response.</p><p>The simulations are then compared to measurements to verify the correctness of the models. Measurements were performed on a magnetic material and on a winding, and were compared with obtained results from the models. It was found that the model developed for the core material predicts the behavior of the magnetic field for frequencies higher than 100 Hz, and that the model for the winding predicts the FRA response of the winding for frequencies up to 20 MHz.</p>

Current transformer saturation will cause the secondary current distortion. When saturation occurs, the secondary current will not be linearly proportional to the primary current, which may lead to maloperation of protection devices. This thesis researches and tests two detecting methods: Fast Fourier Transform (FFT) and Wavelet Transform based methods. Comparing these two methods, FFT has a better performance in steady state saturation, and Wavelet Transform can determine singularity to provide the moment of distortion. The Jiles-Atherton (J-A) theory of ferromagnetic hysteresis is one approach used in electromagnetics transient modeling. With decades of development, the J-A model has evolved into different versions. The author summarizes the different models and implements J-A model in both MATLAB and Simulink.