Federated learning poses a statistical challenge when training on highly heterogeneous sequence data. For example, time-series telecom data collected over long intervals regularly shows mixed fluctuations and patterns. These distinct distributions are an inconvenience when a node not only plans to contribute to the creation of the global model but also plans to apply it on its local dataset. In this scenario, adopting a one-fits-all approach might be inadequate, even when using state-of-the-art machine learning techniques for time series forecasting, such as Long Short-Term Memory (LSTM) networks, which have proven to be able to capture many idiosyncrasies and generalise to new patterns. In this work, we show that by clustering the clients using these patterns and selectively aggregating their updates in different global models can improve local performance with minimal overhead, as we demonstrate through experiments using realworld time series datasets and a basic LSTM model. / Federated Learning utgör en statistisk utmaning vid träning med starkt heterogen sekvensdata. Till exempel så uppvisar tidsseriedata inom telekomdomänen blandade variationer och mönster över längre tidsintervall. Dessa distinkta fördelningar utgör en utmaning när en nod inte bara ska bidra till skapandet av en global modell utan även ämnar applicera denna modell på sin lokala datamängd. Att i detta scenario införa en global modell som ska passa alla kan visa sig vara otillräckligt, även om vi använder oss av de mest framgångsrika modellerna inom maskininlärning för tidsserieprognoser, Long Short-Term Memory (LSTM) nätverk, vilka visat sig kunna fånga komplexa mönster och generalisera väl till nya mönster. I detta arbete visar vi att genom att klustra klienterna med hjälp av dessa mönster och selektivt aggregera deras uppdateringar i olika globala modeller kan vi uppnå förbättringar av den lokal prestandan med minimala kostnader, vilket vi demonstrerar genom experiment med riktigt tidsseriedata och en grundläggande LSTM-modell.

Time Series data has become ubiquitous thanks to affordable edge devices and sensors. Much of this data is valuable for decision making. In order to use these data for the forecasting task, the conventional centralized approach has shown deficiencies regarding large data communication and data privacy issues. Furthermore, Neural Network models cannot make use of the extra information from the time series, thus they usually fail to provide time series specific results. Both issues expose a challenge to large-scale Time Series Forecasting with Neural Network models. All these limitations lead to our research question:Can we realize decentralized time series forecasting with a Federated Learning mechanism that is comparable to the conventional centralized setup in forecasting performance?In this work, we propose a Federated Series Forecasting framework, resolving the challenge by allowing users to keep the data locally, and learns a shared model by aggregating locally computed updates. Besides, we design a hybrid model to enable Neural Network models utilizing the extra information from the time series to achieve a time series specific learning. In particular, the proposed hybrid outperforms state-of-art baseline data-central models with NN5 and Ericsson KPI data. Meanwhile, the federated settings of purposed model yields comparable results to data-central settings on both NN5 and Ericsson KPI data. These results together answer the research question of this thesis. / Tidseriedata har blivit allmänt förekommande tack vare överkomliga kantenheter och sensorer. Mycket av denna data är värdefull för beslutsfattande. För att kunna använda datan för prognosuppgifter har den konventionella centraliserade metoden visat brister avseende storskalig datakommunikation och integritetsfrågor. Vidare har neurala nätverksmodeller inte klarat av att utnyttja den extra informationen från tidsserierna, vilket leder till misslyckanden med att ge specifikt tidsserierelaterade resultat. Båda frågorna exponerar en utmaning för storskalig tidsserieprognostisering med neurala nätverksmodeller. Alla dessa begränsningar leder till vår forskningsfråga:Kan vi realisera decentraliserad tidsserieprognostisering med en federerad lärningsmekanism som presterar jämförbart med konventionella centrala lösningar i prognostisering?I det här arbetet föreslår vi ett ramverk för federerad tidsserieprognos som löser utmaningen genom att låta användaren behålla data lokalt och lära sig en delad modell genom att aggregera lokalt beräknade uppdateringar. Dessutom utformar vi en hybrid modell för att möjliggöra neurala nätverksmodeller som kan utnyttja den extra informationen från tidsserierna för att uppnå inlärning av specifika tidsserier. Den föreslagna hybrida modellen presterar bättre än state-of-art centraliserade grundläggande modeller med NN5och Ericsson KPIdata. Samtidigt ger den federerade ansatsen jämförbara resultat med de datacentrala ansatserna för både NN5och Ericsson KPI-data. Dessa resultat svarar tillsammans på forskningsfrågan av denna avhandling.

Time-series forecasting is an area of machine learning that can be applied to many real-life problems. It is used in areas such as water level forecasting, which aims to help people evacuate on time for floods. This thesis aims to contribute to the research area of time-series forecasting, by introducing a simple but novel ensemble model: Force Linear to Exponential (FLEX). A FLEX ensemble first forecasts points that are exponentially further into the forecasting horizon. After this, the gaps between forecasted points are produced from said forecasted points, as well as the entire data history. This simple model is able to outperform all base models considered in this thesis, even when having the same amount of parameters to tune. / Tidsserieprognoser är ett område för maskininlärning som kan tillämpas på många verkliga problem. Det används i områden som vattenståndsprognoser, som syftar till att hjälpa människor att evakuera i tid för översvämningar. Denna uppsats syftar till att bidra till forskningsområdet tidsserieprognoser genom att introducera en enkel men ny ensemblemodell: Force Linear to Exponential (FLEX). En FLEX-ensemble prognostiserar först punkter som ligger exponentiellt längre in i prognoshorisonten. Efter detta produceras gapen mellan prognostiserade punkter från nämnda prognostiserade punkter, såväl som hela datahistoriken. Denna enkla modell kan överträffa alla basmodeller som behandlas i denna uppsats, även när den har samma mängd parametrar att ställa in.

Time- series Generative Adversarial Networks (TimeGAN) is proposed to overcome the GAN model’s insufficiency in producing synthetic samples that inherit the predictive ability of the original timeseries data. TimeGAN combines the unsupervised adversarial loss in the GAN framework with a supervised loss adopted from an autoregressive model. However, TimeGAN is like another GANbased model that only learns from the set of smaller sequences extracted from the original time-series. This behavior yields a severe consequence when encountering data augmentation for time-series with multiple seasonal patterns, as found in the mobile telecommunication network data. This study examined the effectiveness of the TimeGAN model with the help of Dynamic Time Warping (DTW) and different types of RNN as its architecture to produce synthetic mobile telecommunication network data, which can be utilized to improve the forecasting performance of the statistical and deep learning models relative to the baseline models trained only on the original data. The experiment results indicate that DTW helps TimeGAN maintaining the multiple seasonal attributes. In addition, either LSTM or Bidirectional LSTM as TimeGAN architecture ensures the model is robust to mode collapse problem and creates synthetic data that are diversified and indistinguishable from the original time-series. Finally, merging both original and synthetic time-series becomes a compelling way to significantly improve the deep learning model’s forecasting performance but fails to do so for the statistical model. / Time-series Generative Adversarial Networks (TimeGAN) föreslås för att övervinna GAN-modellens brist att kunna producera syntetisk data som ärver de prediktiva förmåga från den ursprungliga tidsseriedatan. TimeGAN kombinerar den icke-vägledande förlusten i GAN-ramverket tillsammans med den vägledande förlusten från en autoregressiv modell. TimeGAN liknar en vanlig GAN-baserad modell, men behöver bara en mindre uppsättning sekvenser från den ursprungliga tidsserien för att lära sig. Denna egenskap kan dock leda till allvarliga konsekvenser när man stöter på dataförstoring för tidsserier med flera säsongsmönster, vilket återfinns i mobilnätverksdata. Denna studie har undersökt effektiviteten av TimeGAN-modellen med hjälp av Dynamic Time Warping (DTW) och olika typer av RNN som dess arkitektur för att producera syntetisk mobilnätverksdata. Detta kan användas för att förbättra statistiska och djupinlärningsmodellers prognostisering relativt till modeller som bara har tränat på orginaldata. De experimentella resultaten indikerar att DTW hjälper TimeGAN att bibehålla de olika säsongsattributen. Dessutom, TimeGAN med antingen LSTM eller Bidirectional LSTM som arkitektur säkerställer att modellen är robust för lägesfallsproblem och skapar syntetisk data som är diversifierade och inte kan urskiljas från den ursprungliga tidsserien. Slutligen, en sammanslagning av både ursprungliga och syntetiska tidsserier blir ett övertygande sätt att avsevärt förbättra djupinlärningsmodellens prestanda men misslyckas med detta för den statistiska modellen.

Predicting time series is a crucial task for organizations, since decisions are often based on uncertain information. Many forecasting models are designed from a generic statistical point of view. However, each real-world application requires domain-specific adaptations to obtain high-quality results. All such specifics are summarized by the term of context. In contrast to current approaches, we want to integrate context as the primary driver in the forecasting process. We introduce context-driven time series forecasting focusing on two exemplary domains: renewable energy and sparse sales data. In view of this, we discuss the challenge of context integration in the individual process steps.

As the era of Industry 4.0 arises, the number of devices that are connectedto a network has increased. The devices continuously generate data that hasvarious information from power consumption to the configuration of thedevices. Since the data have the raw information about each local node inthe network, the manipulation of the information brings a potential to benefitthe network with different methods. However, due to the large amount ofnon-IID data generated in each node, manual operations to process the dataand tune the methods became challenging. To overcome the challenge, therehave been attempts to apply automated methods to build accurate machinelearning models by a subset of collected data or cluster network nodes byleveraging clustering algorithms and using machine learning models withineach cluster. However, the conventional clustering algorithms are imperfectin a distributed and dynamic network due to risk of data privacy, the nondynamicclusters, and the fixed number of clusters. These limitations ofthe clustering algorithms degrade the performance of the machine learningmodels because the clusters may become obsolete over time. Therefore, thisthesis proposes a three-phase clustering algorithm in dynamic environmentsby leveraging 1) GAN-based clustering, 2) cluster calibration, and 3) divisiveclustering in federated learning. GAN-based clustering preserves data becauseit eliminates the necessity of sharing raw data in a network to create clusters.Cluster calibration adds dynamics to fixed clusters by continuously updatingclusters and benefits methods that manage the network. Moreover, the divisiveclustering explores the different number of clusters by iteratively selectingand dividing a cluster into multiple clusters. As a result, we create clustersfor dynamic environments and improve the performance of machine learningmodels within each cluster. / ett nätverk ökat. Enheterna genererar kontinuerligt data som har varierandeinformation, från strömförbrukning till konfigurationen av enheterna. Eftersomdatan innehåller den råa informationen om varje lokal nod i nätverket germanipulation av informationen potential att gynna nätverket med olika metoder.På grund av den stora mängden data, och dess egenskap av att vara icke-o.l.f.,som genereras i varje nod blir manuella operationer för att bearbeta data ochjustera metoderna utmanande. För att hantera utmaningen finns försök med attanvända automatiserade metoder för att bygga precisa maskininlärningsmodellermed hjälp av en mindre mängd insamlad data eller att gruppera nodergenom att utnyttja klustringsalgoritmer och använda maskininlärningsmodellerinom varje kluster. De konventionella klustringsalgoritmerna är emellertidofullkomliga i ett distribuerat och dynamiskt nätverk på grund av risken fördataskydd, de icke-dynamiska klusterna och det fasta antalet kluster. Dessabegränsningar av klustringsalgoritmerna försämrar maskininlärningsmodellernasprestanda eftersom klustren kan bli föråldrade med tiden. Därför föreslårdenna avhandling en trefasklustringsalgoritm i dynamiska miljöer genom attutnyttja 1) GAN-baserad klustring, 2) klusterkalibrering och 3) klyvning avkluster i federerad inlärning. GAN-baserade klustring bevarar dataintegriteteneftersom det eliminerar behovet av att dela rådata i ett nätverk för att skapakluster. Klusterkalibrering lägger till dynamik i klustringen genom att kontinuerligtuppdatera kluster och fördelar metoder som hanterar nätverket. Dessutomdelar den klövlande klustringen olika antal kluster genom att iterativt välja ochdela ett kluster i flera kluster. Som ett resultat skapar vi kluster för dynamiskamiljöer och förbättrar prestandan hos maskininlärningsmodeller inom varjekluster.

Time series forecasting and volatility forecasting is a particularly active research field within financial mathematics. More recent studies extend well-established forecasting methods with machine learning. This thesis will evaluate and compare the standard Generalized Autoregressive Conditional Heteroskedasticity (GARCH) model and some of its extensions to a proposed Long Short-Term Memory (LSTM) model on historic data from five Swedish stocks. It will also explore hybrid models that combine the two techniques to increase prediction accuracy over longer horizons. The results show that the predictability increases when switching from univariate GARCH and LSTM models to hybrid models combining them both. Combining GARCH, Glosten, Jagannathan, and Runkle GARCH (GJR-GARCH), and Fractionally Integrated GARCH (FIGARCH) yields the most accurate result with regards to mean absolute error and mean square error. The forecasting errors decreased with 10 to 50 percent using the hybrid models. Comparing standard GARCH to the hybrid models, the biggest gains were seen at the longest horizon, while comparing the LSTM to the hybrid models, the biggest gains were seen for the shorter horizons. In conclusion, the prediction ability increases using the hybrid models compared to the regular models. / Tidsserieprognostisering, och volatilitetsprognostiering i synnerhet, är ett växande fält inom finansiell matamatik som kontinereligt står inför implementation av nya tekniker. Det som en gång startade med klassiksa tidsseriemodeller som ARCH har nu utvecklats till att dra fördel av maskininlärning och neurala nätverk. Detta examensarbetet uvärderar och jämför Generalized Autoregressive Conditional Heteroskedasticity (GARCH) modeller och några av dess vidare tillämpningar med Long Short-Term Memory (LSTM) modeller på fem svenska aktier. ARbetet kommer även gå närmare inpå hybridmodeller som kombinerar dessa två tekniker för att öka tillförlitlig prognostisering under längre tidshorisonter. Resultaten visar att förutsägbarheten ökar genom att byta envariata GARCH och LSTM modeller till hybridmodeller som kombinerar båda delarna. De mest korrekta resultaten kom från att kombinera GARCH, Glosten, Jagannathan, och Runkle GARCH (GJR-GARCH) och Fractionally Integrated GARCH (FIGARCH) modeller med ett LSTM nätverk. Prognostiseringsfelen minskade med 10 till 50 procent med hybridmodellerna. Specifikt, vid jämförelse av GARCH modellerna till hybridmodellerna sågs de största förbättringarna för de längre tidshorisonterna, medans jämförelse mellan LSTM och hybridmodellerna sågs den mesta förbättringen hos de kortare tidshorisonterna. Sammanfattningsvis öker prognostiseringsförmågan genom användning av hybridmodeller i jämförelse med standardmodellerna.

The focus of this study explores the adaptations of the Lag-Llama univariate time series forecasting approach [8] to handle multivariate time series, named LSTM2Lag-Llama. This extension is motivated by the increasing necessity to deal with datasets containing many variables of interest, particularly in the healthcare sector. A novel approach is introduced that harnesses the capabilities of the Long Short-Term Memory (LSTM) model. The baseline LSTM model takes multivariate input data and has been used widely to capture long-range dependencies within time series data. These features make it an ideal candidate for our task of expanding the Lag-Llama model to handle multivariate time series. The research process involves a detailed and systematic LSTM2Lag-Llama model to accommodate multiple input and output variables. This adaptation process is not a straightforward task. It requires careful consideration of the model architecture, loss function, and training methodologies. The performance of the LSTM2Lag-Llama model is then evaluated using a real-world dataset on early sepsis predictions. This dataset presents a challenging yet practical scenario for time series forecasting, making it an ideal testbed for our LSTM2Lag-Llama model. The results of this research demonstrate the feasibility and effectiveness of the proposed approach, representing a significant step towards exploring multivariate adaptations of the Lag-Llama model. The LSTM2Lag-Llama model not only handles multivariate data but also leverages the LSTM model’s ability to capture multivariate relationships in its hidden states. While the study does not directly implement the model in a practical healthcare setting, it underscores the potential of such advancements in time series forecasting techniques. This research represents a significant contribution to the field of time series forecasting in healthcare. It opens up new avenues for future research and has the potential to significantly impact the way healthcare professionals use time series data for prediction and decision-making. / L'objectif de cette étude est d'explorer les adaptations de l'approche de prévision univariée des séries temporelles Lag-Llama [8] pour traiter les séries temporelles multivariées, nommée LSTM2Lag-Llama. Cette extension est motivée par la nécessité croissante de traiter des ensembles de données contenant de nombreuses variables d'intérêt, notamment dans le secteur de la santé. Une nouvelle approche est introduite qui exploite les capacités du modèle Long Short-Term Memory (LSTM). Le modèle LSTM de base prend des données d'entrée multivariées et a été largement utilisé pour capturer les dépendances à long terme dans les données de séries temporelles. Ces caractéristiques en font un candidat idéal pour notre tâche d'extension du modèle Lag-Llama pour gérer les séries temporelles multivariées. Le processus de recherche implique un modèle LSTM2Lag-Llama détaillé et systématique pour accueillir plusieurs variables d'entrée et de sortie. Ce processus d'adaptation n'est pas une tâche simple. Il nécessite une considération minutieuse de l'architecture du modèle, de la fonction de perte et des méthodologies d'apprentissage. Les performances du modèle LSTM2Lag-Llama sont ensuite évaluées à l'aide d'un ensemble de données du monde réel sur les prédictions précoces de septicémie. Cet ensemble de données présente un scénario difficile mais pratique pour la prévision de séries temporelles, ce qui en fait un banc d'essai idéal pour notre modèle LSTM2Lag-Llama. Les résultats de cette recherche démontrent la faisabilité et l'efficacité de l'approche proposée, ce qui représente une étape importante vers l'exploration des adaptations multivariées du modèle Lag-Llama. Le modèle LSTM2Lag-Llama gère non seulement les données multivariées, mais tire également parti de la capacité du modèle LSTM à capturer les relations multivariées dans ses états cachés. Bien que l'étude n'implémente pas directement le modèle dans un contexte pratique de soins de santé, elle souligne le potentiel de telles avancées dans les techniques de prévision des séries temporelles. Cette recherche représente une contribution significative au domaine de la prévision des séries temporelles dans le domaine de la santé. Elle ouvre de nouvelles voies pour la recherche future et a le potentiel d'impact significatif sur la manière dont les professionnels de la santé utilisent les données de séries temporelles pour la prédiction et la prise de décision.

The object of the study is automated machine learning packages for forecasting time series. The subject of the study is hyperparameter optimization algorithms used in a number of selected packages. The purpose of the work is to compare automated machine learning packages in the context of the problem of forecasting time series and to identify the features of approaches to optimizing hyperparameters used in each package. Research methods: conducting a theoretical analysis of the available literature on the topic of the study, studying the documentation for the automatic machine learning packages involved in the work, conducting experiments, comparing and evaluating the forecasting results using the constructed pipelines, generalizing and interpreting the results. Results of the work: features in the implementation of hyperparameter optimization algorithms for the libraries under consideration are highlighted. / Объект исследования — пакеты автоматизированного машинного обучения для прогнозирования временных рядов. Предмет исследования — алгоритмы оптимизации гиперпараметров применяемые в ряде выбранных пакетов. Цель работы — проведение сравнения пакетов автоматизированного машинного обучения в контексте задачи прогнозирования временных рядов и выявление особенностей подходов к оптимизации гиперпараметров используемых в каждом пакете. Методы исследования: проведение теоретического анализа доступной литературы по теме исследования, изучение документации к задействованным в работе пакетам автоматического машинного обучения, проведение экспериментов, сравнение и оценка результатов прогнозирования с помощью построенных конвейеров, обобщение и интерпретация полученных результатов. Результаты работы: выделены особенности в реализации алгоритмов оптимизации гиперпараметров для рассматриваемых библиотек.

The idea of time series forecasting techniques is that the past has certain information about future. So, the question of how the information is encoded in the past can be interpreted and later used to extrapolate events of future constitute the crux of time series analysis and forecasting. Several methods such as qualitative techniques (e.g., Delphi method), causal techniques (e.g., least squares regression), quantitative techniques (e.g., smoothing method, time series models) have been developed in the past in which the concept lies in establishing a model either theoretically or mathematically from past observations and estimate future from it. Of all the models, time series methods such as autoregressive moving average (ARMA) process have gained popularity because of their simplicity in implementation and accuracy in obtaining forecasts. But, these models were formulated based on certain properties that a time series is assumed to possess. Classical decomposition techniques were developed to supplement the requirements of time series models. These methods try to define a time series in terms of simple patterns called trend, cyclical and seasonal patterns along with noise. So, the idea of decomposing a time series into component patterns, later modeling each component using forecasting processes and finally combining the component forecasts to obtain actual time series predictions yielded superior performance over standard forecasting techniques. All these methods involve basic principle of moving average computation. But, the developed classical decomposition methods are disadvantageous in terms of containing fixed number of components for any time series, data independent decompositions. During moving average computation, edges of time series might not get modeled properly which affects long range forecasting. So, these issues are to be addressed by more efficient and advanced decomposition techniques such as Wavelets and Empirical Mode Decomposition (EMD). Wavelets and EMD are some of the most innovative concepts considered in time series analysis and are focused on processing nonlinear and nonstationary time series. Hence, this research has been undertaken to ascertain the predictability of nonstationary time series using wavelet and Empirical Mode Decomposition (EMD) based ARMA models. The development of wavelets has been made based on concepts of Fourier analysis and Window Fourier Transform. In accordance with this, initially, the necessity of involving the advent of wavelets has been presented. This is followed by the discussion regarding the advantages that are provided by wavelets. Primarily, the wavelets were defined in the sense of continuous time series. Later, in order to match the real world requirements, wavelets analysis has been defined in discrete scenario which is called as Discrete Wavelet Transform (DWT). The current thesis utilized DWT for performing time series decomposition. The detailed discussion regarding the theory behind time series decomposition is presented in the thesis. This is followed by description regarding mathematical viewpoint of time series decomposition using DWT, which involves decomposition algorithm. EMD also comes under same class as wavelets in the consequence of time series decomposition. EMD is developed out of the fact that most of the time series in nature contain multiple frequencies leading to existence of different scales simultaneously. This method, when compared to standard Fourier analysis and wavelet algorithms, has greater scope of adaptation in processing various nonstationary time series. The method involves decomposing any complicated time series into a very small number of finite empirical modes (IMFs-Intrinsic Mode Functions), where each mode contains information of the original time series. The algorithm of time series decomposition using EMD is presented post conceptual elucidation in the current thesis. Later, the proposed time series forecasting algorithm that couples EMD and ARMA model is presented that even considers the number of time steps ahead of which forecasting needs to be performed. In order to test the methodologies of wavelet and EMD based algorithms for prediction of time series with non stationarity, series of streamflow data from USA and rainfall data from India are used in the study. Four non-stationary streamflow sites (USGS data resources) of monthly total volumes and two non-stationary gridded rainfall sites (IMD) of monthly total rainfall are considered for the study. The predictability by the proposed algorithm is checked in two scenarios, first being six months ahead forecast and the second being twelve months ahead forecast. Normalized Root Mean Square Error (NRMSE) and Nash Sutcliffe Efficiency Index (Ef) are considered to evaluate the performance of the proposed techniques. Based on the performance measures, the results indicate that wavelet based analyses generate good variations in the case of six months ahead forecast maintaining harmony with the observed values at most of the sites. Although the methods are observed to capture the minima of the time series effectively both in the case of six and twelve months ahead predictions, better forecasts are obtained with wavelet based method over EMD based method in the case of twelve months ahead predictions. It is therefore inferred that wavelet based method has better prediction capabilities over EMD based method despite some of the limitations of time series methods and the manner in which decomposition takes place. Finally, the study concludes that the wavelet based time series algorithm could be used to model events such as droughts with reasonable accuracy. Also, some modifications that could be made in the model have been suggested which can extend the scope of applicability to other areas in the field of hydrology.