The battery energy storage system (BESS) holds the promise of becoming an essential element in our energy landscape. With the increasing need for renewable energy and electrification, a BESS serves as backup power, grid frequency balancing, and playing a crucial role in achieving 100% renewable electricity production by 2040 in Sweden. This thesis aims to study intelligent energy trading algorithms for BESS with two markets. The algorithms would allow BESS to trade simultaneously with two markets, day-ahead (energy) market and FCR-N.The problem of trading is solved using reinforcement learning (RL) particularly, multi-agent reinforcement learning (MARL), are proposed as potential solutions for learning energy trading strategies across multiple markets, addressing a gap in current research. In this study two main algorithms are used: Deep Q-networks (DQN), and Advantage Actor-Critic (A2C). These two algorithms are adapted to the MARL’s paradigms. This thesis answers three main questions. First, if any of the MARL variations of the two mentioned algorithms have any advantage over the others. The results suggest that the CTDE variation of A2C performs the best, followed by centralized variation of A2C. Second, the discrete action spaces and continuous action spaces are compared. The algorithms with continuous action spaces achieved higher revenues. The continuous action spaces let the agents decide the exact volumes of the energy to trade. This is while in the discrete action spaces the agents can only choose the volumes from a defined set of values. Third and last, the results from the experiments suggest that trading with two markets results in higher revenue than trading with one market. All the MARL algorithms have higher revenue compared to the simple hard-rule strategy designed for trading with two markets. This thesis shows that the RL and MARL algorithms can be used for creating profitable trading agents and for identifying successful trading strategies.

The rapid advancement of artificial intelligence (AI) has broadened its applications across various sectors, with finance being a prominent area of focus. In financial trading, AI is primarily utilized to detect patterns and facilitate trading decisions. However, challenges such as noisy data, poor model generalization, and overfitting due to high variability in underlying assets continue to hinder its effectiveness. This study introduces a framework that builds on previous research at the intersection of AI and finance, implemented at AP1. It outlines the benefits and limitations of applying AI to trade derivatives rather than single company stocks and serves as a guide for building such trading algorithms. Furthermore, the research identifies an under-explored niche at the intersection of AI and derivative trading. By developing and applying this framework, the study not only addresses this gap but also evaluates the role of AI algorithms in enhancing derivative trading strategies, demonstrating their potential and limitations within this domain. / Den snabba utvecklingen av artificiell intelligens (AI) har breddat dess tillämpningar över olika sektorer, med finans som ett framträdande fokusområde. Inom finansiell handel används AI främst för att upptäcka mönster och underlätta handelsbeslut. Men utmaningar som bullriga data, dålig modellgeneralisering och överanpassning på grund av stor variation i underliggande tillgångar fortsätter dock att hindra dess effektivitet. Denna studie introducerar ett ramverk som bygger på tidigare forskning i skärningspunkten mellan AI och finans, implementerad på AP1. Den beskriver fördelarna och begränsningarna med att tillämpa AI för handel med derivat snarare än aktier i enskilda företag och fungerar som en guide för att bygga sådana handelsalgoritmer. Dessutom identifierar forskningen en underutforskad nisch i skärningspunkten mellan AI och derivathandel. Genom att utveckla och tillämpa detta ramverk tar studien inte bara upp denna lucka utan utvärderar också rollen av AI-algoritmer för att förbättra derivathandelsstrategier, och visar deras potential och begränsningar inom denna domän.