No description available.

No description available.

Systems consisting of multiple robots are traditionallydifficult to optimize. This project considers such a systemin a simulated warehouse setting, where the robots are todeliver boxes while avoiding collisions. Adding such collisionconstraints complicates the problem. For dynamical multi-agentsystems as these, reinforcement learning algorithms are oftenappropriate. We explore and implement a reinforcement learningalgorithm, called multi-agent rollout, that allows for re-planningduring operation. The algorithm is paired with a base policyof following the shortest path. Simulation results with up to10 robots indicates that the algorithm is promising for largescalemulti-robot systems. We have also discussed the possibilityof using neural networks and partitioning to further increaseperformance. / System med flera robotar har traditionellt sett ansetts mycket svåra att optimera. I detta projekt undersöks ett sådant system i en simulerad lagerlokal, där robotarna skall förflytta lådor samtidigt som de undviker kollisioner. För dessa dynamiska system med flera robotar är förstärkande inlärning ofta lämpligt. Vi undersöker och implementerar en förstärkandeinlärningsalgoritm kallad ”multi-agent rollout” vilken möjliggör omdirigering under drift. Algoritmen används tillsammans med en så kallad ”base policy” som alltid väljer kortaste vägen. Baserat på simulationsresultaten med upp till tio robotar verkar algoritmen lovande för storskaliga flerrobotsystem. Det diskuteras även om möjligheten av att använda neurala nätverk och partitionering för att vidare öka prestandan. / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

A major challenge in the transportation industry is how to reduce the emissions of greenhouse gases. One way of achieving this in vehicles is to drive more fuel-efficiently. One recently developed technique that has been successful in reducing the fuel consumption is the look-ahead cruise controller, which utilizes future conditions such as road topography. In this this thesis, similar methods are used in order to reduce the fuel consumption of heavy-duty vehicles driving in environments where the required and desired velocity vary. The main focus is on vehicles in urban driving, which must alter their velocity due to, for instance, changing legal speed restrictions and the presence of intersections. The driving missions of such vehicles are here formulated as optimal control problems. In order to restrict the vehicle to drive in a way that does not deviate too much from a normal way of driving, constraints on the velocity are imposed based on statistics from real truck operation. In a first approach, the vehicle model is based on forces and the cost function involves the consumed energy. This problem is solved both offline using Pontryagin's maximum principle and online using a model predictive controller with a quadratic program formulation. Simulations show that 7 % energy can be saved without increasing the trip time nor deviating from a normal way of driving. In a second approach, the vehicle model is extended to include an engine and a gearbox with the objective of minimizing the fuel consumption. A fuel map for the engine and a polynomial function for the gearbox losses are extracted from experimental data and used in the model. This problem is solved using dynamic programming taking into consideration gear changes, coasting with gear and coasting in neutral. Simulations show that by allowing the use of coasting in neutral gear, 13 % fuel can be saved without increasing the trip time or deviating from a normal way of driving. Finally, an implementation of a rule-based controller into an advanced vehicle model in highway driving is performed. The controller identifies sections of downhills where fuel can be saved by coasting in neutral gear. / En stor utmaning för transportsektorn är hur utsläppen av växthusgaser ska minskas. Detta kan åstadkommas i fordon genom att köra bränslesnålare. En nyligen utvecklad teknik som har varit framgångsrik i att minska bränsleförbrukningen är framförhållningsreglering, som använder framtida förhållanden så som vägtopografi. I denna avhandling används liknande metoder för att minska bränsleförbrukningen i tunga fordon som kör i miljöer där önskad och tvingad hastighet varierar. Fokus ligger framförallt på fordon i stadskörning, där hastigheten måste varieras beroende på bland annat hastighetsbegränsningar och korsningar. Denna typ av körning formuleras här som optimala reglerproblem. För att hindra fordonet från att avvika för mycket från ett normalt körbeteende sätts begränsningar på tillåten hastighet baserat på statistik från verklig körning. Problemet angrips först genom att använda en fordonsmodell baserad på krafter och en kriteriefunktion innehållande energiförbrukning. Problemet löses både offline med Pontryagin's maximum princip och online med modellprediktiv reglering baserad på kvadratisk programmering. Simuleringar visar att 7 % energi kan sparas utan att öka körtiden eller avvika från ett normalt körbeteende. Problemet angrips sedan genom att utöka fordonsmodellen till att också innehålla motor och växellåda med målet att minimera bränsleförbrukningen. Specifik bränsleförbrukning och en polynomisk approximation av förlusterna i växellådan är extraherade från experiment och används i simuleringarna. Problemet löses genom dynamisk programmering som tar hänsyn till växling, släpning och frirullning. Simuleringar visar att 13 % bränsle kan sparas utan att öka körtid eller avvika från normalt körbeteende genom att tillåta frirullning. Slutligen görs en implementering av en regelbaserad regulator på en avancerad fordonsmodell för ett fordon i motorvägskörning. Regulatorn identifierar sektioner med nedförsbackar där bränsle kan sparas genom frirulllning. / <p>QC 20171011</p>

Game theory is the study of tactical interactions involving conflicts and cooperations among multiple decision makers called players with applications in diverse disciplines such as economics, biology, management, communication networks, electric power systems and control. This dissertation studies a statistical differential game problem where finite N players optimize their system performance by shaping the distribution of their cost function through cost cumulants. This research integrates game theory with statistical optimal control theory and considers a statistical Nash non-cooperative nonzero-sum game for a nonlinear dynamic system with nonquadratic cost functions. The objective of the statistical Nash game is to find the equilibrium solution where no player has the incentive to deviate once other players maintain their equilibrium strategy. The necessary condition for the existence of the Nash equilibrium solution is given for the m-th cumulant cost optimization using the Hamilton-Jacobi-Bellman (HJB) equations. In addition, the sufficient condition which is the verification theorem for the existence of Nash equilibrium solution is given for the m-th cumulant cost optimization using the Hamilton-Jacobi-Bellman (HJB) equations. However, solving the HJB equations even for relatively low dimensional game problem is not trivial, we propose to use neural network approximate method to find the solution of the HJB partial differential equations for the statistical game problem. Convergence proof of the neural network approximate method solution to exact solution is given. In addition, numerical examples are provided for the statistical game to demonstrate the applicability of the proposed theoretical developments. / Electrical and Computer Engineering

In an optimal visiting problem, we want to control a trajectory that has to pass as close as possible to a collection of target points or regions. We introduce a hybrid control-based approach for the classic problem where the trajectory can switch between a group of discrete states related to the targets of the problem. The model is subsequently adapted to a mean-field game framework, that is when a huge population of agents plays the optimal visiting problem with a controlled dynamics and with costs also depending on the distribution of the population. In particular, we investigate a single continuity equation with possible sinks and sources and the field possibly depending on the mass of the agents. The same problem is also studied on a network framework. More precisely, we study a mean-field game model by proving the existence of a suitable definition of an approximated mean-field equilibrium and then we address the passage to the limit.

In this thesis, we consider an optimal stopping problem interpreted as the task of valuating two so called real options written on an underlying asset following the dynamics of an observable geometric Brownian motion with non-observable drift; we have incomplete information. After exercising the first real option, however, the value of the underlying asset becomes observable with reduced noise; we obtain partial information. We then state some theoretical properties of the value function such as convexity and monotonicity. Furthermore, numerical solutions for the value functions are obtained by stating and solving a linear complementary problem. This is done in a Python implementation using the 2nd order backward differentiation formula and summation-by-parts operators for finite differences combined with an operator splitting method.

Agent-based models are computer models made up of agents that can exist in a finite number of states. The state of the system at any given time is determined by rules governing agents' interaction. The rules may be deterministic or stochastic. Optimization is the process of finding a solution that optimizes some value that is determined by simulating the model. Optimal control of an agent-based model is the process of determining a sequence of control inputs to the model that steer the system to a desired state in the most efficient way. In large and complex models, the number of possible control inputs is too large to be enumerated by computers; hence methods must be developed for use with these models in order to find solutions without searching the entire solution space. Heuristic algorithms have been applied to such models with some success. Such algorithms are discussed; case studies of examples from biology are presented. The lack of a standard format for agent-based models is a major issue facing the study of agent-based models; presentation as polynomial dynamical systems is presented as a viable option. Algorithms are adapted and presented for use in this framework. / Master of Science

Heterogeneity, influenced by diverse factors such as age, gender, immunity, behavior, and spatial distribution, plays a critical role in the dynamics of infectious disease transmission. Discrete mathematical structures, including matrices and graphs, can offer effective tools for modeling the interactions among these diverse factors, resulting heterogeneous epidemiological models. This dissertation explores analytical approaches, specifically utilizing eigenvalues and eigenvectors of discrete structures, to characterize heterogeneity within mathematical models of infectious diseases. Theoretical results, along with numerical simulations, enhance our understanding of heterogeneous epidemiological processes and their significant implications for disease control strategies. In this dissertation, we introduce a unified approach to establish the final size formula in heterogeneous epidemic models, based on a new concept of “total infectious contacts” as an eigenvector-based aggregation of disease compartments. This approach allows us to identify the peak of total infectious contacts, offering a novel method to pinpoint the turning point of a disease outbreak. Furthermore, we examine spatial heterogeneity through two distinct mathematical frameworks: the Lagrangian and Eulerian models. The Lagrangian model assesses the epidemiological consequences of spatio-temporal residence time matrices, while the Eulerian model investigates “Turing instability” as a new underlying mechanism for spatial heterogeneity observed in disease prevalence data.

2025 / Over the last decade, the application of Wireless Sensor Networks (WSNs) has surged in fields such as environmental monitoring, human health, and smart cities. With this wealth of technologies comes the challenge of how to extract volumes of data collected by such sensor nodes distributed over large, often remote, geographical regions. Data harvesting is the problem of extracting measurements from the remote nodes of WSNs using mobile agents such as ground vehicles or drones. The use of mobile agents can significantly reduce the energy consumption of sensor nodes relative to other modes of extracting the data, extending the lifetime and capabilities of the WSN. Moreover, in remote areas where GPS may not be feasible due to limited power resources on the sensor nodes, the need for accurate sensor node localization and signal broadcasting model estimation becomes critical. Therefore, designing the trajectory of mobile agents is crucial for rapid data collection and information gathering while adhering to vehicle constraints such as dynamics and energy usage. In this thesis, we focus on the application of optimal control methods to design trajectories for mobile agents in data harvesting. This thesis makes contributions in three areas: the creation of a parameterized optimal control policy, the application of a Deep Reinforcement Learning (DRL) based control, and the use of Fisher Information (FI) as a cost matrix in a Receding Horizon Control (RHC) method. Parameterized Optimal Control Policy: Our contributions in this area begin by considering a data harvesting problem in 1-D space. We use a Hamiltonian analysis to show that the optimal control can be described using a parameterized policy and then develop a gradient descent scheme using Infinitesimal Perturbation Analysis (IPA) to calculate the gradients of the cost function with respect to the control parameters. We also consider this problem in a multi-agent setting. To avoid collisions between agents, we apply a Control Barrier Function (CBF) technique to ensure the agents closely track the desired optimal trajectory to complete their mission while avoiding any collisions. Finally, we extend the problem to a mobile sensor scenario. In this more complicated setting we demonstrate that the optimization problem for the control policy parameters can be effectively solved using a heuristic approach. Deep-Reinforcement-Learning based Control: The parametric optimal control approach cannot be easily extended from the 1-D setting to 2-D space. For this reason, we turn to DRL techniques. We utilize Hamiltonian analysis again to get the necessary conditions for optimal control and then translate the problem to a Markov Decision Process (MDP) in discrete time. We apply reinforcement learning techniques, including double deep Q-learning and Proximal Policy Optimization (PPO), to find high-performing solutions across different scenarios. We demonstrate the effectiveness of these methods in 2-D simulations. Fisher-Information-based Receding Horizon Control: For the data harvesting problem in large scale unknown environments, estimating the parameters defining the broadcast model and the location of all the nodes in the environment is critical for efficient extraction of the data. To address that, we start with a Received Signal Strength (RSS) model that relies on a Line-of-Sight (LoS) path-loss model with measurements that are corrupted by Gaussian distributed noise. We first consider a single agent tasked with estimating these unknown parameters in discrete time, and then develop a Fisher Information Matrix (FIM) Receding Horizon (RH) controller for agent motion planning in real time. We also design a Neural Network (NN)-based controller to approximate the optimal solution to the Hamilton-Jacobi-Bellman (HJB) problem, maximizing information gain along a continuous time trajectory. Additionally, a two-stage formation-based RH controller is designed for multi-agent scenarios. The experiments demonstrate that the optimal control policy contribute to the high performance of data collection and the FI-based RHC methods enhance the estimation accuracy in various simulation environments.