Multi-agent information gathering (MA-IG) enables autonomous robots to cooperatively collect information in an unfamiliar area. In some scenarios, the focus is on gathering the true mapping of a physical quantity such as temperature or magnetic field. This thesis proposes a computationally efficient algorithm known as multi-agent RRT-clustered Stackelberg game (MA-RRTc-SG) to solve MA-IG. During exploration, measurements are taken along robot paths to update the belief of a Gaussian process (GP), which gives a continuous estimation of the physical process. To seek informative paths, agents first resort to self-planning: one individually generates a number of choices using sampling-based algorithms and preserves informative ones. Then, paths from different robots are combined and investigated based on a multi-player Stackelberg game. The Stackelberg game ensures robots select the combination of paths that yield maximum system reward. The reward function plays an important role in the aforementioned two steps. In our work, robots are awarded for selecting informative paths and punished for hazardous movements and large control inputs. In experiments, we first conduct variation studies to investigate the influence of key parameters in the proposed algorithm. Then, the algorithm is tested in a simulation case to map the radiation intensity in a nuclear plant. Results show that using our algorithm, robots are able to collect information in an efficient and cooperative way compared to random exploration. / Multi-agent informationsinsamling gör det möjligt för autonoma robotar att samarbeta samla in information i ett okänt område. I vissa scenarier ligger fokus på att samla in den verkliga kartläggningen av en fysisk storhet som temperatur eller magnetfält. Den här avhandlingen föreslår en beräkningseffektiv algoritm som kallas multi-agent RRT-clustered Stackelberg game (MA-RRTc-SG) för att lösa multi-agent informationsinsamling. Under prospektering görs mätningar längs robotbanor för att uppdatera tron på en Gaussisk process, vilket ger en kontinuerlig uppskattning av den fysiska processen. För att söka informativa vägar tillgriper agenter först självplanering: man genererar individuellt ett antal val med hjälp av samplingsbaserade algoritmer och bevarar informativa. Sedan kombineras och undersöks vägar från olika robotar utifrån en Stackelberg spel för flera spelare. Stackelberg spelet säkerställer att robotar väljer kombinationen av vägar som ger maximal systembelöning. Belöningsfunktionen spelar en viktig roll i de ovan nämnda två stegen. I vårt arbete belönas robotar för att välja informativa vägar och straffas för osäkra rörelser och stora kontrollingångar. I experiment genomför vi först variationsstudier för att undersöka inverkan av nyckelparametrar i den föreslagna algoritmen. Därefter testas algoritmen i ett simuleringsfall för att kartlägga strålningsintensiteten i ett kärnkraftverk. Resultaten visar att med vår algoritm kan robotar samla in information på ett effektivt och samarbetssätt jämfört med slumpmässig utforskning.

This project concerns optimizing the behavior ofmultiple dispatching robots in a virtual warehouse environment.Q-learning and deep Q-learning algorithms, two establishedmethods in reinforcement learning, were used for this purpose.Simulations were run during the project, implementing andcomparing different algorithms on environments with up to fourrobots. The efficiency of a given algorithm was assessed primarilyby the number of packages it enabled the robots to deliver andhow fast the solution converged. The simulation results revealedthat a Q-learning algorithm could solve problems in environmentswith up to two active robots efficiently. To solve more complexproblems in environments with more than two robots, deep Qlearninghad to be implemented to avoid prolonged computationsand excessive memory usage. / Detta projekt handlar om att optimera rörelserna för ett flertal robotar i en virtuell miljö. Q-learning och deep Q-learning-algoritmer, två väletablerade metoder inom maskininlärning, användes för detta. Under projektet utfördes simuleringar där de olika algoritmerna jämfördes i miljöer med upp till fyra robotar. En given algoritms prestanda bedömdes med avseende på hur många paket robotarna kunde leverera i miljön samt hur snabbt en lösning konvergerade. Resultaten visade att Q-learning kunde lösa problem i miljöer med upp 2 robotar effektivt. För större problem användes deep Q-learning för att undvika långvariga beräkningar och stor minnesåtgång. / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

One of the main focus of robotics is to integraterobotic tasks and motion planning, which has an increasedsignificance due to their growing number of application fieldsin transportation, navigation, warehouse management and muchmore. A crucial step towards this direction is to have robotsautomatically plan its trajectory to accomplish the given task.In this project a multi-layered approach was implemented toaccomplish it. Our framework consists of a discrete high-levelplanning layer that is designed for planning, and a continuouslow-level search layer that uses a sampling-based method for thetrajectory searching. The layers will interact with each otherduring the search for a solution. In order to coordinate formulti-agent system, velocity tuning is used to avoid collisions, anddifferent priority are assigned to each robot to avoid deadlocks.As a result, the framework trades off completeness for efficiency.The main aim of this project is to study and learn about high-level motion planning and multi-agent system, as an introductionto robotics and computer science. / En viktig aspekt inom robotik är att integrera robotuppgifter med rörelseplanering, som har en ökande be- tydelse för samhället på grund av dess applikationsområde inom t.ex. transport, navigering och lagerhantering. Ett avgörande steg till detta är att få robotarna automatiskt planerar sin bana för att utföra de givna uppgifterna. I detta projekt implementerades “Multi-layered” metod för att uppnå detta. Metoden består av ett hög-nivå diskret planeringslager som är designad för planering, och ett kontinuerligt låg-nivå sökningslager som använder ”sampling-based” algoritmer för sökning av bana. Lagerna interageras med varandra under den tiden där metoden söker efter en önskvärd bana som satisfiera uppdraget. För att koordinera samtliga robotar används den frikopplat approachen där hastigheter för olika robotar justeras till att undvika kollisioner, samt olika prioriteringar tilldelas för varje robot för att undvika ett blockerat låsläge. ”Sampling-based” algoritmer och den frikopplat approachen är oftast mer effektivt tidsmässigt men garantera inte att lösning kommer att hittas även om den existerar. Syftet med detta projekt är att studera och lära sig om rörelseplanering på högt-nivå och multi-agentsystem, som en introduktion till robotik och datavetenskap. / Kandidatexjobb i elektroteknik 2020, KTH, Stockholm

[ES] La Industria 4.0 se enfrenta a importantes retos a la hora de perseguir la transformación digital y la eficiencia operativa. La creciente complejidad de los entornos industriales modernos lleva a la necesidad de desplegar tecnologías digitales y, sobre todo, la automatización de la Industria. Sin embargo, este camino hacia la innovación va acompañado de numerosos obstáculos, ya que el entorno cambia constantemente. Por lo tanto, para adaptarse a esta evolución, es necesario emplear planteamientos más flexibles. Estos planteamientos están estrechamente relacionados con el uso de la AI y RL, ya que surgen como soluciones clave para abordar los retos cruciales de la navegación cooperativa de agentes dentro de entornos dinámicos. Mientras tanto, los algoritmos RL se enfrentan a las complejidades que implica la transmisión y el procesamiento de grandes cantidades de datos, para hacer frente a este desafío, la tecnología 5G emerge como un habilitador clave para las soluciones de escenarios de problemas evolutivos. Entre las principales ventajas de la 5G están que ofrece una transmisión rápida y segura de grandes volúmenes de datos con una latencia mínima. Al ser la única tecnología hasta ahora capaz de ofrecer estas capacidades, 5G se convierte en un componente esencial para desplegar servicios en tiempo real como la navegación cooperativa. Además, otra ventaja es que proporciona la infraestructura necesaria para intercambios de datos robustos y contribuye a la eficiencia del sistema y a la seguridad de los datos en entornos industriales dinámicos. A la vista de lo anterior, es evidente que la complejidad de los entornos industriales conduce a la necesidad de proponer sistemas basados en nuevas tecnologías como las redes AI y 5G, ya que su combinación proporciona una potente sinergia. Además, aparte de abordar los retos identificados en la navegación cooperativa, también abre la puerta a la implementación de fábricas inteligentes, dando lugar a mayores niveles de automatización, seguridad y productividad en las operaciones industriales. Es importante destacar que la aplicación de técnicas de AI conlleva la necesidad de utilizar software de simulación para probar los algoritmos propuestos en entornos virtuales. Esto permite abordar cuestiones esenciales sobre la validez de los algoritmos, reducir los riesgos de daños en el hardware y, sobre todo, optimizar las soluciones propuestas. Con el fin de proporcionar una solución a los retos fundamentales en la automatización de fábricas, esta Tesis se centra en la integración de la robótica móvil en la nube, especialmente en el contexto de la Industria 4.0. También abarca la investigación de las capacidades de las redes 5G, la evaluación de la viabilidad de simuladores como ROS y Gazebo, y la fusión de datos de sensores y el diseño de algoritmos de planificación de trayectorias basados en RL. En otras palabras, esta Tesis no solo identifica y aborda los retos clave de la Industria 4.0, sino que también presenta soluciones innovadoras e hipótesis concretas para la investigación. Además, promueve la combinación de AI y 5G para desplegar servicios en tiempo real, como la navegación cooperativa. Así, aborda retos críticos y demuestra que la colaboración tecnológica redefine la eficiencia y la adaptabilidad en la industria moderna. / [CA] La Indústria 4.0 s'enfronta a importants reptes a l'hora de perseguir la transformació digital i l'eficiència operativa. La creixent complexitat dels entorns industrials moderns porta a la necessitat de desplegar tecnologies digitals i, sobretot, l'automatització de la Indústria. No obstant això, este camí cap a la innovació va acompanyat de nombrosos obstacles, ja que l'entorn canvia constantment. Per tant, per a adaptar-se a esta evolució, és necessari emprar plantejaments més flexibles. Estos plantejaments estan estretament relacionats amb l'ús de l'AI i RL, ja que sorgixen com a solucions clau per a abordar els reptes crucials de la navegació cooperativa d'agents dins d'entorns dinàmics. Mentrestant, els algorismes RL s'enfronten a les complexitats que implica la transmissió i el processament de grans quantitats de dades, per a fer front a este desafiament, la tecnologia 5G emergix com un habilitador clau per a les solucions d'escenaris de problemes evolutius. Entre els principals avantatges de la 5G estan que oferix una transmissió ràpida i segura de grans volums de dades amb una latència mínima. A l'ésser l'única tecnologia fins ara capaç d'oferir estes capacitats, 5G es convertix en un component essencial per a desplegar servicis en temps real com la navegació cooperativa. A més, un altre avantatge és que proporciona la infraestructura necessària per a intercanvis de dades robustes i contribuïx a l'eficiència del sistema i a la seguretat de les dades en entorns industrials dinàmics. A la vista de l'anterior, és evident que la complexitat dels entorns industrials conduïx a la necessitat de proposar sistemes basats en noves tecnologies com les xarxes AI i 5G, ja que la seua combinació proporciona una potent sinergia. A més, a part d'abordar els reptes identificats en la navegació cooperativa, també obri la porta a la implementació de fabriques intel·ligents, donant lloc a majors nivells d'automatització, seguretat i productivitat en les operacions industrials. És important destacar que l'aplicació de tècniques d'AI comporta la necessitat d'utilitzar programari de simulació per a provar els algorismes proposats en entorns virtuals. Això permet abordar qüestions essencials sobre la validesa dels algorismes, reduir els riscos de dona'ns en el maquinari i, sobretot, optimitzar les solucions proposades. Amb la finalitat de proporcionar una solució als reptes fonamentals en l'automatització de fabriques, esta Tesi se centra en la integració de la robòtica mòbil en el núvol, especialment en el context de la Indústria 4.0. També abasta la investigació de les capacitats de les xarxes 5G, l'avaluació de la viabilitat de simuladors com ROS i Gazebo, i la fusió de dades de sensors i el disseny d'algorismes de planificació de trajectòries basats en RL. En altres paraules, esta Tesi no sols identifica i aborda els reptes clau de la Indústria 4.0, sinó que també presenta solucions innovadores i hipòtesis concretes per a la investigació. A més, promou la combinació d'AI i 5G per a desplegar servicis en temps real, com la navegació cooperativa. Així, aborda reptes crítics i demostra que la col·laboració tecnològica redefinix l'eficiència i l'adaptabilitat en la indústria moderna. / [EN] Industry 4.0 faces significant challenges in pursuing digital transformation and operational efficiency. The increasing complexity of modern industrial environments leads to the need to deploy digital technologies and, above all, Industry automation. However, this path to innovation is accompanied by numerous obstacles, as the environment constantly changes. Therefore, to adapt to this evolution, it is necessary to employ more flexible approaches. These approaches are closely linked to the use of Artificial Intelligence (AI) and Reinforcement Learning (RL), as they emerge as pivotal solutions to address the crucial challenges of cooperative agent navigation within dynamic environments. Meanwhile, RL algorithms face the complexities involved in transmitting and processing large amounts of data. To address this challenge, Fifth Generation (5G) technology emerges as a key enabler for evolutionary problem scenario solutions. Among the main advantages of 5G is that it offers fast and secure transmission of large volumes of data with minimal latency. As the only technology so far capable of delivering these capabilities, 5G becomes an essential component for deploying real-time services such as cooperative navigation. Furthermore, another advantage is that it provides the necessary infrastructure for robust data exchanges and contributes to system efficiency and data security in dynamic industrial environments. In view of the above, it is clear that the complexity of industrial environments leads to the need to propose systems based on new technologies such as AI and 5G networks, as their combination provides a powerful synergy. Moreover, aside from tackling the challenges identified in cooperative navigation, it also opens the door to the implementation of smart factories, leading to higher levels of automation, safety, and productivity in industrial operations. It is important to note that the application of AI techniques entails the need to use simulation software to test the proposed algorithms in virtual environments. This makes it possible to address essential questions about the validity of the algorithms, reduce the risks of damage to the hardware, and, above all, optimize the proposed solutions. In order to provide a solution to the fundamental challenges in factory automation, this Thesis focuses on integrating mobile robotics in the cloud, especially in the context of Industry 4.0. It also covers the investigation of the capabilities of 5G networks, the evaluation of the feasibility of simulators such as Robot Operating System (ROS) and Gazebo, and the fusion of sensor data and the design of path planning algorithms based on RL. In other words, this Thesis not only identifies and addresses the key challenges of Industry 4.0 but also presents innovative solutions and concrete hypotheses for research. Furthermore, it promotes the combination of AI and 5G to deploy real-time services, such as cooperative navigation. Thus, it addresses critical challenges and demonstrates that technological collaboration redefines efficiency and adaptability in modern industry. / This research was funded by the Research and Development Grants Program (PAID-01-19) of the Universitat Politècnica de València. The research stay of the author at Technischen Universit¨at Darmstadt (Germany) was funded by the Program of Grants for Student Mobility of doctoral students at the Universitat Politècnica de València in 2022 from Spain and by Erasmus+ Student Mobility for Traineeship 2022 / Palacios Morocho, ME. (2024). Unleashing Technological Collaboration: AI, 5G, and Mobile Robotics for Industry 4.0 Advancements [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/204748

The introduction of communication technology into the electric power grid has made the grid more reliable. Power system operators gain visibility over the power system and are able to resolve operational issues remotely via Supervisory Control And Data Acquisition (SCADA) technology. This reduces outage periods. Nonetheless, the remote-control capability has rendered the power grid vulnerable to cyberattacks. In December 2015, over 200,000 people in Ukraine became victims of the first publicly reported cyberattack on the power grid. Consequently, cyber-physical security research for the power system as a critical infrastructure is in critical need. Research on cybersecurity for power grids has produced a diverse literature; the multi-faceted nature of the grid makes it vulnerable to different types of cyberattacks, such as direct power grid, supply chain and ransom attacks. The attacks may also target different levels of grid operation, such as the transmission system, distribution system, microgrids, and generation. As these levels are characterized by varying operational constraints, the literature may be categorized not only according to the type of attack it targets, but also according to the level of power system operation under consideration. It is noteworthy that cybersecurity research for the transmission system dominates the literature, although the distribution system is noted to have a larger attack surface. For the distribution system, a notable attack type is the so-called direct switching attack, in which an attacker aims to disrupt power supply by compromising switching devices that connect equipment such as generators, and power grid lines. To maximize the damage, this attack tends to be coordinated as the attacker optimally selects the nodes and switches to attack. This decision-making process is often a bi- or tri-level optimization problem which models the interaction between the attacker and the power system defender. It is necessary to detect attacks and establish coordination/correlation among them. Determining coordination is a necessary step to predict the targets of an attack before attack completion, and aids in the mitigation strategy that ensues. While the literature has addressed the direct switching attack on the distribution system in different ways, there are also shortcomings. These include: (i) techniques to establish coordination among attacks are centralized, making them prone to single-point failures; (ii) techniques to establish coordination among attacks leverage only power system models, ignoring the influence of communication network vulnerabilities and load criticality in the decisions of the attacker; (iii) attacker-defender optimization models assume specific knowledge of the attacker resources and constraints by the defender, a strong unrealistic assumption that reduces their usability; (iv) and, mitigation strategies tend to be static and one-sided, being implemented only at the physical level, or at the communication network level. In light of this, this dissertation culminates in major contributions concerning real-time decentralized correlation of detected direct switching attacks and hybrid mitigation for electric power distribution systems. Concerning this, four novel contributions are presented: (i) a framework for decentralized correlation of attacks and mitigation; (ii) an attacker-defender optimization model that accounts for power system laws, load criticality, and cyber vulnerabilities in the decision-making process of the attacker; (iii) a real-time learning-based mechanism for determining correlation among detected attacks and predicting attack targets, and which does not assume knowledge of the attacker's resources and constraints by the power system defender; (iv) a hybrid mitigation strategy optimized in real-time based on information learned from detected attacks, and which combines both physical level and communication network level mitigation. Since the execution of intrusion detection systems and mechanisms such as the ones proposed in this dissertation may deter attackers from directly attacking the power grid, attackers may perform a supply chain cyberattack to yield the same results. Although, supply chain cyberattacks have been acknowledged as potentially far-reaching, and compliance directives put forward for this, the detection of supply chain cyberattacks is in a nascent stage. Consequently, this dissertation also proposes a novel method for detecting supply chain cyberattacks. To the best of the knowledge of the author, this work is the first preliminary work on supply chain cyberattack detection. / Doctor of Philosophy / The electric power grid is the network that transports electricity from generation to consumers, such as homes and factories. The power grid today is highly remote-monitored and controlled. Should there be a fault on the grid, the human operator, often remotely located, may only need to resolve it by sending a control signal to telemetry points, called nodes, via a communication network. This significantly reduces outage periods and improves the reliability of the grid. Nonetheless, the high level connectivity also exposes the grid to cyberattacks. The cyber connectivity between the power grid and the human operator, like all communication networks, is vulnerable to cyberattacks that may allow attackers to gain control of the power grid. If and when successful, wide-spread and extended outages, equipment damage, etc. may ensue. Indeed, in December 2015, over 200,000 people in Ukraine became victims to the first publicly reported cyberattack on a power grid. As a critical infrastructure, cybersecurity for the power grid is, therefore, in critical need. Research on cybersecurity for power grids has produced a diverse literature; the multi-faceted nature of the grid makes it vulnerable to different types of cyberattacks, such as direct power grid, supply chain and ransom attacks. Notable is the so-called direct switching attack, in which an attacker aims to compromise the power grid communication network in order to toggle switches that connect equipment such as generators, and power grid lines. The aim is to disrupt electricity service. To maximize the damage, this attack tends to be coordinated; the attacker optimally selects several grid elements to attack. Thus, it is necessary to both detect attacks and establish coordination among them. Determining coordination is a necessary step to predict the targets of an attack before attack completion. This aids the power grid owner to intercept and mitigate attacks. While the literature has addressed the direct switching attack in different ways, there are also shortcomings. Three outstanding ones are: (i) techniques to determine coordination among attacks and predict attack targets are centralized, making them prone to single-point failures; (ii) techniques to establish coordination among attacks leverage only power system physical laws, ignoring the influence of communication network vulnerabilities in the decisions of the attacker; (iii) and, studies on the interaction between the attacker and the defender (i.e., power grid owner) assume specific knowledge of the attacker resources and constraints by the defender, a strong unrealistic assumption that reduces their usability. This research project addresses several of the shortcomings in the literature, particularly the aforementioned. The work focuses on the electric distribution system, which is the power grid that connects directly to consumers. Indeed, this choice is ideal, as the distribution system has a larger attack surface than other parts of the grid and is characterized by computing devices with more constrained computational capability. Thus, adaptability to simple computing devices is a priority. The contributions of this dissertation provide leverage to the power grid owner to intercept and mitigate attacks in a resilient manner. The original contributions of the work are: (i) a novel realistic model that shows the decision making process of the attacker and their interactions with the defender; (ii) a novel decentralized mechanism for predicting the targets of coordinated cyberattacks on the electric distribution grid in real-time and which is guided by the attack model, (iii) and a novel hybrid optimized mitigation strategy that provides security to the power grid at both the communication network level and the physical power grid level. Since the power grid is constructed with smart equipment from various vendors, attackers may launch effective attacks by compromising the devices deployed in the power grid through a compromised supply chain. By nature, such an attack is evasive to traditional intrusion detection systems and algorithms such as the aforementioned. Therefore, this work also provides a new method to defend the grid against supply chain attacks, resulting in a mechanism for its detection in a critical power system communication device.

Coordination is a desirable feature in many multi-agent systems, such as robotic, social and economic networks, allowing the execution of tasks that would be impossible by individual agents. This thesis addresses two problems in stochastic coordination where each agent make decisions strategically, taking into account the decisions of its neighbors over a regular network. In the first problem, we study the coordination in a team of strategic agents choosing to undertake one of the multiple tasks. We adopt a stochastic framework where the agents decide between two distinct tasks whose difficulty is randomly distributed and partially observed. We show that a Nash equilibrium with a simple and intuitive linear structure exists for textit{diffuse} prior distributions on the task difficulties. Additionally, we show that the best response of any agent to an affine strategy profile can be nonlinear when the prior distribution is not diffuse. Then, we state an algorithm that allows us to efficiently compute a data-driven Nash equilibrium within the class of affine policies. In the second problem, we assume that the payoff structure of the coordination game corresponds to a single task allocation scenario whose difficulty is perfectly observed. Since there are multiple Nash equilibria in this game, the agents must use a distributed stochastic algorithm know as textit{log linear learning} to play it multiple times. First, we show that this networked coordination game is a potential game. Moreover, we establish that for regular networks, the convergence to a Nash equilibrium depends on the ratio between the task-difficulty parameter and the connectivity degree according to a threshold rule. We investigate via simulations the interplay between rationality and the degree of connectivity of the network. Our results show counter-intuitive behaviors such as the existence of regimes in which agents in a network with larger connectivity require less rational agents to converge to the Nash equilibrium with high probability. Simultaneously, we examined the characteristics of both regular graphical coordination games and non-regular graphical games using this particular bi-matrix game model. / Master of Science / This thesis focuses on addressing two problems in stochastic coordination among strategic agents in multi-agent systems, such as robotic, social, and economic networks. The first problem studies the coordination among agents when they need to choose between multiple tasks whose difficulties are randomly distributed and partially observed. The thesis shows the existence of a Nash equilibrium with a linear structure for certain prior distributions, and presents an algorithm to efficiently compute a data-driven Nash equilibrium within a specific class of policies. The second problem assumes a single task allocation scenario, whose difficulty is perfectly observed, and investigates the use of a distributed stochastic algorithm known as log-linear learning to converge to a Nash equilibrium. The thesis shows that the convergence to a Nash equilibrium depends on the task-difficulty parameter and the connectivity degree of the network, and explores the influence of rationality of the agents and the connectivity of the network on the learning process. Overall, the thesis provides insights into the challenges and opportunities in achieving coordination among strategic agents in multi-agent systems.

<p dir="ltr">In multiplayer games, a collection of self-interested players aims to optimize their individual cost functions in a non-cooperative manner. The cost function of each player depends not only on its own actions but also on the actions of others. In addition, players' actions may also collectively satisfy some global constraints. The study of this problem has grown immensely in the past decades with applications arising in a wide range of societal systems, including strategic behaviors in power markets, traffic assignment of strategic risk-averse users, engagement of multiple humanitarian organizations in disaster relief, etc. Furthermore, with machine learning models playing an increasingly important role in practical applications, the robustness of these models becomes another prominent concern. Investigation into the solutions of multiplayer games and Nash equilibrium problems (NEPs) can advance the algorithm design for fitting these models in the presence of adversarial noises. </p><p dir="ltr">Most of the existing methods for solving multiplayer games assume the presence of a central coordinator, which, unfortunately, is not practical in many scenarios. Moreover, in addition to couplings in the objectives and the global constraints, all too often, the objective functions contain uncertainty in the form of stochastic noises and unknown model parameters. The problem is further complicated by the following considerations: the individual objectives of players may be unavailable or too complex to model; players may exhibit reluctance to disclose their actions; players may experience random delays when receiving feedback regarding their actions. To contend with these issues and uncertainties, in the first half of the thesis, we develop several algorithms based on the theory of operator splitting and stochastic approximation, where the game participants only share their local information and decisions with their trusted neighbors on the network. In the second half of the thesis, we explore the bandit online learning framework as a solution to the challenges, where decisions made by players are updated based solely on the realized objective function values. Our future work will delve into data-driven approaches for learning in multiplayer games and we will explore functional representations of players' decisions, in a departure from the vector form. </p>

為了解決代理者程式在開放式網際網路上溝通的問題，我們採用了語意網中本體論的技術。目的是透過可信度等多個本體論的使用，代理者程式可以進行具有語意程度的溝通以完成代理者程式的互信。研究中，我們利用了DAML+OIL這種具有強大的表達能力的語言來描述數位電子憑証及安全相關的字彙，以及代理者程式彼此間的關係以便於檢驗代理者可信度，最後我們藉代理者程式的認証、授權等的溝通協定來達成資訊網服務的資源及可信度的控管機制。 / We use ontology technology from the Semantic Web to solve the agent communication problem. The idea is to build trust and other ontologies for the multi-agent system to ensure semantic level agent communication on agent trust control. In this research, we use a powerful ontology language, i.e. DAML+OIL to explicitly describe a variety of digital certificates and the relationship among agents for agent trust verification. Furthermore, we fulfill the resource and trust control mechanism using agent authentication/authorization communication protocols on the Web Services environment.

The seamless retrieval of information distributed across networks has been one of the key goals of many systems. Early solutions involved the use of single static agents which would retrieve the unfiltered data and then process it. However, this was deemed costly and inefficient in terms of the bandwidth since complete files need to be downloaded when only a single value is often all that is required. As a result, mobile agents were developed to filter the data in situ before returning it to the user. However, mobile agents have their own associated problems, namely security and control. The Agent Trees Multi-Agent System (AT-MAS) has been developed to provide the remote processing and filtering capabilities but without the need for mobile code. It is implemented as a Peer to Peer (P2P) network of static intelligent cooperating agents, each of which control one or more data sources. This dissertation describes the two key technologies have directly influenced the design of ATMAS, Peer-to-Peer (P2P) systems and Multi-Agent Systems (MAS). P2P systems are conceptually simple, but limited in power, whereas MAS are significantly more complex but correspondingly more powerful. The resulting system exhibits the power of traditional MAS systems while retaining the simplicity of P2P systems. The dissertation describes the system in detail and analyses its performance.

This work focuses on the need for new knowledge to allow hierarchical multi-project management to be conducted in the construction industry, which is characterised by high uncertainty, fragmentation, complex decisions, dynamic changes and long-distance communication. A dynamic integrated project management approach is required at strategic, tactical and operational levels in order to achieve adaptability. The work sees the multi-project planning and control problem in the context of supply chain management at main contractor companies. A portfolio manager must select and prioritise the projects, bid and negotiate with a wide range of clients, while project managers are dealing with subcontractors, suppliers, etc whose relationships and collaborations are critical to the optimisation of schedules in which time, cost and safety (etc) criteria must be achieved. Literature review and case studies were used to investigate existing approaches to hierarchical multi-project management, to identify the relationships and interactions between the parties concerned, and to investigate the possibilities for integration. A system framework was developed using a multi-agent-system architecture and utilising procedures adapted from literature to deal with short, medium and long-term planning. The framework is based on in-depth case study and integrates time-cost trade-off for project optimisation with multi-attribute utility theory to facilitate project scheduling, subcontractor selection and bid negotiation at the single project level. In addition, at the enterprise level, key performance indicator rule models are devised to align enterprise supply chain configuration (strategic decision) with bid selection and bid preparation/negotiation (tactical decision) and project supply chain selection (operational decision). Across the hierarchical framework the required quantitative and qualitative methods are integrated for project scheduling, risk assessment and subcontractor evaluation. Thus, experience sharing and knowledge management facilitate project planning across the scattered construction sites. The mathematical aspects were verified using real data from in-depth case study and a test case. The correctness, usefulness and applicability of the framework for users was assessed by creating a prototype Multi Agent System-Decision Support System (MAS-DSS) which was evaluated empirically with four case studies in national, international, large and small companies. The positive feedback from these cases indicates strong acceptance of the framework by experienced practitioners. It provides an original contribution to the literature on planning and supply chain management by integrating a practical solution for the dynamic and uncertain complex multi-project environment of the construction industry.