Decentralized decision-making is an active research topic in artificial intelligence. In a distributed system, a number of individually acting agents coexist. If they strive to accomplish a common goal, the establishment of coordinated cooperation between the agents is of utmost importance. With this in mind, our focus is on multi-agent reinforcement learning (RL) methods which allow for automatically acquiring cooperative policies based solely on a specification of the desired joint behavior of the whole system.The decentralization of the control and observation of the system among independent agents, however, has a significant impact on problem complexity. Therefore, we address the intricacy of learning and acting in multi-agent systems by two complementary approaches.First, we identify a subclass of general decentralized decision-making problems that features regularities in the way the agents interact with one another. We show that the complexity of optimally solving a problem instance from this class is provably lower than solving a general one.Although a lower complexity class may be entered by sticking to certain subclasses of general multi-agent problems, the computational complexitymay be still so high that optimally solving it is infeasible. Hence, our second goal is to develop techniques capable of quickly obtaining approximate solutions in the vicinity of the optimum. To this end, we will develop and utilize various model-free reinforcement learning approaches.Many real-world applications are well-suited to be formulated in terms of spatially or functionally distributed entities. Job-shop scheduling represents one such application. We are going to interpret job-shop scheduling problems as distributed sequential decision-making problems, to employ the multi-agent RL algorithms we propose for solving such problems, and to evaluate the performance of our learning approaches in the scope of various established scheduling benchmark problems.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第18408号 / 情博第523号 / 新制||情||92(附属図書館) / 31266 / 京都大学大学院情報学研究科社会情報学専攻 / (主査)教授 多々納 裕一, 教授 石田 亨, 准教授 畑山 満則 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

Market globalization and mass customization requirements are forcing companies towards automation of their product development processes. Many task-specific software solutions provide localized automation. Coordinating these local solutions to automate higher-level processes requires significant software maintenance costs due to the incompatibility of the software tools and the dynamic nature of the product development environment. Current automation methods do not provide the required level of flexibility to operate in this dynamic environment. An autonomic product development process automation strategy is proposed in order to provide a flexible, standardized approach to product development process automation and to significantly reduce the software maintenance costs associated with traditional automation methods. Key elements of the strategy include a formal approach to decompose product development processes into services, a method to describe functional and quality attributes of services, a process modeling algorithm to configure processes composed of services, a method to evaluate process utility based on quality metrics and user preferences, and an implementation that allows a user to instantiate the optimal process. Because the framework allows a user to rapidly reconfigure and select optimal processes as new services are introduced or as requirements change, the framework should reduce burdensome software maintenance costs associated with traditional automation methods and provide a more flexible approach.

Coalitions are collections of agents that join together to solve a common problem that either cannot be solved individually or can be solved more efficiently as a group. Each individual agent has capabilities that can benefit the group when working together as a coalition. Typically, individual capabilities are joined together in an additive way when forming a coalition. This work will introduce a new operator that is used when combining capabilities, and suggest that the behavior of the operator is contextual, depending on the nature of the capability itself. This work considers six different capabilities of Unmanned Air Vehicles (UAV) and determines the nature of the new operator in the context of each capability as coalitions (squadrons) of UAVs are formed. Coalitions are formed using three different search algorithms, both with and without heuristics: Depth-First, Depth-First Iterative Deepening, and Genetic Algorithm (GA). The effectiveness of each algorithm is evaluated. Multi agent-based UAV simulation software was developed and used to test the ideas presented. In addition to coalition formation, the software aims to address additional multi-agent issues such as agent identity, mutability, and communication as applied to UAV systems, in a realistic simulated environment. Social potential fields provide a means of modeling a clustering attractive force at the same time as a collision-avoiding repulsive force, and are used by the simulation to maintain aircraft position relative to other UAVs.

The numerous previous attempts to simulate financial markets tended to be based on strong assumptions about markets or their participants. This thesis describes a more general kind of model - one in which deep reinforcement learning is used to train agents to make a profit while trading with each other on a virtual exchange. Such a model carries less inductive bias than most others - in theory, a neural network is capable of learning arbitrary decision rules. The model itself led to very simple results, but the conclusions from its construction will hopefully be of guidance to anyone implementing such a model in the future. / De många tidigare försöken att simulera finansmarknader har ofta byggt på starka antaganden om marknaderna eller deras deltagare. I den här avhandlingen beskrivs en mer allmän typ av modell - en modell där djup förstärkningsinlärning används för att träna agenter att göra vinst när de handlar med varandra på en virtuell börs. En sådan modell har mindre induktiva fördomar än de flesta andra - i teorin kan ett neuralt nätverk lära sig godtyckliga beslutsregler. Själva modellen ledde till mycket enkla resultat, men slutsatserna från dess konstruktion kommer förhoppningsvis att vara vägledande för alla som tillämpar en sådan modell i framtiden.

The selection of the most appropriate traffic control actions to solve non-recurrent traffic congestion is a complex task which requires significant expert knowledge and experience. In this thesis we develop and investigate the application of an intelligent traffic control decision support system for road traffic management to assist the human operator to identify the most suitable control actions in order to deal with non-recurrent and non-predictable traffic congestion in a real-time situation. Our intelligent system employs a Fuzzy Neural Networks (FNN) Tool that combines the capabilities of fuzzy reasoning in measuring imprecise and dynamic factors and the capabilities of neural networks in terms of learning processes. In this work we present an effective learning approach with regard to the FNN-Tool, which consists of three stages: initializing the membership functions of both input and output variables by determining their centres and widths using self-organizing algorithms; employing an evolutionary Genetic Algorithm (GA) based learning method to identify the fuzzy rules; tune the derived structure and parameters using the back-propagation learning algorithm. We evaluate experimentally the performance and the prediction capability of this three-stage learning approach using well-known benchmark examples. Experimental results demonstrate the ability of the learning approach to identify all relevant fuzzy rules from the training data. A comparative analysis shows that the proposed learning approach has a higher degree of predictive capability than existing models. We also address the scalability issue of our intelligent traffic control decision support system by using a multi-agent based approach. The large network is divided into sub-networks, each of which has its own associated agent. Finally, our intelligent traffic control decision support system is applied to a number of road traffic case studies using the traffic network in Riyadh, in Saudi Arabia. The results obtained are promising and show that our intelligent traffic control decision support system can provide an effective support for real-time traffic control.

Mycorrhizal Networks (MNs) facilitate the exchange of resources including energy, water, nutrients, and information between trees and plants in forest ecosystems. This work explored MNs as an inspiration for new market models in transactive energy networks, which similarly involve exchanges of energy and information between buildings in local communities. Specific insights from the literature on the structure and function of MNs were translated into an energy model with the aim of addressing challenges associated with the proliferation of distributed energy resources (DERs) at the grid edge and the incorporation of DER aggregations into wholesale energy markets. First, a systematic review of bio-inspired computing interventions applied to microgrids and their interactions with modern energy markets established a technical knowledge base within the context of distributed electrical systems. Second, a bio-inspired design process built on this knowledge base to yield a structural and functional blueprint for a computational mycorrhizal energy market simulation. Lastly, that computational model was implemented and simulated on a blockchain-compatible, multi-agent software platform to determine the effect that mycorrhizal strategies have on transactive energy market performance. The structural translation of a mapped ectomycorrhizal network of Douglas-firs in Oregon, USA called the 'wood-wide web' created an effective framework for the organization of a novel mycorrhizal energy market model that enabled participating buildings to redistribute percentages of their energy assets on different competing exchanges throughout a series of week-long simulations. No significant changes in functional performance –- as determined by economic, technical, and ecological metrics – were observed when the mycorrhizal results were compared to those of a baseline transactive energy community without periodic energy asset redistribution. Still, the model itself is determined to be a useful tool for further exploration of innovative, automated strategies for DER integration into modern energy market structures and electrical infrastructure in the age of Web3, especially as new science emerges to better explain trigger and feedback mechanisms for carbon exchange through MNs and how mycorrhizae adapt to changes in the environment. This dissertation concludes with a brief discussion of policy implications and an analysis applying the ecological principles of robustness, biodiversity, and altruism to the collective energy future of the human species. / Doctor of Philosophy / Beneath the forest floor, a network of fungi connects trees and plants and allows them to exchange energy and other resources. This dissertation compares this mycorrhizal network (mycorrhiza = fungus + root) to a group of solar-powered buildings generating energy and exchanging it in a local community marketplace (transactive energy markets). In the analogy, the buildings become the plants, the solar panels become the leaves, and the electrical grid represents the mycorrhizal network. Trees and plants produce their own energy through photosynthesis and then send large portions of it down to the roots, where they can trade it or send it to neighbors via the mycorrhizal network. Similarly, transactive energy markets are designed to allow buildings to sell the energy they produce on-site to neighbors, usually at better rates. This helps address a major infrastructure challenge that is arising with more people adding roof-top solar to their homes. The grid that powers our buildings is old now and it was designed to send power from a central power plant out to its edges where most homes and businesses are located. When too many homes produce solar power at the same time, there is nowhere for it to go, and it can easily overload the grid leading to fires, equipment failures, and power outages. Mycorrhizal networks solve this problem in part through local energy balancing driven by cooperative feedback patterns that have evolved over millennia to sustain forest ecosystems. This work applies scientific findings on the structure and function of mycorrhizal networks (MNs) to energy simulation methods in order to better understand the potential for building bio-inspired energy infrastructure in local communities. Specifically, the mapped structure of a MN of douglas-fir trees in Oregon, USA was adapted into a digital transactive energy market (TEM) model. This adaptation process revealed that a single building can connect to many TEMs simultaneously and that the number of connections can change over time just as symbiotic connections between organisms grow, decay, and adapt to a changing environment. The behavior of MNs in terms of when those connections are added and subtracted informed the functionality of the TEM model, which adds connections when community energy levels are high and subtracts connections when energy levels are low. The resulting 'mycorrhizal' model of the TEM was able to change how much energy each connected household traded on it by changing the number of connections (more connections mean more energy and vice versa). Though the functional performance of the mycorrhizal TEM did not change significantly from that of a typical TEM when they were the context of decentralized computer networks (blockchains) and distributed artificial intelligence. A concluding discussion addresses ways in which elements of this new model could transform energy distribution in communities and improve the resilience of local energy systems in the face of a changing climate.

<p dir="ltr">This dissertation presents decentralized and agent-based solutions for organizing machine learning resources, such as datasets and learning models. It aims to democratize the analysis of these resources through a simple yet flexible query structure, automate common ML tasks such as training, testing, model selection, and hyperparameter tuning, and enable privacy-centric building of ML models over distributed datasets. Based on networked multi-agent systems, the proposed approach represents ML resources as autonomous and self-reliant entities. This representation makes the resources easily movable, scalable, and independent of geographical locations, alleviating the need for centralized control and management units. Additionally, as all machine learning and data mining tasks are conducted near their resources, providers can apply customized rules independently of other parts of the system. </p><p><br></p>

[pt] Nesta dissertação, apresenta-se uma abordagem para regular a interação dos agentes que fazem parte de um sistema multi-agente aberto. Em sistemas abertos, os agentes podem ser não-cooperativos, estão imersos em um ambiente altamente imprevisível e, freqüentemente, os outros agentes que compõem o sistema não são conhecidos a priori. Para algumas classes de aplicações, esta imprevisibilidade não é adequada, podendo levar a falhas de software. Desta forma, é proposta uma abordagem baseada em leis de interação para construir sistemas multi-agentes abertos, onde um controle sobre o comportamento dos agentes é esperado. Propõe-se um modelo conceitual para a especificação da forma como as interações são reguladas em um sistema multi-agente. Este modelo conceitual trata conceitos como cenas, normas e restrições de forma integrada. Além disso, a interação entre os agentes deve ser monitorada e as leis que foram especificadas devem ser aplicadas. Para isto, propõe-se uma linguagem declarativa para a especificação da interação de acordo com os elementos do modelo conceitual e uma infra-estrutura de software que age como mediador das interações garantindo que elas estejam de acordo com as especificações. / [en] In this work, we propose an approach for regulating agents interaction on an open multi-agent system. In open systems, agents are immersed in a highly unpredictable environment, they can be self-interested, and other agents are frequently unknown beforehand. We argue that, in some applications, unexpected behavior may lead to system faults. For this reason, we propose a law enforcement approach to build open multi-agent systems where a certain degree of control over agents behavior is desirable. A conceptual model is proposed to specify how the interactions of an open multi-agent system should happen. This model deals with concepts such as norms, constraints and scenes in a integrated way. We also propose a declarative language that allows the interaction s specification according to the elements that compose the conceptual model, and a software infrastructure that acts as a mediator monitoring and enforcing agents interaction.

Tesis por compendio / [ES] El transporte de personas y bienes supone un problema complejo a la vez que un servicio esencial en la sociedad moderna. Entre los distintos modos de transporte, el transporte rodado supone ventajas y retos únicos, gracias a su flexibilidad y operación tanto urbana como interurbana. La creciente preocupación social respecto al medio ambiente afecta también al transporte rodado, pues los vehículos a motor son una gran fuente de emisiones de gases de efecto invernadero. Sin embargo, la digitalización de la sociedad y la aparición de nuevos modelos de transporte indican el potencial de mejora del transporte rodado, que podría adaptarse mejor a sus usuarios a la vez que operar de forma más sostenible. En esta tesis afrontamos la mejora del transporte rodado mediante técnicas de computación e inteligencia artificial. Esto incluye el modelado de sistemas de transporte mediante sistemas multiagente y su posterior simulación virtual. La operación de las flotas de transporte está determinada por la distribución de tareas, la planificación de las acciones de cada vehículo y su posterior coordinación. Exploramos distintas técnicas y desarrollamos propuestas que mejoran la operación de distintos sistemas de transporte rodado considerando tres puntos de vista: el del operador, el del usuario y, finalmente, el de la sostenibilidad. En otras palabras, apuntamos a obtener sistemas con mayor rendimiento económico y calidad de servicio a la par que un reducido impacto medioambiental. El objetivo de la mejora del transporte rodado se lleva a cabo desde tres frentes. Primero, se propone un marco de trabajo para el modelado efectivo y la simulación de sistemas de transporte. Esta aportación nos sirve como herramienta para la experimentación del resto de la investigación. Después, la investigación se centra en el transporte urbano, caso de uso para el que modelamos la ciudad como un escenario con recursos compartidos. Proponemos el uso de flotas de vehículos descentralizados para una mayor reactividad del sistema. Mediante un modelado de autointerés, se incentiva a los vehículos a proveer de un mejor servicio a los usuarios a la vez que evitan la congestión de los recursos. Finalmente, con la intención de aportar soluciones innovadoras también a las áreas rurales, se adaptan nuestras propuestas previas para el caso de uso del transporte rural interurbano. En este caso, observamos la necesidad de transporte público flexible y adaptado a los usuarios, con especial importancia en su sostenibilidad económica. Nuestras propuestas de sistema siguen estos principios atendiendo al paradigma del transporte adaptable a la demanda. Los resultados de esta tesis aportan soluciones prácticas para la mejora de distintos sistemas de transporte rodado, contribuyendo a un futuro de movilidad flexible más sostenible y adaptada al usuario. Como aportación en el ámbito de la inteligencia artificial, las técnicas desarrolladas tienen el potencial de ser adaptadas a campos más allá del transporte como soluciones generales para la distribución de tareas y la coordinación de elementos distribuidos. / [CA] El transport de persones i béns suposa un problema complex alhora que un servei essencial en la societat moderna. Entre els diferents modes de transport, el transport rodat suposa avantatges i reptes únics, gràcies a la seua flexibilitat i operació tant urbana com interurbana. La creixent preocupació social respecte al medi ambient afecta també al transport rodat, doncs els vehicles de motor són una gran font d'emissions de gasos d'efecte d'hivernacle. No obstant això, la digitalització de la societat i l'aparició de nous models de transport indiquen el potencial de millora del transport rodat, que podria adaptar-se millor als seus usuaris alhora que operar de forma més sostenible. En esta tesi afrontem la millora del transport rodat mitjançant tècniques de computació i intel·ligència artificial. Això inclou el modelatge de sistemes de transport mitjançant sistemes multiagent i la seua posterior simulació virtual. L'operació de les flotes de transport està determinada per la distribució de tasques, la planificació de les accions de cada vehicle i la seua posterior coordinació. Explorem diferents tècniques i desenvolupem propostes que milloren l'operació de diferents sistemes de transport rodat considerant tres punts de vista: el de l'operador, el de l'usuari i, finalment, el de la sostenibilitat. En altres paraules, apuntem a obtindre sistemes amb major rendiment econòmic i qualitat de servei al mateix temps que un reduït impacte mediambiental. L'objectiu de la millora del transport rodat es duu a terme des de tres fronts. Primer, es proposa un marc de treball per al modelatge efectiu i la simulació de sistemes de transport. Esta aportació ens serveix com a eina per a l'experimentació de la resta de la investigació. Després, la investigació se centra en el transport urbà, cas d'ús per al qual modelem la ciutat com un escenari amb recursos compartits. Proposem l'ús de flotes de vehicles descentralitzats per a una major reactivitat del sistema. Mitjançant un modelatge d'autointerés, s'incentiva als vehicles a proveir d'un millor servei als usuaris alhora que eviten la congestió dels recursos. Finalment, amb la intenció d'aportar solucions innovadores també a les àrees rurals, s'adapten les nostres propostes prèvies per al cas d'ús del transport rural interurbà. En este cas, observem la necessitat de transport públic flexible i adaptat als usuaris, amb especial importància en la seua sostenibilitat econòmica. Les nostres propostes de sistema segueixen estos principis atés el paradigma del transport adaptable a la demanda. Els resultats d'esta tesi aporten solucions pràctiques per a la millora de diferents sistemes de transport rodat, contribuint a un futur de mobilitat flexible més sostenible i adaptada a l'usuari. Com a aportació en l'àmbit de la intel·ligència artificial, les tècniques desenvolupades tenen el potencial de ser adaptades a camps més enllà del transport com a solucions generals per a la distribució de tasques i la coordinació d'elements distribuïts. / [EN] The transportation of people and goods is both a complex problem and an essential service in modern society. Among the various modes of transportation, road transport offers unique advantages and challenges, thanks to its flexibility and operation in both urban and interurban areas. The growing social concern for the environment also affects road transportation, as motor vehicles are a major source of greenhouse gas emissions. However, the digitalisation of society and the emergence of new transport models indicate the potential for improvement in transportation, which could be better adapted to its users while operating in a more sustainable way. In this thesis, we address the improvement of road transportation by means of computational techniques and artificial intelligence. This includes the modelling of transportation through multi-agent systems and their subsequent simulation. The operation of transportation fleets is determined by the distribution of tasks, the planning of the actions of each vehicle and their subsequent coordination. We explore different techniques and develop proposals that improve the operation of different transportation systems by considering three points of view: that of the operator, that of the user and, finally, that of sustainability. In other words, we aim to obtain systems with higher economic performance and quality of service while reducing their environmental impact. The objective of improving road transportation is pursued on three fronts. First, a framework for the effective modelling and simulation of transportation systems is proposed. This contribution serves as a tool for the experimentation of the rest of the research. Next, the research focuses on urban transportation, a use case for which we model the city as a shared resource scenario. We propose the use of decentralised vehicle fleets for greater reactivity of the system. Through self-interested modelling, vehicles are incentivised to provide a better service to users while avoiding resource congestion. Finally, with the intention of bringing innovative solutions also to rural areas, our previous proposals are adapted to the use case of rural interurban transportation. In this case, we note the need for flexible and user-friendly public transportation, with special emphasis on its economic sustainability. Our system proposals follow these principles following the demand-responsive transportation paradigm. The results of this thesis provide practical solutions for the enhancement of different road transportation systems, contributing to a future of more sustainable and user-tailored flexible mobility. As a contribution to the field of artificial intelligence the developed techniques have the potential to be adapted to fields beyond transportation, providing general solutions for the task allocation and the coordination of distributed elements. / Martí Gimeno, P. (2024). Towards Sustainable and Efficient Road Transportation: Development of Artificial Intelligence Solutions for Urban and Interurban Mobility [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/203076 / Compendio