No / Many recent studies have convincingly demonstrated that network traffic exhibits a noticeable self-similar nature which has a considerable impact on queuing performance. However, the networks used in current multicomputers have been primarily designed and analyzed under the assumption of the traditional Poisson arrival process, which is inherently unable to capture traffic self-similarity. Consequently, it is crucial to reexamine the performance properties of multicomputer networks in the context of more realistic traffic models before practical implementations show their potential faults. In an effort toward this end, this paper proposes the first analytical model for wormhole-switched k-ary n-cubes in the presence of self-similar traffic. Simulation experiments demonstrate that the proposed model exhibits a good degree of accuracy for various system sizes and under different operating conditions. The analytical model is then used to investigate the implications of traffic self-similarity on network performance. This study reveals that the network suffers considerable performance degradation when subjected to self-similar traffic, stressing the great need for improving network performance to ensure efficient support for this type of traffic.

no / In this paper, a performance analysis of adaptive routing is presented for packet-switched inter-satellite link (ISL)networks, based on shortest path routing and two alternate link routing forwarding policies. The selected routing algorithm and link-cost function are evaluated for a low earth orbit satellite system, using a demographically weighted traffic flow model. Two distinct traffic flow patterns are modelled: hot spot and regional. Performance analysis, in terms of quality of service and quantity of service, is derived using specifically developed simulation software to model the ISL network, taking into account topology adaptive routing only, or topology and traffic adaptive routing.

One of the fundamental challenges in building Peer-to-Peer (P2P) applications is to locate resources across a dynamic set of nodes without centralised servers. Structured overlay networks solve this challenge by proving a key-based routing (KBR) layer that maps keys to nodes. The performance of KBR is strongly influenced by the dynamic and unpredictable conditions of P2P environments. To cope with such conditions a node must maintain its routing state. Routing state maintenance directly influences both lookup latency and bandwidth consumption. The more vigorously that state information is disseminated between nodes, the greater the accuracy and completeness of the routing state and the lower the lookup latency, but the more bandwidth that is consumed. Existing structured P2P overlays provide a set of configuration parameters that can be used to tune the trade-off between lookup latency and bandwidth consumption. However, the scale and complexity of the configuration space makes the overlays difficult to optimise. Further, it is increasingly difficult to design adaptive overlays that can cope with the ever increasing complexity of P2P environments. This thesis is motivated by the vision that adaptive P2P systems of tomorrow, would not only optimise their own parameters, but also generate and adapt their own design. This thesis studies the effects of using an adaptive technique to automatically adapt state dissemination cost and lookup latency in structured overlays under churn. In contrast to previous adaptive approaches, this work investigates the algorithmic adaptation of the fundamental data dissemination protocol rather than tuning the parameter values of a protocol with fixed design. This work illustrates that such a technique can be used to design parameter-free structured overlays that outperform other structured overlays with fixed design such as Chord in terms of lookup latency, bandwidth consumption and lookup correctness. A large amount of experimentation was performed, more than the space allows to report. This thesis presents a set of key findings. The full set of experiments and data is available online at: http://trombone.cs.st-andrews.ac.uk/thesis/analysis.

Les progrès dans les technologies à base de semi-conducteurs et la demande croissante de puissance de calcul poussent vers une intégration dans une même puce de plus en plus de processeurs intégrés. Par conséquent les réseaux sur puce remplacent progressivement les bus de communication, ceux-ci offrant plus de débit et permettant une mise à l'échelle simplifiée. Parallèlement, la réduction de la finesse de gravure entraine une augmentation de la sensibilité des circuits au processus de fabrication et à son environnement d'utilisation. Les défauts de fabrication et le taux de défaillances pendant la durée de vie du circuit augmentent lorsque l'on passe d'une technologie à une autre. Intégrer des techniques de tolérance aux fautes dans un circuit devient indispensable, en particulier pour les circuits évoluant dans un environnement très sensible (aérospatial, automobile, santé, ...). Nous présentons dans ce travail de thèse, des techniques permettant d'améliorer la tolérance aux fautes des micro-réseaux intégrés dans des circuits évoluant dans un environnement difficile. Le NoC doit ainsi être capable de s'affranchir de la présence de nombreuses fautes. Les travaux publiés jusqu'ici proposaient des solutions pour un seul type de faute. En considérant les contraintes de surface et de consommation du domaine de l'embarqué, nous avons proposé un algorithme de routage adaptatif tolérant à la fois les fautes intermittentes, transitoires et permanentes. En combinant et adaptant des techniques existantes de retransmission de flits, de fragmentation et de regroupement de paquet, notre approche permet de s'affranchir de nombreuses fautes statiques et dynamiques. Les très nombreuses simulations réalisées ont permis de montrer entre autre que, l'algorithme proposé permet d'atteindre un taux de livraison de paquets de 97,68% pour un NoC 16x16 en maille 2D en présence de 384 liens défectueux simultanés, et 93,40% lorsque 103 routeurs sont défaillants. Nous avons étendu l'algorithme aux topologies de type tore avec des résultats bien meilleurs.Une autre originalité de cette thèse est que nous avons inclus dans cet algorithme une fonction de gestion de la congestion. Pour cela nous avons défini une nouvelle métrique de mesure de la congestion (Flit Remain) plus pertinente que les métriques utilisées et publiées jusqu'ici. Les expériences ont montré que l'utilisation de cette métrique permet de réduire la latence (au niveau du pic de saturation) de 2,5 % à 16,1 %, selon le type de trafic généré, par rapport à la plus efficace des métriques existante. La combinaison du routage adaptatif tolérant les fautes statiques et dynamiques et la gestion de la congestion offrent une solution qui permet d'avoir un NoC et par extension un circuit beaucoup plus résilient. / The quest for higher-performance and low-power consumption has driven the microelectronics' industry race towards aggressive technology scaling and multicore chip designs. In this many-core era, the Network-on-chip (NoCs) becomes the most promising solution for on-chip communication because of its performance scaling with the number of IPs integrated in the chip.Fault tolerance becomes mandatory as the CMOS technology continues shrinking down. The yield and the reliability are more and more affected by factors such as manufacturing defects, process variations, environment variations, cosmic radiations, and so on. As a result, the designs should be able to provide full functionality (e.g. critical systems), or at least allow degraded mode in a context of high failure rates. To accomplish this, the systems should be able to adapt to manufacturing and runtime failures.In this thesis, some techniques are proposed to improve the fault tolerance ability of NoC based circuits working in harsh environments. As previous works allow the handling of one type of fault at a time, we propose here a solution where different kinds of faults can be tolerated concurrently.Considering constraints such as area and power consumption, a fault tolerant adaptive routing algorithm was proposed, which can cope with transient, intermittent and permanent faults. Combined with some existing techniques, like flit retransmission and packet fragmentation, this approach allows tolerating numerous static and dynamic faults. Simulations results show that the proposed solution allows a high packet delivery success rate: for a 16x16 2D Mesh NoC, 97.68% in the presence of 384 simultaneous link faults, and 93.40% with the presence of 103 simultaneous router faults. This success rate is even higher when this algorithm is extended to NoCs with Tore topology. Another contribution of this thesis is the inclusion of a congestion management function in the proposed routing algorithm. For this purpose, we introduce a novel metric of congestion measurement named Flit Remain. The experimental results show that using this new congestion metric allows a reduction of the average latency of the Network on Chip from 2.5% to 16.1% when compared to the existing metrics.The combination of static and dynamic fault tolerant and adaptive routing and the congestion management offers a solution, which allows designing a NoC highly resilient.

No description available.

Disruption Tolerant Networking (DTN) is a network architecture that enables communication between devices in challenging environments that may never have a contemporaneous end-to-end path. Routing in DTN is challenging since every node decides autonomously, based on local information, which bundles should be forwarded when two devices have a contact opportunity (i.e., when they meet). Frequently, devices exchange locally and transitively collected information (properties) during contact opportunities to support their routing algorithms. In the context of Public Transport Networks (PTN), devices may be vehicles (e.g., tram, bus) or stations that use vehicular mobility as a data carrier. Vehicles in a PTN tend to move back and forth, creating repetitive patterns. We assumed that routing algorithms capable of exploring those characteristics would likely maximize the desired metric, e.g., increasing delivery probability (DP) or minimizing latency. In the event of unexpected mobility changes, routing algorithms that perform well under normal mobility may perform poorly during this period. In this case, property exchange can provide context awareness, supporting a device in choosing the right moment to adapt the routing algorithm or tuning it at runtime. However, mobility adaptation and adaptive parametrization were not investigated in the context of DTN for PTN. This thesis proposes an adaptive routing module that supports multiple routing algorithms chosen independently, per bundle, at runtime. Our literature research revealed that many adaptive algorithms had been proposed. Still, only a few provided routing adaptation, and those approaches have not been addressed in the context of PTNs. From an extensive review of property exchange in DTN, we classified properties that algorithms exchanged at runtime. Finally, we provided an overview of the current DTN frameworks, showing that at least one allows the choice of the routing algorithm independently; however, the high coupling between routing and information exchange precludes the usage of multiple routing algorithms concurrently in practice. Therefore, our novel approach decouples property exchange from the routing algorithm, i.e., exchanging a set of properties consistently during contact opportunities independently of the routing algorithm used. From the different classes identified in the literature research, resource class is the most suitable for PTN as exchanging its properties enables reproducing the most common routing algorithms and providing context-awareness. The history of contacts is stored together with information about devices resources' state during each contact opportunity and spread transitively in the network. Since this information is immutable, it incrementally allows every device to improve its understanding of network behavior over time. Another improvement in our approach is exchanging raw information as an ordered list of the instantaneous information about the device state, allowing every device to derive the needed information to the routing support locally.Commonly, devices exchange summarized information (e.g., frequency of contacts, centrality, delivery predictability): an interesting approach for single routing, as it allows the exchange of the minimal amount of information; however, it is less attractive for a multi routing approach due to the high coupling between routing and property exchange. Creating an adaptative mechanism for a routing algorithm at runtime claims a careful choice considering multiple aspects of the development cycle: modeling, dynamic adaptation, and system extension. Runtime adaptability and extensibility support have been integrated into some programming languages. In some cases, it is also possible to provide those capabilities through development techniques and design patterns. However, at the modeling level, conceptual modeling languages, as the Entity-Relationship Model (ER) or the Unified Modeling Language (UML) cannot express the dynamics that arise from the context changes and behavioral interaction at runtime (e.g., the influence of congestion control techniques in the outcome of the routing algorithm according to a set of goals or intent of the network). Therefore, we used Role-based modeling languages to express the behavioral and relational nature of the proposed system. In a PTN, the router (vehicle or station) plays the role \textit{route} (executes an objective function) depending on the sensed context. The \textit{route} role can be further influenced by roles that encapsulate routing strategies, as congestion control (limiting the number of replication) or fairness. Routing algorithms are grouped in compartments and selected based on rules that determine what routing algorithm should be selected for a given context and how to tune the routing algorithm. A proof of concept was developed using SCROLL and integrated into the ONE simulator, allowing adaptive simulations. On the one hand, this solution is flexible, allowing algorithms to be designed independently. On the other hand, the integration between Java and SCALA reduced the system's flexibility due to the impossibility of using implicit types and the simulation time increased considerably. The potential performance improvement caused by routing adaptation depends on several variables, e.g., the scenario, the vehicular mobility, and the routing algorithms. Based on the evaluation of common routing algorithms proposed in DTN that can be reproduced by exchanging information about resources, we show that algorithms should be compared under average load. On the one hand, if the network load is low, all routing algorithms perform similarly, since all bundles are delivered; On the other hand, the network is congested regardless of the routing algorithm if the load is too high. Our experiments also indicate that routing algorithms able to explore the runtime behavior of the network are likely to achieve higher DP for an average load and highlighted the importance of proper tuning of each routing algorithm for the target scenario. The adaptive simulations showed that adapting the routing algorithm at runtime could increase the desired metric. Our simulations considered 24 hours of the tram PTN of Freiburg, in which we changed the mobility for a specific period of the day. In both experiments, we compare a historically based non-adaptive algorithm (PRoPHET) with an adaptive algorithm that modifies the behavior (PRoPHET and Epidemic) according to the context. In the first scenario, we simulate the effect of a disaster in which vehicular mobility is replaced by first aid vehicles. The adaptive variant resulted in a DP increase of up to 9.85\% at the end of the adaptation period. In the second scenario, we simulated mobility adaptation due to an accident that split two tramlines. Also, in this use case, the adaptation at runtime allowed the DP to increase up to 11.12\%. In both cases, the ability to perceive the context modification and adapt accordingly is essential. We evaluated the overhead of information exchange based on a concrete example for PTN and found that, for the chosen scenarios and multiple communication technologies, the network overhead caused by information exchange at runtime would be less than 1.2\%. The main contributions of this thesis are summarized as follows: we developed tools to support the creation and simulation of DTN in PTN scenarios; provided a classification of properties exchanged in the DTN literature to provide context awareness and support routing algorithms; pointed out a design decision in current frameworks (coupling between routing and property exchange) that precludes the adoption of current routing algorithms at runtime; reinforced, based on extensive simulations, the importance of tuning each routing algorithm to the target use-case; proposed a method to exchange and store properties that could enable nodes to collect historical data according to its resources (computing and storage space); demonstrated, for a specific property group how to break information into basic units and how to derive complex metrics for multiple purposes; last but not least, we demonstrated that, in scheduled networks, such as PTN, adaptive algorithms can increase the desired metric by adapting the routing algorithm at runtime. As future work remains the following question: how to effectively provide context awareness through the exchange of properties at runtime?

Nous nous intéressons dans ce travail au guidage optimal des usagers dans un réseau routier. Plus précisément, nous nous focalisons sur les stratégies adaptatives de guidage avec des garanties en termes de fiabilité des temps de parcours, et en termes de robustesse de ces stratégies. Nous nous basons sur une approche stochastique où des distributions de probabilités sont associées aux temps de parcours sur les liens du réseau. Le guidage est adaptatif et individuel. L'objectif de ce travail de recherche est le développement de stratégies « robustes » de guidage des usagers dans un réseau de transport routier. Une stratégie de guidage d’un nœud origine vers un nœud destination est dite robuste, ici, si elle minimise la détérioration de sa valeur maximale calculée au départ de l’origine, contre d’éventuelles reconfigurations du réseau dues à des coupures de liens (accidents, travaux, etc.) La valeur de la stratégie de guidage est maximisée par rapport à la moyenne et à la fiabilité des temps de parcours associées à la stratégie. Deux principales parties sont distinguées dans ce travail. Nous commençons par l’aspect statique du guidage, où la dynamique du trafic n’est pas prise en compte. Nous proposons une extension d’une approche existante de guidage, pour tenir compte de la robustesse des itinéraires calculés. Dans une deuxième étape, nous combinons notre nouvel algorithme avec un modèle microscopique du trafic pour avoir l’effet de la dynamique du trafic sur le calcul d’itinéraires robustes / In this work, we are interested in the optimal guidance of users on road networks. More precisely, we are focused on the adaptive strategies of guidance with guarantees in terms of the travel time reliability and in terms of the robustness of the strategies. We base here on a stochastic approach, where probability distributions are associated to travel times on the links of the network. The guidance is adaptive and user-based. The objective of this work is the development of "robust" strategies for user guidance in a road network. A guidance strategy is said to be robust, here, if it minimizes the deterioration of its maximum value calculated at the origin, against eventual reconfigurations of the network due to link failures (accidents, works, etc.) The value of a guidance strategy is maximized with respect to the mean travel time and its reliability. Two main parts are distinguished in this work. We start with the static aspect of the guidance, where the traffic dynamics are not taken into account. We propose an extension of an existing guidance approach, to take into account the robustness of the calculated itineraries. In a second step, we combine our new guidance algorithm with a microscopic traffic model in order to have the effect of the traffic dynamics on the robust route calculation

La red en el chip (NoC) se han convertido en el elemento clave para la comunicación eficiente entre los núcleos dentro de los chip multiprocesador (CMP). Tanto el uso de aplicaciones paralelas en los CMPs como el incremento de la cantidad de memoria necesitada por las aplicaciones, ha impulsado que la red de comunicación gane una mayor importancia. La NoC es la encargada de transportar toda la información requerida por los núcleos. Además, el incremento en el número de núcleos en los CMPs impulsa las NoC a ser diseñadas de forma escalable, pero al mismo tiempo sin que esto afecte a las prestaciones de la red (latencia y productividad). Por tanto, el diseño de la red en el chip se convierte en crítico. Esta tesis presenta diferentes propuestas que atacan el problema de la mejora de las prestaciones de la red en tres escenarios distintos. Los tres escenarios en los que se centran nuestras propuestas son: 1) NoCs que implementan un algoritmo de encaminamiento adaptativo, 2) escenarios con necesidad de tiempos de acceso a memoria bajos y 3) sistemas con previsión de seguridad a nivel de aplicación. Las primeras propuestas se centran en el aumento de la productividad en la red utilizando algoritmos de encaminamiento adaptativos mediante un mejor uso de los recursos de la red, primera propuesta SUR, y evitando que se ramifique la congestión cuando existe tráfico intenso hacia un único destinatario, segunda propuesta EPC. La tercera y principal contribución de esta tesis se centra la problemática de reducir el tiempo de acceso a memoria. PROSA, mediante un diseño híbrido de conmutación de paquete y conmuntación de circuito, permite reducir la latencia de la red aprovechando la latencia de acceso a memoria para establecer circuitos. De esta forma cuando la información llega a la NoC, esta es servida sin retardos. Por último, la propuesta Token Based TDM se centra en el escenario con redes de interconexión seguras. En este tipo de NoC las aplicaciones esta divididas en dominios y la red debe garantizar que no existen interferencias entre los diferentes dominios para evitar de este modo la intrusión de posibles aplicaciones maliciosas. Token-based TDM permite el aislamiento de los dominios sin tener impacto en el diseño de los conmutados de la NoC. Los resultados obtenidos demuestran como estas propuestas han servido para mejorar las prestaciones de la red en los diferentes escenarios. La implementación y la simulación de las propuestas muestra como mediante el balanceado de la utilización de los recursos de la red, los CMPs con algoritmos de encaminamiento adaptativos son capaces de aumentar el tráfico soportado por la red. Además, el uso de un filtro para limitar el encaminamiento adaptativo en situaciones de congestión previene a los mensajes de la ramificación de la congestión a lo largo de la red. Por otra parte, los resultados demuestran que el uso combinado de la conmutación de paquete y conmutación de circuito reduce muy significativa de la latencia de red acceso a memoria, contribuyendo a una reducción significativa del tiempo de ejecución de la aplicación. Por último, Token-Based TDM incrementa las prestaciones de las redes TDM debido a su alta flexibilidad dado que no requiere ninguna modificación en la red para soportar una cantidad diferente de dominios mientras mejora la latencia de la red y mantiene un aislamiento perfecto entre los tráficos de las aplicaciones. / The Network on Chip (NoC) has become the key element for an efficient communication between cores within the multiprocessor chip (CMP). The use of parallel applications in CMPs and the increase in the amount of memory needed by applications have pushed the network communication to gain importance. The NoC is in charge of transporting all the data needed by the processors cores. Moreover, the increase in the number of cores pushes the NoCs to be designed in a scalable way, but at the same time, without affecting network performance (latency and productivity). Thus, network-on-chip design becomes critical. This thesis presents different proposals that attack the problem of improving the network performance in three different scenarios. The three scenarios in which our proposals are focused are: 1) NoCs with an adaptive routing algorithm, 2) scenarios with low memory access time needs, and 3) high-assurance NoCs. The first proposals focus on increasing network throughput with adaptive routing algorithms via the improvement of the network resources utilization, the first proposal SUR, and avoiding congestion spreading when an intense traffic to a single destination occurs, second proposal ECP. The third one and main contribution of this thesis focuses on the problem of reducing memory access latency. PROSA, through a hybrid circuit-packet switching architecture design, reduces the network latency by getting benefit of the memory access latency slack and to establishing circuits during that delay. In this way the information when arrives to the NoC is served without any delay. Finally, the proposal Token-Based TDM focuses on the scenario with high assurance networks on chips. In this type of NoCs the applications are divided into domains and the network must guarantee that there are no interferences between the different domains avoiding this way intrusion of possible malicious applications. Token-based TDM allows domain isolation with no design impact on NoC routers. The results show how these proposals improve the performance of the network in each different scenario. The implementation and simulations of the proposals show the efficient use of network resources in CMPs with adaptive routing algorithms which leads to an increasement of the injected traffic supported by the network. In addition, using a filter to limit the adaptivity of the routing algorithm under congested situations prevents messages from spreading the congestion along the network. On the other hand, the results show that the combined use of circuit and packet switching reduces the memory access latency significantly, contributing to a significant reduction in application execution time. Finally, Token-Based TDM increases network performance of TDM networks due to its high flexibility and efficient arbitration. Moreover, Token-Based TDM does not require any modification in the network to support a different number of domains while improving latency and keeping a strong traffic isolation from different domains. / La xarxa en el xip (NoC) s'ha convertit en un element clau per a una comunicació eficient entre els diferents nuclis dins d'un xip multiprocessador (CMP). Tant la utilització d'aplicacions paral·leles en el CMP com l'increment de la quantitat de memòria necessitada per les aplicacions, hi ha produït que la xarxa de comunicació tinga una major importància. La NoC és l'encarregada de transportar tota la informació necessària pels nuclis. A més, l'increment del nombre de nuclis dins del CMP fa que la NoC haja de ser dissenyada d'una forma escalable, sense que afecte les prestacions de la xarxa (latència i productivitat). Per tant, el disseny de la xarxa en el xip es converteix crític. Aquesta tesi presenta diferents propostes que ataquen el problema de la millora de les prestacions de la xarxa en tres escenaris distints. Els tres escenaris en els quals se centren les nostres propostes són: 1) NoCs que implementen un algoritme d'encaminament adaptatiu, 2) escenaris amb necessitat de temps baix d'accés a memòria i 3) sistemes amb previsió de seguretat en l'àmbit d'aplicació. Les primeres propostes se centren en l'augment de la productivitat en la xarxa utilitzant algoritmes d'encaminament adaptatiu mitjançant una millor utilització dels recursos de la xarxa, primera proposta SUR, i evitant que es ramifique la congestió quan existeix un trànsit intens cap a un únic destinatari, segona proposta EPC. La tercera i principal contribució d'aquesta tesi es basa en la problemàtica de reduir el temps d'accés a memòria. PROSA, mitjançant un disseny híbrid de commutació de paquet i commutació de circuit, redueix la latència de la xarxa aprofitant la latència d'accés a memòria i establint els circuits durant aquesta latència. D'aquesta forma la informació quan arriba a la NoC pot ser enviada sense cap retràs. Per últim, la proposta Token-based TDM se centra en l'escenari amb xarxes d'interconnexió d'alta seguretat. En aquest tipus de NoC les aplicacions estan dividides en dominis i la xarxa deu garantir que no existeixen interferències entre els diferents dominis per a evitar d'aquesta forma la intrusió de possibles aplicacions malicioses. Token-based TDM permet l'aïllament dels dominis sense tindre impacte en el disseny dels encaminadors de la NoC. Els resultats demostren com aquestes propostes han servit per a millorar les prestacions de la xarxa en els diferents escenaris. La seua implementació i simulació demostra com mitjançant el balancejat de la utilització dels recursos de la xarxa, els CMP amb algoritmes d'encaminament adaptatiu són capaços d'augmentar el trànsit suportat per la xarxa. A més, l'ús d'un filtre per a limitar l'adaptabilitat de l'encaminament adaptatiu en situacions de congestió permet prevenir els missatges de la congestió al llarg de la xarxa. Per altra banda, els resultats demostren que l'ús combinat de la commutació de paquet i commutació de circuit redueix molt significativament de la latència d'accés a memòria, contribuint en una reducció significativa del temps d'execució de l'aplicació. Per últim, Token-based TDM incrementa les prestacions de les xarxes TDM debut a la seua alta flexibilitat donat que no requereix cap modificació en la xarxa per a suportar una quantitat diferent de dominis mentre millora la latència de la xarxa i mantén un aïllament perfecte entre els trànsits de les aplicacions. / Gorgues Alonso, M. (2018). Improving Network-on-Chip Performance in Multi-Core Systems [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/107336 / TESIS

Grâce aux avancées conjointes des systèmes microélectroniques, des technologies sans fil et de la microélectronique embarquée, les réseaux de capteurs sans fil (RCsF) ont récemment pu voir le jour. Très sophistiqués et en interaction directe avec leur environnement, ces systèmes informatiques et électroniques communiquent principalement à travers des réseaux radio qui en font des objets communicants autonomes. Ils offrent l'opportunité de prendre en compte les évolutions temporelles et spatiales du monde physique environnant. Les RCsF se retrouvent donc au cœur de nombreuses applications couvrant des domaines aussi variés que la santé, la domotique, l'intelligence ambiante, les transports, la sécurité, l'agronomie et l'environnement. Ils connaissent un véritable essor et ce dans divers domaines des STIC : hardware, système d'exploitation, conception d'antenne, système d'information, protocoles réseaux, théorie des graphes, algorithmique distribuée, sécurité, etc. L’intérêt des communautés issues de la recherche et de l’industrie pour ces RCsF s’est accru par la potentielle fiabilité, précision, flexibilité, faible coût ainsi que la facilité de déploiement de ces systèmes. La spontanéité, l’adaptabilité du réseau et la dynamicité de sa topologie dans le déploiement des RCsF soulèvent néanmoins de nombreuses questions encore ouvertes. Dans le cadre de cette thèse, nous nous sommes intéressés aux aspects liés à la problématique du routage dans un RCsF, l’objectif étant de proposer des approches algorithmiques permettant de faire du routage adaptatif multi critères dans un RCsF. Nous nous sommes concentrés sur deux critères principaux : la consommation d’énergie dans les capteurs et le délai d’acheminement des informations collectées par les capteurs. Nous avons proposé ainsi un nouveau protocole de routage, appelé EDEAR (Energy and Delay Efficient Adaptive Routing), qui se base sur un mécanisme d’apprentissage continu et distribué permettant de prendre en compte la dynamicité du réseau. Celui-ci utilise deux types d’agents explorateurs chargés de la collecte de l’information pour la mise à jour des tables de routage. Afin de réduire la consommation d’énergie et la surcharge du réseau, nous proposons également un processus d’exploration des routes basé sur une diffusion optimisée des messages de contrôle. Le protocole EDEAR calcule les routes qui minimisent simultanément l’énergie consommée et le délai d’acheminement des informations de bout en bout permettant ainsi de maximiser la durée de vie du réseau. L’apprentissage se faisant de manière continue, le routage se fait donc de façon évolutive et permet ainsi une réactivité aux différents évènements qui peuvent intervenir sur le réseau. Le protocole proposé est validé et comparé aux approches traditionnelles, son efficacité au niveau du routage adaptatif est mise particulièrement en évidence aussi bien dans le cas de capteurs fixes que de capteurs mobiles. En effet, celui-ci permet une meilleure prise en compte de l'état du réseau contrairement aux approches classiques / Through the joint advanced microelectronic systems, wireless technologies and embedded microelectronics, wireless sensor networks have recently been possible. Given the convergence of communications and the emergence of ubiquitous networks, sensor networks can be used in several applications and have a great impact on our everyday life. There is currently a real interest of research in wireless sensor networks; however, most of the existing routing protocols propose an optimization of energy consumption without taking into account other metrics of quality of service. In this thesis, we propose an adaptive routing protocol called "EDEAR" which takes into account both necessary criteria to the context of communications in sensor networks, which are energy and delay of data delivery. We are looking the routes for optimizing a nodes’ lifetime in the network, these paths are based on joint optimization of energy consumption and delay through a multi criteria cost function. The proposed algorithm is based on the use of the dynamic state-dependent policies which is implemented with a bio-inspired approach based on iterative trial/error paradigm. Our proposal is considered as a hybrid protocol: it combines on demand searching routes concept and proactive exploration concept. It uses also a multipoint relay mechanism for energy consumption in order to reduce the overhead generated by the exploration packets. Numerical results obtained with NS simulator for different static and mobility scenario show the efficiency of the adaptive approaches compared to traditional approaches and proves that such adaptive algorithms are very useful in tracking a phenomenon that evolves over time

[pt] Esta dissertação investiga a aplicação dos métodos de Reinforcement Learning na descoberta de rotas ótimas em uma rede de comunicação. Uma rede de comunicação real possui um comportamento dinâmico, mudando seu estado com o tempo. Os algoritmos de roteamento devem, portanto, oferecer rapidez na resposta às mudanças do estado da rede. O objetivo do trabalho é avaliar a aplicação de técnicas de Reinforcement Learning (RL) como base de algoritmos adaptativos de roteamento de pacotes. O problema de roteamento de pacotes sob a visão de RL consiste na definição de cada nó na rede como um agente RL, sendo que este agente deve definir ações de forma a minimizar uma função objetivo que pode ser o tempo de roteamento dos pacotes. Um dos objetivos do RL é precisamente aprender a tomar as ações que minimizem uma função. O trabalho consistiu de 4 etapas principais: um estudo sobre a área de Reinforcement Learning (RL); um estudo sobre a área de redes de comunicação e roteamento de pacotes; a modelagem do problema de roteamento como um sistema RL e implementação de diferentes métodos de RL para obter algoritmos de roteamento; e o estudo de casos. O estudo na área de Reinforcement Learning abrangeu desde as definições mais fundamentais: suas características, os elementos de um sistema RL e modelagem do ambiente como um Processo de Decisão de Markov, até os métodos básicos de solução: Programação Dinâmica, método de Monte Carlo, e o método de Diferenças Temporais. Neste último método, foram considerados dois algoritmos específicos: TD e Q-Learning. Em seguida, foi avaliado o parâmetro Eligibility Traces como uma alternativa para apressar o processo de aprendizado, obtendo o TD(lambda) e o Q(lambda) respectivamente. O estudo sobre Redes de Comunicação e Roteamento de pacotes envolveu os conceitos básicos de redes de comunicações, comutação por pacotes, a questão do roteamento de pacotes e os algoritmos existentes adaptativos e não adaptativos, que são utilizados na atualidade. Nas redes de comunicação, definidas como um conjunto de nós ligados através de enlaces de comunicação, para se enviar uma mensagem de um nó a outro, geralmente, a mensagem é quebrada em pedaços, chamados pacotes, e enviados através de outros nós, até chegar ao destino. Deste modo surge o problema de escolher os nós que levem o pacote o mais rápido possível até o nó destino. Os algoritmos analisados foram: Shortest Path Routing que procura os caminhos com menor número de nós intermediários, não sendo sensível às mudanças na carga nem na topologia da rede; Weighted Shortest Path Routing, que oferece um melhor desempenho a partir de uma visão global do estado da rede, que nem sempre é fácil de obter em redes reais e o algoritmo de Bellman-Ford, baseado em decisões de roteamento locais e atualizações periódicas, com algumas limitações para obter políticas em altas cargas. Este último é um dos algoritmos mais utilizados na atualidade, sendo base de muitos protocolos de roteamento existentes. A modelagem do problema de roteamento como um sistema RL foi inspirada por uma característica na definição de um sistema RL: um agente que interage com o ambiente e aprende a atingir um objetivo. Assim, a modelagem dos algoritmos tem como objetivo aprender a descobrir as rotas que minimizem o tempo de roteamento de pacotes desde uma origem até um dado destino. A avaliação de uma rota escolhida não pode ser obtida antes que o pacote alcance o seu destino final. Este fato faz com que os processos de aprendizado supervisionado tenham dificuldade de se aplicar a esse problema. Por outro lado, o Reinforcement Learning não necessita de um par entrada-resposta para fazer o aprendizado, permitindo-lhe abordar o problema com relativa facilidade. Na modelagem efetuada, cada nó na rede se comporta como um agente de RL que age na própria rede, a qual é o ambiente. A informação das rotas é armazenada nas funções de valor existentes em todos os nós da rede para / [en] This dissertation investigates the application of Reinforcement Learning methods to the discovery of optimal routes in communication networks. Any current communication network displays dynamic behavior, changing its states over time. Therefore, the routing algorithms must react swiftly to changes in the network status. The objective of this work is to evaluate the application of some Reinforcement Learning techniques to define adaptive packet routing algorithms. The packet routing problem under the RL vision consists in the definition of each node on network as an RL agent. Thus, each agent must take actions in order to minimize an objective function such as end to end packet routing delay. One main objective of the RL is precisely learning to take the actions that minimize a given function. This thesis is consists of 4 main parts: first, a study of Reinforcement Learning (RL); a study of the communication networks and packet routing; the routing problem model as a RL system and the implementation of several RL methods in order to obtain some routing algorithms; e finally, the case study. The study of Reinforcement Learning extends from the more basic definitions, Reinforcement Learning features, elements of a RL system and environment modeling as a Markovian Decision Process, to the basic methods of solution: Dynamic Programming, Monte Carlo methods and Temporal Differences methods. In this last case, two specific algorithms have been considered: TD and Q-Learning, and, finally, the Eligibility Traces are evaluated as a useful tool that permits us to accelerate the learning process leading to the TD(lambda) and the Q(lambda) routing algorithms. The study on communication networks and packet routing involves the foundations of communication networks, packet switching, the packet routing problem, and adaptive and non- adaptive routing algorithms used at the present time. Communication networks are defined as a set of nodes connected through communication links. In order to send a message from a source node to a destination node usually the message is broken into segments called packets, and these are sent through other nodes until arriving at the destination. In this way the problem appears to choose the path which takes the shortest possible time for the packet to reach the destination node. The following algorithms have been analyzed: Shortest Path Routing that looks for paths with minimal hop number, not being sensible to the changes of load level and network topology; Weighted Shortest Path Routing that offers better performance from a global vision of the state of the network, which is not always easy to get in real networks; on the other hand, the Bellman- Ford routing algorithm was studied, this is based on local routing decisions and periodic updates, with some limitations to obtain policies in high load conditions. Bellman-Ford is one of the algorithms most used at the present time, being the basis for many existing routing protocols. The modeling of the routing problem as a RL system was inspired by one of the main features of the definition of an RL system: an agent who interacts with the environment and learns to reach an objective; therefore, the modeling of the routing algorithms has as its objective to learn to discover the paths that minimize packet routing time from an origin to an destination. The evaluation of a chosen route cannot be completed before the package reaches its final destination. This fact implies that supervised learning cannot be applied to the routing problem. On the other hand, Reinforcement Learning does not need a input-output pair for the learning process, allowing it to approach the problem with relative ease. In the modeling, each network node is viewed as a RL agent that acts in the same network; the network is the environment. The routing information is stored in the existing value functions in all nodes in the network, for each node and all another destination node / [es] Esta disertación investiga la aplicación de los métodos de Reinforcement Learning en la determinación de rutas óptimas en una red de comunicación. Una red de comunicación real posee un comportamiento dinámico, donde su estado varia en el tiempo. Los algoritmos de ruta óptima deben, por lo tanto, ofrecer rapidez en la respuesta a las variaciones del estado de la red. El objetivo de este trabajo es evaluar la aplicación de técnicas de Reinforcement Learning (RL) como base de algoritmos adaptativos de problemas de ruteamiento en redes. Este problema consiste en la definición de cada nodo de la red como un agente RL. Este agente debe definir acciones de modo a minimizar una función objetivo que puede ser el tiempo de ruteamiento. El trabajo consta de 4 etapas principais: un estudio sobre el área de Reinforcement Learning (RL); un estudio sobre redes de comunicación y problema de ruteamiento; el modelo de ruta óptima como un sistema RL y la implementación de diferentes métodos de RL para obtener algoritmos de ruta óptima; y un estudio de casos. El estudio en el área de Reinforcement Learning va desde las definiciones fundamentales: características, elementos de un sistema RL y modelaje del ambiente como un Proceso de Decisión de Markov, hasta los métodos básicos de solución: Programación Dinámica, método de Monte Carlo, y método de Diferencias Temporales. En este último método, fueron considerados dos algoritmos específicos: TD e Q-Learning. A seguir, fue evaluado el parámetro Eligibility Traces como una alternativa para agilizar el proceso de aprendizaje, obteniendo el TD(lambda) y el Q(lambda) respectivamente. El estudio sobre Redes de Comunicación y Problema de Transporte incluye los conceptos básicos de redes de comunicaciones, la cuestión de la ruta óptima y los algoritmos adaptativos y no adaptativos existentes, que se utilizan actualmente. Los algoritmos analizados fueron: Shortest Path Routing, que busca los caminos con menor número de nodos intermedios, no siendo sensible a variaciones en la carga ni en la topología de la red; Weighted Shortest Path Routing, que ofrece un mejor desempeño a partir de una visión global del estado de la red, que no siempre es fácil de obtener en redes reales; y el algoritmo de Bellman-Ford, que tiene como base decisiones de rutas locales y actualizaciones periódicas, con algunas limitaciones para obtener políticas en altas cargas. Este último es uno de los algoritmos más utilizados en la actualidad, siendo base de muchos protocolos de trazado de ruta existentes. La solución para modelar el problema de ruteamiento como un sistema RL fue inspirada por una característica en la definición de un sistema RL: un agente que interactúa con el ambiente y aprende a alcanzar un objetivo. Así, el modelo tiene como objetivo aprender a determinar las rutas que minimizen el timpo desde el origen hasta un destino dado. La evaluación de uma ruta seleccionada no puede ser obtenida antes que el paquete alcance su destino final. Esto hace que los procesos de aprendizaje supervisionado tengan dificultades para ser aplicados a este problema. Por otro lado, Reinforcement Learning no necesita de un par entrada-salida para el aprendizaje, permitiendo así, abordar el problema con relativa facilidad. En el modelo establecido, cada nodo en la red se comporta como un agente de RL que actúa en la propria red. La información de las rutas se almacena en las funciones de valor existentes en todos los nodos de la red para cada nodo destino diferente. Esta información contiene un valor estimado del tiempo requerido para un paquete para llegar hasta el nodo destino. La actualización de esos valores se realiza durante la transición del paquete hasta el vecino seleccionado. En este trabajo se implementaron varios algoritmos de ruta óptima. Cada uno de los algoritmos aplica características de las técnicas en Reinforcement Learning: o Q(lambda)-Routing, y el TD-Routing. En el estudio d