O CAN (Controller Area Network) é um padrão no barramento de comunicação, amplamente difundido em aplicações industriais, particularmente em sistemas automotivos. Atualmente, um dos principais problemas no ramo automotivo é que esse barramento está com muitas mensagens no barramento, resultado da incorporação incremental de sistemas eletrônicos em automóveis, visto que há uma exigência maior de conectividade devido às exigências da sociedade e mercado. Como alternativa, vem sendo desenvolvida uma nova rede de comunicação, conhecida como CAN with Flexible Data-Rate (CAN-FD), que é um barramento com velocidade de transmissão de informação mais alta e maior capacidade de transporte de dados. Este projeto tem por objetivo principal explorar as funcionalidades da rede CAN-FD, através de simulações do trânsito de mensagens numa rede CAN-FD usando os dados de uma rede real CAN, e verificando a previsibilidade de ambas no âmbito de um protocolo que possa atender à demanda de sistemas complexos. A comparação é executada a partir de um conjunto de mensagens adicionadas na rede, para verificar os limites de transmissão de cada uma das redes, e os respectivos tempos de atraso das mensagens. Como um segundo estudo de caso, uma rede de controle em malha fechada foi desenvolvida, conectada a um barramento CAN e um barramento CAN-FD. Essa técnica de controle permitiu eliminar os ruídos que interferem no controle, e checar o limite em que o protocolo de comunicação consegue manter em uma malha de controle funcionando. Os resultados mostraram que é possível transmitir uma imensa quantidade de dados com o menor uso do busload (quantidade de mensagens transmitidas) no veículo através do uso do barramento CAN-FD, porém ainda não foi lançado no mercado um controlador do CAN-FD para realizar essa tarefa. Por outro lado, os dois protocolos, CAN-FD e CAN, tem suas previsibilidades comprometidas pois não conseguem enviar a mensagem quando o barramento está superior a 98,86% de carga. / The CAN (Controller Area Network) is a standard in the communication bus, widespread in industrial applications, particularly in automotive systems. Currently, one of the main problems in the automotive industry is that this bus is with many messages on the bus, the result of incremental incorporation of electronic systems in automobiles, since there is a greater demand for connectivity due to the demands of society and the market. Alternatively, it has been developed a new communications network, known as CAN with Flexible Data-Rate (CAN-FD), which is a bus with transmission speeds higher and higher capacity data transport information. This project\'s main objective is to explore the features of the network CAN-FD, through simulations of the traffic of messages on a CAN network FD using data from a real CAN network, and verifying the predictability both in the context of a protocol that can meet the demand complex systems. The comparison is performed from a set of messages added to the network to verify the boundaries of each of the transmission networks and the respective delay times of the messages. As a second case study, a network of closed-loop control was developed, connected to a CAN bus and CAN bus FD. This control technique has eliminated the noises that interfere with the control and check the extent that the communication protocol can keep a control loop running. The results showed that it is possible to transmit a huge amount of data with the lowest usage busload (amount of transmitted messages) to the vehicle through the use of CAN bus FD, but not yet released to market a CAN controller FD to accomplish this task . Moreover, both protocols, CAN-FD and CAN has its predictability compromised because they are unable to send the message when the bus is more than 98.86% load.

O crescente aumento da demanda de tráfego nas redes celulares vem aumentando a necessidade de uma melhor utilização dos recursos do sistema, já que sua expansão é custosa. Nas estações rádio base (ERB), a disponibilidade de largura de faixa de freqüências é limitada e desta maneira, em uma rede de comunicação móvel celular, o controle de admissão de chamadas exerce grande influência no desempenho do sistema, pois determina a utilização de banda das ERBs e se uma determinada quantidade de recursos (banda) será alocado ou não para uma determinada chamada. O desempenho da rede pode ser atrelado a determinados parâmetros, como a probabilidade de bloqueio de novas chamadas, probabilidade de bloqueio de chamadas handoff e a utilização de banda da rede. Este trabalho propõe um controle de admissão de chamadas que, no atendimento de uma chamada, faz o empréstimo de banda de chamadas em andamento na célula no caso de banda insuficiente. O sistema adota um mecanismo heurístico que determina a banda disponível para novas chamadas conforme os valores de certos parâmetros do sistema. O empréstimo de banda é realizado em chamadas em andamento nas células até níveis mínimos estabelecidos para cada tipo de chamada, que se diferenciam pelas necessidades de banda de cada uma. O algoritmo foi aplicado às bandas e características de uma rede de terceira geração (3G), que possui chamadas de voz, videoconferência, interação multimídia, e-mail, downloads e transferência de arquivos e a uma rede GSM/GPRS (global system for mobile communications/ general packet radio service), que possui chamadas de voz e de dados. Os resultados mostram melhorias na probabilidade de bloqueio de novas chamadas, probabilidade de bloqueio de handoff e na utilização de banda do sistema. / The recent growth in traffic loads in cellular networks has seen the need for a better use of system resources as its expansion is expensive. In the base transceiver station (BTS), the bandwidth availability is limited. Thus, in cellular networks the call admission control greatly influences the system performance because it determines the bandwidth use of the BTSs and if an amount of resources will or will not be allocated to a call. The network performance can be evaluated by parameters such as blocking probability of new calls, dropping probability of handoff calls and bandwidth use. This work proposes a call admission control that carries out the bandwidth borrowing when a call arrives and there is not enough bandwidth. The system makes use of a heuristic mechanism that determines the available bandwidth for the new calls according to some parameter values of the system. The bandwidth borrowing is applied to the cell ongoing calls until the minimum levels for each type are met. The algorithm was applied to the bandwidths and characteristics of a third generation cellular network, which supports voice calls, videoconference, multimedia interaction, e-mails, downloads and file transfers. It was also applied to a GSM/GPRS (global system for mobile communications/ general packet radio service), which supports voice and data calls. The results show improvements in the blocking probability of new calls, dropping probability of handoff calls and in the bandwidth use of the system.

Opportunistic selection is a practically appealing technique that is often used in multi-node wireless systems such as scheduling and rate adaptation in cellular systems and opportunistic wireless local area networks, wireless sensor networks, cooperative communications, and vehicular networks. In it, each node maintains a local preference number called metric that is function of its channel gains, and the best node with the highest metric is selected. Identifying the best node is challenging as the information about a node's metric is available only locally at each node. In our work, we focus on the popular, simple, and low feedback timer scheme for selection. In it, each node sets a timer as a function of its metric and transmits a packet when the timer expires. The metric-to-timer mapping maps larger metric values to smaller timer values, which ensures that the best node's timer expires first. However, it can fail to select the best node if another node transmits a packet within D s of the transmission by the best node. In this thesis, we make three contributions to the design and understanding of the timer-based selection scheme. Firstly, we introduce feedback overhead-aware contention resolution in the timer-based selection scheme. The outcome is a novel selection scheme that is faster than the splitting scheme and more reliable than the timer-based selection scheme. We analyze and minimize the average time required by the scheme to select the best node. Secondly, we characterize the optimal metric-to-timer mapping when the number of nodes in the system is not known, as is the case in several practical deployments. When the prior distribution of the nodes is known, we propose an optimal mapping that maximizes the success probability averaged over the distribution on the number of nodes. When even the prior distribution is not known, we propose a robust mapping that maximizes the worst case average success probability over all possible probability distributions on the number of nodes. In both cases, we show that the timers can expire only at 0, D, 2D, ... in the optimal timer mapping. For the known prior case, we develop recursive techniques to effectively compute the optimal timer mapping for binomial and Poisson priors. Lastly, we consider a discrete rate adaptive system and design an optimal timer scheme to maximize the end-to-end performance measure of system throughput. We derive several novel, insightful results about the optimal mapping that culminate in an iterative algorithm to compute it. We show that the design of the selection scheme is intimately related to the rate adaptation rule and the selection policy used. In all cases, extensive benchmarking with several ad hoc schemes proposed in the literature shows the significant gains that the proposed designs can deliver.

Les systèmes de commande en réseau (SCR) sont de plus en plus répandus dans le milieu industriel. Ils procurent en effet de nombreux avantages en termes de coût, de flexibilité, de maintenance, etc. Cependant,l’introduction d’un réseau, qui par nature est composé de ressources partagées, impacte considérablement les performances temporelles des systèmes de commande. Un signal de commande par exemple n’arrive à destination qu’après un certain délai. Pour s’assurer que ce délai soit inférieur à un certain seuil de sécurité ou du respect d’autres contraintes temps réels de ces systèmes, une évaluation au préalable, avant la mise en service d’un SCR, s’avère donc nécessaire. Dans nos travaux de recherche, nous nous intéressons à la réactivité des SCR client/serveur et évaluons leur temps de réponse.Notre contribution dans ces travaux est d’adopter une approche analytique à base de l’algèbre (Max,+) et remédier aux problèmes des méthodes existantes comme l’explosion combinatoire de la vérification formelle ou de la non exhaustivité des approches par simulation. Après modélisation des SCR client/serveur à l’aide de Graphe d’Evénements Temporisés puis représentation de leurs dynamiques à l’aides d’équations (Max,+) linéaires, nous obtenons des formules de calcul direct du temps de réponse. Plus précisément, nous adoptons une analyse déterministe pour calculer les bornes, minimale et maximale, du temps de réponse puis une analyse stochastique pour calculer la fonction de sa distribution. De plus, nous prenons en compte dans nos travaux tous les délais élémentaires qui composent le temps de réponse, y compris les délais de bout-en-bout, dus à la traversée du seul réseau de communication. Ce dernier étant naturellement composé de ressources partagées, rendant l’utilisation des modèles (Max,+) classiques impossibles, nous introduisons une nouvelle approche de modélisation à base du formalisme (Max,+) mais prenant en compte le concept de conflit ou ressource partagée.L’exemple d’un réseau de type Ethernet est considéré pour évaluer ces délais de bout-en-bout. Par ailleurs, cette nouvelle méthode (Max,+) est assez générique et reste applicable à de nombreux systèmes impliquant des ressources partagées, au delà des seuls réseaux de communication. Enfin, pour vérifier la validité des résultats obtenus dans nos travaux, notamment la formule de la borne maximale du temps de réponse, une compagne de mesures expérimentales sont menées sur une plateforme dédiée. Différentes configurations et conditions de trafic dans un réseau Ethernet sont considérées. / Networked automation systems (NAS) are more and more used in industry, given the several advantages they provide like flexibility, low cost, ease of maintenance, etc. However, the use of a communication network in SCR means in essence sharing some resources and therefore strikingly impacts their time performances. For instance, a control signal does get to its destination (actuator) only after a non zero delay. So, to guarantee that such a delay is shorter than a given threshold or other time constraints well respected, an a priori evaluation is necessary before operating the SCR. In our research activities, we are interested in client/server SCR reactivity and the evaluation of their response time.Our contribution in this investigation is the introduction of a (Max,+) Algebra-based analytic approach to solve some problems, faced in the existing methods like state explosion of model checking or the non exhaustivity of simulation. So, after getting Timed Event Graphs based models of the SCR and their linear state (Max,+) representation, we obtain formulae that enables to calculate straightforwardly the SCR response times. More precisely, we obtain formulae of the bounds of response time by adopting a deterministic analysis and other formulae to calculate the probability density of response time by considering a stochastic analysis. Moreover, in our investigation we take into account every single elementary delay involved in the response time, including the end-to-end delays, due exclusively to crossing the communication network. This latter being however constituted of shared resources, making by the way the use of TEG and (Max,+) Algebra impossible, we introduce a novel approach to model the communication network. This approach brings to life a new class of Petri nets, called Conflicting Timed Event Graphs (CTEG), which enables us to solve the problem of the shared resources. We also manage to represent the CTEG dynamics using recurrent (Max,+) equations and therefore calculate the end to-end delays. An Ethernet-based network is studied as an example to apply this novel approach. Note by the way that the field of application of this approach borders largely communication networks and is quite possible when dealing with other systems.Finally, to validate the different results of our research activities and the related hypotheses, especially the maximal bound of response time formula, we carry out lots of experimental measurements on a lab facility. We compare the measures to the formula predictions and check their agreement under different conditions.

This master’s thesis deals abou the transmission of voice in communications networks. The theoretical part describes criteria for optimizing voice, such as quality of service, type of service, level of service, service type, and mean opinion score. Next I describe the Internet Protocol, comparing IPv4 and IPv6, VoIP, including security, protocols and parameters necessary for transmission. Other part is about neural networks. There are basically described the neural network, Hopfield neural network and Kohenen neural network. The research is based on a comparison of the network without ensuring the quality of service and with ensuring quality of service. Then, there are compared two types of switches. Classical switch-controlled sequentially, and switch controlled by neural networks. The overall simulation program is implemented in Opnet Modeler. The conclusion deals with the creation of laboratory tasks in this program to compare the different systems of ensuring quality of service.

Les outils informatiques (comme les routeurs et calculateurs entre autres) sont des consommateurs accrus d'énergie. Cette problématique a été déjà prise en compte dans les réseaux mobiles. La question de l'énergie commence juste à être prise en compte pour les systèmes "fixes" à grande échelle qui atteignent de nos jours des tailles impressionnantes. L'objectif de cette thèse est de traiter le problème de la consommation de l'énergie dans les réseaux de communication filaires: fournir un certain niveau de qualité de service (QdS) par rapport à la perte des paquets, la vitesse de réponse et la robustesse par rapport aux différentes périodes d'échantillonnages tout en contrôlant la puissance consommée du système. Le but est de concevoir une méthode à partir de la théorie de la commande, qui consiste à garantir un certain nombre de paramètres de QdS. Cette technique est appliquée au niveau local d'un équipement réseau (routeur, switch ...). La loi de commande permet de distribuer temporellement le trafic qui traverse un nœud contrôlé dans les réseaux de communication filaires. Dans ce travail, nous avons considéré que les nœuds de communications sont des routeurs de type ALR. Pour traiter le problème de la consommation énergétique dans les réseaux de communication filaires, nous avons proposé un modèle énergétique ALR étendu adapté à la théorie de commande. Pour ce modèle, nous avons besoin de choisir deux paramètres (ß, ?), permettant de choisir la taille de file d'attente de référence qref et sa fenêtre temporelle d'actualisation Tqref .Ce deux paramètres ont été choisis à partir de plusieurs simulations avec différentes combinaisons des paramètres (ß, ?). Nous avons vu que la variation de ces deux paramètres permet d'agir énormément sur la QdS ainsi que sur la quantité d'énergie réduite. Les résultats théoriques sont ensuite testés sur Matlab-Simulink, puis sur le simulateur de réseaux NS-2. Les simulations ont montré que la consommation énergétique dans les réseaux de communication est bien réduite tout en garantissant un certain niveau de QdS. / The computer tools (as the routers and calculators among others) present a high energy consumption. This problem has been already included in mobile networks. The question of energy is just beginning to be considered for "fixed" large-scale systems that reach nowadays high sizes. The objective of this thesis is to address the problem of energy consumption in wired communication networks: provide a certain level of quality of service (QoS) with respect to the packet lost, response speed and robustness with respect to different sampling periods while controlling power consumption of the system. The goal is to design a method from the theory of control, which guarantees these QoS. This technique is applied locally to a network equipment (router, switch ...) and the control law used to distribute temporally the traffic through a controlled node in the wired communications networks. In this work, we considere that the communication between nodes are performed by routers ALR type. In order to deal with energy reduction problem, we propose an extended ALR energy model adapted to control theory. For this model, we need to choose two parameters (ß, ?) allowing to choose the queue length reference, qref, and the related update time-window, Tqref. These parameters have been chosen after performing some simulations with different combinations of parameters (ß, ?). We have seen that the variation of these two parameters provide an impact over the QoS as well as the energy reduction. The theoretical results are then tested in Matlab-Simulink as well as some experiments under the simulator NS-2. Simulations showed that the energy consumption in communications networks is reduced while ensuring a certain level of QoS.

[pt] Esta dissertação investiga a aplicação dos métodos de Reinforcement Learning na descoberta de rotas ótimas em uma rede de comunicação. Uma rede de comunicação real possui um comportamento dinâmico, mudando seu estado com o tempo. Os algoritmos de roteamento devem, portanto, oferecer rapidez na resposta às mudanças do estado da rede. O objetivo do trabalho é avaliar a aplicação de técnicas de Reinforcement Learning (RL) como base de algoritmos adaptativos de roteamento de pacotes. O problema de roteamento de pacotes sob a visão de RL consiste na definição de cada nó na rede como um agente RL, sendo que este agente deve definir ações de forma a minimizar uma função objetivo que pode ser o tempo de roteamento dos pacotes. Um dos objetivos do RL é precisamente aprender a tomar as ações que minimizem uma função. O trabalho consistiu de 4 etapas principais: um estudo sobre a área de Reinforcement Learning (RL); um estudo sobre a área de redes de comunicação e roteamento de pacotes; a modelagem do problema de roteamento como um sistema RL e implementação de diferentes métodos de RL para obter algoritmos de roteamento; e o estudo de casos. O estudo na área de Reinforcement Learning abrangeu desde as definições mais fundamentais: suas características, os elementos de um sistema RL e modelagem do ambiente como um Processo de Decisão de Markov, até os métodos básicos de solução: Programação Dinâmica, método de Monte Carlo, e o método de Diferenças Temporais. Neste último método, foram considerados dois algoritmos específicos: TD e Q-Learning. Em seguida, foi avaliado o parâmetro Eligibility Traces como uma alternativa para apressar o processo de aprendizado, obtendo o TD(lambda) e o Q(lambda) respectivamente. O estudo sobre Redes de Comunicação e Roteamento de pacotes envolveu os conceitos básicos de redes de comunicações, comutação por pacotes, a questão do roteamento de pacotes e os algoritmos existentes adaptativos e não adaptativos, que são utilizados na atualidade. Nas redes de comunicação, definidas como um conjunto de nós ligados através de enlaces de comunicação, para se enviar uma mensagem de um nó a outro, geralmente, a mensagem é quebrada em pedaços, chamados pacotes, e enviados através de outros nós, até chegar ao destino. Deste modo surge o problema de escolher os nós que levem o pacote o mais rápido possível até o nó destino. Os algoritmos analisados foram: Shortest Path Routing que procura os caminhos com menor número de nós intermediários, não sendo sensível às mudanças na carga nem na topologia da rede; Weighted Shortest Path Routing, que oferece um melhor desempenho a partir de uma visão global do estado da rede, que nem sempre é fácil de obter em redes reais e o algoritmo de Bellman-Ford, baseado em decisões de roteamento locais e atualizações periódicas, com algumas limitações para obter políticas em altas cargas. Este último é um dos algoritmos mais utilizados na atualidade, sendo base de muitos protocolos de roteamento existentes. A modelagem do problema de roteamento como um sistema RL foi inspirada por uma característica na definição de um sistema RL: um agente que interage com o ambiente e aprende a atingir um objetivo. Assim, a modelagem dos algoritmos tem como objetivo aprender a descobrir as rotas que minimizem o tempo de roteamento de pacotes desde uma origem até um dado destino. A avaliação de uma rota escolhida não pode ser obtida antes que o pacote alcance o seu destino final. Este fato faz com que os processos de aprendizado supervisionado tenham dificuldade de se aplicar a esse problema. Por outro lado, o Reinforcement Learning não necessita de um par entrada-resposta para fazer o aprendizado, permitindo-lhe abordar o problema com relativa facilidade. Na modelagem efetuada, cada nó na rede se comporta como um agente de RL que age na própria rede, a qual é o ambiente. A informação das rotas é armazenada nas funções de valor existentes em todos os nós da rede para / [en] This dissertation investigates the application of Reinforcement Learning methods to the discovery of optimal routes in communication networks. Any current communication network displays dynamic behavior, changing its states over time. Therefore, the routing algorithms must react swiftly to changes in the network status. The objective of this work is to evaluate the application of some Reinforcement Learning techniques to define adaptive packet routing algorithms. The packet routing problem under the RL vision consists in the definition of each node on network as an RL agent. Thus, each agent must take actions in order to minimize an objective function such as end to end packet routing delay. One main objective of the RL is precisely learning to take the actions that minimize a given function. This thesis is consists of 4 main parts: first, a study of Reinforcement Learning (RL); a study of the communication networks and packet routing; the routing problem model as a RL system and the implementation of several RL methods in order to obtain some routing algorithms; e finally, the case study. The study of Reinforcement Learning extends from the more basic definitions, Reinforcement Learning features, elements of a RL system and environment modeling as a Markovian Decision Process, to the basic methods of solution: Dynamic Programming, Monte Carlo methods and Temporal Differences methods. In this last case, two specific algorithms have been considered: TD and Q-Learning, and, finally, the Eligibility Traces are evaluated as a useful tool that permits us to accelerate the learning process leading to the TD(lambda) and the Q(lambda) routing algorithms. The study on communication networks and packet routing involves the foundations of communication networks, packet switching, the packet routing problem, and adaptive and non- adaptive routing algorithms used at the present time. Communication networks are defined as a set of nodes connected through communication links. In order to send a message from a source node to a destination node usually the message is broken into segments called packets, and these are sent through other nodes until arriving at the destination. In this way the problem appears to choose the path which takes the shortest possible time for the packet to reach the destination node. The following algorithms have been analyzed: Shortest Path Routing that looks for paths with minimal hop number, not being sensible to the changes of load level and network topology; Weighted Shortest Path Routing that offers better performance from a global vision of the state of the network, which is not always easy to get in real networks; on the other hand, the Bellman- Ford routing algorithm was studied, this is based on local routing decisions and periodic updates, with some limitations to obtain policies in high load conditions. Bellman-Ford is one of the algorithms most used at the present time, being the basis for many existing routing protocols. The modeling of the routing problem as a RL system was inspired by one of the main features of the definition of an RL system: an agent who interacts with the environment and learns to reach an objective; therefore, the modeling of the routing algorithms has as its objective to learn to discover the paths that minimize packet routing time from an origin to an destination. The evaluation of a chosen route cannot be completed before the package reaches its final destination. This fact implies that supervised learning cannot be applied to the routing problem. On the other hand, Reinforcement Learning does not need a input-output pair for the learning process, allowing it to approach the problem with relative ease. In the modeling, each network node is viewed as a RL agent that acts in the same network; the network is the environment. The routing information is stored in the existing value functions in all nodes in the network, for each node and all another destination node / [es] Esta disertación investiga la aplicación de los métodos de Reinforcement Learning en la determinación de rutas óptimas en una red de comunicación. Una red de comunicación real posee un comportamiento dinámico, donde su estado varia en el tiempo. Los algoritmos de ruta óptima deben, por lo tanto, ofrecer rapidez en la respuesta a las variaciones del estado de la red. El objetivo de este trabajo es evaluar la aplicación de técnicas de Reinforcement Learning (RL) como base de algoritmos adaptativos de problemas de ruteamiento en redes. Este problema consiste en la definición de cada nodo de la red como un agente RL. Este agente debe definir acciones de modo a minimizar una función objetivo que puede ser el tiempo de ruteamiento. El trabajo consta de 4 etapas principais: un estudio sobre el área de Reinforcement Learning (RL); un estudio sobre redes de comunicación y problema de ruteamiento; el modelo de ruta óptima como un sistema RL y la implementación de diferentes métodos de RL para obtener algoritmos de ruta óptima; y un estudio de casos. El estudio en el área de Reinforcement Learning va desde las definiciones fundamentales: características, elementos de un sistema RL y modelaje del ambiente como un Proceso de Decisión de Markov, hasta los métodos básicos de solución: Programación Dinámica, método de Monte Carlo, y método de Diferencias Temporales. En este último método, fueron considerados dos algoritmos específicos: TD e Q-Learning. A seguir, fue evaluado el parámetro Eligibility Traces como una alternativa para agilizar el proceso de aprendizaje, obteniendo el TD(lambda) y el Q(lambda) respectivamente. El estudio sobre Redes de Comunicación y Problema de Transporte incluye los conceptos básicos de redes de comunicaciones, la cuestión de la ruta óptima y los algoritmos adaptativos y no adaptativos existentes, que se utilizan actualmente. Los algoritmos analizados fueron: Shortest Path Routing, que busca los caminos con menor número de nodos intermedios, no siendo sensible a variaciones en la carga ni en la topología de la red; Weighted Shortest Path Routing, que ofrece un mejor desempeño a partir de una visión global del estado de la red, que no siempre es fácil de obtener en redes reales; y el algoritmo de Bellman-Ford, que tiene como base decisiones de rutas locales y actualizaciones periódicas, con algunas limitaciones para obtener políticas en altas cargas. Este último es uno de los algoritmos más utilizados en la actualidad, siendo base de muchos protocolos de trazado de ruta existentes. La solución para modelar el problema de ruteamiento como un sistema RL fue inspirada por una característica en la definición de un sistema RL: un agente que interactúa con el ambiente y aprende a alcanzar un objetivo. Así, el modelo tiene como objetivo aprender a determinar las rutas que minimizen el timpo desde el origen hasta un destino dado. La evaluación de uma ruta seleccionada no puede ser obtenida antes que el paquete alcance su destino final. Esto hace que los procesos de aprendizaje supervisionado tengan dificultades para ser aplicados a este problema. Por otro lado, Reinforcement Learning no necesita de un par entrada-salida para el aprendizaje, permitiendo así, abordar el problema con relativa facilidad. En el modelo establecido, cada nodo en la red se comporta como un agente de RL que actúa en la propria red. La información de las rutas se almacena en las funciones de valor existentes en todos los nodos de la red para cada nodo destino diferente. Esta información contiene un valor estimado del tiempo requerido para un paquete para llegar hasta el nodo destino. La actualización de esos valores se realiza durante la transición del paquete hasta el vecino seleccionado. En este trabajo se implementaron varios algoritmos de ruta óptima. Cada uno de los algoritmos aplica características de las técnicas en Reinforcement Learning: o Q(lambda)-Routing, y el TD-Routing. En el estudio d

Innovations in optical communication are expected to transform the landscape of global communications, internet and datacenter networks. This dissertation investigates several important issues in optical communication such as fairness, throughput, blocking probability and differentiated quality of service (QoS). Novel algorithms and new approaches have been presented to improve the performance of optical circuit switching (OCS) and optical burst switching (OBS) for long haul, and datacenter networks. Extensive simulations tests have been conducted to evaluate the effectiveness of the proposed algorithms. These simulation tests were performed over a number of network topologies such as ring, mesh and U.S. Long-Haul, some high processing computing (HPC) topologies such as 2D and 6D mesh torus topologies and modern datacenter topologies such as FatTree and BCube. Two new schemes are proposed for long haul networks to improve throughput and hop count fairness in OBS networks. The idea is motivated by the observation that providing a slightly more priority to longer bursts over short bursts can significantly improve the throughput of the OBS networks without adversely affecting hop-count fairness. The results of extensive performance tests have shown that proposed schemes improve the throughput of optical OBS networks and enhance the hop-count fairness. Another contribution of this dissertation is the research work on developing routing and wavelength assignment schemes in multimode fiber networks. Two additional schemes for long haul networks are presented and evaluated over multimode fiber networks. First for alleviating the fairness problem in OBS networks using wavelength-division multiplexing as well as mode-division multiplexing while the second scheme for achieving higher throughput without sacrificing hop count fairness. We have also shown the significant benefits of using both mode division multiplexing and wavelength division multiplexing in real-life short-distance optical networks such as the optical circuit switching networks used in the hybrid electronic-optical switching architectures for datacenters. We evaluated four mode and wavelength assignment heuristics and compared their throughput performance. We also included preliminary results of impact of the cascaded mode conversion constraint on network throughput. Datacenter and high performance computing networks share a number of common performance goals. Another highly efficient adaptive mode wavelength- routing algorithm is presented over OBS networks to improve throughput of these networks. The effectiveness of the proposed model has been validated by extensive simulation results. In order to optimize bandwidth and maximize throughput of datacenters, an extension of TCP called multipath-TCP (MPTCP) has been evaluated over an OBS network using dense interconnect datacenter topologies. We have proposed a service differentiation scheme using MPTCP over OBS for datacenter traffic. The scheme is evaluated over mixed workload traffic model of datacenters and is shown to provide tangible service differentiation between flows of different priority levels. An adaptive QoS differentiation architecture is proposed for software defined optical datacenter networks using MPTCP over OBS. This scheme prioritizes flows based on current network state.

Along with playing an ever-increasing role in the integration of other communication networks and expanding in application diversities, the current Internet suffers from serious overuse and congestion bottlenecks. Efficient congestion control is fundamental to ensure the Internet reliability, satisfy the specified Quality-of-Service (QoS) constraints and achieve desirable performance in response to varying application scenarios. Active Queue Management (AQM) is a promising scheme to support end-to-end Transmission Control Protocol (TCP) congestion control because it enables the sender to react appropriately to the real network situation. Analytical performance models are powerful tools which can be adopted to investigate optimal setting of AQM parameters. Among the existing research efforts in this field, however, there is a current lack of analytical models that can be viewed as a cost-effective performance evaluation tool for AQM in the presence of heterogeneous traffic, generated by various network applications. This thesis aims to provide a generic and extensible analytical framework for analyzing AQM congestion control for various traffic types, such as non-bursty Poisson and bursty Markov-Modulated Poisson Process (MMPP) traffic. Specifically, the Markov analytical models are developed for AQM congestion control scheme coupled with queue thresholds and then are adopted to derive expressions for important QoS metrics. The main contributions of this thesis are listed as follows: iii ¿ Study the queueing systems for modeling AQM scheme subject to single-class and multiple-classes Poisson traffic, respectively. Analyze the effects of the varying threshold, mean traffic arrival rate, service rate and buffer capacity on the key performance metrics. ¿ Propose an analytical model for AQM scheme with single class bursty traffic and investigate how burstiness and correlations affect the performance metrics. The analytical results reveal that high burstiness and correlation can result in significant degradation of AQM performance, such as increased queueing delay and packet loss probability, and reduced throughput and utlization. ¿ Develop an analytical model for a single server queueing system with AQM in the presence of heterogeneous traffic and evaluate the aggregate and marginal performance subject to different threshold values, burstiness degree and correlation. ¿ Conduct stochastic analysis of a single-server system with single-queue and multiple-queues, respectively, for AQM scheme in the presence of multiple priority traffic classes scheduled by the Priority Resume (PR) policy. ¿ Carry out the performance comparison of AQM with PR and First-In First-Out (FIFO) scheme and compare the performance of AQM with single PR priority queue and multiple priority queues, respectively.

An investigation into the impact of generalised maximum entropy solutions on the study of queues with or without arrival balking and their applications to congestion management in communication networks Keywords: Queues, Balking, Maximum Entropy (ME) Principle, Global Balance (GB), Queue Length Distribution (QLD), Generalised Geometric (GGeo), Generalised Exponential (GE), Generalised Discrete Half Normal (GdHN), Congestion Management, Packet Dropping Policy (PDP) Generalisations to links between discrete least biased (i.e. maximum entropy (ME)) distribution inferences and Markov chains are conjectured towards the performance modelling, analysis and prediction of general, single server queues with or without arrival balking. New ME solutions, namely the generalised discrete Half Normal (GdHN) and truncated GdHN (GdHNT) distributions are characterised, subject to appropriate mean value constraints, for inferences of stationary discrete state probability distributions. Moreover, a closed form global balance (GB) solution is derived for the queue length distribution (QLD) of the M/GE/1/K queue subject to extended Morse balking, characterised by a Poisson prospective arrival process, i.i.d. generalised exponential (GE) service times and finite capacity, K. In this context, based on comprehensive numerical experimentation, the latter GB solution is conjectured to be a special case of the GdHNT ME distribution. ii Owing to the appropriate operational properties of the M/GE/1/K queue subject to extended Morse balking, this queueing system is applied as an ME performance model of Internet Protocol (IP)-based communication network nodes featuring static or dynamic packet dropping congestion management schemes. A performance evaluation study in terms of the model’s delay is carried out. Subsequently, the QLD’s of the GE/GE/1/K censored queue subject to extended Morse balking under three different composite batch balking and batch blocking policies are solved via the technique of GB. Following comprehensive numerical experimentation, the latter QLD’s are also conjectured to be special cases of the GdHNT. Limitations of this work and open problems which have arisen are included after the conclusions