Dissertation for the degree of Doctor of Science in Technology--Helsinki University of Technology, Espoo, 2002. / "ISSN 0781-5816." Includes bibliographical references (p. 65-71). Available online as a PDF file via the World Wide Web.

Thesis (Ph. D.)--Virginia Polytechnic Institute and State University, 1996. / Vita. Abstract. Includes bibliographical references (leaves 161-166). Also available via the Internet.

Um melhor uso da infraestrutura da rede de transporte é um ponto fundamental para atenuar os efeitos dos congestionamentos no trânsito. Este trabalho utiliza aprendizado por reforço multiagente (MARL) para melhorar o uso da infraestrutura e, consequentemente, mitigar tais congestionamentos. A partir disso, diversos desafios surgem. Primeiro, a maioria da literatura assume que os motoristas aprendem (semáforos não possuem nenhum tipo de aprendizado) ou os semáforos aprendem (motoristas não alteram seus comportamentos). Em segundo lugar, independentemente do tipo de classe de agentes e do tipo de aprendizado, as ações são altamente acopladas, tornando a tarefa de aprendizado mais difícil. Terceiro, quando duas classes de agentes co-aprendem, as tarefas de aprendizado de cada agente são de natureza diferente (do ponto de vista do aprendizado por reforço multiagente). Finalmente, é utilizada uma modelagem microscópica, que modela os agentes com um alto nível de detalhes, o que não é trivial, pois cada agente tem seu próprio ritmo de aprendizado. Portanto, este trabalho não propõe somente a abordagem de co-aprendizado em agentes que atuam em ambiente compartilhado, mas também argumenta que essa tarefa precisa ser formulada de forma assíncrona. Além disso, os agentes motoristas podem atualizar os valores das ações disponíveis ao receber informações de outros motoristas. Os resultados mostram que a abordagem proposta, baseada no coaprendizado, supera outras políticas em termos de tempo médio de viagem. Além disso, quando o co-aprendizado é utilizado, as filas de veículos parados nos semáforos são menores. / A better use of transport network infrastructure is a key point in mitigating the effects of traffic congestion. This work uses multiagent reinforcement learning (MARL) to improve the use of infrastructure and, consequently, to reduce such congestion. From this, several challenges arise. First, most literature assumes that drivers learn (traffic lights do not have any type of learning) or the traffic lights learn (drivers do not change their behaviors). Second, regardless of the type of agent class and the type of learning, the actions are highly coupled, making the learning task more difficult. Third, when two classes of agents co-learn, the learning tasks of each agent are of a different nature (from the point of view of multiagent reinforcement learning). Finally, a microscopic modeling is used, which models the agents with a high level of detail, which is not trivial, since each agent has its own learning pace. Therefore, this work does not only propose the co-learnig approach in agents that act in a shared environment, but also argues that this taks needs to be formulated asynchronously. In addtion, driver agents can update the value of the available actions by receiving information from other drivers. The results show that the proposed approach, based on co-learning, outperforms other policies regarding average travel time. Also, when co-learning is use, queues of stopped vehicles at traffic lights are lower.

Um melhor uso da infraestrutura da rede de transporte é um ponto fundamental para atenuar os efeitos dos congestionamentos no trânsito. Este trabalho utiliza aprendizado por reforço multiagente (MARL) para melhorar o uso da infraestrutura e, consequentemente, mitigar tais congestionamentos. A partir disso, diversos desafios surgem. Primeiro, a maioria da literatura assume que os motoristas aprendem (semáforos não possuem nenhum tipo de aprendizado) ou os semáforos aprendem (motoristas não alteram seus comportamentos). Em segundo lugar, independentemente do tipo de classe de agentes e do tipo de aprendizado, as ações são altamente acopladas, tornando a tarefa de aprendizado mais difícil. Terceiro, quando duas classes de agentes co-aprendem, as tarefas de aprendizado de cada agente são de natureza diferente (do ponto de vista do aprendizado por reforço multiagente). Finalmente, é utilizada uma modelagem microscópica, que modela os agentes com um alto nível de detalhes, o que não é trivial, pois cada agente tem seu próprio ritmo de aprendizado. Portanto, este trabalho não propõe somente a abordagem de co-aprendizado em agentes que atuam em ambiente compartilhado, mas também argumenta que essa tarefa precisa ser formulada de forma assíncrona. Além disso, os agentes motoristas podem atualizar os valores das ações disponíveis ao receber informações de outros motoristas. Os resultados mostram que a abordagem proposta, baseada no coaprendizado, supera outras políticas em termos de tempo médio de viagem. Além disso, quando o co-aprendizado é utilizado, as filas de veículos parados nos semáforos são menores. / A better use of transport network infrastructure is a key point in mitigating the effects of traffic congestion. This work uses multiagent reinforcement learning (MARL) to improve the use of infrastructure and, consequently, to reduce such congestion. From this, several challenges arise. First, most literature assumes that drivers learn (traffic lights do not have any type of learning) or the traffic lights learn (drivers do not change their behaviors). Second, regardless of the type of agent class and the type of learning, the actions are highly coupled, making the learning task more difficult. Third, when two classes of agents co-learn, the learning tasks of each agent are of a different nature (from the point of view of multiagent reinforcement learning). Finally, a microscopic modeling is used, which models the agents with a high level of detail, which is not trivial, since each agent has its own learning pace. Therefore, this work does not only propose the co-learnig approach in agents that act in a shared environment, but also argues that this taks needs to be formulated asynchronously. In addtion, driver agents can update the value of the available actions by receiving information from other drivers. The results show that the proposed approach, based on co-learning, outperforms other policies regarding average travel time. Also, when co-learning is use, queues of stopped vehicles at traffic lights are lower.

No description available.

No description available.

This dissertation discusses network wide signal control strategies base on connected vehicle technology. Traffic congestion on arterials has become one of the largest threats to economic competitiveness, livability, safety, and long-term environmental sustainability in the United States. In addition, arterials usually experience more blockage than freeways, specifically in terms of intersection congestion. There is no doubt that emerging technologies provide unequaled opportunities to revolutionize “retiming” and mitigate traffic congestion. Connected vehicle technology provides unparalleled safety benefits and holds promise in terms of alleviating both traffic congestion and the environmental impacts of future transportation systems. The objective of this research is to improve the mobility, safety and environmental effects at signalized arterials with connected vehicles. The proposed solution of this dissertation is to formulate traffic signal control models for signalized arterials based on connected vehicle technology. The models optimize offset, split, and cycle length to minimize total queue delay in all directions of coordinated intersections. Then, the models are implemented in a centralized system—including closed-loop systems—first, before expanding the results to distributed systems. The benefits of the models are realized at the infant stage of connected vehicle deployment when the penetration rate of connected vehicles is around 10%. Furthermore, the benefits incentivize the growth of the penetration rate for drivers. In addition, this dissertation contains a performance evaluation in traffic delay, volume throughput, fuel consumption, emission, and safety by providing a case study of coordinated signalized intersections. The case study results show the solution of this dissertation could adapt early deployment of connected vehicle technology and apply to future connected vehicle technology development.

The capacity of today’s transportation network in Alexandria cannot handle the increasing number of vehicles. The root of traffic congestion in Alexandria is most likely as result of urban planning failure, an inadequate public transportation network, strong population growth, lack of enforcement of traffic laws, complex spatial street network and poor road quality.The aim with this thesis is to simulate various traffic scenarios for the purpose of studying, evaluating and improving the traffic conditions on the intersection Al-Ibrahimeya, however, for long-term efficiency and improvement in Al-Ibrahimeya, the traffic conditions in the entire system be improved.In this thesis, a model was built in the microscopic software VISSIM and macroscopic software SYNCHRO. The model is calibrated based on data collected from video recording. Two alternative solutions to overcome the congested traffic conditions were tested. Signalization has been the main alternative solution performed for intersection Al- Ibrahimeya.The simulation results showed some improvements in terms of delay and travel time. Roundabouts are usually suitable for under low to mid traffic conditions. When there is a heavy traffic flow, as in this case, roundabouts are not suitable and may cause problem in the network. Implementing signals led to some decrease in the entering and exiting flows because the total network became more controlled. However, an organized and well-functioning signalization program helped to reduce delay and travel time in most of routes.

Sub-optimal automated transportation control systems incur high mobility, human health and environmental costs. With society reliant on its transportation systems for the movement of individuals, goods and services, minimizing these costs benefits many. Intersection traffic signal controllers are an important element of modern transportation systems that govern how vehicles traverse road infrastructure. Many types of traffic signal controllers exist; fixed time, actuated and adaptive. Adaptive traffic signal controllers seek to minimize transportation costs through dynamic control of the intersection. However, many existing adaptive traffic signal controllers rely on heuristic or expert knowledge and were not originally designed for scalability or for transportation’s big data future. This research addresses the aforementioned challenges by developing a scalable system for adaptive traffic signal control model development using deep reinforcement learning in traffic simulation. Traffic signal control can be modelled as a sequential decision-making problem; reinforcement learning can solve sequential decision-making problems by learning an optimal policy. Deep reinforcement learning makes use of deep neural networks, powerful function approximators which benefit from large amounts of data. Distributed, parallel computing techniques are used to provide scalability, with the proposed methods validated on a simulation of the City of Luxembourg, Luxembourg, consisting of 196 intersections. This research contributes to the body of knowledge by successfully developing a scalable system for adaptive traffic signal control model development and validating it on the largest traffic microsimulator in the literature. The proposed system reduces delay, queues, vehicle stopped time and travel time compared to conventional traffic signal controllers. Findings from this research include that using reinforcement learning methods which explicitly develop the policy offers improved performance over purely value-based methods. The developed methods are expected to mitigate the problems caused by sub-optimal automated transportation signal controls systems, improving mobility and human health and reducing environmental costs. / Thesis / Doctor of Philosophy (PhD) / Inefficient transportation systems negatively impact mobility, human health and the environment. The goal of this research is to mitigate these negative impacts by improving automated transportation control systems, specifically intersection traffic signal controllers. This research presents a system for developing adaptive traffic signal controllers that can efficiently scale to the size of cities by using machine learning and parallel computation techniques. The proposed system is validated by developing adaptive traffic signal controllers for 196 intersections in a simulation of the City of Luxembourg, Luxembourg, successfully reducing delay, queues, vehicle stopped time and travel time.

Dilemma zone (DZ), also called decision zone in other literature, is an area where drivers face an indecisiveness of stopping or crossing at the yellow onset. The DZ issue is a major reason for the crashes at high-speed signalized intersections. As a result, how to prevent approaching vehicles from being caught in the DZ is a widely concerning issue. In this dissertation, the author addressed several DZ-associated issues, including the new stochastic safety measure, namely dilemma hazard, that indicates the vehicles' changing unsafe levels when they are approaching intersections, the optimal advance detector configurations for the multi-detector green extension systems, the new dilemma zone protection algorithm based on the Markov process, and the simulation-based optimization of traffic signal systems with the retrospective approximation concept. The findings include: the dilemma hazard reaches the maximum when a vehicle moves in the dilemma zone and it can be calculated according the caught vehicles' time to the intersection; the new (optimized) GES design can significantly improve the safety, but slightly improve the efficiency; the Markov process can be used in the dilemma zone protection, and the Markov-process-based dilemma zone protection system can outperform the prevailing dilemma zone protection system, the detection-control system (D-CS). When the data collection has higher fidelity, the new system will have an even better performance. The retrospective approximation technique can identify the sufficient, but not excessive, simulation efforts to model the true system and the new optimization algorithm can converge fast, as well as accommodate the requirements by the RA technique. / Ph. D.