No description available.

No description available.

No description available.

No description available.

<p> </p> <p>Formulating RL with MDPs work typically works for a single objective, and hence, they are not readily applicable where the policies need to optimize multiple objectives or to satisfy certain constraints while maximizing one or multiple objectives, which can often be conflicting. Further, many applications such as robotics or autonomous driving do not allow for violating constraints even during the training process. Currently, existing algorithms do not simultaneously combine multiple objectives and zero-constraint violations, sample efficiency, and computational complexity. To this end, we study sample efficient Reinforcement Learning with concave objective and convex constraints, where an agent maximizes a concave, Lipschitz continuous function of multiple objectives while satisfying a convex cost objective. For this setup, we provide a posterior sampling algorithm which works with a convex optimization problem to solve for the stationary distribution of the states and actions. Further, using our Bellman error based analysis, we show that the algorithm obtains a near-optimal Bayesian regret bound for the number of interaction with the environment. Moreover, with an assumption of existence of slack policies, we design an algorithm that solves for conservative policies which does not violate  constraints and still achieves the near-optimal regret bound. We also show that the algorithm performs significantly better than the existing algorithm for MDPs with finite states and finite actions.</p>

Managing traffic during oversaturated conditions is a current challenge for practitioners due to the lack of adequate tools that can handle such situations. Unlike under-saturated conditions, operation of traffic signal systems during congestion requires careful consideration and analysis of the underlying causes of the congestion before developing mitigation strategies. The objectives of this research are to provide a practical guidance for practitioners to identify oversaturated scenarios and to develop a multi-objective methodology for selecting and evaluating mitigation strategy/ or combinations of strategies based on a guiding principles. The research focused on traffic control strategies that can be implemented by traffic signal systems. The research did not considered strategies that deals with demand reduction or seek to influence departure time choice, or route choice. The proposed timing methodology starts by detecting network's critical routes as a necessary step to identify the traffic patterns and potential problematic scenarios. A wide array of control strategies are defined and categorized to address oversaturation problematic scenarios. A timing procedure was then developed using the principles of oversaturation timing in cycle selection, split allocation, offset design, demand overflow, and queue allocation in non-critical links. Three regimes of operation were defined and considered in oversaturation timing: (1) loading, (2) processing, and (3) recovery. The research also provides a closed-form formula for switching control plans during the oversaturation regimes. The selection of optimal control plan is formulated as linear integer programming problem. Microscopic simulation results of two arterial test cases revealed that traffic control strategies developed using the proposed framework led to tangible performance improvements when compared to signal control strategies designed for operations in under-saturated conditions. The generated control plans successfully manage to allocate queues in network links. / Ph. D.

In this dissertation, applications of thermodynamics at the macroscopic and quantum levels of description are developed. Within the macroscopic level, an upper-level Sustainability Assessment Framework (SAF) is proposed for evaluating the sustainable and resilient synthesis/design and operation of sets of small renewable and non-renewable energy production technologies coupled to power production transmission and distribution networks via microgrids. The upper-level SAF is developed in accord with the four pillars of sustainability, i.e., economic, environmental, technical and social. A superstructure of energy producers with a fixed transmission network initially available is synthesized based on the day with the highest energy demand of the year, resulting in an optimum synthesis, design, and off-design network configuration. The optimization is developed in a quasi-stationary manner with an hourly basis, including partial-load behavior for the producers. Since sustainability indices are typically not expressed in the same units, multicriteria decision making methods are employed to obtain a composite sustainability index. Within the quantum level of description, steepest-entropy-ascent quantum thermodynamics (SEA-QT) is used to model the phenomenon of decoherence. The two smallest microscopic composite systems encountered in Nature are studied. The first of these is composed of two two-level-type particles, while the second one is composed of a two-level-type particle and an electromagnetic field. Starting from a non-equilibrium state of the composite and for each of the two different composite systems, the time evolution of the state of the composite as well as that of the reduced and locally-perceived states of the constituents are traced along their relaxation towards stable equilibrium at constant system energy. The modeling shows how the initial entanglement and coherence between constituents are reduced during the relaxation towards a state of stable equilibrium. When the constituents are non-interacting, the initial coherence is lost once stable equilibrium is reached. When they are interacting, the coherence in the final stable equilibrium state is only that due to the interaction. For the atom-photon field composite system, decoherence is compared with data obtained experimentally by the CQED group at Paris. The SEA-QT method applied in this dissertation provides an alternative and comprehensive explanation to that obtained with the "open system" approach of Quantum Thermodynamics (QT) and its associated quantum master equations of the Kossakowski-Lindblad-Gorini-Sudarshan type. / Ph. D.

Increasingly tighter budgets, changes in government role/function, declines in staff resources, and demands for increased accountability in the transportation field have brought unprecedented challenges for state transportation officials at all management levels. Systematic methodologies for effective management of a specific type of infrastructure (e.g., pavement and bridges) as well as for holistically managing all types of infrastructure assets are being developed to approach these challenges. In particular, the intrinsic characteristics of highway system make the use of multi-objective optimization techniques particularly attractive for managing highway assets. Recognizing the need for effective tradeoff tools and the limitations of state-of-practice analytical models and tools in highway asset management, the main objective of this dissertation was to develop a performance-based asset management framework that uses multi-objective optimization techniques and consists of stand-alone but logically interconnected optimization models for different management levels. Based on a critical review of popular multi-objective optimization techniques and their applications in highway asset management, a synergistic integration of complementary multi-criteria optimization techniques is recommended for the development of practical and efficient decision-supporting tools. Accordingly, the dissertation first proposes and implements a probabilistic multi-objective model for performance-based pavement preservation programming that uses the weighting sum method and chance constraints. This model can handle multiple incommensurable and conflicting objectives while considering probabilistic constraints related to the available budget over the planning horizon, but is found more suitable to problems with small number of objective functions due to its computational intensity. To enhance the above model, a hybrid model that requires less computing time and systematically captures the decision maker's preferences on multiple objectives is developed by combining the analytic hierarchy process and goal programming. This model is further extended to also capture the relative importance existent within optimization constraints to be suitable for allocations of funding across multiple districts for a decentralized state department of transportation. Finally, as a continuation of the above proposed models for the succeeding management level, a project selection model capable of incorporating qualitative factors (e.g. equity, user satisfaction) into the decision making is developed. This model combines k-means clustering, analytic hierarchy process and integer linear programming. All the models are logically interconnected in a comprehensive resource allocation framework. Their feasibility, practicality and potential benefits are illustrated through various case studies and recommendations for further developments are provided. / Ph. D.

This research involves the development of a comprehensive decision support system for energy planning through the increased use of renewable energy sources, while still considering the role of existing electricity generating facilities. This dissertation focuses on energy planning at the regional level, with the Greater Southern Appalachian Mountain region chosen for analysis due to the dependence on coal as the largest source of generation and the availability of wind and solar resources within the region. The first stage of this planning utilizes a geographic information system (GIS) for the discovery of renewable energy sources. This GIS model analyzes not just the availability of wind and solar power based on resource strength, but also considers the geographic, topographic, regulatory, and other constraints that limit the use of these resources. The model determines potential wind and solar sites within the region based on these input constraints, and finally the model calculates the cost and generation characteristics for each site. The results of the GIS model are then input into the second section of the model framework which utilizes a multi-objective linear programming (MOLP) model to determine the optimal mix of new renewable energy sources and existing fossil fuel facilities. In addition to the potential wind and solar resources discovered in the GIS, the MOLP model considers the implementation of solid wood waste biomass for co-fire at coal plants. The model consists of two competing objectives, the minimization of annual generation cost and the minimization of annual greenhouse gas emissions, subject to constraints on electricity demand and capital investment, amongst others. The model uses the MiniMax function in order to find solutions that consider both of the objective functions. The third major section of this dissertation analyzes three potential public policies — renewable portfolio standard, carbon tax, and renewable energy production tax credit — that have been used to foster increased renewable energy usage. These policies require minor modifications to the MOLP model for implementation. The results of these policy cases are then analyzed to determine the impact that these policies have on generation cost and pollution emissions within the region. / Ph. D.

Within the past two decades, the planning arena has changed considerably and increasing awareness of the impacts of utility generation, intensifying pressure from the public and regulators, and growing competition from other energy and electricity suppliers have made the utility planning process rather complex. The variety of players in utility planning has introduced new priorities and a new set of competing objectives. Increased resource scarcity, the requirement for economic efficiency and the need to view the electricity production and utilization process in its entirety also necessitate an integrated resource planning approach, resulting in a wider array of expansion alternatives that must be evaluated. Another characteristic that makes planning so complicated is the uncertainty in the factors that influence the cost of the power system plan. Public concern for the environment has resulted in a series of legislations for controlling emissions of acid rain precursors (SO₂ and NO_x) and other pollutants. More and more regulators are also requiring electric utilities to internalize environmental externalities in their planning processes. The potential for new legislation on currently uncontrolled effluents like CO₂ likewise remains. There is thus a need to examine the modeling of emissions that would reflect not only the cost of control but the environmental impacts of these emissions as well. This thesis combines the features of the trade-off and decision analysis techniques to address the multiplicity of objectives and the uncertainties of planning. It draws on Saaty's analytic hierarchy process (AHP) and the interdependent data analysis (IDA) technique developed at Virginia Tech to develop priority weights among objectives and probability distributions of uncertainties. It elucidates the relationship between the competing techniques of trade-off analysis and the method of weights in terms of the economic theory of the firm. The confidence intervals determined with the IDA technique are then used to obtain a range of alternatives that satisfy the requirements of both approaches for evaluation by the decision maker (DM). Special attention is given to the environmental impacts of the generation plan and the model accounts for these issues as attributes in the planning process as well as being legislation uncertainties. / Ph. D.