Motivated by the management of sales representatives who visit customers to develop customer relationships, we present a stochastic orienteering problem on a network of queues, in which a hard time window is associated with each customer and the representative may experience uncertain wait time resulting from a queueing process at the customer. In general, given a list of potential customers and a time horizon consisting of several periods, the sales representative needs to decide which customers to visit in each period and how to visit customers within the period, with an objective to maximize the total reward collected by the end of the horizon. We start our study with a daily orienteering problem, which is a subproblem of the general problem. We focus on developing a priori and dynamic routing strategies for the salesperson to implement during a day. In the a priori routing case, the salesperson visits customers in a pre-planned order, and we seek to construct a static sequence of customers that maximizes the expected value collected. We consider two types of recourse actions. One is to skip a customer specified by an a priori route if the representative will arrive late in the customer's time window. The other type is to leave a customer immediately after arriving if observing a sufficiently long queue (balking) or to leave after waiting in queue for a period of time without meeting with the customer (reneging). We propose customer-specific decision rules to facilitate the execution of recourse actions and derive an analytical formula to compute the expected sales from the a priori route. We tailor a variable neighborhood search (VNS) heuristic to find a priori routes. In the dynamic routing case, the salesperson decides which customer to visit and how long to wait at each customer based on realized events. To seek dynamic routing policies, we propose an approximate dynamic programming approach based on rollout algorithms. The method introduces a two-stage heuristic estimation that we refer to as compound rollout. In the first stage, the algorithm decides whether to stay at the current customer or go to another customer. If departing the current customer, it chooses the customer to whom to go in the second stage. We demonstrate the value of our modeling and solution approaches by comparing the dynamic policies to a priori solutions with recourse actions. Finally, we address the multi-period orienteering problem. We consider that each customer's likelihood of adopting the representative's product stochastically evolves over time and is not fully observed by the representative. The representative can only estimate the adoption likelihood by meeting with the customer and the estimation may not be accurate. We model the problem as a partially observed Markov decision process with an objective to maximize the expected sales at the end of the horizon. We propose a heuristic that decomposes the problem into an assignment problem to schedule customers for a period and a routing problem to decide how to visit the scheduled customers within the period.

When a package is shipped, the customer often requires the delivery to be made within a particular time window or by a deadline. However, meeting such time requirements is difficult, and delivery companies may not always know ahead of time which customers will need a delivery. In this thesis, we present models and solution approaches for two stochastic last-mile delivery problems in which customers have delivery time constraints and customer presence is known in advance only according to a probability distribution. Our solutions can help reduce the operational costs of delivery while improving customer service. The first problem is the probabilistic traveling salesman problem with time windows (PTSPTW). In the PTSPTW, customers have both a time window and a probability of needing a delivery on any given day. The objective is to find a pre-planned route with an expected minimum cost. We present computational results that characterize the PTSPTW solutions. We provide insights for practitioners on when solving the PTSPTW is beneficial compared to solving the deterministic analogue of the problem. The second problem is the same-day delivery problem (SDDP). The SDDP is a dynamic and stochastic pick-up and delivery problem. In the SDDP, customers make delivery requests throughout the day and vehicles are dispatched from a warehouse or brick and mortar store to serve the requests. Associated with each request is a request deadline or time window. In order to make better-informed decisions, our solution approach incorporates information about future requests into routing decisions by using a sample scenario planning approach with a consensus function. We also introduce an analytical result that identifies when it is beneficial for vehicles to wait at the depot. We present a wide range of computational experiments that demonstrate the value of our approaches.

Data-driven decision making has become more popular in today’s businesses including logistics and vehicle routing. Leveraging historical data, companies can achieve goals such as customer satisfaction management, scalable and efficient operation, and higher overall revenue. In the management of customer satisfaction, logistics companies use consistent assignment of their drivers to customers over time. Creating this consistency takes time and depends on the history experienced between the company and the customer. While pursuing this goal, companies trade off the cost of capacity with consistency because demand is unknown on a daily basis. We propose concepts and methods that enable a parcel delivery company to balance the trade-off between cost and customer satisfaction. We use clustering methods that use cumulative historical service data to generate better consistency using the information entropy measure. Parcel delivery companies route many vehicles to serve customer requests on a daily basis. While clustering was important to the development of early routing algorithms, modern solution methods rely on metaheuristics, which are not easily deployable and often do not have open source code bases. We propose a two-stage, shape-based clustering approach that efficiently obtains a clustering of delivery request locations. Our solution technique is based on creating clusters that form certain shapes with respect to the depot. We obtain a routing solution by ordering all locations in every cluster separately. Our results are competitive with a state-of-the-art vehicle routing solver in terms of quality. Moreover, the results show that the algorithm is more scalable and is robust to problem parameters in terms of runtime. Fish trawling can be considered as a vehicle routing problem where the main objective is to maximize the amount of fish (revenue) facing uncertainty on catch. This uncertainty creates an embedded prediction problem before deciding where to harvest. Using previous catch data to train prediction models, we solve the routing problem a fish trawler faces using dynamically updated routing decisions allowing for spatiotemporal correlation in the random catch. We investigate the relationship between the quality of predictions and the quality of revenue generated as a result.

This dissertation explores workforce planning in manufacturing and healthcare systems. In manufacturing systems, the existing workforce planning models often lack fidelity with respect to the mechanism of learning. Learning refers to that employees’ productivity increases as they gain more experience. Workforce scheduling in the short term has a longer term impact on organizations’ capacity. The mathematical representations of learning are usually nonlinear. This nonlinearity complicates the planning models and provides opportunities to develop solution methodologies for realistically-sized instances. This research formulates the workforce planning problem as a mixed-integer nonlinear program (MINLP) and overcomes the limitations of cur- rent solution methods. Specifically, this research develops a reformulation technique that converts the MINLP to a mixed integer linear program (MILP) and proposes several techniques to speed up the solution time of solving the MILP. In organizations that use group work, workers learn not only by individual learning but also from knowledge transferred from team members. Managers face the decision of how to pair or team workers such that organizations benefit from this transfer of learning. Using a mathematical representation that incorporates both in- dividual learning and knowledge transfer between workers, this research considers the problem of grouping workers to teams and assigning teams to sets of jobs based on workers’ learning and knowledge transfer characteristics. This study builds a Mixed- integer nonlinear programs (MINP) for parallel systems with the objective of maximizing the system throughput and propose exact and heuristic solution approaches for solving the MINLP. In healthcare systems, we focus on managing medical technicians in medical laboratories, in particular, the phlebotomists. Phlebotomists draw specimens from patients based on doctors’ orders, which arrive randomly in a day. According to the literature, optimizing scheduling and routing in hospital laboratories has not been regarded as a necessity for laboratory management. This study is motivated by a real case at University of Iowa Hospital and Clinics, where there is a team of phlebotomists that cannot fulfill doctors requests in the morning shift. The goal of this research is routing these phlebotomists to patient units such that as many orders as possible are fulfilled during the shift. The problem is a team orienteering problem with stochastic rewards and service times. This research develops an a priori approach which applies a variable neighborhood search heuristic algorithm that improves the daily performance compared to the hospital practice.

Public health data show the tremendous economic and societal impact of pandemic influenza in the past. Currently, the welfare of society is threatened by the lack of planning to ensure an adequate response to a pandemic. This preparation is difficult because the characteristics of the virus that would cause the pandemic are unknown, but primarily because the response requires tools to support decision-making based on scientific methods. The response to the next pandemic influenza will likely include extensive use of antiviral drugs, which will create an unprecedented selective pressure for the emergence of antiviral resistant strains. Nevertheless, the literature has insufficient exhaustive models to simulate the spread and mitigation of pandemic influenza, including infection by an antiviral resistant strain. We are building a large-scale simulation optimization framework for development of dynamic antiviral strategies including treatment of symptomatic cases and chemoprophylaxis of pre- and post-exposure cases. The model considers an oseltamivir-sensitive strain and a resistant strain with low/high fitness cost, induced by the use of the several antiviral measures. The mitigation strategies incorporate age/immunitybased risk groups for treatment and pre-/post-exposure chemoprophylaxis, and duration of pre-exposure chemoprophylaxis. The model is tested on a hypothetical region in Florida, U.S., involving more than one million people. The analysis is conducted under different virus transmissibility and severity scenarios, varying intensity of non-pharmaceutical interventions, measuring the levels of antiviral stockpile availability. The model is intended to support pandemic preparedness and response policy making.

Intelligent agents are designed to interact with, and learn about, their environment so that they can act purposefully towards a goal. One class of problems encountered in building such agents is learning how to respond to dynamic systems with a continuous state space. The goals of this dissertation are to develop a framework for understanding the behaviour of partitioned dynamic systems with continuous underlying state and to translate this framework into algorithms which adaptively form a partition of the continuous space such that the partitioned system is more easily learned and controlled, and such that the control law may be easily explained in intuitive ways. Currently, algorithms which learn a control policy for partitioned continuous state space systems treat the partitioned system as an approximation to a Markov chain. I give conditions for the partitioned system to be a Markov chain, a semi-Markov process and a new class of system, a weak-semi-Markov process. The weak-semi-Markov model is shown to model partitioned dynamic systems with greater economy than other surveyed models. The behaviour of a partitioned state space system in the area around the region boundaries is also considered. I use the theory of sliding surfaces, and some heuristic arguments to recommend region boundary shape and position. The concept of 'staying on the boundary' then becomes a robust and relatively easy subgoal within the control algorithm. The concept of 'reaching the sliding surface' as a subgoal is used as the basis for an intuitive explanation of the learnt controller. I present an algorithm based on this concept which explains the behaviour of a learnt controller in ways not previously available to a machine learning algorithms. Finally, the Markov Property and the theory of Sliding Mode Control are used as the basis of a class of recursive algorithms. These algorithms adaptively find a partition, and simultaneously use this partition in conjunction with one of five reinforcement learning algorithms to find a control policy based on that partition. This technique is shown to work very well in learning, controlling and explaining a variety of physical systems, from a monorail to a container crane.

<p>The situation when driving in dense traffic and at low speeds is called Stop and Go. A controller for automatic following of the car in front could under these conditions reduce the driver's workload and keep a safety distance to the preceding vehicle through different choices of gear and engine torque. The aim of this thesis is to develop such a controller, with an additional focus on lowering the fuel consumption. With help of GPS, 3D-maps and sensors information about the slope of the road and the preceding vehicle can be obtained. Using this information the controller is able to predict future possible control actions and an optimization algorithm can then find the best inputs with respect to some criteria. The control method used is Model Predictive Control (MPC) and as the name indicate a model of the control object is required for the prediction. To find the optimal sequence of inputs, the optimization method Dynamic Programming choose the one which lead to the lowest fuel consumption and satisfactory following. Simulations have been made using a reference trajectory which was measured in a real traffic jam. The simulations show that it is possible to follow the preceding vehicle in a good way and at the same time reduce the fuel consumption with approximately 3 %.</p>

<p>This thesis is an approach to improve a two-mode hybrid electric vehicle, which is currently under development by GM, with respect to fuel consumption. The study is not only restricted to the specific two-mode HEV but also presents results regarding parallel as well as serial HEV’s. GM whishes to verify if the online-based controller in the prototype vehicle utilizes the most of the HEV ability and if there is more potential to lower the fuel consumption. The purpose is that the results and conclusions from this work are to be implemented in the controller to further improve the vehicle’s performance. To analyze the behavior of the two-mode HEV and to see where improvements can be made, models of its driveline and components are developed with a focuson losses and efficiency. The models are implemented in MATLAB together with an optimization algorithm based on Dynamic Programming. The models are validated against data retrieved from the prototype vehicle and various cases with different inputs is set up and optimized over the NEDC cycle. Compensation for cold starts and NOx emissions are also implemented in the final model. Deliberate simplifications are made regarding the modeling of the power split’s functionality due to the limited amount of time available for this thesis. The optimizations show that there is potential to lower the fuel consumptionfor the two-mode HEV. The results are further analyzed and the behavior of the engine, motors/generators and battery are compared with recorded data from a prototype vehicle and summarized to a list of suggestions to improve fuel economy.</p>

<p>The department REI/EP at DaimlerChrysler Research and Technology and the Laboratory for Efficient Energy Systems at Trier University of Applied Science, are developing control functions and fuel optimal strategies for low speed conditions. The goal of this thesis project was to further develop the fuel optimal operating strategies, and implement them into a test vehicle equipped with a dSPACE environment. This was accomplished by making optimal reference signals using dynamic programming. Optimal, in this case, means signals that results in low fuel consumption, comfortable driving, and a proper distance to the preceding vehicle. These reference signals for the velocity and distance are used by an MPC controller (Model Predictive Control) to control the car. In every situation a suitable reference path is chosen, depending on the velocities of both vehicles, and the distance. The controller was able to follow another vehicle in a proper way. The distance was kept, the driving was pleasant, and it also seems like it is possible to save fuel. When accepting some deviations in distance to the preceding car, a fuel reduction of 8 % compared to the car in front can be achieved.</p>

<p>The use of positioning systems in a vehicle is a research intensive field. In the first part of this thesis an increase in new applications is disclosed through a mapping of patent documents on how positioning systems can support adaptive cruise control, gear changing systems and engine control. Many ideas are presented and explained and the ideas are valued. Furthermore, a new method for selective catalytic reduction (SCR) control using a positioning system is introduced. It is concluded that look-ahead control, where the vehicle position in relation to the upcoming road section is utilized could give better fuel efficiency, lower emissions and less brake, transmission and engine wear.</p><p>In the second part of this thesis a real time test platform for predictive speed control algorithms has been developed and tested in a real truck. Previously such algorithms could</p><p>only be simulated. In this thesis an algorithm which utilizes model predictive control (MPC) and dynamic programming (DP) been implemented and evaluated. An initial comparative fuel test shows a reduction in fuel consumption when the MPC algorithm is used.</p>