In an optimal visiting problem, we want to control a trajectory that has to pass as close as possible to a collection of target points or regions. We introduce a hybrid control-based approach for the classic problem where the trajectory can switch between a group of discrete states related to the targets of the problem. The model is subsequently adapted to a mean-field game framework, that is when a huge population of agents plays the optimal visiting problem with a controlled dynamics and with costs also depending on the distribution of the population. In particular, we investigate a single continuity equation with possible sinks and sources and the field possibly depending on the mass of the agents. The same problem is also studied on a network framework. More precisely, we study a mean-field game model by proving the existence of a suitable definition of an approximated mean-field equilibrium and then we address the passage to the limit.

Demonstrating strength and agility on virtual and real humanoids has been an important goal in computer graphics and robotics. However, developing physics- based controllers for various agile motor skills requires a tremendous amount of prior knowledge and manual labor due to complex mechanisms of the motor skills. The focus of the dissertation is to develop a set of computational tools to expedite the design process of physics-based controllers that can execute a variety of agile motor skills on virtual and real humanoids. Instead of designing directly controllers real humanoids, this dissertation takes an approach that develops appropriate theories and models in virtual simulation and systematically transfers the solutions to hardware systems. The algorithms and frameworks in this dissertation span various topics from spe- cific physics-based controllers to general learning frameworks. We first present an online algorithm for controlling falling and landing motions of virtual characters. The proposed algorithm is effective and efficient enough to generate falling motions for a wide range of arbitrary initial conditions in real-time. Next, we present a robust falling strategy for real humanoids that can manage a wide range of perturbations by planning the optimal contact sequences. We then introduce an iterative learning framework to easily design various agile motions, which is inspired by human learn- ing techniques. The proposed framework is followed by novel algorithms to efficiently optimize control parameters for the target tasks, especially when they have many constraints or parameterized goals. Finally, we introduce an iterative approach for exporting simulation-optimized control policies to hardware of robots to reduce the number of hardware experiments, that accompany expensive costs and labors.

Master of Arts / Department of Economics / Steven P. Cassou / This report shows the mathematics behind the solution to continuous time optimization problems. It shows how to specify the Hamiltonian function, how to use the Hamiltonian to obtain the optimal conditions for a typical economic optimal control problem and applies these techniques to several optimal control problems commonly encountered in macroeconomics. An appendix shows how to set up the optimal conditions for the case in which the state and co-state variables are both vectors. A second appendix shows how to approach the control situation for a system of optimal control problems where the co-state variable for the first sub-optimal control problem is the state variable for the second sub-optimal control problem.

An aneurysm is a balloon-like bulge in the wall of blood vessels, occurring in major arteries from the heart and brain. Biodegradable stent-assisted coiling is expected to be the ideal treatment of wide-neck complex aneurysms. A number of biodegradable stents are promising, but also with issues and/or several limitations to be addressed. From the design point of view, biodegradable stents are typically designed without structure optimization. The drawbacks of these stents often cause weaker mechanical properties than native arterial vessels. From the fabrication point of view, the conventional methods of the fabricating stent are time-consuming and expensive, and also lack precise control over the stent microstructure. As an emerging fabrication technique, dispensing-based rapid prototyping (DBRP) allows for more accurate control over the scaffold microstructure, thus facilitating the fabrication of stents as designed. This thesis is aimed at developing methods for fabrication and optimal design of biodegradable stents for treating aneurysms. Firstly, a method was developed to fabricate biodegradable stents by using the DBRP technique. Then, a compression test was carried out to characterize the radial deformation of the stents fabricated. The results illustrated the stent with a zigzag structure has a higher radial stiffness than the one with a coil structure. On this basis, the stent with a zigzag structure was chosen to develop a finite element model for simulating the real compression tests. The result showed the finite element model of biodegradable stents is acceptable within a range of radial deformation around 20%. Furthermore, an optimization of the zigzag structure was performed with ANSYS DesignXplorer, and the results indicated that the total deformation could be decreased by 35% by optimizing the structure parameters, which would represent a significant advance of the radial stiffness of biodegradable stents. Finally, the optimized stent was used to investigate its deformation in a blood vessel. The deformation is found to be 0.25 mm in the simulation, and the rigidity of biodegradable stents is 7.22%, which is able to support the blood vessel all. It is illustrated that the finite element analysis indeed helps in designing stents with new structures and therefore improved mechanical properties.

A fast and efficient method to calculate time schedules for internal and boundary PRVs and flow modulation curves has been developed and implemented. Both time and flow modulation can be applied to a single inlet DMA. The time modulation methodology is based on solving a nonlinear programming problem (NLP). In addition, Genetic Algorithms (GA) has been proposed and investigated to calculate the optimal coefficients of a second order relationship between the flow and the outlet pressure for a PRV to minimize the background leakage. The obtained curve can be subsequently implemented using a flow modulation controller in a feedback control scheme. The Aquai-Mod® is a hydraulic device to control and modulate the outlet pressure of a PRV according to the valve flow. The controller was experimentally tested to assess its performance and functionality in different conditions and operating ranges. The mathematical model of the controller has been developed and solved, in both steady state and dynamic conditions. The results of the model have been compared with the experimental data and showed a good agreement in the magnitude and trends. A new method for combined energy and pressure management via integration and coordination of pump scheduling with pressure control aspects has been created. The method is based on formulating and solving an optimisation NLP problem and involves pressure dependent leakage. The cost function of the optimisation problem represents the total cost of water treatment and pumping energy. Developed network scheduling algorithm consists of two stages. The first stage involves solving a continuous problem, where operation of each pump is described by continuous variable. Subsequently, the second stage continuous pump schedules are discretised using heuristic algorithm. Another area of research has been developing optimal feedback rules using GA to control the operation of pump stations. Each pump station has a rule described by two water levels in a downstream reservoir and a value of pump speed for each tariff period. The lower and upper water switching levels of the downstream reservoir correspond to the pump being “ON” or “OFF”. The achieved similar energy cost per 1 Ml of pumped water. In the considered case study, the optimal feedback rules had advantage of small number of ON/OFF switches, which increase the pump stations lifetime and reduce the maintenance cost as well.

For robotic systems to continue to move towards ubiquity, robots need to be more autonomous. More autonomy dictates that robots need to be able to make better decisions. Control theory and machine learning are fields of robotics that focus on the decision making process. However, each of these fields implements decision making at different levels of abstraction and at different time scales. Control theory defines low-level decisions at high rates, while machine learning defines high-level decision at low rates. The objective of this research is to integrate tools from both machine leaning and control theory to solve higher dimensional, complex problems, and to optimize the decision making process. Throughout this research, multiple algorithms were created that use concepts from both control theory and machine learning, which provide new tools for robots to make better decisions. One algorithm enables a robot to learn how to optimally explore an unknown space, and autonomously decide when to explore for new information or exploit its current information. Another algorithm enables a robot to learn how to locomote with complex dynamics. These algorithms are evaluated both in simulation and on real robots. The results and analysis of these experiments are presented, which demonstrate the utility of the algorithms introduced in this work. Additionally, a new notion of “learnability” is introduced to define and determine when a given dynamical system has the ability to gain knowledge to optimize a given objective function.

Vehicle powertrain electrification, i.e. combining the internal combustion engine (ICE) with an electric motor (EM), is a potential way of meeting the increased demands for efficient and low emission transportation, at a price of increased powertrain complexity since more degrees of freedom (DoF) have been introduced. Optimal control is used in a series of studies of how to best exploit the additional DoFs. In a diesel-electric powertrain the absence of a secondary energy storage and mechanical connection between the ICE and the wheels means that all electricity used by the EMs needs to be produced simultaneously by the ICE, whose rotational speed is a DoF. This in combination with the relatively slow dynamics of the turbocharger in the ICE puts high requirements on good transient control. In optimal control studies, accurate models with good extrapolation properties are needed. For this aim two nonlinear physics based models are developed and made available that fulfill these requirements, these are also smooth in the region of interest, to enable gradient based optimization techniques. Using optimal control and one of the developed models, the turbocharger dynamics are shown to have a strong impact on how to control the powertrain and neglecting these can lead to erroneous estimates both in the response of the powertrain as well as how the powertrain should be controlled. Also the objective, whether time or fuel is to be minimized, influences the engine speed-torque path to be used, even though it is shown that the time optimal solution is almost fuel optimal. To increase the freedom of the powertrain control, a small energy storage can be added to assist in the transients. This is shown to be especially useful to decrease the response time of the powertrain, but the manner it is used, depends on the time horizon of the optimal control problem. The resulting optimal control solutions are for certain cases oscillatory when stationary controls would have been expected. This is shown to be neither an artifact of the discretization used nor a result of the modeling assumptions used. Instead it is for the formulated problems actually optimal to use periodic control in certain stationary operating points. Measurements show that the pumping torque is different depending on whether the controls are periodic or constant despite the same average value. Whether this is beneficial or not depends on the operating point and control frequency, but can be predicted using optimal periodic control theory. In hybrid electric vehicles (HEV) the size of the energy storage reduces the impact of poor transient control, since the battery can compensate for the slower dynamics of the ICE. For HEVs the problem instead is how and when to use the battery to ensure good fuel economy. An adaptive map-based equivalent consumption minimization strategy controller using battery state of charge for feedback control is designed and tested in a real vehicle with good results, even when the controller is started with poor initial values. In a plug-in HEV (PHEV) the battery is even larger, enabling all-electric drive, making it it desirable to use the energy in the battery during the driving mission. A controller is designed and implemented for a PHEV Benchmark and is shown to perform well even for unknown driving cycles, requiring a minimum of future knowledge. / Elektrifiering av drivlinan i fordon är ett sätt att möta kraven på transporter med hög effektivitet och låga utsläpp. Att byta ut förbränningsmotorn mot en elmotor kan ge vinningar avseende effektivitet, prestanda och utsläpp, men till en kostnad av lägre mobilitet på grund av eletriska energilagers relativt låga energitäthet i jämförelse med fossila bränslen. Att istället komplettera förbränningsmotorn med en elmotor erbjuder möjligheten att kombinera de två systemens fördelar och samtidigt undvika nackdelarna. Att använda mer än en motor i drivlinan ökar komplexiteten eftersom fler frihetsgrader har introducerats. Detta ställer ökade krav på utformningen av reglersystemet för att få ut det mesta av potentialen i drivlinan. I optimal styrning använder man matematiska modeller och optimeringsalgoritmer för att beräkna hur man bäst styr det modellerade systemet. Storleken på det elektriska energilagret påverkar dock valet av optimal styrnings-metod samt vilken detaljnivå på modellerna som behövs. I avhandlingen används optimal styrning i en serie studier av hur man bäst utnyttjar de extra frihetsgraderna som elektrifieringen har introducerat. I en diesel-elektrisk drivlina finns det ingen mekanisk koppling mellan motorn och hjulen, likt en växellåda i ett vanligt fordon, vilket gör att dieselmotorns varvtal är en frihetsgrad som måste styras. Avsaknaden av elektriskt energilager leder också till att all elektrisk energi till elmotorn måste produceras av förbränningsmotorn exakt då den behövs. Dessa två egenskaper, i kombination med den långsamma dynamiken hos turboaggregatet, ställer detta höga krav på god transientreglering. För att studera optimal styrning krävs bra modeller med goda extrapoleringsegenskaper. Med avseende på detta utvecklas två fysik-baserade modeller som uppfyller dessa krav och dessutom är tillräckligt glatta i det relevanta arbetsområdet för att möjliggöra gradient-baserade optimeringstekniker. Med optimal styrning och en av de utvecklade modellerna visas turbons dynamik ha stor påverkan på hur drivlinan bör styras. Att försumma turbodynamiken kan leda till felaktiga uppskattningar, både av drivlinans responstid, men även hur den bör styras. Kriteriet, det vill säga om bränsle eller tidsåtgången minimeras, påverkar också vilken motorvarvtal-motormoment-väg som är optimal, även om det visas att den tidsoptimala lösningen är nästan bränsleoptimal. För att ytterligare öka frihetsgraden i drivlinan kan ett elektriskt energilager användas för att assistera i transienterna. Detta visar sig vara särskilt användbart för att minska responstiden hos drivlinan, men hur det ska använda beror på tidshorisonten på optimeringsproblemet De resulterande optimala styrsignalerna är i vissa fall oscillerande där konstanta styrsignaler förväntas. Detta visas vara vare sig en effekt av den använda diskretiseringen eller modelleringsvalen som är gjorda. Istället är det för de lösta problemen faktiskt optimalt att använda periodiska styrsignaler för vissa stationära arbetspunkter. I experiment visas att pumparbetet skiljer sig beroende på om periodiska eller konstanta styrsignaler används, även om medelvärdet är detsamma. Huruvida detta ökar effektiviteten eller inte beror på arbetspunkt och periodtid. För hybridelektriska fordon (HEV) så minskar batteriets storlek effekten av dålig transientreglering då batteriet kan användas för att kompensera för den långsamma förbränningsmotordynamiken. Istället blir problemet i huvudsak hur mycket och när batteriet ska användas för att få god bränsleekonomi. En adaptiv mapp-baserad ekvivalentförbruknings-minimerande styrlag (ECMS) med återkopplad reglering baserad på batteriets laddningsnivå, utvecklas och testas i riktigt fordon med gott resultat, även vid dålig initialisering av regulatorn. För plug-in hybrider (PHEV) är batteriet större och kan dessutom laddas från elnätet, vilket medför möjlighet till rent elektrisk drift och att det är önskvärt att använda energin i batteriet under köruppdraget. För att minska energiåtgången är det däremot ofta lönsamt att blanda energin från bränsle och batteriet kontinuerligt under köruppdraget och se till att batteriet töms lagom till slutet av köruppdraget. För att åstadkomma detta måste då även urladdningstakten bestämmas. En regulator utvecklas för att minimera energiåtgången för en PHEV, det vill säga som försöker använda lagom av batteriet så det ska räcka hela vägen, men inte längre. Denna regulator implementeras för ett referensproblem, med gott resultat även för okända körcykler, trots ett minimum av framtidskunskap.

An enhanced method in optimization rooted in direct collocation is formulated to treat the finite set optimal control problem. This is motivated by applications in which a hybrid dynamical system is subject to ordinary differential continuity constraints, but control variables are contained within finite spaces. Resulting solutions display control discontinuities as variables switch between one feasible value to another. Solutions derived are characterized as optimal switching schedules between feasible control values. The methodology allows control switches to be determined over a continuous spectrum, overcoming many of the limitations associated with discretized solutions. Implementation details are presented and several applications demonstrate the method’s utility and capability. Simple applications highlight the effectiveness of the methodology, while complicated dynamic systems showcase its relevance. A key example considers the challenges associated with libration point formations. Extensions are proposed for broader classes of hybrid systems. / text

No description available.

No description available.