No description available.

Object recognition in the real world is a big challenge in the field of computer vision. Given the potentially enormous size of the search space it is essential to be able to make intelligent decisions about where in the visual field to obtain information from to reduce the computational resources needed. In this report a POMDP (Partially Observable Markov Decision Process) learning framework, using a policy gradient method and information rewards as a training signal, has been implemented and used to train fixation policies that aim to maximize the information gathered in each fixation. The purpose of such policies is to make object recognition faster by reducing the number of fixations needed. The trained policies are evaluated by simulation and comparing them with several fixed policies. Finally it is shown that it is possible to use the framework to train policies that outperform the fixed policies for certain observation models.

Safety verification of continuous dynamical systems require the computationof the reachable set. The reachable set comprises those states the systemcan reach at a specific point in time. The present work aims to compute thisreachable set for the marine vessel, in the presence of uncertainties in thedynamic modeling of the system and in the presence of external disturbancesin the form of wind, waves and currents. The reachable set can then be usedto check if the vessel collides with an obstacle. The dynamic model used isthat of a nonlinear maneuvering model for the marine vessel. The dynamicson the azipod actuators are also considered.Several methods are considered to solve the reachability problem for themarine vessel. The first method considered is that of the Hamilton JacobiReachability analysis, where a dynamic game between the control input andthe disturbance input is played. This results in a dynamic programmingproblem known as the Hamilton Jacobi Bellman Isaacs (HJBI) equation. Itis solved using the Level-Set method, but it suffers from the curse of dimensionality.The other method considered is the use of set-theoretic approach,where an over-approximation of the reachable set is computed, in the contextof safety verification. But on the downside, large sets of admissible controlyields highly over-approximated reachable sets, which cannot be usedIn order to overcome the disadvantages posed by the first two methods,emphasizing on the real-time computation, a third method is developed, wherea supervised classification algorithm is used to compute the reachable setboundary. The dataset required for the classification algorithm is computedby solving a 2 Point Boundary Value Optimal Control Problem for the marinevessel. The features for classification algorithm can be extended, so as toinclude the uncertainties and disturbances in the system. The computationtime is greatly reduced and the accuracy of the method is comparable to theexact reachable set computation. / Säkerhetsverifiering av kontinuerliga dynamiska system kräver beräkningav mängden av tillstånd som kan nås vid en specifik tidpunkt, givet dess initialtillstånd.Detta arbete fokuserar påatt bestämma denna mängd av nåbaratillstånd för ett marint fartyg under modellosäkerheter och externa störningari form av vind, vågor och strömmar. Den nåbara mängden av tillstånd användssedan för att kontrollera om fartyget riskerar att kollidera med hinder.Den dynamiska modell som används i våra studier är en icke-linjär modelldär även dynamiken hos azipod-ställdonen betraktas.Arbetet studerar flera metoder för att lösa problemet: en klassisk Hamilton-Jacobi nåbarhetsanalys, en mängd-teoretisk teknik, samt en ny metod baseradpåmaskininlärning. Numeriska simuleringsstudier bekräftar att den föreslagnamaskininlärningsmetoden är snabbare än de tvåalternativen.

Applying reinforcement learning to control systems enables the use of machine learning to develop elegant and efficient control laws. Coupled with the representational power of neural networks, reinforcement learning algorithms can learn complex policies that can be difficult to emulate using traditional control system design approaches. In this thesis, three different model-free reinforcement learning algorithms, including Monte Carlo Control, REINFORCE with baseline, and Guided Policy Search are compared in simulated, continuous action-space environments. The results show that the Guided Policy Search algorithm is able to learn a desired control policy much faster than the other algorithms. In the inverted pendulum system, it learns an effective policy up to three times faster than the other algorithms. In the cartpole system, it learns an effective policy up to nearly fifteen times faster than the other algorithms.

<p> Within recent years, there has been a renewed interest in developing data-driven learning based algorithms for solving longstanding challenging control problems. This interest is primarily motivated by the availability of ubiquitous data and an increase in computational resources of modern machines.  However, there is a prevailing concern on the lack of provable performance guarantees on data-driven/model-free learning based control algorithms. This dissertation focuses the following key aspects: i) with what facility can state-of-the-art learning-based control methods eke out successful performance for challenging flight control applications such as aerobatic maneuvering?; and ii) can we leverage well-established tools and techniques in control theory to provide some provable guarantees for different types of learning-based algorithms?  </p> <p>To these ends, a deep RL-based controller is implemented, via high-fidelity simulations, for Fixed-Wing aerobatic maneuvering. which shows the facility with which learning-control methods can eke out successful performances and further encourages the development of learning-based control algorithms with an eye towards providing provable guarantees.<br> </p> <p>Two learning-based algorithms are also developed: i) a model-free algorithm which learns a stabilizing optimal control policy for the bilinear biquadratic regulator (BBR) which solves the regulator problem with a biquadratic performance index given an unknown bilinear system; and ii) a model-free inverse reinforcement learning algorithm, called the Model-Free Stochastic inverse LQR (iLQR) algorithm, which solves a well-posed semidefinite programming optimization problem to obtain unique solutions on the linear control gain and the parameters of the quadratic performance index given zero-mean noisy optimal trajectories generated by a linear time-invariant dynamical system. Theoretical analysis and numerical results are provided to validate the effectiveness of all proposed algorithms.</p>

There is a considerable scope for improving the temperature control of exothermic processes. In this thesis, a sub-optimal control strategy is developed through utilizing the dynamic, simulation tool. This scheme is built around easily obtained knowledge of the system and still retains flexibility. It can be applied to both exothermic batch and fed-batch processes. It consists of servo and regulatory modes, where a Generalized Predictive Controller (GPC) was used to provide self-tuning facilities. The methods outlined allow for limited thermal runaway whilst keeping some spare cooling capacity to ensure that operation at constraints are not violated. A special feature of the method proposed is that switching temperatures and temperature profiles can be readily found from plant trials whilst the addition rate profile Is capable of fairly straightforward computation. The work shows that It is unnecessary to demand stability for the whole of the exothermic reaction cycle, permitting a small runaway has resulted in a fast temperature response within the given safety margin. The Idea was employed for an exothermic single Irreversible reaction and also to a set of complex reactions. Both are carried out in a vessel with a heating/cooling coil. Two constraints are Imposed; (1) limited heat transfer area, and (11) a maximum allowable reaction temperature Tmax. The non-minimum phase problem can be considered as one of the difficulties in managing exothermic fed-batch process when cold reactant Is added to vessel at the maximum operating temperature. The control system coped with this within limits, a not unexpected result. In all cases, the new strategy out-performed the conventional controller and produced smoother variations in the manipulated variable. The simulation results showed that batch to batch variations and disturbances In cooling were successfully handled. GPC worked well but can be susceptible to measurement noise. / Higher Education Ministry and Scientific Research

Though multirotor unmanned aerial vehicles (UAVs) have become widely used during the past decade, challenges in autonomy have prevented their widespread use when moving vehicles act as their base stations. Emerging use cases, including maritime surveillance, package delivery and convoy support, require UAVs to autonomously operate in this scenario. This thesis presents improved solutions to both the state estimation and control problems that must be solved to enable robust, autonomous landing of multirotor UAVs onto moving vehicles.Current state-of-the-art UAV landing systems depend on the detection of visual fiducial markers placed on the landing target vehicle. However, in challenging conditions, such as poor lighting, occlusion, or extreme motion, these fiducial markers may be undected for significant periods of time. This thesis demonstrates a state estimation algorithm that tracks and estimates the locations of unknown visual features on the target vehicle. Experimental results show that this method significantly improves the estimation of the state of the target vehicle while the fiducial marker is not detected.This thesis also describes an improved control scheme that enables a multirotor UAV to accurately track a time-dependent trajectory. Rooted in Lie theory, this controller computes the optimal control signal based on an error-state formulation of the UAV dynamics. Simulation and hardware experiments of this control scheme show its accuracy and computational efficiency, making it a viable solution for use in a robust landing system.

Water heating in a typical home in the U.S. accounts for a significant portion (between 14% and 25%) of the total home's annual energy consumption. The objective of considerably reducing the home's energy consumption from the utilities calls for the use of onsite renewable energy systems. Integral Collector Storage (ICS) solar domestic water heating systems are an alternative to help meet the hot water energy demands in a household. In order to evaluate the potential benefits and contributions from the ICS system, it is important that the parameter values included in the model used to estimate the system's performance are as accurate as possible. The overall heat loss coefficient (Uloss) in the model plays an important role in the performance prediction methodology of the ICS. This work presents a new and improved methodology to determine Uloss as a function of time in an ICS system using a systematic optimal control theoretic approach. This methodology is based on the derivation of a new nonlinear state space model of the system, and the formulation of a quadratic performance function whose minimization yields estimates of Uloss values that can be used in computer simulations to improve the performance prediction of the ICS system, depending on the desired time of the year and hot water draw profile. Simulation results show that predictions of the system's performance based on these estimates of Uloss are considerably more accurate than the predictions based on current existing methods for estimating Uloss.

No description available.

No description available.