A fundamental question in a sequential decision making setting under uncertainty is “how to allocate resources amongst competing entities so as to maximize the rewards accumulated in the long run?”. The resources allocated may be either abstract quantities such as time or concrete quantities such as manpower. The sequential decision making setting involves one or more agents interacting with an environment to procure rewards at every time instant and the goal is to find an optimal policy for choosing actions. Most of these problems involve multiple (infinite) stages and the objective function is usually a long-run performance objective. The problem is further complicated by the uncertainties in the sys-tem, for instance, the stochastic noise and partial observability in a single-agent setting or private information of the agents in a multi-agent setting. The dimensionality of the problem also plays an important role in the solution methodology adopted. Most of the real-world problems involve high-dimensional state and action spaces and an important design aspect of the solution is the choice of knowledge representation. The aim of this thesis is to answer important resource allocation related questions in different real-world application contexts and in the process contribute novel algorithms to the theory as well. The resource allocation algorithms considered include those from stochastic optimization, stochastic control and reinforcement learning. A number of new algorithms are developed as well. The application contexts selected encompass both single and multi-agent systems, abstract and concrete resources and contain high-dimensional state and control spaces. The empirical results from the various studies performed indicate that the algorithms presented here perform significantly better than those previously proposed in the literature. Further, the algorithms presented here are also shown to theoretically converge, hence guaranteeing optimal performance. We now briefly describe the various studies conducted here to investigate problems of resource allocation under uncertainties of different kinds: Vehicular Traffic Control The aim here is to optimize the ‘green time’ resource of the individual lanes in road networks that maximizes a certain long-term performance objective. We develop several reinforcement learning based algorithms for solving this problem. In the infinite horizon discounted Markov decision process setting, a Q-learning based traffic light control (TLC) algorithm that incorporates feature based representations and function approximation to handle large road networks is proposed, see Prashanth and Bhatnagar [2011b]. This TLC algorithm works with coarse information, obtained via graded thresholds, about the congestion level on the lanes of the road network. However, the graded threshold values used in the above Q-learning based TLC algorithm as well as several other graded threshold-based TLC algorithms that we propose, may not be optimal for all traffic conditions. We therefore also develop a new algorithm based on SPSA to tune the associated thresholds to the ‘optimal’ values (Prashanth and Bhatnagar [2012]). Our thresh-old tuning algorithm is online, incremental with proven convergence to the optimal values of thresholds. Further, we also study average cost traffic signal control and develop two novel reinforcement learning based TLC algorithms with function approximation (Prashanth and Bhatnagar [2011c]). Lastly, we also develop a feature adaptation method for ‘optimal’ feature selection (Bhatnagar et al. [2012a]). This algorithm adapts the features in a way as to converge to an optimal set of features, which can then be used in the algorithm. Service Systems The aim here is to optimize the ‘workforce’, the critical resource of any service system. However, adapting the staffing levels to the workloads in such systems is nontrivial as the queue stability and aggregate service level agreement (SLA) constraints have to be complied with. We formulate this problem as a constrained hidden Markov process with a (discrete) worker parameter and propose simultaneous perturbation based simulation optimization algorithms for this purpose. The algorithms include both first order as well as second order methods and incorporate SPSA based gradient estimates in the primal, with dual ascent for the Lagrange multipliers. All the algorithms that we propose are online, incremental and are easy to implement. Further, they involve a certain generalized smooth projection operator, which is essential to project the continuous-valued worker parameter updates obtained from the SASOC algorithms onto the discrete set. We validate our algorithms on five real-life service systems and compare their performance with a state-of-the-art optimization tool-kit OptQuest. Being ��times faster than OptQuest, our scheme is particularly suitable for adaptive labor staffing. Also, we observe that it guarantees convergence and finds better solutions than OptQuest in many cases. Wireless Sensor Networks The aim here is to allocate the ‘sleep time’ (resource) of the individual sensors in an intrusion detection application such that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We model this sleep–wake scheduling problem as a partially-observed Markov decision process (POMDP) and propose novel RL-based algorithms -with both long-run discounted and average cost objectives -for solving this problem. All our algorithms incorporate function approximation and feature-based representations to handle the curse of dimensionality. Further, the feature selection scheme used in each of the proposed algorithms intelligently manages the energy cost and tracking cost factors, which in turn, assists the search for the optimal sleeping policy. The results from the simulation experiments suggest that our proposed algorithms perform better than a recently proposed algorithm from Fuemmeler and Veeravalli [2008], Fuemmeler et al. [2011]. Mechanism Design The setting here is of multiple self-interested agents with limited capacities, attempting to maximize their individual utilities, which often comes at the expense of the group’s utility. The aim of the resource allocator here then is to efficiently allocate the resource (which is being contended for, by the agents) and also maximize the social welfare via the ‘right’ transfer of payments. In other words, the problem is to find an incentive compatible transfer scheme following a socially efficient allocation. We present two novel mechanisms with progressively realistic assumptions about agent types aimed at economic scenarios where agents have limited capacities. For the simplest case where agent types consist of a unit cost of production and a capacity that does not change with time, we provide an enhancement to the static mechanism of Dash et al. [2007] that effectively deters misreport of the capacity type element by an agent to receive an allocation beyond its capacity, which thereby damages other agents. Our model incorporates an agent’s preference to harm other agents through a additive factor in the utility function of an agent and the mechanism we propose achieves strategy proofness by means of a novel penalty scheme. Next, we consider a dynamic setting where agent types evolve and the individual agents here again have a preference to harm others via capacity misreports. We show via a counterexample that the dynamic pivot mechanism of Bergemann and Valimaki [2010] cannot be directly applied in our setting with capacity-limited alim¨agents. We propose an enhancement to the mechanism of Bergemann and V¨alim¨aki [2010] that ensures truth telling w.r.t. capacity type element through a variable penalty scheme (in the spirit of the static mechanism). We show that each of our mechanisms is ex-post incentive compatible, ex-post individually rational, and socially efficient

This work is focused on single phase inverters, which are used for the conversion of the direct current to the alternating current and are nowdays used especially in systems of back-up power supply. The specific aim of this work is implementation of design hight power circuit of inverter include calculation of control algorithm. It describes the complete solution of power circuit. Next step is a analysis of problems concerning the digital control with help of signal processor which is used for solution of regulator structure. Check of the design and checkout of control algorithm is made in the form of simulation in the MATLAB Simulink. Debugged program algorithm is subsequently implemented into the signal microprocessor. The work results rate estimation functionality of inverter and solution of control algorithm.

The master thesis encompasses the possibilities of position evaluation and drive control with the aid of SMI work enviroment. Furthermore the thesis is directed to create a program through a designed control algorithm. The work is realised on the elevator model with electronically commuted motor. An incremental scanner is used as the position sensor. The motor control unit is set up and programmed in the SmartMotorInterface software. In the next part, measurements with altered parameters are conducted. On the basis of these measurements the influence of parameters on the positioning process and its accuracy is evaluated. At the conclusion of the work, a design of laboratory task for educational purposes is created. The laboratory task is composed in such a way, that students would become familiar with the SMI work enviroment and would be able to practicaly test the setup of incremental position sensor and motor control with the aid of entered algorithm.

The aim of the thesis is to improve the control quality of the adaptive systems (Self Tuning Controllers). The thesis mainly deals with problematical identification part of the adaptive system. This part demonstrates a weak point for existing adaptive systems. Paradoxically, the quality of the adaptive system depends mainly on the identification part because on the basis of the process model obtained by identification are worked out parameters of a control part, afterwards the control action plan is established. Knowledge of the modern control methods is used and a new identification algorithm for closed loop identification is proposed. This simple, fast and efficient algorithm overcomes all disadvantages of current classical identification methods based on least mean-square algorithms. The possibility of the choice of a short sample time, one tuning parameter ability to adjust the control process, the ability to identify processes in real use belong to its main goals. This algorithm was built in the adaptive system and then it was tested on a set of simulation and real models with surprisingly excellent results. The successful implementation of the algorithm into the programmable logic controller was also realized. One part of the thesis introduces a new universal graphics environment for testing and verifying control algorithms.

The electrification era has been marked up by an increase in volume of electric vehicles which are directly or indirectly powered by electricity. Railways, roadways and airways are being electrified as we speak at their own respective rate. In addition to that upcoming concepts for transport solution such as hyperloop also described as the fifth mode of transportation will be electrified. The current thesis work is based on developing the model and control of the propulsion system of a scaled Hyperloop pod designed by student team KTH Hyperloop representing KTH. The team competes in Hyperloop competition organized by Spacex and the goal is to achieve the highest possible speed in a given distance and track designed by SpaceX. In order to achieve the goal of being the fastest, the scaled pod uses a Double Sided Linear Induction Motor (DSLIM) as mentioned in the subsequent chapter. The motor modelling is done on Simulink and is similar to a rotary induction motor (RIM). However the presence of end effect in DSLIM makes it different from RIM and has been discussed subsequently. The control strategy uses a synchronous frame PI control for the current control and sensor based speed control for controlling the speed of the pod.The speed control output is a reference current which is used as an input to the current controller which finally gives voltage as the control output. The corresponding bandwidth for the various loops have been calculated based on motor parameters as discussed in the method section. The validation of the motor model and the corresponding controller has been discussed in the result section, where the accuracy of the controller for the designed modelled is discussed. / Elektrifieringstiden har präglats av en ökning i volym av elfordon som direkt eller indirekt drivs med el. Järnvägar, vägar och luftvägar elektrifieras just nu med deras respektive takt. Utöver det kommer kommande koncept för transportlösning som hyperloop som också beskrivs som det femte transportsättet att elektrifieras. Detta examensarbete bygger på att utveckla modellen och regleringen av framdrivningssystemet för en nedskalad Hyperloop-pod utvecklad av studentteamet KTH Hyperloop som representerar KTH. Teamet tävlar i Hyperloop-tävlingen organiserad av SpaceX och målet är att uppnå högsta möjliga hastighet på ett visst avstånd och spår framtaget av SpaceX. För att uppnå målet om att vara snabbast använder den nedskalade podden en dubbelsidig elektrisk linjär induktionsmotor (DSLIM) som nämns i det följande kapitlet. Den elektriska motormodelleringen görs i Simulink och liknar en roterande induktionsmotor(RIM). Men närvaron av ’end effect’ i DSLIM gör den annorlunda än RIM och har diskuterats därefter. Styrstrategin använder en synkron ram-PI-styrning för strömstyrning och sensorbaserad hastighetsreglering för att styra hastigheten på podden. Varvtalsstyrningsutgången är en referensström som används som en ingång till den nuvarande styrenheten som slutligen ger spänning som slutling styrning. Motsvarande bandbredd för de olika slingorna har beräknats baserat på elektriska motorparametrar som diskuterats i metodavsnittet.Valideringen av elmotormodellen och motsvarande styrenhet har diskuterats i resultatsektionen, där noggrannheten hos styrenheten för den konstruerade modellerna diskuteras.

thermal manikin, thermophysiological model, thermal comfort, climatic chamber, clothing thermal resistence