The main objective of the present thesis focuses on the integration of the in-wheel electric motors into the conception and control of road vehicles. The present thesis is the subject of the grant 186-654 (2010-2013) between the Laboratory of Signals and Systems (L2S-CNRS) and the French Institute of Petrol and New Energies (IFPEN). The thesis work has originally started from a vehicular electrification project, equipped with in-wheel electric motors at the rear axle, to obtain a full electric urban use and a standard extra-urban use with energy recovery at the rear axle in braking phases. The standard internal combustion engines have the disadvantage that complex estimation techniques are necessary to compute the instantaneous engine torque. At the same time, the actuators that control the braking system have some delays due to the hydraulic and mechanical circuits. These aspects represent the primary motivation for the introduction and study of the integration of the electric motor as unique propelling source for the vehicle. The advantages brought by the use of the electric motor are revealed and new techniques of control are set up to maximize its novelty. Control laws are constructed starting from the key feature of the electric motor, which is the fact that the torque transmitted at the wheel can be measured, depending on the current that passes through the motor. Another important feature of the electric motor is its response time, the independent control, as well as the fact that it can produce negative torques, in generator mode, to help decelerate the vehicle and store energy at the same time. Therefore, the novelty of the present work is that the in-wheel electric motor is considered to be the only control actuator signal in acceleration and deceleration phases, simplifying the architecture of the design of the vehicle and of the control laws. The control laws are focused on simplicity and rapidity in order to generate the torques which are transmitted at the wheels. To compute the adequate torques, estimation strategies are set up to produce reliable maximum friction estimation. Function of this maximum adherence available at the contact between the road and the tires, an adequate torque will be computed in order to achieve a stable wheel behavior in acceleration as well as in deceleration phases. The critical issue that was studied in this work was the non-linearity of the tire-road interaction characteristics and its complexity to estimate when it varies. The estimation strategy will have to detect all changes in the road-surface adherence and the computed control law should maintain the stability of the wheel even when the maximum friction changes. Perturbations and noise are also treated in order to test the robustness of the proposed estimation and control approaches.

L'objectif principal de cette thèse est l'étude de l'exploitation de systèmes moteurs-roues (machines électriques intégrées à la roue) pour le contrôle de la dynamique véhicule. Cette thèse est issue d'un co-financement (numéro 186-654, 2010-2013) entre le Laboratoire des Signaux et Systèmes (CNRS) et l'Institut Français du Pétrole et Énergies Nouvelles (IFPEN). Les avantages apportés par l'utilisation du moteur électrique sont avérés et de nouvelles techniques de contrôle sont développées pour optimiser son utilisation. Les lois de contrôle basent généralement sur la grandeur principale du moteur électrique: le couple transmis, qui peut être mesuré via le courant consommé. Une autre caractéristique importante du moteur électrique est son temps de réponse, avec le fait qu'il peut produire des couples négatifs, pour ralentir le véhicule, tout en stockant l'énergie. La nouveauté du présent travail est de considérer le moteur-roue électrique comme seul signal de contrôle dans des phases d'accélération et des phases de ralentissement, simplifiant l'architecture de la conception du véhicule et des lois de contrôle. Pour répondre à la demande conducteur tout en préservant un comportement sain du véhicule, des stratégies d'estimation de la limite d'adhérence seront présentées. En fonction de cette adhérence maximale disponible entre la route et les pneus, un couple adéquat sera calculé pour assurer un comportement stable dans des phases d'accélération aussi bien que de freinage. L'aspect critique étudié dans ce travail est la non-linéarité des caractéristiques d'interaction entre la route et le pneu et la complexité de son estimation dans des conditions variables. La stratégie d'estimation devra détecter tous les changements d'adhérence de route et la loi de contrôle calculée devra maintenir la stabilité véhicule même lorsque la friction maximale change. Certaines formes de perturbation et de bruit seront également prises en compte afin de tester la robustesse des approches d'estimation et de contrôle proposés. Parmi les systèmes de sécurité active les plus importants en phase d'accélération, le système de contrôle de traction (TCS) rétablit la traction si les roues commencent à patiner et le programme de stabilité électronique (ESP) intervient pour prévenir une perte menaçante du contrôle latéral du véhicule. Dans le cas du freinage, le système décisif est le système d'antiblocage (ou ABS), qui empêche le blocage des roues. On peut trouver d'autres systèmes embarqués, comme le système de distribution de force de freinage électronique (EBD), qui assure une distribution optimale de la force de freinage transmise aux roues, pour éviter de déraper et assure un ralentissement stable du véhicule. Les systèmes embarqués qui fournissent les estimations doivent être robustes aux bruits de mesure et aux perturbations. A fortiori, ces calculs doivent être faits en temps réel, donc une complexité réduite et une réponse rapide de la loi de contrôle sont nécessaires. Enfin, l'environnement dans lequel le véhicule fonctionne est dynamique, les caractéristiques d'adhérence peuvent varier en fonction de l'état de la route et de la météo. Ainsi, on ne peut prévoir les réactions du conducteur pouvant influencer la réponse globale du véhicule dans des situations d'urgence. Le contrôleur devrait prendre en compte tous ces aspects pour préserver un comportement stable du véhicule. Bien que le contrôle latéral du véhicule présente une importance majeure dans la stabilité globale du véhicule, le présent travail est concentré sur le contrôle longitudinal du véhicule, puisqu'il représente le point de départ de la dynamique véhicule. / The main objective of the present thesis focuses on the integration of the in-wheel electric motors into the conception and control of road vehicles. The present thesis is the subject of the grant 186-654 (2010-2013) between the Laboratory of Signals and Systems (L2S-CNRS) and the French Institute of Petrol and New Energies (IFPEN). The thesis work has originally started from a vehicular electrification project, equipped with in-wheel electric motors at the rear axle, to obtain a full electric urban use and a standard extra-urban use with energy recovery at the rear axle in braking phases. The standard internal combustion engines have the disadvantage that complex estimation techniques are necessary to compute the instantaneous engine torque. At the same time, the actuators that control the braking system have some delays due to the hydraulic and mechanical circuits. These aspects represent the primary motivation for the introduction and study of the integration of the electric motor as unique propelling source for the vehicle. The advantages brought by the use of the electric motor are revealed and new techniques of control are set up to maximize its novelty. Control laws are constructed starting from the key feature of the electric motor, which is the fact that the torque transmitted at the wheel can be measured, depending on the current that passes through the motor. Another important feature of the electric motor is its response time, the independent control, as well as the fact that it can produce negative torques, in generator mode, to help decelerate the vehicle and store energy at the same time. Therefore, the novelty of the present work is that the in-wheel electric motor is considered to be the only control actuator signal in acceleration and deceleration phases, simplifying the architecture of the design of the vehicle and of the control laws. The control laws are focused on simplicity and rapidity in order to generate the torques which are transmitted at the wheels. To compute the adequate torques, estimation strategies are set up to produce reliable maximum friction estimation. Function of this maximum adherence available at the contact between the road and the tires, an adequate torque will be computed in order to achieve a stable wheel behavior in acceleration as well as in deceleration phases. The critical issue that was studied in this work was the non-linearity of the tire-road interaction characteristics and its complexity to estimate when it varies. The estimation strategy will have to detect all changes in the road-surface adherence and the computed control law should maintain the stability of the wheel even when the maximum friction changes. Perturbations and noise are also treated in order to test the robustness of the proposed estimation and control approaches.

A fundamental question in a sequential decision making setting under uncertainty is “how to allocate resources amongst competing entities so as to maximize the rewards accumulated in the long run?”. The resources allocated may be either abstract quantities such as time or concrete quantities such as manpower. The sequential decision making setting involves one or more agents interacting with an environment to procure rewards at every time instant and the goal is to find an optimal policy for choosing actions. Most of these problems involve multiple (infinite) stages and the objective function is usually a long-run performance objective. The problem is further complicated by the uncertainties in the sys-tem, for instance, the stochastic noise and partial observability in a single-agent setting or private information of the agents in a multi-agent setting. The dimensionality of the problem also plays an important role in the solution methodology adopted. Most of the real-world problems involve high-dimensional state and action spaces and an important design aspect of the solution is the choice of knowledge representation. The aim of this thesis is to answer important resource allocation related questions in different real-world application contexts and in the process contribute novel algorithms to the theory as well. The resource allocation algorithms considered include those from stochastic optimization, stochastic control and reinforcement learning. A number of new algorithms are developed as well. The application contexts selected encompass both single and multi-agent systems, abstract and concrete resources and contain high-dimensional state and control spaces. The empirical results from the various studies performed indicate that the algorithms presented here perform significantly better than those previously proposed in the literature. Further, the algorithms presented here are also shown to theoretically converge, hence guaranteeing optimal performance. We now briefly describe the various studies conducted here to investigate problems of resource allocation under uncertainties of different kinds: Vehicular Traffic Control The aim here is to optimize the ‘green time’ resource of the individual lanes in road networks that maximizes a certain long-term performance objective. We develop several reinforcement learning based algorithms for solving this problem. In the infinite horizon discounted Markov decision process setting, a Q-learning based traffic light control (TLC) algorithm that incorporates feature based representations and function approximation to handle large road networks is proposed, see Prashanth and Bhatnagar [2011b]. This TLC algorithm works with coarse information, obtained via graded thresholds, about the congestion level on the lanes of the road network. However, the graded threshold values used in the above Q-learning based TLC algorithm as well as several other graded threshold-based TLC algorithms that we propose, may not be optimal for all traffic conditions. We therefore also develop a new algorithm based on SPSA to tune the associated thresholds to the ‘optimal’ values (Prashanth and Bhatnagar [2012]). Our thresh-old tuning algorithm is online, incremental with proven convergence to the optimal values of thresholds. Further, we also study average cost traffic signal control and develop two novel reinforcement learning based TLC algorithms with function approximation (Prashanth and Bhatnagar [2011c]). Lastly, we also develop a feature adaptation method for ‘optimal’ feature selection (Bhatnagar et al. [2012a]). This algorithm adapts the features in a way as to converge to an optimal set of features, which can then be used in the algorithm. Service Systems The aim here is to optimize the ‘workforce’, the critical resource of any service system. However, adapting the staffing levels to the workloads in such systems is nontrivial as the queue stability and aggregate service level agreement (SLA) constraints have to be complied with. We formulate this problem as a constrained hidden Markov process with a (discrete) worker parameter and propose simultaneous perturbation based simulation optimization algorithms for this purpose. The algorithms include both first order as well as second order methods and incorporate SPSA based gradient estimates in the primal, with dual ascent for the Lagrange multipliers. All the algorithms that we propose are online, incremental and are easy to implement. Further, they involve a certain generalized smooth projection operator, which is essential to project the continuous-valued worker parameter updates obtained from the SASOC algorithms onto the discrete set. We validate our algorithms on five real-life service systems and compare their performance with a state-of-the-art optimization tool-kit OptQuest. Being ��times faster than OptQuest, our scheme is particularly suitable for adaptive labor staffing. Also, we observe that it guarantees convergence and finds better solutions than OptQuest in many cases. Wireless Sensor Networks The aim here is to allocate the ‘sleep time’ (resource) of the individual sensors in an intrusion detection application such that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We model this sleep–wake scheduling problem as a partially-observed Markov decision process (POMDP) and propose novel RL-based algorithms -with both long-run discounted and average cost objectives -for solving this problem. All our algorithms incorporate function approximation and feature-based representations to handle the curse of dimensionality. Further, the feature selection scheme used in each of the proposed algorithms intelligently manages the energy cost and tracking cost factors, which in turn, assists the search for the optimal sleeping policy. The results from the simulation experiments suggest that our proposed algorithms perform better than a recently proposed algorithm from Fuemmeler and Veeravalli [2008], Fuemmeler et al. [2011]. Mechanism Design The setting here is of multiple self-interested agents with limited capacities, attempting to maximize their individual utilities, which often comes at the expense of the group’s utility. The aim of the resource allocator here then is to efficiently allocate the resource (which is being contended for, by the agents) and also maximize the social welfare via the ‘right’ transfer of payments. In other words, the problem is to find an incentive compatible transfer scheme following a socially efficient allocation. We present two novel mechanisms with progressively realistic assumptions about agent types aimed at economic scenarios where agents have limited capacities. For the simplest case where agent types consist of a unit cost of production and a capacity that does not change with time, we provide an enhancement to the static mechanism of Dash et al. [2007] that effectively deters misreport of the capacity type element by an agent to receive an allocation beyond its capacity, which thereby damages other agents. Our model incorporates an agent’s preference to harm other agents through a additive factor in the utility function of an agent and the mechanism we propose achieves strategy proofness by means of a novel penalty scheme. Next, we consider a dynamic setting where agent types evolve and the individual agents here again have a preference to harm others via capacity misreports. We show via a counterexample that the dynamic pivot mechanism of Bergemann and Valimaki [2010] cannot be directly applied in our setting with capacity-limited alim¨agents. We propose an enhancement to the mechanism of Bergemann and V¨alim¨aki [2010] that ensures truth telling w.r.t. capacity type element through a variable penalty scheme (in the spirit of the static mechanism). We show that each of our mechanisms is ex-post incentive compatible, ex-post individually rational, and socially efficient