Many everyday human skills can be framed in terms of performing some task subject to constraints imposed by the task or the environment. Constraints are usually unobservable and frequently change between contexts. In this thesis, we explore the problem of learning control policies from data containing variable, dynamic and non-linear constraints on motion. We show that an effective approach for doing this is to learn the unconstrained policy in a way that is consistent with the constraints. We propose several novel algorithms for extracting these policies from movement data, where observations are recorded under different constraints. Furthermore, we show that, by doing so, we are able to learn representations of movement that generalise over constraints and can predict behaviour under new constraints. In our experiments, we test the algorithms on systems of varying size and complexity, and show that the novel approaches give significant improvements in performance compared with standard policy learning approaches that are naive to the effect of constraints. Finally, we illustrate the utility of the approaches for learning from human motion capture data and transferring behaviour to several robotic platforms.

I många industriella tillämpningar används robotar för tunga och repetetiva uppgifter. För dessa tillämpningar är iterative learning control (ILC) ett sätt att fånga upp och utnyttja repeterbarheten för att förbättra någon form av referenseföljning. I det här examensarbetet har det tagits fram ett ramverk som ska hjälpa en användare att kunna untyttja ILC. Det visas handgripliga exempel på hur man enkelt kan avända ramverket. Övergången från den betydligt mer vanliga diskreta ILC algoritmen till det kontinuerliga tillvägagångssättet som anänds av ramverket underlättas av teroretisk  underbygga inställningsregler. Den uppnåeliga prestandan demonstreras med hjälp av ramverkets inbyggda plotfunktioner. / In many industrial applications robots are used for heavy and repetitive tasks. For these applications iterative learning control (ILC) is a way to capture the repetitive nature and use it to improve some kind of reference tracking. In this master thesis a framework has been developed to help a user getting started with ILC. Some hands-on examples are given on how to easily use the framework. The transition from the far more common discrete time domain to the continuous time domain used by the framework is eased by tuning theory. The achievable performance is demonstrated with the help of the built-in plot functions of the framework.

Behavioural cloning is a simple and effective technique for automatically and non-intrusively producing comprehensible and implementable models of human control skill. Behavioural cloning applies machine learning techniques to behavioural trace data, in a transparent manner, and has been very successful in a wide range of domains. The limitations of early behavioural cloning work are: that the clones lack goal-structure, are not robust to variation, are sensitive to the nature of the training data and often produce complicated models of the control skill. Recent behavioural cloning work has sought to address these limitations by adopting goal-structured task decompositions and combining control engineering representations with more sophisticated machine learning algorithms. These approaches have had some success but by compromising either transparency or robustness. This thesis addresses these limitations by investigating: new behavioural cloning representations, control structures, data processing techniques, machine learning algorithms, and performance estimation and testing techniques. First a novel hierarchical decomposition of control is developed, where goal settings and the control skill to achieve them are learnt. This decomposition allows feedback control mechanisms to be combined with modular goal-achievement. Data processing limitations are addressed by developing data-driven, correlative and sampling techniques, that also inform the development of the learning algorithm. The behavioural cloning process is developed by performing experiments on simulated aircraft piloting tasks, and then the generality of the process is tested by performing experiments on simulated gantry-crane control tasks. The performance of the behavioural cloning process was compared to existing techniques, and demonstrated a marked improvement over these methods. The system is capable of handling novel goal-settings and task structure, under high noise conditions. The ability to produce successful controllers was greatly improved by using the developed control representation, data processing and learning techniques. The models produced are compact but tend to abstract the originating control behaviour. In conclusion, the control representation and cloning process address current limitations of behavioural cloning, and produce reliable, reusable and readable clones.

Behavioural cloning is a simple and effective technique for automatically and non-intrusively producing comprehensible and implementable models of human control skill. Behavioural cloning applies machine learning techniques to behavioural trace data, in a transparent manner, and has been very successful in a wide range of domains. The limitations of early behavioural cloning work are: that the clones lack goal-structure, are not robust to variation, are sensitive to the nature of the training data and often produce complicated models of the control skill. Recent behavioural cloning work has sought to address these limitations by adopting goal-structured task decompositions and combining control engineering representations with more sophisticated machine learning algorithms. These approaches have had some success but by compromising either transparency or robustness. This thesis addresses these limitations by investigating: new behavioural cloning representations, control structures, data processing techniques, machine learning algorithms, and performance estimation and testing techniques. First a novel hierarchical decomposition of control is developed, where goal settings and the control skill to achieve them are learnt. This decomposition allows feedback control mechanisms to be combined with modular goal-achievement. Data processing limitations are addressed by developing data-driven, correlative and sampling techniques, that also inform the development of the learning algorithm. The behavioural cloning process is developed by performing experiments on simulated aircraft piloting tasks, and then the generality of the process is tested by performing experiments on simulated gantry-crane control tasks. The performance of the behavioural cloning process was compared to existing techniques, and demonstrated a marked improvement over these methods. The system is capable of handling novel goal-settings and task structure, under high noise conditions. The ability to produce successful controllers was greatly improved by using the developed control representation, data processing and learning techniques. The models produced are compact but tend to abstract the originating control behaviour. In conclusion, the control representation and cloning process address current limitations of behavioural cloning, and produce reliable, reusable and readable clones.

In many  applications industrial robots perform the same motion  repeatedly. One way of compensating the repetitive part of the error  is by using iterative learning control (ILC). The ILC algorithm  makes use of the measured errors and iteratively calculates a  correction signal that is applied to the system. The main topic of the thesis is to apply an ILC algorithm to a  dynamic system where the controlled variable is not measured. A  remedy for handling this difficulty is to use additional sensors in  combination with signal processing algorithms to obtain estimates of  the controlled variable. A framework for analysis of ILC algorithms  is proposed for the situation when an ILC algorithm uses an estimate  of the controlled variable. This is a relevant research problem in  for example industrial robot applications, where normally only the  motor angular positions are measured while the control objective is  to follow a desired tool path. Additionally, the dynamic model of  the flexible robot structure suffers from uncertainties. The  behaviour when a system having these difficulties is controlled by  an ILC algorithm using measured variables directly is illustrated  experimentally, on both a serial and a parallel robot, and in  simulations of a flexible two-mass model. It is shown that the  correction of the tool-position error is limited by the accuracy of  the robot model. The benefits of estimation-based ILC is illustrated for cases when  fusing measurements of the robot motor angular positions with  measurements from an additional accelerometer mounted on the robot  tool to form a tool-position estimate. Estimation-based ILC is  studied in simulations on a flexible two-mass model and on a  flexible nonlinear two-link robot model, as well as in experiments  on a parallel robot. The results show that it is possible to improve  the tool performance when a tool-position estimate is used in the  ILC algorithm, compared to when the original measurements available  are used directly in the algorithm. Furthermore, the resulting  performance relies on the quality of the estimate, as expected. In the last part of the thesis, some implementation aspects of ILC  are discussed. Since the ILC algorithm involves filtering of signals  over finite-time intervals, often using non-causal filters, it is  important that the boundary effects of the filtering operations are  appropriately handled when implementing the algorithm. It is  illustrated by theoretical analysis and in simulations that the  method of implementation can have large influence over stability and  convergence properties of the algorithm. / Denna avhandling behandlar reglering genom iterativ inlärning, ILC  (från engelskans iterative learning control). Metoden har sitt  ursprung i industrirobottillämpningar där en robot utför samma  rörelse om och om igen. Ett sätt att kompensera för felen är genom  en ILC-algoritm som beräknar en korrektionssignal, som läggs på  systemet i nästa iteration. ILC-algoritmen kan ses som ett  komplement till det befintliga styrsystemet för att förbättra  prestanda. Det problem som särskilt studeras är då en ILC-algoritm appliceras  på ett dynamiskt system där reglerstorheten inte mäts. Ett sätt att  hantera dessa svårigheter är att använda ytterligare sensorer i  kombination med signalbehandlingsalgoritmer för att beräkna en  skattning av reglerstorheten som kan användas i ILC-algoritmen. Ett  ramverk för analys av skattningsbaserad ILC föreslås i avhandlingen.  Problemet är relevant och motiveras utifrån experiment på både en  seriell och en parallel robot. I konventionella robotstyrsystem  mäts endast de enskilda motorpositionerna, medan verktygspositionen  ska följa en önskad bana. Experimentresultat visar att en  ILC-algoritm baserad på motorpositionsfelen kan reducera dessa fel  effektivt. Dock behöver detta inte betyda en förbättrad  verktygsposition, eftersom robotmotorerna styrs mot felaktiga värden  på grund av att modellerna som används för att beräkna dessa  referensbanor inte beskriver den verkliga robotdynamiken helt. Skattningsbaserad ILC studeras både i simulering av en flexibel  tvåmassemodell och en olinjär robotmodell med flexibla leder, och i  experiment på en parallell robot. I studierna sammanvägs  motorpositionsmätningar med mätningar från en accelerometer på  robotverktyget till en skattning av verktygspositionen som används i  ILC-algoritmen. Resultaten visar att det är möjligt att förbättra  verktygspositionen med skattningsbaserad ILC, jämfört med när  motorpositionsmätningarna används direkt i  ILC-algoritmen. Resultatet beror också på skattningskvaliteten, som  förväntat. Slutligen diskuteras några implementeringsaspekter. Alla värden i  uppdateringssignalen läggs på systemet samtidigt, vilket gör det  möjligt att använda icke-kausal filtering där man utnyttjar framtida  signalvärden i filteringen. Detta gör att det är viktigt hur  randeffekterna (början och slutet av signalen) hanteras när man  implementerar ILC-algoritmen. Genom teoretisk analys och  simuleringsexempel illustreras att implementeringsmetoden kan ha  stor betydelse för egenskaperna hos ILC-algoritmen.

Abstract In this thesis a set of new algorithms is introduced for Iterative Learning Control (ILC) and Repetitive Control (RC). Both areas of study are relatively new in control theory, and the common denominator for them is that they concentrate on controlling systems that include either reference signals or disturbances which are periodic. This provides opportunities for using past information or experience so that the control system learns the control action that results in good performance in terms of reference tracking or disturbance rejection. The first major contribution of the thesis is the algebraic analysis of ILC systems. This analysis shows that in the discrete-time case ILC algorithm design can be considered as designing a multivariable controller for a multivariable static plant and the reference signal that has to be tracked is a multivariable step function. Furthermore, the algebraic analysis reveals that time-varying algorithms should be used instead of time-invariant ones in order to guarantee monotonic convergence of the error in norm. However, from the algebraic analysis it is not clear how to select the free parameters of a given ILC algorithm. Hence in this thesis optimisation methods are used to automate this design phase. Special emphasis is placed on the so called Norm-Optimal Iterative Learning Control (NOILC) that was originally developed in (Amann:1996) as a new result it is shown that a convex modification of the existing predictive algorithm will result in a considerable improvement in convergence speed. Because the NOILC algorithm is computationally quite complex, a new set of Parameter-Optimal ILC algorithms are derived that converge under certain assumptions on the original plant. Three of these new algorithms will result in monotonic convergence to zero tracking error for an arbitrary discrete-time, linear, time-invariant plant. This a very strong property that has been earlier reported for only a small number of ILC algorithms. In the RC case it is shown that an existing RC algorithm that has been widely analysed and used in the research literature is in fact highly unrobust if the algorithm is implemented using sampled-data processing. Consequently, in this thesis a new optimality based discrete-time RC algorithm is derived, which converges to zero tracking error asymptotically for an arbitrary linear, time-invariant discrete-time plant under mild controllability and observability conditions.

The focus of Location-Aware Adaptive Vehicle Dynamics System (LAAVDS) research is to develop a system to avoid situations in which the vehicle exceeds its handling capabilities. The proposed method is predictive, estimating the ability of the vehicle to successfully navigate upcoming terrain, and it is assumed that the future vehicle states and local driving environment is known. An Intervention Strategy must be developed such that the vehicle is navigated successfully along a road via modest changes to the driver's commands and do so in a manner that is in harmony with the driver's intentions and not in a distracting or irritating manner. Clearly this research relies on the numerous new technologies being developed to capture and convey information about the local driving environment (e.g., bank angle, elevation changes, curvature, and friction coefficient) to the vehicle and driver. / Ph. D.

No / In this paper, we investigate trajectory tracking in a multi-input nonlinear system, where there is little knowledge of the system parameters and the form of the nonlinear function. An identification-based iterative learning control (ILC) scheme to repetitively estimate the linearity in a neighborhood of a desired trajectory is presented. Based on this estimation, the original nonlinear system can track the desired trajectory perfectly by the aid of a regional training scheme. Just like in adaptive control, a singularity exists in ILC when the input coupling matrix is estimated. Singularity avoidance is discussed. A new parameter modification procedure for ILC is presented such that the determinant of the estimate of the input coupling matrix is uniformly bounded from below. Compared with the scheme used for adaptive control of a MIMO system, the proposed scheme reduces the computation load greatly. It is used in a robotic visual system for manipulator trajectory tracking without any information about the camera-robot relationship. The estimated image Jacobian is updated repetitively and then its inverse is used to calculate the manipulator velocity without any singularity.

Submitted by PPG Engenharia El?trica (engenharia.pg.eletrica@pucrs.br) on 2018-03-26T18:49:00Z No. of bitstreams: 1 MATHEUS_SENNA_OLIVEIRA_DIS.pdf: 2367932 bytes, checksum: 927039b4746ebdc5d7da25318435b24a (MD5) / Approved for entry into archive by Tatiana Lopes (tatiana.lopes@pucrs.br) on 2018-04-06T17:26:05Z (GMT) No. of bitstreams: 1 MATHEUS_SENNA_OLIVEIRA_DIS.pdf: 2367932 bytes, checksum: 927039b4746ebdc5d7da25318435b24a (MD5) / Made available in DSpace on 2018-04-06T17:38:56Z (GMT). No. of bitstreams: 1 MATHEUS_SENNA_OLIVEIRA_DIS.pdf: 2367932 bytes, checksum: 927039b4746ebdc5d7da25318435b24a (MD5) Previous issue date: 2018-01-31 / Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior - CAPES / Este documento apresenta um trabalho de disserta??o sobre o estudo de estrat?gias de controle para o seguimento eficiente de padr?es espirais. Estes padr?es podem ser aplicados em muitas ?reas, como por exemplo, a Microscopia de For?a At?mica, onde padr?es de referenciais r?pidos e suaves s?o requeridos. Para realizar com sucesso o seguimento destas refer?ncias, que s?o compostas de sinais senoidais de amplitude e frequ?ncia vari?vel, estrat?gias de controle avan?adas foram investigadas. O Princ?pio do Modelo Interno ? uma abordagem tradicional para o seguimento de sinais, mas ele n?o pode ser aplicado diretamente em sinais com frequ?ncia variante. Logo, o presente trabalho prop?s uma estrat?gia de controle robusto onde o Princ?pio do Modelo Interno foi aplicado como um Controlador Ressonante em uma estrutura aumentada e variante no tempo. O sistema aumentado e os valores da frequ?ncia foram organizados usando uma representa??o polit?pica e estruturados como um problema de otimiza??o sujeito a restri??es na forma de Desigualdades Matriciais Lineares. Esta s?ntese foi avaliada atrav?s de um conjunto de simula??es, usando um modelo num?rico de um Microsc?pio de For?a At?mica e um novo padr?o de refer?ncia para escaneamento apropriado. Al?m disso, usando a premissa que estes sinais de refer?ncia s?o aplicados m?ltiplas vezes, um Controle por Aprendizagem Iterativa tamb?m foi projetado para melhorar o desempenho do seguimento da estrat?gia principal proposta. Resultados num?ricos demonstraram que o controlador projetado atingiu resultados satisfat?rios, em compara??o com o controlador tradicional dispon?vel na ?rea. / This document presents a dissertation work regarding the study of control strategies for the efficient tracking of spiral patterns. Such patterns arise in many areas, as for example the Atomic Force Microscopy, where fast and smooth reference signals are required. In order to successfully track the above mentioned references, which are composed of amplitude and frequency-varying sinusoidal signals, advanced control strategies were investigated. The Internal Model Principle is a traditional approach to track reference signals, but it cannot be directly applied in frequency-varying signals. Therefore, the present work proposed a robust control strategy where the Internal Model Principle was applied as a Resonant Control in an augmented time-varying structure. The augmented system and the reference frequency values were organized using a polytopic representation and structured as an optimization problem subject to constraints in the form of Linear Matrix Inequalities. This synthesis was evaluated through a set of simulations, using a numerical model of an Atomic Force Microscope and a new suitable scanning reference pattern. Moreover, using the premise that the same reference signals are tracked multiple times, an Iterative Learning Controller was also designed in order to improve the tracking performance of the proposed main strategy. Numerical results demonstrated that the designed controller achieved satisfactory results, in comparison to the traditional controller available in the area.

No description available.