Keepaway is a simpler subtask of robot soccer where three `keepers' attempt to keep possession of the ball while a `taker' tries to steal it from them. This is a less complex task than full robot soccer, and lends itself well as a testbed for multi-agent systems. This thesis does a comprehensive evaluation of various learning methods using neuroevolution with Enforced Sub-Populations (ESP) with the robocup soccer simulator. Both single and multi-component ESP are evaluated using various learning methods on homogeneous and heterogeneous teams of agents. In particular, the effectiveness of modularity and task decomposition for evolving keepaway teams is evaluated. It is shown that in the robocup soccer simulator, homogeneous agents controlled by monolithic networks perform the best. More complex learning approaches like layered learning, concurrent layered learning and co-evolution decrease the performance as does making the agents heterogeneous. The results are also compared with previous results in the keepaway domain. / text

This thesis presents a new design process in which design specifications and task distribution are determined from a parallel multi user prospective. Using this method, projects are more easily decomposed into tasks that can be performed concurrently, thus decreasing the design time. Also, a framework is provided to determine the correct distribution of available talent and stakeholders that can be utilized on a given project. The research suggests that by involving the necessary stakeholders in a multi user setting, changes can be made quickly and without additional approval wait time. By including individuals from the various areas of required talent, persons of expertise will be able to work together in a mode of shared design rather than an iterative design process. Decreasing iterations as well as reducing wait time for approval will reduce the overall design time significantly. This method has been tested and validated utilizing controlled tests simulating real life situations of much larger scale. The validation results show that the new method does in fact improve design time and overall achievement of initial design goals

A distributed reinforcement-learning system is designed and implemented on a mobile robot for the study of complex task decomposition in real robot learning environments. The Distributed Q-learning Classifier System (DQLCS) is evolved from the standard Learning Classifier System (LCS) proposed by J.H. Holland. Two of the limitations of the standard LCS are its monolithic nature and its complex apportionment of credit scheme, the bucket brigade algorithm (BBA). The DQLCS addresses both of these problems as well as the inherent difficulties faced by learning systems operating in real environments. We introduce Q-learning as the apportionment of credit component of the DQLCS, and we develop a distributed learning architecture to facilitate complex task decomposition. Based upon dynamic programming, the Q-learning update equation is derived and its advantages over the complex BBA are discussed. The distributed architecture is implemented to provide for faster learning by allowing the system to effectively decrease the size of the problem space it must explore. Holistic and monolithic shaping approaches are used to distribute reward among the learning modules of the DQLCS in a variety of real robot learning experiments. The results of these experiments support the DQLCS as a useful reinforcement learning paradigm and suggest future areas of study in distributed learning systems. / Master of Science

Submitted by Fabio Sobreira Campos da Costa (fabio.sobreira@ufpe.br) on 2017-04-05T15:39:48Z No. of bitstreams: 2 license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) everson_verissimo_dissertacao.pdf: 2502424 bytes, checksum: 0f98e5de2dc7eab2b63e0c0ccd1a6703 (MD5) / Made available in DSpace on 2017-04-05T15:39:48Z (GMT). No. of bitstreams: 2 license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) everson_verissimo_dissertacao.pdf: 2502424 bytes, checksum: 0f98e5de2dc7eab2b63e0c0ccd1a6703 (MD5) Previous issue date: 2013-08-14 / CAPES / Apesar dos avanços em técnicas da Aprendizagem de Máquina, muito esforço ainda é despendido na concepção de um classificador que consiga aprender bem uma dada tarefa. Váriasabordagenssurgiramparaatenuaresseesforçoatravésdacombinaçãodeclassificadores. A combinação de classificadores permite que o projetista do sistema não necessite escolher o classificador mais eficiente dentre vários, nem descartar classificadores que podem possuir informaçãoimportantesobreatarefa. Estratégiasdecombinaçãopermitemqueváriosalgoritmos trabalhem em conjunto a fim de melhorar a precisão de todo o sistema numa dada tarefa. O objetivodestetrabalhoéproporummétododecombinaçãodeclassificadoresqueagregueas vantagensdeduasabordagens: máquinasdecomitêemisturasdeespecialistas. Asmáquinasde comitêvisamcombinarclassificadoresqueresolvempadrõesdetodooespaçodecaracterísticas. Quandocombinados,lidammelhorcomsuperfíciesdedecisãocomplexasqueumclassificador individualmente e são capazes de incorporar novos classificadores mesmo após o uso. Nas MisturasdeEspecialistas,cadaumdosclassificadoreséumespecialistaemumadeterminada áreadoespaçodecaracterísticaseemboraresolvapadrõesdetodooespaçodecaracterísticas,se dedicaaresolverproblemasbemmaissimples,atingindoumdesempenhosuperioremrelaçãoa umclassificadorsópararesolveroproblematodo. OmétodopropostoéchamadodeComitê de Misturas de Especialistas e corresponde a uma máquina de comitês formada por misturas de especialistas. Assim, o método herda a escalabilidade e a tolerância a erros das máquinas decomitêeasimplicidadedetreinamentodasmisturasdeespecialistas. Experimentosforam realizadosparaverificarasuperioridadedocomitêdemisturasdeespecialistassobretrêsfatores de mudanças entre as misturas: técnicas de decomposição de tarefas, número de grupos e características. / Despite the advance of the techniques in Machine Learning, much effort is taken to conceiveaclassifierthatlearnswellaparticulartask. Severalapproacheshavebeenproposed to attenuate this effort through combination of classifiers. Combination of classifiers allows thatnotonlythemosteffectiveclassifiersbechosenamongseveral,nordiscardclassifiersthat mayhaveimportantinformationaboutthetask. Strategiesallowthatseveralalgorithmswork togetherinordertoimproveaccuracyofthewholesystemgivenatask. Thegoalofthiswork is to propose a method to combine classifiers that put together advantages of two approaches: committeemachinesandmixtureofexperts. CommitteeMachinesaimtocombineclassifiersthat solvepatternsfromalloverthespace. Whencombined,theydealbetterwithcomplexdecision boundaries than a single classifier and they are capable of incorporating new classifiers even aftertheuse. Inthemixtureofexperts,eachoneoftheclassifiersisanexpertinacertainregion ofthefeaturespaceand,althoughitsolvespatternsfromthewholefeaturespace,theclassifier is dedicated to solve well simpler problems, reaching a better performance in comparison to a unique classifier to solve the entire problem. Also, there is a hybrid approach, the mixture of experts, in which each classifier solves patterns from the entire space as a committe, but it is trained with patterns from a smaller region, similarly to modular neural networks. The proposedmethodisentitledCommitteeofMixtureofExpertsandcorrespondstoacommittee machineformedbymixtureofexperts. So,themethodinheritsscalabilityanderrortolerance from committee machines and training simplicity from the mixture of experts. Experiments weremadetoverifythesuperiorityofthecommitteeofmixturesofexpertsoverthreefactorsof changingamongthemixtures: taskdecompositionmethods,numberofgroupsandfeatures.

Cette thèse considère l'apprentissage de motifs récurrents dans la perception multimodale. Elle s'attache à développer des modèles robotiques de ces facultés telles qu'observées chez l'enfant, et elle s'inscrit en cela dans le domaine de la robotique développementale.Elle s'articule plus précisément autour de deux thèmes principaux qui sont d'une part la capacité d'enfants ou de robots à imiter et à comprendre le comportement d'humains, et d'autre part l'acquisition du langage. A leur intersection, nous examinons la question de la découverte par un agent en développement d'un répertoire de motifs primitifs dans son flux perceptuel. Nous spécifions ce problème et établissons son lien avec ceux de l'indétermination de la traduction décrit par Quine et de la séparation aveugle de source tels qu'étudiés en acoustique.Nous en étudions successivement quatre sous-problèmes et formulons une définition expérimentale de chacun. Des modèles d'agents résolvant ces problèmes sont également décrits et testés. Ils s'appuient particulièrement sur des techniques dites de sacs de mots, de factorisation de matrices et d'apprentissage par renforcement inverse. Nous approfondissons séparément les trois problèmes de l'apprentissage de sons élémentaires tels les phonèmes ou les mots, de mouvements basiques de danse et d'objectifs primaires composant des tâches motrices complexes. Pour finir nous étudions le problème de l'apprentissage d'éléments primitifs multimodaux, ce qui revient à résoudre simultanément plusieurs des problèmes précédents. Nous expliquons notamment en quoi cela fournit un modèle de l'ancrage de mots acoustiques / This thesis focuses on learning recurring patterns in multimodal perception. For that purpose it develops cognitive systems that model the mechanisms providing such capabilities to infants; a methodology that fits into thefield of developmental robotics.More precisely, this thesis revolves around two main topics that are, on the one hand the ability of infants or robots to imitate and understand human behaviors, and on the other the acquisition of language. At the crossing of these topics, we study the question of the how a developmental cognitive agent can discover a dictionary of primitive patterns from its multimodal perceptual flow. We specify this problem and formulate its links with Quine's indetermination of translation and blind source separation, as studied in acoustics.We sequentially study four sub-problems and provide an experimental formulation of each of them. We then describe and test computational models of agents solving these problems. They are particularly based on bag-of-words techniques, matrix factorization algorithms, and inverse reinforcement learning approaches. We first go in depth into the three separate problems of learning primitive sounds, such as phonemes or words, learning primitive dance motions, and learning primitive objective that compose complex tasks. Finally we study the problem of learning multimodal primitive patterns, which corresponds to solve simultaneously several of the aforementioned problems. We also details how the last problems models acoustic words grounding.