Developing artificial agents able to autonomously discover new goals, to select them and learn the related skills is an important challenge for robotics. This becomes even crucial if we want robots to interact with real environments where they have to face many unpredictable problems and where it is not clear which skills will be the more suitable to solve them. The ability to learn and store multiple skills in order to use them when required is one of the main characteristics of biological agents: forming ample repertoires of actions is important to widen the possibility for an agent to better adapt to different environments and to improve its chance of survival and reproduction. Moreover, humans and other mammals explore the environment and learn new skills not only on the basis of reward-related stimuli but also on the basis of novel or unexpected neutral stimuli. The mechanisms related to this kind of learning processes have been studied under the heading of “Intrinsic Motivations” (IMs), and in the last decades the concept of IMs have been used in developmental and autonomous robotics to foster an artificial curiosity that can improve the autonomy and versatility of artificial agents. In the research presented in this thesis I focus on the development of open-ended learning robots able to autonomously discover interesting events in the environment and autonomously learn the skills necessary to reproduce those events. In particular, this research focuses on the role that IMs can play in fostering those processes and in improving the autonomy and versatility of artificial agents. Taking inspiration from recent and past research in this field, I tackle some of the interesting open challenges related to IMs and to the implementation of intrinsically motivated robots. I first focus on the neurophysiology underlying IM learning signals, and in particular on the relations between IMs and phasic dopamine (DA). With the support of a first computational model, I propose a new hypothesis that addresses the dispute over the nature and the functions of phasic DA activations: reconciling two contrasting theories in the literature and taking xi into account the different experimental data, I suggest that phasic DA can be considered as a reinforcement prediction error learning signal determined by both unexpected changes in the environment (temporary, intrinsic reinforcements) and biological rewards (permanent, extrinsic reinforcements). The results obtained with my computational model support the presented hypothesis, showing how such a learning signal can serve two important functions: driving both the discovery and acquisition of novel actions and the maximisation of rewards. Moreover, those results provide a first example of the power of IMs to guide artificial agents in the cumulative learning of complex behaviours that would not be learnt simply providing a direct reward for the final tasks. In a second work, I move to investigate the issues related to the implementation of IMs signal in robots. Since the literature still lacks a specific analysis of which is the best IM signal to drive skill acquisition, I compare in a robotic setup different typologies of IMs, as well as the different mechanisms used to implement them. The results provide two important contributions: 1) they show how IM signals based on the competence of the system are able to generate a better guidance for skill acquisition with respect to the signals based on the knowledge of the agent; 2) they identify a proper mechanism to generate a competence-based IM signal, showing that the stronger the link between the IM signal and the competence of the system, the better the performance. Following the aim of widening the autonomy and the versatility of artificial agents, in a third work I focus on the improvement of the control architecture of the robot. I build a new 3-level architecture that allows the system to select the goals to pursue, to search for the best way to achieve them, and acquire the related skills. I implement this architecture in a simulated iCub robot and test it in a 3D experimental scenario where the agent has to learn, on the basis of IMs, a reaching task where it is not clear which arm of the robot is the most suitable to reach the different targets. The performance of the system is compared to the one of my previous 2-level architecture system, where tasks and computational resources are associated at design time. The better performance of the system endowed with the new 3-level architecture highlights the importance of developing robots with different levels of autonomy, and in particular both the high-level of goal selection and the low-level of motor control. Finally, I focus on a crucial issue for autonomous robotics: the development of a system that is able not only to select its own goals, but also to discover them through the interaction with the environment. In the last work I present GRAIL, a Goal-discovering Robotic Architecture for Intrisically-motivated Learning. Building on the insights provided by my previous research, GRAIL is a 4-level hierarchical architecture that for the first time assembles in unique system different features necessary for the development of truly autonomous robots. GRAIL is able to autonomously 1) discover new goals, 2) create and store representations of the events associated to those goals, 3) select the goal to pursue, 4) select the computational resources to learn to achieve the desired goal, and 5) self-generate its own learning signals on the basis of the achievement of the selected goals. I implement GRAIL in a simulated iCub and test it in three different 3D experimental setup, comparing its performance to my previous systems, showing its capacity to generate new goals in unknown scenarios, and testing its ability to cope with stochastic environments. The experiments highlight on the one hand the importance of an appropriate hierarchical architecture for supporting the development of autonomous robots, and on the other hand how IMs (together with goals) can play a crucial role in the autonomous learning of multiple skills.

Apprendre de nouvelles connaissances et savoir-faire sensorimoteurs dans des environnements réels entraine un grand nombre de défis majeurs pour les robots d'aujourd'hui. Pour acquérir de nouveaux comportements, ceux-ci ont besoin d'explorer des espaces sensorimoteurs qui possèdent généralement les caractéristiques d'être de grande dimensionnalité, de grands volumes, redondants, et de comporter des zones de complexités différentes. Dans cette thèse qui entre dans le cadre de la robotique développementale, nous proposons différents processus permettant de guider et contraindre une acquisition autonome de comportements sensorimoteurs nouveaux dans de tels espaces. Nous proposons une approche unifiée de résolution de ces problèmes qui prend inspiration des phénomènes de contraintes développementales présentés en biologie et psychologie, et plus particulièrement des motivations intrinsèques et des contraintes maturationnelles. Après la formalisation de cadres computationnels basés sur ces notions, nous présentons trois architectures algorithmiques différentes, chacune réutilisée de manière intégrée dans la suivante:La première, appelée RIAC, pour Robust-Intelligent Adaptive Curiosity, correspond à l'implémentation d'un algorithme d'apprentissage actif développemental permettant d'orienter l'exploration dans des espaces bornés et de dimensionnalité connue, possédant des régions de différents niveaux de complexités. Ce système, qui utilise des heuristiques prenant inspiration des mécanismes de motivations intrinsèques basées sur les connaissances, permet de diriger efficacement une exploration progressive de nouvelles connaissances sensorimotrices, qui correspondent à l'apprentissage de modèles directs. Il entraine aussi l'émergence de trajectoires développementales auto-organisées relatives à l'orientation de l'exploration sensorimotrice vers des activités de complexités intermédiaires.Ensuite, nous proposons l'algorithme SAGG-RIAC, pour Self-Adaptive Goal Generation - RIAC, en tant que mécanisme d'exploration intrinsèquement motivée basée sur les compétences, qui permet à des robots dont les espaces sensorimoteurs sont de grandes dimensions, hautement redondants, et possédant des schémas corporels différents, d'apprendre efficacement et activement de nouveaux comportements moteurs dans leurs espaces de tâches. L'idée principale de cet algorithme est d'orienter le robot à effectuer un babillage actif dans un espace des tâches de faible dimensionnalité, en opposition à un babillage moteur effectué dans un espace de contrôle de plus grande dimension, en auto-générant activement et adaptivement des objectifs dans les régions de l'espace des tâches qui fournissent les meilleures améliorations de compétences, pour l'atteinte d'objectifs précédemment tentés. Enfin, nous introduisons l'algorithme McSAGG-RIAC, pour Maturationally-Constrained SAGG-RIAC, qui repose sur le couplage de modèles computationnels de motivations intrinsèques et de contraintes maturationnelles physiologiques. Nous argumentons que ces mécanismes peuvent avoir des interactions bidirectionnelles complexes permettant le contrôle actif de l'augmentation de la complexité du développement sensorimoteur, afin de diriger une exploration et un apprentissage efficaces. Nous introduisons plus particulièrement un modèle fonctionnel des contraintes maturationnelles inspiré par le processus de myélinisation chez les humains, et montrons comment celui-ci peut être couplé avec l'algorithme SAGG-RIAC. Nous montrons qualitativement et quantitativement que cette approche intégrée des trois architectures présentées pendant cette thèse permet de répondre à certaines des problématiques des environnements réels, en contrôlant la complexité, le volume, la dimensionnalité et la redondance des comportements explorés de manière intrinsèque au robot, diminuant de manière importante la nécessité de contraindre et préparer l'environnement de manière externe. / Learning new sensorimotor knowledge and know-how in real environments leads to an important number of challenges for today's robots. In order to learn new skills, they need to explore sensorimotor spaces which are generally high-dimensional, high-volume, redundant, and possess areas of heterogenous levels of complexity. In this thesis, introduced within the developmental robotics domain, we propose different processes in order to guide and constrain the autonomous acquisition of new sensorimotor skills in such spaces. We propose an unified approach in order to resolve these problems which takes inspiration from phenomenon of developmental constraints introduced in biology and psychology, and more particularly intrinsic motivations and maturational constraints. After formalizing a computational framework based on these notions, we present three different algorithmic architectures, each one reused in an integrated manner within the next one:The first one, called RIAC, for Robust-Intelligent Adaptive Curiosity, corresponds to the implementation of an active learning algorithm which orients the exploration in bounded spaces whose dimensionality is known and which possess regions of different levels of complexity. This system, which uses heuristics taking inspiration from knowledge based intrinsic motivations mechanisms, effectively directs a progressive exploration of new sensorimotor knowledge, which corresponds to the learning of forward models. It also leads to the emergence of self-organized developmental trajectories related to the orientation of the sensorimotor exploration toward activities of intermediate complexity. Then, we propose the SAGG-RIAC algorithm for Self-Adaptive Goal Generation - RIAC, as a competence based intrinsic motivations exploration mechanism, which allows highly-redundant robots whose sensorimotor spaces are high-dimensional to learn effectively and actively new motor skills in their task spaces. The main idea of this algorithm is to guide the robot to do active babbling in a low-dimensional task space, in contrast with a motor babbling carried out in a higher-dimensional control space, by actively and adaptively self-generating goals in regions of the task space which bring the highest improvement of competences for reaching previously attempted goals.Finally, we introduce the McSAGG-RIAC algorithm for Maturationally-Constrained SAGG-RIAC, which is based on a coupling of computational models of intrinsic motivation and physiological maturational constraints. We argue that these mechanisms may have complex bidirectional interactions allowing the active control of the increase of complexity in the sensorimotor development, in order to direct efficient learning and exploration processes. We introduce more particularly a functional model of maturational constraints inspired by the biological process of myelination, and show how this can be coupled with the SAGG-RIAC algorithm. We show qualitatively and quantitatively that this integrated approach of the three architectures introduced in this thesis answers some problematics raised by real environments, by controlling the complexity, volume, dimensionality and redundancy of skills explored in a manner intrinsic to the robot, thus decreasing in an important extent the necessity of constraining and preparing the environment in en external manner.

One of the most critical issues facing government over the next decade will be filling management positions vacated by Baby Boomers. The purpose of this quantitative correlational research study was to examine how intrinsic and extrinsic motivations affect job satisfaction among different age cohorts in the public workforce. The public workforce is comprised of Baby Boomers (born 1946- 1964), Generation X (born 1965- 1980) and Generation Y (born1981 to 1996). The theoretical framework for this study was Herzberg's motivation-hygiene theory. A random sample of 213 participants: Generation Y = 40, Generation X = 77, and Baby Boomers = 96, participated in an online SurveyMonkey government panel. The panel was composed of local, state, and federal employees. Participants answered the survey using the Career Goals Scale, the Job Satisfaction Scale, and a brief demographics scale. Data were analyzed using descriptive statistics as a measure of central tendency. Also, inferential statistics using Pearson product-moment correlations, simple linear regressions, and one-way multivariate analysis of variance (MANOVA) were conducted to answer three central research questions. Results revealed that both intrinsic and extrinsic motivations affect job satisfaction. Also, results of the individual one-way ANOVAs did not indicate significant differences in intrinsic motivation or job satisfaction among the age cohorts. Finally, pairwise comparisons determined that there were significant differences in extrinsic motivation between Baby Boomers and Generation Y. The information for this study may inform human resource managers in the public sector, about factors that would affect benefit plan policy, and improve recruitment and retention of employees.

No description available.

Cette thèse répond à un double objectif : analyser théoriquement, à l’aide d’un modèle d’arbitrage travail-loisir, la réaction du temps de travail du médecin aux différents modes de rémunération lorsque seul le nombre de consultations et de patients peut faire l’objet d’un contrat entre le médecin et sa tutelle ; et observer empiriquement comment le temps de travail des médecins généralistes français évoluerait suite à une revalorisation du tarif de la consultation si l’on tient compte de leurs différents registres de motivations.Dans la première partie, nous présentons tout d’abord le modèle nous servant de base pour analyser l’offre de travail des médecins généralistes dans différents contextes (chapitre 1). Ensuite, nous montrons comment celui-ci peut être « enrichi » pour analyser les problèmes d’inégalités de santé, à travers le choix du médecin en termes de durée de consultation (chapitre 2). Enfin, nous analysons théoriquement comment l’offre de travail des médecins réagit à un choc de vieillissement de la population, selon le schéma de paiement en vigueur, paiement à l’acte ou capitation (chapitre 3).Dans la seconde partie, nous présentons tout d’abord une revue de littérature retraçant l’émergence du concept de motivation intrinsèque en économie de la santé (chapitre 4). Ensuite, nous regardons comment la théorie standard de l’offre de travail peut intégrer l’interaction possible entre les motivations intrinsèques et extrinsèques (chapitre 5). Enfin,nous cherchons à identifier empiriquement l’existence possible d’un effet contreproductif des incitations économiques sur les motivations intrinsèques, dans le champ de la promotion de la santé (chapitre 6). / This PhD dissertation has a twofold objective: to theoretically analyse, by using a workleisure trade-off model, the sensibility of physician’s working time to different payment systems when only the number of consultations and patients are contractible; and to empirically observe how French GP’s working time would change if consultation fee increased, in the specific case where different range of human motives are considered.The first part is dedicated to the presentation of the main properties of the basic model we use to study the labour supply behaviour of self-employed GPs in various contexts (chapter 1). Then, we show how this model can be ‘fitted’ to deal with the issue of health inequalities through physician’s choice in terms of consultation length (chapter 2). Finally, we theoretically investigate how physicians’ labour supply reacts to a population ageing shock, according to the effective payment scheme - fee-for service or capitation (chapter 3). The second part presents a survey on the emergence of the concept of intrinsic motivations in health economics aiming at specifying the theoretical background with which the remainder of our work is in line (chapter 4). Next, we explore how standard labour supply theory can integrate intrinsic motivations as a key determinant of human behaviour and, more especially, the effect of extrinsic incentives on the former in the field of general practice (chapter 5). Lastly, we intend to check, from an empirical perspective, if economic incentives can have a detrimental effect on intrinsic motivations in the particular field of health promotion, by using two different strategies (chapter 6).

Cette thèse s’intéresse à plusieurs questions posées par l’introduction et la généralisation, en France, d’incitatifs économiques de type P4P appliqués à la médecine générale. Ces schémas incitatifs (CAPI, ROSP) ont pour objectif d’améliorer la qualité des soins, mais ils sont débattus en termes d’efficience et d’effets indésirables potentiels. Dans un premier temps, nous évaluons l’impact du CAPI sur différents indicateurs de la qualité des soins : la durée de la consultation et le dépistage des cancers. Puis, nous étudions les modalités d’une meilleure implication du médecin généraliste (MG) dans le dépistage des cancers, en appliquant la méthodologie des choix discrets. Enfin, nous analysons la relation entre motivations intrinsèques et extrinsèques des MGs français. Nous montrons que le CAPI n’a pas eu d’impact significatif sur les indicateurs de qualité considérés et que les MGs seraient sensibles à d’autres dispositifs non monétaires potentiellement moins coûteux. Enfin, nous mettons en évidence une relation de substituabilité entre motivations intrinsèques et extrinsèques. Ces résultats invitent à davantage de prudence dans la définition des incitatifs économiques en médecine générale. / This thesis addresses several issues raised by the introduction in France of economic incentives such as pay-For-Performance applied to general practice. These incentive schemes are designed to improve the quality of care, but they are discussed both in terms of effectiveness and potential side effects. Initially, we assess the impact of the CAPI scheme on various indicators of quality of care: the consultation length and cancers screening. Then, using the discrete choice experiment methodology, we reveal general practitioners (GPs) preferences for devices aimed at improving the early detection of cancers. Finally, we analyse empirically the relationship between French GPs' intrinsic and extrinsic motivations. We show that the CAPI has not had a significant impact on the selected quality indicators. In addition, GPs would be sensitive to potentially less costly nonmonetary devices. Eventually, we highlight a negative relationship between GPs' intrinsic and extrinsic motivations. Our results call for greater caution in the definition of economic incentives in general practice.