{"id":{"repo_id":"whiterose","oai_identifier":"oai:etheses.whiterose.ac.uk:28366"},"canonical_url":"https://search.ndltd.org/etd/whiterose/oai:etheses.whiterose.ac.uk:28366","repository":{"repo_id":"whiterose","name":"White Rose University Consortium","base_url":"https://etheses.whiterose.ac.uk/cgi/oai2"},"display":{"title":"The Role of Emotions in Autonomous Social Agents","abstract":"The holy grail of both AI and cognitive science is human-level intelligence. Whereas AI relies on computer algorithms to simulate human abilities, cognitive scientists investigate the brain to understand the underlying mechanisms. For most of human history, emotions were thought to be nothing more than disturbances for cognition. Therefore, they were usually ostracized from research on intelligence. As a result, cognitive architectures only partially include emotions in their design, if at all. Recently, though, it was discovered that emotions and cognition are in fact inter-dependent systems. Consequently, before being able to fully replicate human-level intelligence, it is necessary to understand the concept of emotions and its many roles within the brain. In this thesis, working around the lack of definition for emotion, I show that emotions inform the brain as to the nature of a given situation and guide the decision-making process, to increase the survival potential of virtual agents. In particular ProtoEmo, an architecture replicating the circuits found at the base of the forebrain, is shown to have the ability to detect stimuli relevant to the survival of virtual agents. Hence, it outperforms other emotional agents in terms of survival capabilities, which are measured by the size of the remaining population at the end of a resource foraging task. PrimEmo, the architecture born from the integration of ProtoEmo with standard models of the reward and decision-making systems in the brain, displays survival capabilities similar to the advantage actor-critic algorithm. PrimEmo also shows promises for supporting primitive emotions characterized by their level of arousal and valence. After further refinement, PrimEmo could replace the core decision-making module of a cognitive architecture, such as ACT-R or SOAR. Not only would it confer survival capabilities to the architecture, it would also allow for the possibility of investigating full-fledged emotions, and even emotional expression.","abstract_html":"The holy grail of both AI and cognitive science is human-level intelligence. Whereas AI relies on computer algorithms to simulate human abilities, cognitive scientists investigate the brain to understand the underlying mechanisms. For most of human history, emotions were thought to be nothing more than disturbances for cognition. Therefore, they were usually ostracized from research on intelligence. As a result, cognitive architectures only partially include emotions in their design, if at all. Recently, though, it was discovered that emotions and cognition are in fact inter-dependent systems. Consequently, before being able to fully replicate human-level intelligence, it is necessary to understand the concept of emotions and its many roles within the brain. In this thesis, working around the lack of definition for emotion, I show that emotions inform the brain as to the nature of a given situation and guide the decision-making process, to increase the survival potential of virtual agents. In particular ProtoEmo, an architecture replicating the circuits found at the base of the forebrain, is shown to have the ability to detect stimuli relevant to the survival of virtual agents. Hence, it outperforms other emotional agents in terms of survival capabilities, which are measured by the size of the remaining population at the end of a resource foraging task. PrimEmo, the architecture born from the integration of ProtoEmo with standard models of the reward and decision-making systems in the brain, displays survival capabilities similar to the advantage actor-critic algorithm. PrimEmo also shows promises for supporting primitive emotions characterized by their level of arousal and valence. After further refinement, PrimEmo could replace the core decision-making module of a cognitive architecture, such as ACT-R or SOAR. Not only would it confer survival capabilities to the architecture, it would also allow for the possibility of investigating full-fledged emotions, and even emotional expression.","abstract_has_math":false,"creators":["Caspar, Luc"],"institution":"University of Sheffield","degree_name":"Ph.D","degree_level":"doctoral","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["Moore, Roger"],"committee_chairs":[],"committee_members":[],"year":2020,"date_issued":"2020-02","date_published":"2020-02","updated_at":"2026-11-18T22:41:20Z","subjects":[],"languages":["eng"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["uk.bl.ethos.823924"],"render_values":[{"text":"uk.bl.ethos.823924","href":null,"code":true}]}],"person_identifier_entries":[{"key":"person:identifier.orcid","label":"ORCID","values":["0000000285189433"],"render_values":[{"text":"0000-0002-8518-9433","href":"https://orcid.org/0000-0002-8518-9433","code":true}]}]},"links":{"outbound_url":null,"outbound_label":null,"outbound_source":null,"record_url":null,"record_label":null,"record_kind":null,"record_source":null,"download_url":"https://etheses.whiterose.ac.uk/id/eprint/28366/1/thesis_Caspar_Luc_140127441_final_revision.pdf","download_label":"Download PDF","download_source":"dc:identifier.uri","access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://etheses.whiterose.ac.uk/cgi/oai2?verb=GetRecord&metadataPrefix=uketd_dc&identifier=oai%3Aetheses.whiterose.ac.uk%3A28366","prefix":"uketd_dc"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Moore, Roger"]},{"key":"dc:creator","label":"Author","values":["Caspar, Luc"]},{"key":"person:identifier.orcid","label":"ORCID","values":["0000000285189433"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2020-02"]},{"key":"dc:date.issued","label":"Date","values":["2020-02"]},{"key":"dc:publisher.department","label":"Dc Publisher Department","values":["Computer Science (Sheffield)"]},{"key":"dc:publisher.institution","label":"Dc Publisher Institution","values":["University of Sheffield"]},{"key":"dc:relation.isreferencedby","label":"Dc Relation Isreferencedby","values":["https://etheses.whiterose.ac.uk/id/eprint/28366/"]},{"key":"dc:relation.references","label":"Dc Relation References","values":["Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., ... Zheng, X. (2015). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Adams, S., Arel, I., Bach, J., Coop, R., Furlan, R., Goertzel, B., ... Sowa, J. (2012, mar). Mapping the Landscape of Human-Level Artificial General Intelligence. AI Magazine, 33(1), 25. Adolphs, R., Gosselin, F., Buchanan, T. W., Tranel, D., Schyns, P., & Damasio, A. R. (2005, jan). A mechanism for impaired fear recognition after amygdala damage. Nature, 433(7021), 68–72. Aggleton, J. P. (1992). The Amygdala : neurobiological aspects of emotion, memory and mental dysfunction. New York: Wiley-Liss. Alexandrov, Y. I., & Sams, M. E. (2005, oct). Emotion and consciousness: ends of a continuum. Cognitive Brain Research, 25(2), 387–405. Anderson, A. K., Christoff, K., Stappen, I., Panitz, D., Ghahremani, D. G., Glover, G., ... Sobel, N. (2003, feb). Dissociated neural representations of intensity and valence in human olfaction. Nature neuroscience, 6(2), 196–202. Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036–1060. Arbib, M. A. (2003). The handbook of brain theory and neural networks (2nd ed. ed.; M. A. Arbib & I. NetLibrary, Eds.). Cambridge, Mass.: Cambridge, Mass. : MIT Press, c2003. Aristotle. (1926). The Nicomachean Ethics (H. Rackham, Trans.). Cambridge, MA: Harvard University Press. (Original work published 350 B.C) Armony, J. L., Servan-Schreiber, D., Cohen, J. D., & LeDoux, J. E. (1995, apr). An anatomically constrained neural network model of fear conditioning. Behavioral Neuroscience, 109(2), 246–57. Armony, J. L., Servan-Schreiber, D., Cohen, J. D., & LeDoux, J. E. (1997, apr). Computational modeling of emotion: explorations through the anatomy and physiology of fear conditioning. Trends in cognitive sciences, 1(1), 28–34. Armony, J. L., Servan-Schreiber, D., Romanski, L. M., Cohen, J. D., & LeDoux, J. E. (1997, mar). Stimulus generalization of fear responses: effects of auditory cortex lesions in a computational model and in rats. Cerebral cortex (New York, N.Y. : 991), 7(2), 157–65. Arnold, M. B. (1959). From perception to emotion and action. Acta Psychologica, 15, 406–417. Arnold, M. B. (1961). Emotion and personality. London: London : Cassell, 1961. Atallah, H. E., Frank, M. J., & O’Reilly, R. C. (2004, nov). Hippocampus, cortex, and basal ganglia: insights from computational models of complementary learning systems. Neurobiology of learning and memory, 82(3), 253–67. Baars, B. J. (2005). Global workspace theory of consciousness: Toward a cognitive neuroscience of human experience. Progress in Brain Research, 150, 45–53. Bard, P. (1929, aug). The central representation of the sympathetic system: As indicated by certain physiologic observations. Archives of Neurology And Psychiatry, 22(2), 230–246. Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983, sep). Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems. IEEE Transactions on Systems, Man and Cybernetics, SMC-13(5), 834–846. Bellman, R. (1957). Dynamic programming (R. Corporation, Ed.). Princeton, N.J. : London: Princeton, N.J. : Princeton U.P. Berke, J. D. (2018, jun). What does dopamine mean? (Vol. 21) (No. 6). Nature Publishing Group. Bush, P. C., & Sejnowski, T. J. (1995, oct). Models of Cortical Networks. In M. J. Gutnick & I. Mody (Eds.), The cortical neuron (pp. 174–190). Oxford University Press. Cannon, W. B. (1927, dec). The James-Lange Theory of Emotions: A Critical Examination and an Alternative Theory. The American Journal of Psychology, 39(1/4), 106. Cannon, W. B. (1929, dec). Bodily Changes in Pain, Hunger, Fear, and Rage. Archives of Internal Medicine, 44(6), 908. Cannon, W. B. (1931). Again the James-Lange and the thalamic theories of emotion. (Vol. 38). Cannon, W. B., Lewis, J. T., & Britton, S. W. (1927, sep). The Dispensability of the Sympathetic Division of the Autonomic Nervous System. The Boston Medical and Surgical Journal, 197(13), 514–515. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014, jun). Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Emnlp 2014 - 2014 conference on empirical methods in natural language processing, proceedings of the conference (pp. 1724–1734). Chollet, F. (2015). Keras: The Python Deep Learning library. Keras.Io. Collins, A. G. E., & Frank, M. J. (2013). Cognitive control over learning: creating, clustering, and generalizing task-set structure. Psychological review, 120(1), 190–229. Collins, A. G. E., & Frank, M. J. (2014). Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychological review, 121(3), 337–66. Connors, B. W., Landisman, C. E., & Reid, R. (1998, dec). Thalamus: Organization and Function (Vol. 1), Experimental and Clinical Aspects (Vol. 2). Trends in Neurosciences, 21(12), 539–540. Cools, R., Nakamura, K., & Daw, N. D. (2011, jan). Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology, 36(1), 98–13. Dalgleish, T. (2004, jul). The emotional brain. Nature Reviews Neuroscience, 5(7), 583–9. Damasio, A. (1997). The Feeling of What Happens: Body and Emotion in the Making of Consciousness. Vintage. Damasio, A. (2008). Descartes’ error: Emotion, reason and the human brain. Vintage. Darwin, C. (1859). On the Origin of Species (J. Murray, Ed.). London: John Murray. Darwin, C. (1872). The Expression of the Emotions in Man and Animals (J. Murray, Ed.). London: John Murray. Davidson, R. J. (2003, sep). Affective neuroscience and psychophysiology: Toward a synthesis. In Psychophysiology (Vol. 40, pp. 655–665). Society for Psychophysiological Research. Davidson, R. J., Scherer, K. R., & Goldsmith, H. H. (2003). Handbook of affective sciences. Oxford ; New York: Oxford University Press. Daw, N. D., Kakade, S., & Dayan, P. (2002, jun). Opponent interactions between serotonin and dopamine. Neural Networks, 15(4-6), 603–616. Dayan, P., & Abbott, L. F. (2001). Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Massachusetts Institute of Technology Press. den Ouden, H. E. M., Daw, N. D., Fernandez, G., Elshout, J. A., Rijpkema, M., Hoogman, M., ... Cools, R. (2013, nov). Dissociable effects of dopamine and serotonin on reversal learning. Neuron, 80(4), 1090–100. Descartes, R. (1637). Discourse on the Method. Philosophy, 1, xii, 249. Descartes, R. (1649). The Passions of the Soul. Hackett Publishing. Di Pietro, N. C., & Seamans, J. K. (2011, jun). Dopamine and serotonin interactively modulate prefrontal cortex neurons in vitro. Biological psychiatry, 69(12), 1204–11. Douglas, R. J., Koch, C., Mahowald, M., Martin, K. A., & Suarez, H. H. (1995, aug). Recurrent excitation in neocortical circuits. Science, 269(5226), 981–985. Ekman, P. (1973). Darwin and facial expression, a century of research in review (Vol. 23) (No. 4). Academic Press. Ekman, P. (1992). Are There Basic Emotions? Psychological Review, 99(3), 550–553. Eliasmith, C., & Anderson, C. H. (2003). Neural Engineering: Computation Representation and Dyamics in Neurobiological Systems (Vol. 19) (No. 5). MIT Press. Fellous, J.-m. (1999). The Neuromodulatory Basis of Emotion. The Neuroscientist, 5, 283–294. Fellous, J.-M., & Arbib, M. A. (2005a). Who needs emotions?: The Brain Meets the Robot. Oxford University Press. Fellous, J.-M., & Arbib, M. A. (2005b, apr). “Edison” and “Russell”. In J.-M. Fellous & M. A. Arbib (Eds.), Who needs emotions? the brain meets the robot (pp. 3–8). Oxford University Press. Fellous, J.-m., & LeDoux, J. E. (2005, mar). Toward Basic Principles for Emotional Processing. In J.-M. Fellous & M. A. Arbib (Eds.), Who needs emotions?: The brain meets the robot (pp. 79–116). Oxford University Press. Fontaine, J. J. R., Scherer, K. R., & Soriano, C. (2013). Components of Emotional Meaning (J. R. J. Fontaine, K. R. Scherer, & C. Soriano, Eds.). Oxford: Oxford University Press. Fortin, F.-A., De Rainville, F.-M., Gardner, M.-A. G., Parizeau, M., & Gagné, C. (2012, jul). DEAP: Evolutionary Algorithms Made Easy. Journal of Machine Learning Research, 13(1), 2171–2175. Frank, M. J. (2005, jan). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of cognitive neuroscience, 17(1), 51–72. Frank, M. J., & Claus, E. D. (2006, apr). Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychological review, 113(2), 300–26. Frank, M. J., Loughry, B., & O’Reilly, R. C. (2001, jun). Interactions between frontal cortex and basal ganglia in working memory: A computational model. Cognitive, 1(2), 137–160. Franklin, S., Madl, T., D’Mello, S., & Snaider, J. (2014, mar). LIDA: A Systems-level Architecture for Cognition, Emotion, and Learning. IEEE Transactions on Autonomous Mental Development, 6(1), 19–41. Freitas, J. D., Gudwin, R., & Queiroz, J. (2005). Emotion in artificial intelligence and artificial life research: Facing problems. IVA. Frijda, N. H. (1986). The Emotions. Frijda, N. H., & Parrott, W. G. (2011, sep). Basic Emotions or Ur-Emotions? Emotion Review, 3(4), 406–415. Girard, B., Cuzin, V., Guillot, A., Gurney, K. N., & Prescott, T. J. (2003, dec). A basal ganglia inspired model of action selection evaluated in a robotic survival task. Journal of Integrative Neuroscience, 2(2), 179–200. Glorot, X., Bordes, A., & Bengio, Y. (2011). Deep sparse rectifier neural networks. In Journal of machine learning research (Vol. 15, pp. 315–323). Goertzel, B. (2014). Artificial General Intelligence: Concept, State of the Art, and Future Prospects. Journal of Artificial General Intelligence, 5(1), 1–48. Goertzel, B. (2015). Artificial General Intelligence. Goleman, D. (1995). Emotional intelligence. Personality and Individual Differences, 9, 1091–1100. Grillner, S., Robertson, B., & Stephenson-Jones, M. (2013). The evolutionary origin of the vertebrate basal ganglia and its role in action selection. The Journal of Physiology. Gurney, K., Prescott, T. J., & Redgrave, P. (2001a, may). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biological Cybernetics, 84(6), 401–410. Gurney, K., Prescott, T. J., & Redgrave, P. (2001b, may). A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour. Biological cybernetics, 84(6), 411–423. Gurney, K., Prescott, T. J., Wickens, J. R., & Redgrave, P. (2004). Computational models of the basal ganglia: from robots to membranes. Trends in Neurosciences, 27(8), 453–459. Gutnick, M. J., & Mody, I. (2012). The Cortical Neuron (M. J. Gutnick & I. Mody, Eds.). Oxford University Press. Harlow, J. (1848). Passage of an Iron Rod through the Head. The Boston Medical and Surgical Journal, 39(20), 389–393. Harlow, J. M. (1868). Recovery from the passage of an iron bar through the head. Publications of the Massachusetts Medical Society, 2, 327–347. Hawkins, J., & Blakeslee, S. (2007). On intelligence. Times Books. Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2006). Banishing the homunculus: Making working memory work. Neuroscience, 139(1), 105–118. Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2010). Neural mechanisms of acquired phasic dopamine responses in learning. Neuroscience and Biobehavioral Reviews, 34(5), 701– 720. Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2011). Toward an executive without a homunculus: Computational models of the prefrontal cortex/basal ganglia system. In Modelling natural action selection (pp. 239–263). Hewitt, C., Bishop, P., & Steiger, R. (1973). A Universal Modular ACTOR Formalism for Artificial Intelligence. In Proceeding ijcai’73 proceedings of the 3rd international joint conference on artificial intelligence (pp. 235–245). Morgan Kaufmann Publishers Inc. Hochreiter, S., & Schmidhuber, J. (1997, nov). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780. Holovaty, A., & Willson, S. (2017). Django. Django Software Fundation. Houk, J. C. (2011). Action selection and refinement in subcortical loops through basal ganglia and cerebellum. In Modelling natural action selection. Ivanovic, M., Budimac, Z., Radovanovi´c, M., Kurbalija, V., Dai, W., B˘adic˘a, C., ... Mitrovi´c, D. (2015). Emotional agents — state of the art and applications. Computer Science and Information Systems, 12(4), 1121–1148. Izard, C. E. (1993, jan). Four systems for emotion activation: cognitive and noncognitive processes. Psychological review, 100(1), 68–90. James, W. (1884). What is an Emotion? Mind, 9, 188–205. Joel, D., Niv, Y., & Ruppin, E. (2002, jun). Actor-critic models of the basal ganglia: new anatomical and computational perspectives (Vol. 15) (No. 4-6). Pergamon. Jones, E. G. (1985). The thalamus. New York: New York. Karten, H. J., & Shimizu, T. (1991). Are Visual Hierarchies in the Brains of the Beholders? Constancy and Variability in the Visual System of Birds and Mammals. In The changing visual system (pp. 51–59). Boston, MA: Springer US. Kingma, D. P., & Ba, J. L. (2015, dec). Adam: A Method for Stochastic Optimization. In 3rd international conference on learning representations, iclr 2015 - conference track proceedings. International Conference on Learning Representations, ICLR. Kluver, H., & Bucy, P. C. (1937). \"Psychic blindness\" and other symptoms following bilateral temporal lobectomy in rhesus monkeys. American Journal of Physiology, 19, 352–353. Kreibig, S. D. (2010). Autonomic nervous system activity in emotion: A review. Laird, J. E., Newell, A., & Rosenbloom, P. S. (1987, sep). SOAR: An architecture for general intelligence. Artificial Intelligence, 33(1), 1–64. Lange, C. G. (1885). No Title. The Emotions, 33–90. Lazarus, R. S. (1966). Psychological stress and the coping process. New York: New York : McGraw-Hill, (1966). LeDoux, J. E. (1986). Sensory systems and emotion: A model of affective processing. Integrative Psychiatry, 4(4), 237–243. LeDoux, J. E. (1990). Information flow from sensation to emotion plasticity in the neural com- putation of stimulus values. In Learning and computational neuroscience: Foundations of adaptive networks (pp. 3–52). Bradford Books (MIT Press). LeDoux, J. E. (1992, apr). Brain mechanisms of emotion and emotional learning. Current Opinion in Neurobiology, 2(4), 191–197. LeDoux, J. E. (1995, jan). Emotion: clues from the brain. Annual Review of Psychology, 46(1), 209–235. LeDoux, J. E. (1998). The Emotional Brain: The Mysterious Underpinnings of Emotional Life. Phoenix. LeDoux, J. E. (2003). The emotional brain, fear, and the amygdala (Vol. 23). LeDoux, J. E. (2012, feb). Rethinking the emotional brain. Neuron, 73(4), 653–76. LeDoux, J. E. (2014, oct). Low roads and higher order thoughts in emotion. Cortex, 59, 214–215. Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., . . . Wierstra, D. (2016, sep). Continuous control with deep reinforcement learning. In 4th international conference on learning representations, iclr 2016 - conference track proceedings. Litt, A., Eliasmith, C., & Thagard, P. (2006). Why losses loom larger than gains: modeling neural mechanisms of cognitive-affective interaction. Proceedings of the Twenty-eighth Annual Meeting of the Cognitive Science Society, 495–500. Litt, A., Eliasmith, C., & Thagard, P. (2007). A large-scale neurocomputational model of emotional decision making. BMC Neuroscience, 8(Suppl 2), P14. Litt, A., Eliasmith, C., & Thagard, P. (2008). Neural affective decision theory: Choices, brains, and emotions. Cognitive Systems Research, 9(4), 252–273. Maas, A. L., Hannun, A. Y., & Ng, A. Y. (2013). Rectifier Nonlinearities Improve Neural Network Acoustic Models. Proceedings of the 30 th International Conference on Machine Learning, 30(1), 6. MacLean, P. D. (1949, nov). Psychosomatic Disease and the \"Visceral Brain\". Psychosomatic Medicine, 11(6), 338–353. MacLean, P. D. (1952, nov). Some psychiatric implications of physiological studies on frontotemporal portion of limbic system (visceral brain). Electroencephalography and Clinical Neurophysiology, 4(4), 407–18. MacLean, P. D. (1970). The triune brain, emotion and scientific bias. In F. Schmitt (Ed.), The neurosciences: Second study program (pp. 336–349). New York: The Rockefeller University Press. Maia, T. V. (2009). Reinforcement learning, conditioning, and the brain: Successes and challenges. Cognitive, Affective and Behavioral Neuroscience. Marsella, S., Gratch, J., & Petta, P. (2010). Computational models of emotion. In K. R. Scherer, T. Bänziger, & E. Roesch (Eds.), A blueprint for affective computing: A sourcebook and manual. Oxford University Press. Martinez-Miranda, J., & Aldea, A. (2005, mar). Emotions in human and artificial intelligence. Computers in Human Behavior, 21(2), 323–341. McClure, S. M., York, M. K., & Montague, P. R. (2004, jun). The Neural Substrates of Reward Processing in Humans: The Modern Role of fMRI. The Neuroscientist, 10(3), 260–268. McDougall, W. (1921). An Introduction to Social Psychology. J.W. Luce & Company. Mehrabian, A. (1980). Basic dimensions for a general psychological theory: Implications for personality, social, environmental, and developmental studies. (G. &. H. Oelgeschlager, Ed.). Cambridge, Mass. Mehrabian, A. (1996). Pleasure-Arousal-Dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4), 261–292. Miller, B. L., & Goldberg, D. E. (1995). Genetic algorithms, tournament selection, and the effects of noise. Complex Systems, 9, 193–212. Mnih, V., Badia, A. P., Mirza, L., Graves, A., Harley, T., Lillicrap, T. P., ... Kavukcuoglu, K. (2016). Asynchronous Methods for Deep Reinforcement Learning. 33rd International Conference on Machine Learning, ICML 2016, 4, 2850–2869. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013, dec). Playing Atari with Deep Reinforcement Learning. Nagel, T. (1974, oct). What Is It Like to Be a Bat? The Philosophical Review, 83(4), 435. Newell, A. (1973, jan). You can’t play 20 questions with nature and win: Projective comments on the papers of this symposium. Visual Information Processing, 283–308. Niv, Y. (2009, jun). Reinforcement learning in the brain. Journal of Mathematical Psychology, 53(3), 139–154. Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007, apr). Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology, 191(3), 507–20. Northcutt, G. R., & Kaas, J. H. (1995, sep). The emergence and evolution of mammalian neocortex. Trends in Neurosciences, 18(9), 373–379. Oatley, K., & Johnson-laird, P. N. (1987, mar). Towards a Cognitive Theory of Emotions. Cognition & Emotion, 1(1), 29–50. O’Doherty, J. P., Lee, S. W., & McNamee, D. (2015). The structure of reinforcement-learning mechanisms in the human brain (Vol. 1). O’Reilly, R. C., & Frank, M. J. (2006, feb). Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural computation, 18(2), 283–328. O’Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007). PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm. Behavioral Neuroscience, 121(1), 31–49. O’Reilly, R. C., & Munakata, Y. (2013). Computational Cognitive Neuroscience (No. 2001). Wiki Book. O’Reilly, R. C., Wyatte, D., & Rohrlich, J. (2014, jul). Learning Through Time in the Thalamocortical Loops. Ortony, A., Clore, G. L., & Collins, A. (1988). The Cognitive Structure of Emotions (Vol. 18). Cambridge: Cambridge University Press. Ortony, A., Norman, D. A., & Revelle, W. (2005). Affect and Proto-Affect in Effective Functioning. In Who needs emotions?: The brain meets the robot (pp. 173–202). Oxford University Press. Ortony, A., & Turner, T. J. (1990). What’s basic about basic emotions? Psychological Review, 97, 315–331. Osgood, C. E. C. E. (1957). The measurement of meaning (G. J. Suci & P. H. Tannenbaum, Eds.). Urbana [Ill.]: Urbana Ill. Panksepp, J. (1998). Affective Neuroscience: The Foundations of Human and Animal Emotions. Oxford University Press. Panksepp, J. (2005, mar). Affective consciousness: Core emotional feelings in animals and humans. Consciousness and Cognition, 14(1), 30–80. Panksepp, J., Mir, D., Delgado, J. M. R., & Edwards, S. (1982, sep). Toward a general psychobiological theory of emotions. Behavioral and Brain Sciences, 5(03), 407. Papez, J. W. (1937, jan). A proposed mechanism of emotion. The Journal of Neuropsychiatry and Clinical Neurosciences, 7(1), 103–12. Pavlov, I. P. (1927). Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. Oxford University Press: Humphrey Milford. Picard, R. W. (2000). Affective Computing. MIT Press. Plutchik, R. (1958, mar). Section of psychology: outlines of a new theory of emotion*. Transactions of the New York Academy of Sciences, 20(5 Series II), 394–403. Plutchik, R. (1962). The Emotions: Facts and Theories, and a New Model. Plutchik, R., & Conte, H. (1997). Circumplex models of personality and emotions. Potter, M. A., & De Jong, K. A. (2000, mar). Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evolutionary computation, 8(1), 1–29. Prescott, T. J., Gurney, K., Montes-Gonzalez, F., Humphries, M., & Redgrave, P. (2002). The robot basal ganglia: Action selection by an embedded model of the basal ganglia (Vol. 52). Prescott, T. J., Montes González, F. M., Gurney, K., Humphries, M. D., & Redgrave, P. (2006, jan). A robot model of the basal ganglia: Behavior and intrinsic processing. Neural Networks, 19(1), 31–61. Redgrave, P., Prescott, T. J., & Gurney, K. (1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience, 89(4), 1009–1023. Reisenzein, R. (2006, nov). Arnold’s theory of emotion in historical perspective. Cognition & Emotion, 20(7), 920-951. Rescorla, R. (2008, mar). Rescorla-Wagner model. Scholarpedia, 3(3), 2237. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Classical conditioning ii current research and theory (Vol. 21, pp. 64–99). Rohrer, B. (2010). What we might look for in an AGI benchmark. AGI Conference. Rolls, E. T. (2007). Emotion Explained. Oxford University Press. Rolls, E. T. (2013). Emotion and Decision-making Explained. Romanski, L. M., & LeDoux, J. E. (1992). Equipotentiality of thalamo-amygdala and thalamo- cortico-amygdala circuits in auditory fear conditioning. Journal of Neuroscience, 12(11), 4501–4509. Russell, J. a. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110(1), 145–172. Russell, J. A., & Barrett, L. F. (1999). Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. Journal of personality and social psychology, 76(5), 805–819. Ryle, G. (1949). The concept of mind. London: Hutchinson. Sander, D., Grandjean, D., & Scherer, K. R. (2005, may). A systems approach to appraisal mechanisms in emotion. Neural networks : the official journal of the International Neural Network Society, 18(4), 317–52. Saper, C. B. (2009, jan). Hypothalamus. Scholarpedia, 4(1), 2791. Saper, C. B. (2012). Hypothalamus. In J. Mai & G. Paxinos (Eds.), The human nervous system (pp. 548–583). Academic Press. Saper, C. B., & Lowell, B. B. (2014, dec). The hypothalamus. Current Biology, 24(23), R1111–R1116. Schachter, S., & Singer, J. (1962). Cognitive, social, and physiological determinants of emotional state. Psychological review, 69, 379–399. Scherer, K. R. (1984). On the Nature and Function of Emotion: A Component Process Approach. In K. R. Scherer & P. Ekman (Eds.), Approaches to emotion (pp. 293–317). Psychology Press. Scherer, K. R. (2005, dec). What are emotions? And how can they be measured? Social Science Information, 44(4), 695–729. Scherer, K. R. (2009, dec). Emotions are emergent processes: they require a dynamic computational architecture. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 364(1535), 3459–74. Scherer, K. R., Bänziger, T., & Roesch, E. (2010). A Blueprint for Affective Computing: A Sourcebook and Manual. Oxford University Press. Scherer, K. R., & Ekman, P. (1984). Approaches to emotion. L. Erlbaum Associates. Scherer, K. R., Schorr, A., & Johnstone, T. (2001). Appraisal processes in emotion: Theory, methods, research. Series in affective science. Scheutz, M. (2004). Useful roles of emotions in artificial agents: A case study from artificial life. In Proceedings of the 19th national conference on artificial intelligence (pp. 42–47). AAAI Press. Scheutz, M. (2012). The affect dilemma for artificial agents: Should we develop affective artificial agents? IEEE Transactions on Affective Computing, 3(4), 424–433. Schmajuk, N. (2008, mar). Classical conditioning. Scholarpedia, 3(3), 2316. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017, jul). Proximal Policy Optimization Algorithms. Schultz, W. (2000, dec). Multiple reward signals in the brain. Nature reviews. Neuroscience, 1(3), 199-207. Seamans, J. (2007, jun). Dopamine anatomy. Scholarpedia, 2(6), 3737. Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(3), 417–424. Shanahan, M. (2010). Embodiment and the Inner Life: Cognition and Consciousness in the Space of Possible Minds. Oxford University Press. Sherman, S. (2006, sep). Thalamus. Scholarpedia, 1(9), 1583. Sherman, S. M. (2016, mar). Thalamus plays a central role in ongoing cortical functioning. Nature Neuroscience, 16(4), 533–541. Sherman, S. M., & Guillery, R. W. (2002, dec). The role of the thalamus in the flow of information to the cortex. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 357(1428), 1695–708. Sherman, S. M., & Guillery, R. W. (2005). Exploring the Thalamus and Its Role in Cortical Function. Sloman, A. (1999). How Many Separately Evolved Emotional Beasties Live Within Us? Emotions in humans and artifacts. Sloman, A. (2001). Beyond shallow models of emotion. Cognitive Processing: International Quarterly of ..., 2, 177–198. Snaider, J., McCall, R., & Franklin, S. (2011). The LIDA framework as a general tool for AGI. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Vol. 6830 LNAI, pp. 133–142). Stent, G. S. (1973). A Physiological Mechanism for Hebb’s Postulate of Learning. Proceedings of the National Academy of Sciences of the United States of America, 70(4), 997–1001. Suri, R., & Schultz, W. (1999, jul). A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience, 91(3), 871–890. Sutton, R. S. (1984). Temporal Credit Assignment in Reinforcement Learning. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: an Introduction. Swanson, L. W. (2000, dec). Cerebral hemisphere regulation of motivated behavior. Brain Research, 886(1-2), 113–164. Thagard, P. (1989, sep). Explanatory coherence. Behavioral and Brain Sciences, 12(03), 435–502. Thagard, P. (2002). Coherence in Thought and Action. Place of publication not identified MIT Press, 2002. Thagard, P., & Aubie, B. (2008, sep). Emotional consciousness: A neural model of how cognitive appraisal and somatic perception interact to produce qualitative experience. Consciousness and Cognition, 17(3), 811–834. Thorndike, E. L. (1911). Animal intelligence. New York: The Macmillan company. Toga, A. W. (2015). Brain mapping : an encyclopedic reference (Vol. 1-3). Treue, S. (2001, may). Neural correlates of attention in primate visual cortex. Trends in Neurosciences, 24(5), 295–300. Turian, J., Bergstra, J., & Bengio, Y. (2009). Quadratic features and deep architectures for chunking. In Human language technologies: Conference of the north american association of computational linguistics (p. 245). Association for Computational Linguistics (ACL). Verschure, P., & Eds, A. D. (2014). Distributed Adaptive Control: Theory and Practice. Barcelona, Spain. Vitay, J., & Hamker, F. H. (2010). A computational model of Basal Ganglia and its role in memory retrieval in rewarded visual memory tasks. Wagar, B. M., & Thagard, P. (2004, jan). Spiking Phineas Gage: a neurocomputational theory of cognitive-affective integration in decision making. Psychological review, 111(1), 67–79. Wager, T. D., Kang, J., Johnson, T. D., Nichols, T. E., Satpute, A. B., & Barrett, L. F. (2015, apr). A Bayesian Model of Category-Specific Emotional Brain Responses. PLoS computational biology, 11(4), e1004066. Watkins, C. J. C. H. (1989). Learning from delayed rewards (Unpublished doctoral dissertation). Cambridge, England. Wells, F. L. (1925). Reactions to visual stimuli in affective settings. Journal of Experimental Psychology, 8(1), 64–76. Wells, F. L., Forbes, A., & Whitley, M. T. (1912, jan). On Certain Electrical Processes in the Human Body and Their Relation to Emotional Reactions. The American Journal of Psychology, 23(1), 149. Werbos, P. J. (1990). Backpropagation Through Time: What It Does and How to Do It. Proceedings of the IEEE, 78(10), 1550–1560. Whalen, P. J., & Phelps, E. A. (2009). The Human Amygdala. Guilford Press. Williams, R. J. (1992). Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning, 8(3), 229–256. Woergoetter, F., & Porr, B. (2008, mar). Reinforcement learning. Scholarpedia, 3(3), 1448. Woodworth, R. S., & Sherrington, C. S. (1904, jun). A pseudaffective reflex and its spinal path. The Journal of Physiology, 31(3-4), 234–243. Wundt, W. M. (1874). Grundzüge der physiologischen Psychologie, Volume 1. W. Engelman. Yates, J. R., Darna, M., Gipson, C. D., Dwoskin, L. P., & Bardo, M. T. (2015, may). Dissociable roles of dopamine and serotonin transporter function in a rat model of negative urgency. Behavioural brain research, 291, 201–208. Yilmazer-Hanke, D. M. (2012). Amygdala. In J. K. Mai & G. Paxinos (Eds.), The human nervous system (third edition) (Third Edit ed., pp. 759–834). San Diego: Academic Press. Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, 9(2), 1–27. Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences. American Psychologist, 35(2), 151-175."]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"dc:type.qualificationlevel","label":"Dc Type Qualificationlevel","values":["doctoral"]},{"key":"dc:type.qualificationname","label":"Dc Type Qualificationname","values":["Ph.D"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.dcterms_iso639_2","label":"Language (ISO 639-2)","values":["eng"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["uk.bl.ethos.823924"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://etheses.whiterose.ac.uk/id/eprint/28366/1/thesis_Caspar_Luc_140127441_final_revision.pdf"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The holy grail of both AI and cognitive science is human-level intelligence. Whereas AI relies on computer algorithms to simulate human abilities, cognitive scientists investigate the brain to understand the underlying mechanisms. For most of human history, emotions were thought to be nothing more than disturbances for cognition. Therefore, they were usually ostracized from research on intelligence. As a result, cognitive architectures only partially include emotions in their design, if at all. Recently, though, it was discovered that emotions and cognition are in fact inter-dependent systems. Consequently, before being able to fully replicate human-level intelligence, it is necessary to understand the concept of emotions and its many roles within the brain. In this thesis, working around the lack of definition for emotion, I show that emotions inform the brain as to the nature of a given situation and guide the decision-making process, to increase the survival potential of virtual agents. In particular ProtoEmo, an architecture replicating the circuits found at the base of the forebrain, is shown to have the ability to detect stimuli relevant to the survival of virtual agents. Hence, it outperforms other emotional agents in terms of survival capabilities, which are measured by the size of the remaining population at the end of a resource foraging task. PrimEmo, the architecture born from the integration of ProtoEmo with standard models of the reward and decision-making systems in the brain, displays survival capabilities similar to the advantage actor-critic algorithm. PrimEmo also shows promises for supporting primitive emotions characterized by their level of arousal and valence. After further refinement, PrimEmo could replace the core decision-making module of a cognitive architecture, such as ACT-R or SOAR. Not only would it confer survival capabilities to the architecture, it would also allow for the possibility of investigating full-fledged emotions, and even emotional expression."]},{"key":"dc:format","label":"Dc Format","values":["text"]},{"key":"dcterms:accessrights","label":"Dcterms Accessrights","values":["public"]},{"key":"dc:title","label":"Title","values":["The Role of Emotions in Autonomous Social Agents"]}]}],"canonical_facts":{"dc:contributor.advisor":["Moore, Roger"],"dc:creator":["Caspar, Luc"],"dc:date":["2020-02"],"dc:date.issued":["2020-02"],"dc:description.abstract":["The holy grail of both AI and cognitive science is human-level intelligence. Whereas AI relies on computer algorithms to simulate human abilities, cognitive scientists investigate the brain to understand the underlying mechanisms. For most of human history, emotions were thought to be nothing more than disturbances for cognition. Therefore, they were usually ostracized from research on intelligence. As a result, cognitive architectures only partially include emotions in their design, if at all. Recently, though, it was discovered that emotions and cognition are in fact inter-dependent systems. Consequently, before being able to fully replicate human-level intelligence, it is necessary to understand the concept of emotions and its many roles within the brain. In this thesis, working around the lack of definition for emotion, I show that emotions inform the brain as to the nature of a given situation and guide the decision-making process, to increase the survival potential of virtual agents. In particular ProtoEmo, an architecture replicating the circuits found at the base of the forebrain, is shown to have the ability to detect stimuli relevant to the survival of virtual agents. Hence, it outperforms other emotional agents in terms of survival capabilities, which are measured by the size of the remaining population at the end of a resource foraging task. PrimEmo, the architecture born from the integration of ProtoEmo with standard models of the reward and decision-making systems in the brain, displays survival capabilities similar to the advantage actor-critic algorithm. PrimEmo also shows promises for supporting primitive emotions characterized by their level of arousal and valence. After further refinement, PrimEmo could replace the core decision-making module of a cognitive architecture, such as ACT-R or SOAR. Not only would it confer survival capabilities to the architecture, it would also allow for the possibility of investigating full-fledged emotions, and even emotional expression."],"dc:format":["text"],"dc:identifier":["uk.bl.ethos.823924"],"dc:identifier.uri":["https://etheses.whiterose.ac.uk/id/eprint/28366/1/thesis_Caspar_Luc_140127441_final_revision.pdf"],"dc:language.dcterms_iso639_2":["eng"],"dc:publisher.department":["Computer Science (Sheffield)"],"dc:publisher.institution":["University of Sheffield"],"dc:relation.isreferencedby":["https://etheses.whiterose.ac.uk/id/eprint/28366/"],"dc:relation.references":["Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., ... Zheng, X. (2015). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Adams, S., Arel, I., Bach, J., Coop, R., Furlan, R., Goertzel, B., ... Sowa, J. (2012, mar). Mapping the Landscape of Human-Level Artificial General Intelligence. AI Magazine, 33(1), 25. Adolphs, R., Gosselin, F., Buchanan, T. W., Tranel, D., Schyns, P., & Damasio, A. R. (2005, jan). A mechanism for impaired fear recognition after amygdala damage. Nature, 433(7021), 68–72. Aggleton, J. P. (1992). The Amygdala : neurobiological aspects of emotion, memory and mental dysfunction. New York: Wiley-Liss. Alexandrov, Y. I., & Sams, M. E. (2005, oct). Emotion and consciousness: ends of a continuum. Cognitive Brain Research, 25(2), 387–405. Anderson, A. K., Christoff, K., Stappen, I., Panitz, D., Ghahremani, D. G., Glover, G., ... Sobel, N. (2003, feb). Dissociated neural representations of intensity and valence in human olfaction. Nature neuroscience, 6(2), 196–202. Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036–1060. Arbib, M. A. (2003). The handbook of brain theory and neural networks (2nd ed. ed.; M. A. Arbib & I. NetLibrary, Eds.). Cambridge, Mass.: Cambridge, Mass. : MIT Press, c2003. Aristotle. (1926). The Nicomachean Ethics (H. Rackham, Trans.). Cambridge, MA: Harvard University Press. (Original work published 350 B.C) Armony, J. L., Servan-Schreiber, D., Cohen, J. D., & LeDoux, J. E. (1995, apr). An anatomically constrained neural network model of fear conditioning. Behavioral Neuroscience, 109(2), 246–57. Armony, J. L., Servan-Schreiber, D., Cohen, J. D., & LeDoux, J. E. (1997, apr). Computational modeling of emotion: explorations through the anatomy and physiology of fear conditioning. Trends in cognitive sciences, 1(1), 28–34. Armony, J. L., Servan-Schreiber, D., Romanski, L. M., Cohen, J. D., & LeDoux, J. E. (1997, mar). Stimulus generalization of fear responses: effects of auditory cortex lesions in a computational model and in rats. Cerebral cortex (New York, N.Y. : 991), 7(2), 157–65. Arnold, M. B. (1959). From perception to emotion and action. Acta Psychologica, 15, 406–417. Arnold, M. B. (1961). Emotion and personality. London: London : Cassell, 1961. Atallah, H. E., Frank, M. J., & O’Reilly, R. C. (2004, nov). Hippocampus, cortex, and basal ganglia: insights from computational models of complementary learning systems. Neurobiology of learning and memory, 82(3), 253–67. Baars, B. J. (2005). Global workspace theory of consciousness: Toward a cognitive neuroscience of human experience. Progress in Brain Research, 150, 45–53. Bard, P. (1929, aug). The central representation of the sympathetic system: As indicated by certain physiologic observations. Archives of Neurology And Psychiatry, 22(2), 230–246. Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983, sep). Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems. IEEE Transactions on Systems, Man and Cybernetics, SMC-13(5), 834–846. Bellman, R. (1957). Dynamic programming (R. Corporation, Ed.). Princeton, N.J. : London: Princeton, N.J. : Princeton U.P. Berke, J. D. (2018, jun). What does dopamine mean? (Vol. 21) (No. 6). Nature Publishing Group. Bush, P. C., & Sejnowski, T. J. (1995, oct). Models of Cortical Networks. In M. J. Gutnick & I. Mody (Eds.), The cortical neuron (pp. 174–190). Oxford University Press. Cannon, W. B. (1927, dec). The James-Lange Theory of Emotions: A Critical Examination and an Alternative Theory. The American Journal of Psychology, 39(1/4), 106. Cannon, W. B. (1929, dec). Bodily Changes in Pain, Hunger, Fear, and Rage. Archives of Internal Medicine, 44(6), 908. Cannon, W. B. (1931). Again the James-Lange and the thalamic theories of emotion. (Vol. 38). Cannon, W. B., Lewis, J. T., & Britton, S. W. (1927, sep). The Dispensability of the Sympathetic Division of the Autonomic Nervous System. The Boston Medical and Surgical Journal, 197(13), 514–515. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014, jun). Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Emnlp 2014 - 2014 conference on empirical methods in natural language processing, proceedings of the conference (pp. 1724–1734). Chollet, F. (2015). Keras: The Python Deep Learning library. Keras.Io. Collins, A. G. E., & Frank, M. J. (2013). Cognitive control over learning: creating, clustering, and generalizing task-set structure. Psychological review, 120(1), 190–229. Collins, A. G. E., & Frank, M. J. (2014). Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychological review, 121(3), 337–66. Connors, B. W., Landisman, C. E., & Reid, R. (1998, dec). Thalamus: Organization and Function (Vol. 1), Experimental and Clinical Aspects (Vol. 2). Trends in Neurosciences, 21(12), 539–540. Cools, R., Nakamura, K., & Daw, N. D. (2011, jan). Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology, 36(1), 98–13. Dalgleish, T. (2004, jul). The emotional brain. Nature Reviews Neuroscience, 5(7), 583–9. Damasio, A. (1997). The Feeling of What Happens: Body and Emotion in the Making of Consciousness. Vintage. Damasio, A. (2008). Descartes’ error: Emotion, reason and the human brain. Vintage. Darwin, C. (1859). On the Origin of Species (J. Murray, Ed.). London: John Murray. Darwin, C. (1872). The Expression of the Emotions in Man and Animals (J. Murray, Ed.). London: John Murray. Davidson, R. J. (2003, sep). Affective neuroscience and psychophysiology: Toward a synthesis. In Psychophysiology (Vol. 40, pp. 655–665). Society for Psychophysiological Research. Davidson, R. J., Scherer, K. R., & Goldsmith, H. H. (2003). Handbook of affective sciences. Oxford ; New York: Oxford University Press. Daw, N. D., Kakade, S., & Dayan, P. (2002, jun). Opponent interactions between serotonin and dopamine. Neural Networks, 15(4-6), 603–616. Dayan, P., & Abbott, L. F. (2001). Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Massachusetts Institute of Technology Press. den Ouden, H. E. M., Daw, N. D., Fernandez, G., Elshout, J. A., Rijpkema, M., Hoogman, M., ... Cools, R. (2013, nov). Dissociable effects of dopamine and serotonin on reversal learning. Neuron, 80(4), 1090–100. Descartes, R. (1637). Discourse on the Method. Philosophy, 1, xii, 249. Descartes, R. (1649). The Passions of the Soul. Hackett Publishing. Di Pietro, N. C., & Seamans, J. K. (2011, jun). Dopamine and serotonin interactively modulate prefrontal cortex neurons in vitro. Biological psychiatry, 69(12), 1204–11. Douglas, R. J., Koch, C., Mahowald, M., Martin, K. A., & Suarez, H. H. (1995, aug). Recurrent excitation in neocortical circuits. Science, 269(5226), 981–985. Ekman, P. (1973). Darwin and facial expression, a century of research in review (Vol. 23) (No. 4). Academic Press. Ekman, P. (1992). Are There Basic Emotions? Psychological Review, 99(3), 550–553. Eliasmith, C., & Anderson, C. H. (2003). Neural Engineering: Computation Representation and Dyamics in Neurobiological Systems (Vol. 19) (No. 5). MIT Press. Fellous, J.-m. (1999). The Neuromodulatory Basis of Emotion. The Neuroscientist, 5, 283–294. Fellous, J.-M., & Arbib, M. A. (2005a). Who needs emotions?: The Brain Meets the Robot. Oxford University Press. Fellous, J.-M., & Arbib, M. A. (2005b, apr). “Edison” and “Russell”. In J.-M. Fellous & M. A. Arbib (Eds.), Who needs emotions? the brain meets the robot (pp. 3–8). Oxford University Press. Fellous, J.-m., & LeDoux, J. E. (2005, mar). Toward Basic Principles for Emotional Processing. In J.-M. Fellous & M. A. Arbib (Eds.), Who needs emotions?: The brain meets the robot (pp. 79–116). Oxford University Press. Fontaine, J. J. R., Scherer, K. R., & Soriano, C. (2013). Components of Emotional Meaning (J. R. J. Fontaine, K. R. Scherer, & C. Soriano, Eds.). Oxford: Oxford University Press. Fortin, F.-A., De Rainville, F.-M., Gardner, M.-A. G., Parizeau, M., & Gagné, C. (2012, jul). DEAP: Evolutionary Algorithms Made Easy. Journal of Machine Learning Research, 13(1), 2171–2175. Frank, M. J. (2005, jan). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of cognitive neuroscience, 17(1), 51–72. Frank, M. J., & Claus, E. D. (2006, apr). Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychological review, 113(2), 300–26. Frank, M. J., Loughry, B., & O’Reilly, R. C. (2001, jun). Interactions between frontal cortex and basal ganglia in working memory: A computational model. Cognitive, 1(2), 137–160. Franklin, S., Madl, T., D’Mello, S., & Snaider, J. (2014, mar). LIDA: A Systems-level Architecture for Cognition, Emotion, and Learning. IEEE Transactions on Autonomous Mental Development, 6(1), 19–41. Freitas, J. D., Gudwin, R., & Queiroz, J. (2005). Emotion in artificial intelligence and artificial life research: Facing problems. IVA. Frijda, N. H. (1986). The Emotions. Frijda, N. H., & Parrott, W. G. (2011, sep). Basic Emotions or Ur-Emotions? Emotion Review, 3(4), 406–415. Girard, B., Cuzin, V., Guillot, A., Gurney, K. N., & Prescott, T. J. (2003, dec). A basal ganglia inspired model of action selection evaluated in a robotic survival task. Journal of Integrative Neuroscience, 2(2), 179–200. Glorot, X., Bordes, A., & Bengio, Y. (2011). Deep sparse rectifier neural networks. In Journal of machine learning research (Vol. 15, pp. 315–323). Goertzel, B. (2014). Artificial General Intelligence: Concept, State of the Art, and Future Prospects. Journal of Artificial General Intelligence, 5(1), 1–48. Goertzel, B. (2015). Artificial General Intelligence. Goleman, D. (1995). Emotional intelligence. Personality and Individual Differences, 9, 1091–1100. Grillner, S., Robertson, B., & Stephenson-Jones, M. (2013). The evolutionary origin of the vertebrate basal ganglia and its role in action selection. The Journal of Physiology. Gurney, K., Prescott, T. J., & Redgrave, P. (2001a, may). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biological Cybernetics, 84(6), 401–410. Gurney, K., Prescott, T. J., & Redgrave, P. (2001b, may). A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour. Biological cybernetics, 84(6), 411–423. Gurney, K., Prescott, T. J., Wickens, J. R., & Redgrave, P. (2004). Computational models of the basal ganglia: from robots to membranes. Trends in Neurosciences, 27(8), 453–459. Gutnick, M. J., & Mody, I. (2012). The Cortical Neuron (M. J. Gutnick & I. Mody, Eds.). Oxford University Press. Harlow, J. (1848). Passage of an Iron Rod through the Head. The Boston Medical and Surgical Journal, 39(20), 389–393. Harlow, J. M. (1868). Recovery from the passage of an iron bar through the head. Publications of the Massachusetts Medical Society, 2, 327–347. Hawkins, J., & Blakeslee, S. (2007). On intelligence. Times Books. Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2006). Banishing the homunculus: Making working memory work. Neuroscience, 139(1), 105–118. Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2010). Neural mechanisms of acquired phasic dopamine responses in learning. Neuroscience and Biobehavioral Reviews, 34(5), 701– 720. Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2011). Toward an executive without a homunculus: Computational models of the prefrontal cortex/basal ganglia system. In Modelling natural action selection (pp. 239–263). Hewitt, C., Bishop, P., & Steiger, R. (1973). A Universal Modular ACTOR Formalism for Artificial Intelligence. In Proceeding ijcai’73 proceedings of the 3rd international joint conference on artificial intelligence (pp. 235–245). Morgan Kaufmann Publishers Inc. Hochreiter, S., & Schmidhuber, J. (1997, nov). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780. Holovaty, A., & Willson, S. (2017). Django. Django Software Fundation. Houk, J. C. (2011). Action selection and refinement in subcortical loops through basal ganglia and cerebellum. In Modelling natural action selection. Ivanovic, M., Budimac, Z., Radovanovi´c, M., Kurbalija, V., Dai, W., B˘adic˘a, C., ... Mitrovi´c, D. (2015). Emotional agents — state of the art and applications. Computer Science and Information Systems, 12(4), 1121–1148. Izard, C. E. (1993, jan). Four systems for emotion activation: cognitive and noncognitive processes. Psychological review, 100(1), 68–90. James, W. (1884). What is an Emotion? Mind, 9, 188–205. Joel, D., Niv, Y., & Ruppin, E. (2002, jun). Actor-critic models of the basal ganglia: new anatomical and computational perspectives (Vol. 15) (No. 4-6). Pergamon. Jones, E. G. (1985). The thalamus. New York: New York. Karten, H. J., & Shimizu, T. (1991). Are Visual Hierarchies in the Brains of the Beholders? Constancy and Variability in the Visual System of Birds and Mammals. In The changing visual system (pp. 51–59). Boston, MA: Springer US. Kingma, D. P., & Ba, J. L. (2015, dec). Adam: A Method for Stochastic Optimization. In 3rd international conference on learning representations, iclr 2015 - conference track proceedings. International Conference on Learning Representations, ICLR. Kluver, H., & Bucy, P. C. (1937). \"Psychic blindness\" and other symptoms following bilateral temporal lobectomy in rhesus monkeys. American Journal of Physiology, 19, 352–353. Kreibig, S. D. (2010). Autonomic nervous system activity in emotion: A review. Laird, J. E., Newell, A., & Rosenbloom, P. S. (1987, sep). SOAR: An architecture for general intelligence. Artificial Intelligence, 33(1), 1–64. Lange, C. G. (1885). No Title. The Emotions, 33–90. Lazarus, R. S. (1966). Psychological stress and the coping process. New York: New York : McGraw-Hill, (1966). LeDoux, J. E. (1986). Sensory systems and emotion: A model of affective processing. Integrative Psychiatry, 4(4), 237–243. LeDoux, J. E. (1990). Information flow from sensation to emotion plasticity in the neural com- putation of stimulus values. In Learning and computational neuroscience: Foundations of adaptive networks (pp. 3–52). Bradford Books (MIT Press). LeDoux, J. E. (1992, apr). Brain mechanisms of emotion and emotional learning. Current Opinion in Neurobiology, 2(4), 191–197. LeDoux, J. E. (1995, jan). Emotion: clues from the brain. Annual Review of Psychology, 46(1), 209–235. LeDoux, J. E. (1998). The Emotional Brain: The Mysterious Underpinnings of Emotional Life. Phoenix. LeDoux, J. E. (2003). The emotional brain, fear, and the amygdala (Vol. 23). LeDoux, J. E. (2012, feb). Rethinking the emotional brain. Neuron, 73(4), 653–76. LeDoux, J. E. (2014, oct). Low roads and higher order thoughts in emotion. Cortex, 59, 214–215. Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., . . . Wierstra, D. (2016, sep). Continuous control with deep reinforcement learning. In 4th international conference on learning representations, iclr 2016 - conference track proceedings. Litt, A., Eliasmith, C., & Thagard, P. (2006). Why losses loom larger than gains: modeling neural mechanisms of cognitive-affective interaction. Proceedings of the Twenty-eighth Annual Meeting of the Cognitive Science Society, 495–500. Litt, A., Eliasmith, C., & Thagard, P. (2007). A large-scale neurocomputational model of emotional decision making. BMC Neuroscience, 8(Suppl 2), P14. Litt, A., Eliasmith, C., & Thagard, P. (2008). Neural affective decision theory: Choices, brains, and emotions. Cognitive Systems Research, 9(4), 252–273. Maas, A. L., Hannun, A. Y., & Ng, A. Y. (2013). Rectifier Nonlinearities Improve Neural Network Acoustic Models. Proceedings of the 30 th International Conference on Machine Learning, 30(1), 6. MacLean, P. D. (1949, nov). Psychosomatic Disease and the \"Visceral Brain\". Psychosomatic Medicine, 11(6), 338–353. MacLean, P. D. (1952, nov). Some psychiatric implications of physiological studies on frontotemporal portion of limbic system (visceral brain). Electroencephalography and Clinical Neurophysiology, 4(4), 407–18. MacLean, P. D. (1970). The triune brain, emotion and scientific bias. In F. Schmitt (Ed.), The neurosciences: Second study program (pp. 336–349). New York: The Rockefeller University Press. Maia, T. V. (2009). Reinforcement learning, conditioning, and the brain: Successes and challenges. Cognitive, Affective and Behavioral Neuroscience. Marsella, S., Gratch, J., & Petta, P. (2010). Computational models of emotion. In K. R. Scherer, T. Bänziger, & E. Roesch (Eds.), A blueprint for affective computing: A sourcebook and manual. Oxford University Press. Martinez-Miranda, J., & Aldea, A. (2005, mar). Emotions in human and artificial intelligence. Computers in Human Behavior, 21(2), 323–341. McClure, S. M., York, M. K., & Montague, P. R. (2004, jun). The Neural Substrates of Reward Processing in Humans: The Modern Role of fMRI. The Neuroscientist, 10(3), 260–268. McDougall, W. (1921). An Introduction to Social Psychology. J.W. Luce & Company. Mehrabian, A. (1980). Basic dimensions for a general psychological theory: Implications for personality, social, environmental, and developmental studies. (G. &. H. Oelgeschlager, Ed.). Cambridge, Mass. Mehrabian, A. (1996). Pleasure-Arousal-Dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4), 261–292. Miller, B. L., & Goldberg, D. E. (1995). Genetic algorithms, tournament selection, and the effects of noise. Complex Systems, 9, 193–212. Mnih, V., Badia, A. P., Mirza, L., Graves, A., Harley, T., Lillicrap, T. P., ... Kavukcuoglu, K. (2016). Asynchronous Methods for Deep Reinforcement Learning. 33rd International Conference on Machine Learning, ICML 2016, 4, 2850–2869. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013, dec). Playing Atari with Deep Reinforcement Learning. Nagel, T. (1974, oct). What Is It Like to Be a Bat? The Philosophical Review, 83(4), 435. Newell, A. (1973, jan). You can’t play 20 questions with nature and win: Projective comments on the papers of this symposium. Visual Information Processing, 283–308. Niv, Y. (2009, jun). Reinforcement learning in the brain. Journal of Mathematical Psychology, 53(3), 139–154. Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007, apr). Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology, 191(3), 507–20. Northcutt, G. R., & Kaas, J. H. (1995, sep). The emergence and evolution of mammalian neocortex. Trends in Neurosciences, 18(9), 373–379. Oatley, K., & Johnson-laird, P. N. (1987, mar). Towards a Cognitive Theory of Emotions. Cognition & Emotion, 1(1), 29–50. O’Doherty, J. P., Lee, S. W., & McNamee, D. (2015). The structure of reinforcement-learning mechanisms in the human brain (Vol. 1). O’Reilly, R. C., & Frank, M. J. (2006, feb). Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural computation, 18(2), 283–328. O’Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007). PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm. Behavioral Neuroscience, 121(1), 31–49. O’Reilly, R. C., & Munakata, Y. (2013). Computational Cognitive Neuroscience (No. 2001). Wiki Book. O’Reilly, R. C., Wyatte, D., & Rohrlich, J. (2014, jul). Learning Through Time in the Thalamocortical Loops. Ortony, A., Clore, G. L., & Collins, A. (1988). The Cognitive Structure of Emotions (Vol. 18). Cambridge: Cambridge University Press. Ortony, A., Norman, D. A., & Revelle, W. (2005). Affect and Proto-Affect in Effective Functioning. In Who needs emotions?: The brain meets the robot (pp. 173–202). Oxford University Press. Ortony, A., & Turner, T. J. (1990). What’s basic about basic emotions? Psychological Review, 97, 315–331. Osgood, C. E. C. E. (1957). The measurement of meaning (G. J. Suci & P. H. Tannenbaum, Eds.). Urbana [Ill.]: Urbana Ill. Panksepp, J. (1998). Affective Neuroscience: The Foundations of Human and Animal Emotions. Oxford University Press. Panksepp, J. (2005, mar). Affective consciousness: Core emotional feelings in animals and humans. Consciousness and Cognition, 14(1), 30–80. Panksepp, J., Mir, D., Delgado, J. M. R., & Edwards, S. (1982, sep). Toward a general psychobiological theory of emotions. Behavioral and Brain Sciences, 5(03), 407. Papez, J. W. (1937, jan). A proposed mechanism of emotion. The Journal of Neuropsychiatry and Clinical Neurosciences, 7(1), 103–12. Pavlov, I. P. (1927). Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. Oxford University Press: Humphrey Milford. Picard, R. W. (2000). Affective Computing. MIT Press. Plutchik, R. (1958, mar). Section of psychology: outlines of a new theory of emotion*. Transactions of the New York Academy of Sciences, 20(5 Series II), 394–403. Plutchik, R. (1962). The Emotions: Facts and Theories, and a New Model. Plutchik, R., & Conte, H. (1997). Circumplex models of personality and emotions. Potter, M. A., & De Jong, K. A. (2000, mar). Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evolutionary computation, 8(1), 1–29. Prescott, T. J., Gurney, K., Montes-Gonzalez, F., Humphries, M., & Redgrave, P. (2002). The robot basal ganglia: Action selection by an embedded model of the basal ganglia (Vol. 52). Prescott, T. J., Montes González, F. M., Gurney, K., Humphries, M. D., & Redgrave, P. (2006, jan). A robot model of the basal ganglia: Behavior and intrinsic processing. Neural Networks, 19(1), 31–61. Redgrave, P., Prescott, T. J., & Gurney, K. (1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience, 89(4), 1009–1023. Reisenzein, R. (2006, nov). Arnold’s theory of emotion in historical perspective. Cognition & Emotion, 20(7), 920-951. Rescorla, R. (2008, mar). Rescorla-Wagner model. Scholarpedia, 3(3), 2237. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Classical conditioning ii current research and theory (Vol. 21, pp. 64–99). Rohrer, B. (2010). What we might look for in an AGI benchmark. AGI Conference. Rolls, E. T. (2007). Emotion Explained. Oxford University Press. Rolls, E. T. (2013). Emotion and Decision-making Explained. Romanski, L. M., & LeDoux, J. E. (1992). Equipotentiality of thalamo-amygdala and thalamo- cortico-amygdala circuits in auditory fear conditioning. Journal of Neuroscience, 12(11), 4501–4509. Russell, J. a. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110(1), 145–172. Russell, J. A., & Barrett, L. F. (1999). Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. Journal of personality and social psychology, 76(5), 805–819. Ryle, G. (1949). The concept of mind. London: Hutchinson. Sander, D., Grandjean, D., & Scherer, K. R. (2005, may). A systems approach to appraisal mechanisms in emotion. Neural networks : the official journal of the International Neural Network Society, 18(4), 317–52. Saper, C. B. (2009, jan). Hypothalamus. Scholarpedia, 4(1), 2791. Saper, C. B. (2012). Hypothalamus. In J. Mai & G. Paxinos (Eds.), The human nervous system (pp. 548–583). Academic Press. Saper, C. B., & Lowell, B. B. (2014, dec). The hypothalamus. Current Biology, 24(23), R1111–R1116. Schachter, S., & Singer, J. (1962). Cognitive, social, and physiological determinants of emotional state. Psychological review, 69, 379–399. Scherer, K. R. (1984). On the Nature and Function of Emotion: A Component Process Approach. In K. R. Scherer & P. Ekman (Eds.), Approaches to emotion (pp. 293–317). Psychology Press. Scherer, K. R. (2005, dec). What are emotions? And how can they be measured? Social Science Information, 44(4), 695–729. Scherer, K. R. (2009, dec). Emotions are emergent processes: they require a dynamic computational architecture. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 364(1535), 3459–74. Scherer, K. R., Bänziger, T., & Roesch, E. (2010). A Blueprint for Affective Computing: A Sourcebook and Manual. Oxford University Press. Scherer, K. R., & Ekman, P. (1984). Approaches to emotion. L. Erlbaum Associates. Scherer, K. R., Schorr, A., & Johnstone, T. (2001). Appraisal processes in emotion: Theory, methods, research. Series in affective science. Scheutz, M. (2004). Useful roles of emotions in artificial agents: A case study from artificial life. In Proceedings of the 19th national conference on artificial intelligence (pp. 42–47). AAAI Press. Scheutz, M. (2012). The affect dilemma for artificial agents: Should we develop affective artificial agents? IEEE Transactions on Affective Computing, 3(4), 424–433. Schmajuk, N. (2008, mar). Classical conditioning. Scholarpedia, 3(3), 2316. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017, jul). Proximal Policy Optimization Algorithms. Schultz, W. (2000, dec). Multiple reward signals in the brain. Nature reviews. Neuroscience, 1(3), 199-207. Seamans, J. (2007, jun). Dopamine anatomy. Scholarpedia, 2(6), 3737. Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(3), 417–424. Shanahan, M. (2010). Embodiment and the Inner Life: Cognition and Consciousness in the Space of Possible Minds. Oxford University Press. Sherman, S. (2006, sep). Thalamus. Scholarpedia, 1(9), 1583. Sherman, S. M. (2016, mar). Thalamus plays a central role in ongoing cortical functioning. Nature Neuroscience, 16(4), 533–541. Sherman, S. M., & Guillery, R. W. (2002, dec). The role of the thalamus in the flow of information to the cortex. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 357(1428), 1695–708. Sherman, S. M., & Guillery, R. W. (2005). Exploring the Thalamus and Its Role in Cortical Function. Sloman, A. (1999). How Many Separately Evolved Emotional Beasties Live Within Us? Emotions in humans and artifacts. Sloman, A. (2001). Beyond shallow models of emotion. Cognitive Processing: International Quarterly of ..., 2, 177–198. Snaider, J., McCall, R., & Franklin, S. (2011). The LIDA framework as a general tool for AGI. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Vol. 6830 LNAI, pp. 133–142). Stent, G. S. (1973). A Physiological Mechanism for Hebb’s Postulate of Learning. Proceedings of the National Academy of Sciences of the United States of America, 70(4), 997–1001. Suri, R., & Schultz, W. (1999, jul). A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience, 91(3), 871–890. Sutton, R. S. (1984). Temporal Credit Assignment in Reinforcement Learning. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: an Introduction. Swanson, L. W. (2000, dec). Cerebral hemisphere regulation of motivated behavior. Brain Research, 886(1-2), 113–164. Thagard, P. (1989, sep). Explanatory coherence. Behavioral and Brain Sciences, 12(03), 435–502. Thagard, P. (2002). Coherence in Thought and Action. Place of publication not identified MIT Press, 2002. Thagard, P., & Aubie, B. (2008, sep). Emotional consciousness: A neural model of how cognitive appraisal and somatic perception interact to produce qualitative experience. Consciousness and Cognition, 17(3), 811–834. Thorndike, E. L. (1911). Animal intelligence. New York: The Macmillan company. Toga, A. W. (2015). Brain mapping : an encyclopedic reference (Vol. 1-3). Treue, S. (2001, may). Neural correlates of attention in primate visual cortex. Trends in Neurosciences, 24(5), 295–300. Turian, J., Bergstra, J., & Bengio, Y. (2009). Quadratic features and deep architectures for chunking. In Human language technologies: Conference of the north american association of computational linguistics (p. 245). Association for Computational Linguistics (ACL). Verschure, P., & Eds, A. D. (2014). Distributed Adaptive Control: Theory and Practice. Barcelona, Spain. Vitay, J., & Hamker, F. H. (2010). A computational model of Basal Ganglia and its role in memory retrieval in rewarded visual memory tasks. Wagar, B. M., & Thagard, P. (2004, jan). Spiking Phineas Gage: a neurocomputational theory of cognitive-affective integration in decision making. Psychological review, 111(1), 67–79. Wager, T. D., Kang, J., Johnson, T. D., Nichols, T. E., Satpute, A. B., & Barrett, L. F. (2015, apr). A Bayesian Model of Category-Specific Emotional Brain Responses. PLoS computational biology, 11(4), e1004066. Watkins, C. J. C. H. (1989). Learning from delayed rewards (Unpublished doctoral dissertation). Cambridge, England. Wells, F. L. (1925). Reactions to visual stimuli in affective settings. Journal of Experimental Psychology, 8(1), 64–76. Wells, F. L., Forbes, A., & Whitley, M. T. (1912, jan). On Certain Electrical Processes in the Human Body and Their Relation to Emotional Reactions. The American Journal of Psychology, 23(1), 149. Werbos, P. J. (1990). Backpropagation Through Time: What It Does and How to Do It. Proceedings of the IEEE, 78(10), 1550–1560. Whalen, P. J., & Phelps, E. A. (2009). The Human Amygdala. Guilford Press. Williams, R. J. (1992). Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning, 8(3), 229–256. Woergoetter, F., & Porr, B. (2008, mar). Reinforcement learning. Scholarpedia, 3(3), 1448. Woodworth, R. S., & Sherrington, C. S. (1904, jun). A pseudaffective reflex and its spinal path. The Journal of Physiology, 31(3-4), 234–243. Wundt, W. M. (1874). Grundzüge der physiologischen Psychologie, Volume 1. W. Engelman. Yates, J. R., Darna, M., Gipson, C. D., Dwoskin, L. P., & Bardo, M. T. (2015, may). Dissociable roles of dopamine and serotonin transporter function in a rat model of negative urgency. Behavioural brain research, 291, 201–208. Yilmazer-Hanke, D. M. (2012). Amygdala. In J. K. Mai & G. Paxinos (Eds.), The human nervous system (third edition) (Third Edit ed., pp. 759–834). San Diego: Academic Press. Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, 9(2), 1–27. Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences. American Psychologist, 35(2), 151-175."],"dcterms:accessrights":["public"],"dc:title":["The Role of Emotions in Autonomous Social Agents"],"dc:type":["Thesis"],"dc:type.qualificationlevel":["doctoral"],"dc:type.qualificationname":["Ph.D"],"person:identifier.orcid":["0000000285189433"]},"updated_at":"2026-11-18T22:41:20Z"}