Human intelligence, with its many components, has been elusive. Until recently, the emphasis has been on facts and how humans perceive them. Now, it is time to embellish these facts with emotions and commentary. Emotional experiences and expressions play a critical role in human behavior and are influenced by language and cultural diversity. In this thesis, we explore the importance of emotions across multiple languages, such as Arabic, Chinese, and Spanish. In addition, we argue for the importance of collecting diverse emotional experiences including negative ones. We aim to develop AI systems that have a deeper understanding of emotional experiences. We open-source two datasets that emphasize diversity over emotions, language, and culture. ArtELingo contains affective annotations in the aforementioned languages, revealing valuable insights into how linguistic backgrounds shape emotional perception and expression. While ArtEmis 2.0 has a balanced distribution of positive and negative emotional experiences. Studying emotional experiences in AI is crucial for creating applications that genuinely understand and resonate with users.
We identify and tackle challenges in popular existing affective captioning datasets, mainly unbalanced emotion distribution, and generic captions, we pro- pose a contrastive data collection method. This approach results in a dataset with a balanced distribution of emotions, significantly enhancing the quality of trained neural speakers and emotion recognition models. Consequently, our trained speakers generate emotionally accurate and relevant captions, demonstrating the advantages of using a linguistically and emotionally diverse dataset in AI systems.
In addition, we explore the cultural aspects of emotional experiences and
expressions, highlighting the importance of considering cultural differences in the development of AI applications. By incorporating these insights, our research lays the groundwork for future advancements in culturally diverse affective computing.
This thesis establishes a foundation for future research in emotionally and culturally diverse affective computing, contributing to the development of AI applications capable of effectively understanding and engaging with humans on a deeper emotional level, regardless of their cultural background.
| Identifer | oai:union.ndltd.org:kaust.edu.sa/oai:repository.kaust.edu.sa:10754/691829 |
| Date | 03 1900 |
| Creators | MOHAMED, YOUSSEF SHERIF MANSOUR |
| Contributors | Elhoseiny, Mohamed, Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division, Wonka, Peter, Ghanem, Bernard |
| Source Sets | King Abdullah University of Science and Technology |
| Language | English |
| Detected Language | English |
| Type | Thesis |
| Relation | http://artelingo.org/, https://www.artemisdataset-v2.org/ |
Page generated in 0.0028 seconds