University of British Columbia
A bottom-up framework for cross-cultural evaluation of GPT-4o’s social norm biases via implicit narrative invocation
Abstract
dc:descriptionLarge Language Models (LLMs) have been demonstrated to align with the values of Western or North American cultures. Prior work predominantly showed this effect through leveraging surveys that directly ask – originally people and now also LLMs – about their values. However, it is not clear that these explicitly stated beliefs actually correspond to the slant that LLMs take in real tasks. To address that, we take a bottom-up approach, asking LLMs to recall cultural norms invoked by narratives from different cultures. We find that GPT-4o tends to generate norms that, while not necessarily incorrect, are significantly less culture-specific. In addition, while it avoids overtly generating stereotypes, the stereotypical representations of certain cultures are merely hidden rather than suppressed in the model, and such stereotypes can be easily recovered. Addressing these challenges is a crucial step towards developing LLMs that fairly serve their diverse user base.
Degree
thesis:*- Name thesis:degree_name
- Master of Science - MSc
- Level thesis:degree_level
- master's
- Discipline thesis:degree_discipline
- Computer Science
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Liu, Zhuozhuo
- Contributors dc:contributor
-
- Shwartz, Vered
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/92274