Georgia Institute of Technology
What are Foundation Models Cooking in the Post-Soviet World
Abstract
dc:description.abstractFood is a vital component of cultural knowledge, and understanding its nuances is essential for culturally aware models. In this study, we investigate the \textit{cultural food knowledge} of state-of-the-art models using \benchmarkname{}, a multimodal dataset encompassing 1147 and 823 dishes in the Russian and Ukrainian languages centered around the Post-Soviet region. We demonstrate that leading models struggle to correctly identify the origins of Post-Soviet dishes in both text-only and multimodal Question Answering (QA), even as they perform better on more globally recognized cuisines. Through analysis of pretraining data, we show that these results can be explained by misleading dish–origin co-occurrences, along with linguistic phenomena such as Russian–Ukrainian code mixing. Furthermore, to move beyond QA-based assessments, we test models’ ability to produce accurate visual descriptions of dishes. The weak correlation between this task and QA suggests that QA alone may be insufficient as a metric of cultural understanding.
Degree
thesis:*- Level thesis:degree_level
- Masters
- Department dc:contributor.department
- Interactive Computing
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Lavrouk, Anton
- Advisor dc:contributor.advisor
-
- Xu, Wei
- Committee members dc:contributor.committeemember
-
- Ritter, Alan
- Goyal, Kartik
Subjects
dc:subject × 8Rights
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/80140