Back to search

University of Nevada - Reno

Using Large Language Models in the Experimental Analysis of Persuasion and Bias

Abstract

dc:description.abstract

The United States of America currently faces high levels of political polarization, which has resulted in political violence including the events of January 6th, 2021. Traditional cable news networks such as FOX and MSNBC are in part responsible for this worsening polarization among American citizens. As America faces rising political tension in the 2024 election cycle, Artificial Intelligence-powered products such as Large Language Models may have the potential to pour fuel on the fire. This study uses the Relational Frame Theory of language and cognition to conceptualize how media content may contribute to polarization, and what the role of AI may be in the future of polarizing content. This study consisted of two experiments. Experiment 1 scraped transcripts from ABC, FOX, and MSNBC along with asking ChatGPT (an LLM developed by OpenAI) to produce monologues mimicking the hosts of popular shows on this network. These Real or AI transcripts were then compared using the Linguistic Inquiry and Word Count (LIWC) tool and the Inflexitext program to visually compare and contrast the linguistic elements used across these sources of media content. Experiment 2 consisted of a two by two ANOVA and ANCOVA to assess the main and interaction effects of media outlet (FOX or MSNBC) and monologue type (Real or AI) on a sample of 250 participants. The results of this experiment indicated that MSNBC monologues are perceived as more persuasive (p < .001) and credible (p < .001) than FOX monologues, and FOX monologues were viewed as more polarizing (p < .001). Additionally, MSNBC AI monologues were significantly more persuasive than MSNBC Real monologues (p < .05), indicating that LLMs may be capable of providing text that is more persuasive than real writers in some contexts. Implications for polarization in America are discussed.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy
Level thesis:degree_level
Doctorate Degree
Department thesis:degree_department
Psychology
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Falletta-Cowden, Neal A.
Advisor dc:contributor.advisor
  • Hayes, Steven
Committee members dc:contributor.committeemember
  • Houmanfar, Ramona
  • Lewon, Matthew
  • Hand, Emily
  • Yun, Gi

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • CC BY-NC-SA
  • Author(s)
Language dc:language.iso
en_US, English

Identifiers

dc:identifier.*
Repository record dc:identifier.uri
https://scholarwolf.unr.edu/handle/11714/11118

Chain of custody

source
Harvested from
University of Nevada - Reno
Base URL
scholarwolf.unr.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Falletta-Cowden, Neal A.. Using Large Language Models in the Experimental Analysis of Persuasion and Bias. Doctorate Degree thesis, 2024. https://scholarwolf.unr.edu/handle/11714/11118