Back to search

University of South Florida

Are you who AI says you are? Exploring the role of Natural Language Processing algorithms for “predicting” personality traits from text

Abstract

dc:description

Advances in text analytic approaches to measuring personality traits from text and speech have shown great promise for automating the assessment of personality. While a lot of effort has gone into developing robust algorithms for predicting personality ratings, more work is needed in understanding the function and theoretical relationships of these algorithmic ratings. This study proposes a novel application of the Trait-Reputation-Identity model (McAbee and Connelly, 2016), wherein, algorithm-generated ratings form the Reputation factor and are assumed to resemble learned observer ratings, the original self-report ratings form the Identity factor, and the total set of ratings form the consensus Trait factor. To test this model, a set of open-ended text responses to assessment center exercises, along with self-reported personality scale scores will be modelled using two machine learning approaches: a) classification model – XGBoost, and b) deep learning model – RoBERTa. These models generated personality trait scores for a reserved set of text responses by “learning” from the input text. These scores, along with a combined Ensemble score, were compared to the original set of self-report scores for convergence, and were to assess criterion-relatedness on a set of assessment center competency scores. Several traits were shown to moderately replicate the expected direction of convergence with traits and criteria, however, several other traits failed to converge meaningfully. We provide a detailed discussion on the philosophy of thinking about algorithmic scores as potential “observers” with limitations addressed for future researchers to be mindful of when using machine learning for personality assessment. A great need for collaboration between psychometricians and computer scientists is uncovered and heavily advised.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (Ph.D.)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Psychology
Grantor
University of South Florida
Year dc:date
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Raghavan, Mukhunth
Contributors dc:contributor
  • Brenton M. Wiernik, Ph.D.
  • Stephen Stark, Ph.D.
  • Marina A. Bornovalova, Ph.D.
  • Georgia Chao, Ph.D.
  • John Licato, Ph.D.

Subjects

dc:subject × 6

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/10668

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Raghavan, Mukhunth. Are you who AI says you are? Exploring the role of Natural Language Processing algorithms for “predicting” personality traits from text. Dissertation thesis, University of South Florida, 2024. https://digitalcommons.usf.edu/etd/10668