Back to search

University of North Texas

Generalization and Fairness Optimization in Pretrained Language Models

Abstract

dc:description

This study introduces an effective method to address the generalization challenge in pretrained language models (PLMs), which affects their performance on diverse linguistic data beyond their training scope. Improving PLMs' adaptability to out-of-distribution (OOD) data is essential for their reliability and practical utility in real-world applications. Furthermore, we address the ethical imperative of fairness in PLMs, particularly as they become integral to decision-making in sensitive societal sectors. We introduce gender-tuning, to identify and disrupt gender-related biases in training data. This method perturbs gendered terms, replacing them to break associations with other words. Gender-tuning stands as a practical, ethical intervention against gender bias in PLMs. Finally, we present FairAgent, a novel framework designed to imbue small language models (SLMs) with fairness, drawing on the knowledge of large language models (LLMs) without incurring the latter's computational costs. FairAgent operates by enabling SLMs to consult with LLMs, harnessing their vast knowledge to guide the generation of less biased content. This dynamic system not only detects bias in SLM responses but also generates prompts to correct it, accumulating effective prompts for future use. Over time, SLMs become increasingly adept at producing fair responses, enhancing both computational efficiency and fairness in AI-driven interactions.

Degree

thesis:*
Grantor dc:publisher
University of North Texas
Year dc:date
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ghanbar Zadeh, Somayeh
Contributors dc:contributor
  • Huang, Yan
  • Buckles, Bill P., 1942-
  • Blanco, Eduardo
  • Yuan, Jing

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Public
  • Ghanbar Zadeh, Somayeh
  • Copyright
  • Copyright is held by the author, unless otherwise noted. All rights Reserved.
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
local-cont-no: submission_3736
https://digital.library.unt.edu/ark:/67531/metadc2332571/
ark: ark:/67531/metadc2332571

Chain of custody

source
Harvested from
University of North Texas
Base URL
digital.library.unt.edu/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Ghanbar Zadeh, Somayeh. Generalization and Fairness Optimization in Pretrained Language Models. University of North Texas, 2024. https://doi.org/10.12794/metadc2332571