One critical aspect that remains in the application
of state-of-the-art neural networks to text analysis in applied
research is the continued requirement for manual data annotation. In computer science research, there is a strong focus on
maximizing the data efficiency of fine-tuning language models.
This has led to the development of zero-shot text classification
methods, which promise to work effectively without requiring
fine-tuning for the specific task at hand. In this paper, we conduct
an in-depth analysis of aspect-based sentiment analysis in historic
German stock market reports to evaluate the reliability of this
promise. We present a comparison of a zero-shot approach
with a meticulously fine-tuned three-step process of training
and applying text classification models. This study aims to
empirically assess the reliability of zero-shot text classification
and provide justification for the potential benefits it offers in
terms of reducing the burden of data labeling and training for
analysis purposes. The findings of our study demonstrate a strong
correlation between the sentiment time series generated through
aspect-based sentiment analysis using the zero-shot approach and
those derived from the fine-tuned supervised pipeline, validating
the viability of the zero-shot approach. While the zero-shot
pipeline exhibits a tendency to underestimate negative examples,
the overall trend remains discernible. Additionally, a qualitative
analysis of the linguistic patterns reveals no explicit error
patterns. Nevertheless, we acknowledge and discuss the practical
and epistemological obstacles associated with employing zero-shot
algorithms in untested domains.
| Identifer | oai:union.ndltd.org:DRESDEN/oai:qucosa:de:qucosa:92377 |
| Date | 28 June 2024 |
| Creators | Borst-Graetz, Janos, Burghardt, Manuel, Niekler, Andreas, Wehrheim, Lino |
| Publisher | Polish Information Processing Society |
| Source Sets | Hochschulschriftenserver (HSSS) der SLUB Dresden |
| Language | English |
| Detected Language | English |
| Type | info:eu-repo/semantics/publishedVersion, doc-type:conferenceObject, info:eu-repo/semantics/conferenceObject, doc-type:Text |
| Rights | info:eu-repo/semantics/openAccess |
| Relation | https://doi.org/10.15439/2023F3725 |
Page generated in 0.0018 seconds