Back to search

University of West Attica

Textual summaries generation from unstructured data on social media using transformers

Abstract

dc:description

The exponential growth of social media has produced an unprecedented volume of unstructured, user-generated content—informal, noisy, and rich in slang, abbreviations, emojis, and fragmented discourse. Extracting meaningful information from this content at scale is a critical challenge. Existing automatic summarization systems are primarily designed for formal text (news articles, academic papers, legal documents) and fail to handle the linguistic irregularities, redundancy, and platform-specific features that characterize social media interactions. At the same time, no purpose-built system existed for summarizing collections of user comments on social media posts—specifically the pool of replies and reactions that accumulate under a single post—as opposed to summarizing individual posts or news articles. This gap constitutes both the research problem and the primary motivation of the present thesis.The objective of this study is to design, implement, and evaluate the first dedicated system for abstractive summarization of social media comment pools. The system is intended to be robust to social media noise, computationally efficient for real-world deployment, and capable of generating summaries that are coherent, concise, and semantically faithful to the source discussions. A secondary objective is to establish a multidimensional evaluation framework that goes beyond traditional lexical metrics and captures the full quality profile of generated summaries.The proposed system is built upon the Text-to-Text Transfer Transformer (T5) architecture, specifically the T5-Small variant (76.9 million parameters), selected to balance performance with computational feasibility. The methodology encompasses four principal components. First, a domain-specific pre-processing pipeline was developed to normalize social media noise, including emoji-to-text conversion, hashtag segmentation, repeated-character normalization, and structured comment-thread tokenization using separator tokens ([COMMENT], [REPLY]). Second, three targeted architectural enhancements were introduced to the standard T5 model: (i) an adaptive attention scaling mechanism that adjusts attention temperature based on sequence length to prevent over-dispersion on short posts; (ii) a noise-aware attention mask that suppresses high-entropy tokens (e.g., repeated punctuation, hashtag strings) from influencing attention weights; and (iii) a context-aware positional encoding scheme with a trainable frequency parameter τ that reduces positional sensitivity for hashtag-heavy, order-independent content. Third, the training objective was augmented with a coverage loss term to penalize repeated attention to already-summarized input regions, directly addressing the redundancy typical of social media comment pools. Fourth, the model was fine-tuned on a purpose-built dataset of 15,000 Facebook post–comment pool pairs (12,000 training / 1,500 validation / 1,500 test), annotated with human-written reference summaries achieving an inter-annotator agreement of κ = 0.78 (Fleiss' kappa).Experimental results demonstrate that the proposed model achieves strong performance. On the final test set (Experiment 3, T5-Small), the model attains ROUGE-1 F1 = 0.3292, ROUGE-L F1 = 0.1841, and ROUGE-Lsum F1 = 0.1835. These scores reflect the inherent difficulty of social media summarization—where informal, fragmented source text limits n-gram overlap with formal reference summaries—and are consistent with state-of-the-art results reported in comparable tasks. In a direct comparison with ChatGPT 4.0 on the same dataset, the proposed model achieves superior F1 scores across all ROUGE variants (ROUGE-1: 0.3292 vs. 0.2739; ROUGE-L: 0.1841 vs. 0.1528), reflecting higher precision (0.3921 vs. 0.2557 for ROUGE-1) despite ChatGPT 4.0 achieving higher recall. This precision advantage—20.5% improvement in ROUGE-L—indicates that the proposed model generates more focused summaries with fewer extraneous tokens. The multidimensional evaluation framework confirms the model's superiority across a broader quality profile. On reference-free metrics, the proposed model outperforms ChatGPT 4.0 on Shannon Consistency (0.9512 vs. 0.9447), Shannon Coherence (0.9519 vs. 0.9471), SDC (0.5746 vs. 0.2646—a 117% improvement in semantic diversity capture), and SUPERT (0.9990 vs. 0.9977). On human-shaped metrics, Consistency reaches 0.5627 (vs. 0.2490 for ChatGPT 4.0, a 126% improvement), Lexical Diversity scores 0.7787 (vs. 0.6927), and ROUGE-WE (semantic word-embedding overlap) reaches 0.9568 vs. 0.9495. These results are statistically significant (paired t-test, p < 0.01) and confirm that task-specific fine-tuning of a compact encoder-decoder model yields superior domain performance over a general-purpose large language model with over 1 trillion parameters. The principal contributions of this research are as follows. (i) System novelty: The first purpose-built abstractive summarization system for social media comment pools, demonstrated on Facebook data across 15,000 post-comment pairs. (ii) Architectural innovations: Three novel enhancements to the transformer attention and positional encoding mechanisms, each specifically motivated by the properties of social media text and each validated with quantitative ablation evidence. (iii) A coverage-augmented training objective: A coverage loss formulation that directly reduces redundancy in generated summaries and improves consistency scores by 126% relative to ChatGPT 4.0. (iv) A multidimensional evaluation framework: An evaluation methodology combining ROUGE, ROUGE-WE, SUPERT, Shannon entropy metrics, SDC, and six human-shaped criteria (consistency, coherence, semantic coherence, lexical diversity, fluency, and readability), providing a reusable evaluation standard for social media summarization. (v) Empirical evidence of domain-adapted efficiency: Demonstration that a 76.9M-parameter fine-tuned model outperforms a 1T-parameter general-purpose model (ChatGPT 4.0) on a specialized task, with practical implications for cost-effective deployment of summarization systems.

Degree

thesis:*
Grantor dc:publisher
University of West Attica
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Papagiannopoulou, Afroditi
  • Παπαγιαννοπούλου, Αφροδίτη

Subjects

dc:subject × 17

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/62638

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Papagiannopoulou, Afroditi; Παπαγιαννοπούλου, Αφροδίτη. Textual summaries generation from unstructured data on social media using transformers. University of West Attica, 2026. http://hdl.handle.net/10442/hedi/62638