University of West Attica
Textual summaries generation from unstructured data on social media using transformers
Abstract
dc:descriptionThe exponential growth of social media has produced an unprecedented volume of unstructured, user-generated content—informal, noisy, and rich in slang, abbreviations, emojis, and fragmented discourse. Extracting meaningful information from this content at scale is a critical challenge. Existing automatic summarization systems are primarily designed for formal text (news articles, academic papers, legal documents) and fail to handle the linguistic irregularities, redundancy, and platform-specific features that characterize social media interactions. At the same time, no purpose-built system existed for summarizing collections of user comments on social media posts—specifically the pool of replies and reactions that accumulate under a single post—as opposed to summarizing individual posts or news articles. This gap constitutes both the research problem and the primary motivation of the present thesis.The objective of this study is to design, implement, and evaluate the first dedicated system for abstractive summarization of social media comment pools. The system is intended to be robust to social media noise, computationally efficient for real-world deployment, and capable of generating summaries that are coherent, concise, and semantically faithful to the source discussions. A secondary objective is to establish a multidimensional evaluation framework that goes beyond traditional lexical metrics and captures the full quality profile of generated summaries.The proposed system is built upon the Text-to-Text Transfer Transformer (T5) architecture, specifically the T5-Small variant (76.9 million parameters), selected to balance performance with computational feasibility. The methodology encompasses four principal components. First, a domain-specific pre-processing pipeline was developed to normalize social media noise, including emoji-to-text conversion, hashtag segmentation, repeated-character normalization, and structured comment-thread tokenization using separator tokens ([COMMENT], [REPLY]). Second, three targeted architectural enhancements were introduced to the standard T5 model: (i) an adaptive attention scaling mechanism that adjusts attention temperature based on sequence length to prevent over-dispersion on short posts; (ii) a noise-aware attention mask that suppresses high-entropy tokens (e.g., repeated punctuation, hashtag strings) from influencing attention weights; and (iii) a context-aware positional encoding scheme with a trainable frequency parameter τ that reduces positional sensitivity for hashtag-heavy, order-independent content. Third, the training objective was augmented with a coverage loss term to penalize repeated attention to already-summarized input regions, directly addressing the redundancy typical of social media comment pools. Fourth, the model was fine-tuned on a purpose-built dataset of 15,000 Facebook post–comment pool pairs (12,000 training / 1,500 validation / 1,500 test), annotated with human-written reference summaries achieving an inter-annotator agreement of κ = 0.78 (Fleiss' kappa).Experimental results demonstrate that the proposed model achieves strong performance. On the final test set (Experiment 3, T5-Small), the model attains ROUGE-1 F1 = 0.3292, ROUGE-L F1 = 0.1841, and ROUGE-Lsum F1 = 0.1835. These scores reflect the inherent difficulty of social media summarization—where informal, fragmented source text limits n-gram overlap with formal reference summaries—and are consistent with state-of-the-art results reported in comparable tasks. In a direct comparison with ChatGPT 4.0 on the same dataset, the proposed model achieves superior F1 scores across all ROUGE variants (ROUGE-1: 0.3292 vs. 0.2739; ROUGE-L: 0.1841 vs. 0.1528), reflecting higher precision (0.3921 vs. 0.2557 for ROUGE-1) despite ChatGPT 4.0 achieving higher recall. This precision advantage—20.5% improvement in ROUGE-L—indicates that the proposed model generates more focused summaries with fewer extraneous tokens. The multidimensional evaluation framework confirms the model's superiority across a broader quality profile. On reference-free metrics, the proposed model outperforms ChatGPT 4.0 on Shannon Consistency (0.9512 vs. 0.9447), Shannon Coherence (0.9519 vs. 0.9471), SDC (0.5746 vs. 0.2646—a 117% improvement in semantic diversity capture), and SUPERT (0.9990 vs. 0.9977). On human-shaped metrics, Consistency reaches 0.5627 (vs. 0.2490 for ChatGPT 4.0, a 126% improvement), Lexical Diversity scores 0.7787 (vs. 0.6927), and ROUGE-WE (semantic word-embedding overlap) reaches 0.9568 vs. 0.9495. These results are statistically significant (paired t-test, p < 0.01) and confirm that task-specific fine-tuning of a compact encoder-decoder model yields superior domain performance over a general-purpose large language model with over 1 trillion parameters. The principal contributions of this research are as follows. (i) System novelty: The first purpose-built abstractive summarization system for social media comment pools, demonstrated on Facebook data across 15,000 post-comment pairs. (ii) Architectural innovations: Three novel enhancements to the transformer attention and positional encoding mechanisms, each specifically motivated by the properties of social media text and each validated with quantitative ablation evidence. (iii) A coverage-augmented training objective: A coverage loss formulation that directly reduces redundancy in generated summaries and improves consistency scores by 126% relative to ChatGPT 4.0. (iv) A multidimensional evaluation framework: An evaluation methodology combining ROUGE, ROUGE-WE, SUPERT, Shannon entropy metrics, SDC, and six human-shaped criteria (consistency, coherence, semantic coherence, lexical diversity, fluency, and readability), providing a reusable evaluation standard for social media summarization. (v) Empirical evidence of domain-adapted efficiency: Demonstration that a 76.9M-parameter fine-tuned model outperforms a 1T-parameter general-purpose model (ChatGPT 4.0) on a specialized task, with practical implications for cost-effective deployment of summarization systems.
Degree
thesis:*- Grantor dc:publisher
- University of West Attica
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Papagiannopoulou, Afroditi
- Παπαγιαννοπούλου, Αφροδίτη
Subjects
dc:subject × 17- Αφαιρετική Περίληψη
- Μοντέλα Μετασχηματιστή
- T5
- Επεξεργασία Φυσικής Γλώσσας Κοινωνικής Δικτύωσης
- Μεταφορά μάθησης
- Πολυδιάστατη Αξιολόγηση
- Abstractive Summarization
- Transformer models
- Social Media NLP
- Transfer learning
- Multidimensional Evaluation
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Αυτοματισμοί και Συστήματα ελέγχου
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Automation and control systems
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/62638